git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2 13/13] merge-ort, diffcore-rename: employ cached renames when possible

From
Derrick Stolee <stolee@gmail.com>
Date
May 22, 2021, 11:17 UTC
Message-ID
<48f5d6f8-0ad9-0622-58a6-9887f797f4e7@gmail.com>
In-Reply-To
<CABPp-BFdxn9f0-jUjY6Ake_6kX-jeN1EEzpeJeTg+TV4wfepwg@mail.gmail.com>
On 5/19/21 8:36 PM, Elijah Newren wrote:
Show 47 quoted lines
> On Mon, May 17, 2021 at 7:23 AM Derrick Stolee <stolee@gmail.com> wrote:>> As I was reading, I was also thinking that it would be good to
>> have some kind of tracing, if only a summary of how often we
>> relied upon the cached renames.
> 
> That's an interesting thought.  If there are renames that remain
> cached, the code always uses them (whether we need the cached rename
> or not doesn't matter since we already have it and it is cheap to
> use), so a summary would probably be the only thing that would make
> sense.
> 
> The easiest summary of how often we rely upon cached renames, is
> checking if we have enough to avoid calling diffcore_rename_extended()
> entirely.  That's precisely what the code above does that you are
> responding to.
> 
> What more information could we get out?  I guess we could get ever so
> slightly more information by tracking how many times we decide that
> the cache from a previous merge can be re-used (by tracking the number
> of times that cached_valid_side is set to 1 or 2 instead of 0).  Since
> we sometimes have some cached renames but not enough to skip rename
> detection (because source paths that were irrelevant in previous
> commits are marked relevant for the current commit), this would be an
> independent number from the region_enter-diffcore_rename count used
> above.
> 
>> That would present a mechanism
>> for the test cases to verify that the rename cache is behaving
>> as expected
> 
> How so?  I did check something like that above with verifying I was
> able to reduce the number of calls to diffcore_rename_extended() while
> still getting the expected result (which can only be done if renames
> are known).  The only additional thing I can think of we could check
> would be a testcase where renames are cached and can be re-used but
> it's not enough to avoid calling diffcore_rename_extended().  Did you
> have something else in mind?
> 
>> but also provides a way to diagnose any issues that
>> might arise in the future by asking a user to reproduce a
>> problematic rebase/merge with a GIT_TRACE2* target. Such a
>> change would fit as a follow-up, and does not need to insert
>> into an already heavy patch.
> 
> I'm not sure what information could be recorded that would help
> diagnose any such issues.  "9 out of 10 commits in your rebase reused
> cached renames" just doesn't seem granular enough to help.  Is there
> something else you were thinking of recording?

My initial thought was to include basic summary statistics, such as "number of cached renames used" and "number of new renames" and "number of invalidated cached renames" or something, summarized per commit in the list. The information might not be a clear way to find a root cause to a strange rebase, but it would help someone looking for the root cause to know whether this rename cache is involved or not.

As for the use in the test cases, we might be able to see the list of trace2 results and extract the number of each type of statistic, matching them to our expectations. That might make the tests too fragile to future changes, though.

I'm commenting also on your v3 that we don't need to pursue the trace2 idea right now, and can do it if we find it necessary to diagnose any issues with the feature.

Thanks, -Stolee

Previous: Elijah NewrenNext: Elijah Newren via GitGitGadget
Message 39 of 61 in “Optimization batch 11: avoid repeatedly detecting same renames”
  1. 0/7 Optimization batch 11: avoid repeatedly detecting same renamesElijah Newren via GitGitGadget, Mar 24, 2021
  2. 2/7 merge-ort: populate caches of rename detection resultsElijah Newren via GitGitGadget, Mar 24, 2021
  3. 1/7 merge-ort: add data structures for in-memory caching of rename detectionElijah Newren via GitGitGadget, Mar 24, 2021
  4. 4/7 merge-ort: avoid accidental API mis-useElijah Newren via GitGitGadget, Mar 24, 2021
  5. 3/7 merge-ort: add code to check for whether cached renames can be reusedElijah Newren via GitGitGadget, Mar 24, 2021
  6. 5/7 merge-ort: preserve cached renames for the appropriate sideElijah Newren via GitGitGadget, Mar 24, 2021
  7. 6/7 merge-ort: add helper functions for using cached renamesElijah Newren via GitGitGadget, Mar 24, 2021
  8. 7/7 merge-ort, diffcore-rename: employ cached renames when possibleElijah Newren via GitGitGadget, Mar 24, 2021
  9. Junio C HamanoMar 24, 2021
  10. Elijah NewrenMar 24, 2021
  11. Junio C HamanoMar 25, 2021
  12. Elijah NewrenMar 29, 2021
  13. Derrick StoleeMar 30, 2021
  14. 00/13 Optimization batch 11: avoid repeatedly detecting same renamesElijah Newren via GitGitGadget, May 4, 2021
  15. 01/13 t6423: rename file within directory that other side renamedElijah Newren via GitGitGadget, May 4, 2021
  16. 03/13 fast-rebase: change assert() to BUG()Elijah Newren via GitGitGadget, May 4, 2021
  17. 04/13 fast-rebase: write conflict state to working tree, index, and HEADElijah Newren via GitGitGadget, May 4, 2021
  18. Derrick StoleeMay 17, 2021
  19. Elijah NewrenMay 18, 2021
  20. Derrick StoleeMay 18, 2021
  21. 02/13 Documentation/technical: describe remembering renames optimizationElijah Newren via GitGitGadget, May 4, 2021
  22. 07/13 merge-ort: populate caches of rename detection resultsElijah Newren via GitGitGadget, May 4, 2021
  23. Derrick StoleeMay 17, 2021
  24. Elijah NewrenMay 20, 2021
  25. 06/13 merge-ort: add data structures for in-memory caching of rename detectionElijah Newren via GitGitGadget, May 4, 2021
  26. Derrick StoleeMay 17, 2021
  27. Elijah NewrenMay 18, 2021
  28. Derrick StoleeMay 18, 2021
  29. 05/13 t6429: testcases for remembering renamesElijah Newren via GitGitGadget, May 4, 2021
  30. 08/13 merge-ort: add code to check for whether cached renames can be reusedElijah Newren via GitGitGadget, May 4, 2021
  31. Derrick StoleeMay 17, 2021
  32. 10/13 merge-ort: preserve cached renames for the appropriate sideElijah Newren via GitGitGadget, May 4, 2021
  33. 11/13 merge-ort: add helper functions for using cached renamesElijah Newren via GitGitGadget, May 4, 2021
  34. 12/13 merge-ort: handle interactions of caching and rename/rename(1to1) casesElijah Newren via GitGitGadget, May 4, 2021
  35. Derrick StoleeMay 17, 2021
  36. 13/13 merge-ort, diffcore-rename: employ cached renames when possibleElijah Newren via GitGitGadget, May 4, 2021
  37. Derrick StoleeMay 17, 2021
  38. Elijah NewrenMay 20, 2021
  39. Derrick StoleeMay 22, 2021
  40. 09/13 merge-ort: avoid accidental API mis-useElijah Newren via GitGitGadget, May 4, 2021
  41. Derrick StoleeMay 17, 2021
  42. Elijah NewrenMay 14, 2021
  43. Derrick StoleeMay 14, 2021
  44. 00/13 Optimization batch 11: avoid repeatedly detecting same renamesElijah Newren via GitGitGadget, May 20, 2021
  45. 01/13 t6423: rename file within directory that other side renamedElijah Newren via GitGitGadget, May 20, 2021
  46. 02/13 Documentation/technical: describe remembering renames optimizationElijah Newren via GitGitGadget, May 20, 2021
  47. Bagas SanjayaMay 20, 2021
  48. Kerry, RichardMay 20, 2021
  49. Elijah NewrenMay 20, 2021
  50. 03/13 fast-rebase: change assert() to BUG()Elijah Newren via GitGitGadget, May 20, 2021
  51. 04/13 fast-rebase: write conflict state to working tree, index, and HEADElijah Newren via GitGitGadget, May 20, 2021
  52. 05/13 t6429: testcases for remembering renamesElijah Newren via GitGitGadget, May 20, 2021
  53. 08/13 merge-ort: add code to check for whether cached renames can be reusedElijah Newren via GitGitGadget, May 20, 2021
  54. 07/13 merge-ort: populate caches of rename detection resultsElijah Newren via GitGitGadget, May 20, 2021
  55. 06/13 merge-ort: add data structures for in-memory caching of rename detectionElijah Newren via GitGitGadget, May 20, 2021
  56. 09/13 merge-ort: avoid accidental API mis-useElijah Newren via GitGitGadget, May 20, 2021
  57. 10/13 merge-ort: preserve cached renames for the appropriate sideElijah Newren via GitGitGadget, May 20, 2021
  58. 12/13 merge-ort: handle interactions of caching and rename/rename(1to1) casesElijah Newren via GitGitGadget, May 20, 2021
  59. 11/13 merge-ort: add helper functions for using cached renamesElijah Newren via GitGitGadget, May 20, 2021
  60. 13/13 merge-ort, diffcore-rename: employ cached renames when possibleElijah Newren via GitGitGadget, May 20, 2021
  61. Derrick StoleeMay 22, 2021

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.