git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 0/3] Optimization batch 7: use file basenames to guide rename detection

From
Elijah Newren via GitGitGadget <gitgitgadget@gmail.com>
Date
Feb 6, 2021, 22:52 UTC
Message-ID
<pull.843.git.1612651937.gitgitgadget@gmail.com>

This series depends on ort-perf-batch-6[1], which has not yet appeared in seen despite being reviewed by both Junio and Stolee.

This series uses file basenames in a basic fashion to guide rename detection. It represents "Optimization #3" from my Git Merge 2020 talk[2], and is based on the fact that real world repositories tend to have a large majority of the renames they have done in history be ones that do not affect the basenames of the renamed files (in other words, they are simply moving files into different directories). For the testcases mentioned in commit 557ac0350d ("merge-ort: begin performance work; instrument with trace2_region_* calls", 2020-10-28), the changes in just this series improves the performance as follows:

                     Before Series           After Series
no-renames:       13.815 s ±  0.062 s    13.138 s ±  0.086 s
mega-renames:   1799.937 s ±  0.493 s   169.488 s ±  0.494 s
just-one-mega:    51.289 s ±  0.019 s     5.061 s ±  0.017 s

As a reminder, before any merge-ort/diffcore-rename performance work, the performance results we started with (as noted in the same commit message) were:

no-renames-am: 6.940 s ± 0.485 s no-renames: 18.912 s ± 0.174 s mega-renames: 5964.031 s ± 10.459 s just-one-mega: 149.583 s ± 0.751 s

[1] https://lore.kernel.org/git/xmqqlfc4byt6.fsf@gitster.c.googlers.com/ [2] https://github.com/newren/presentations/blob/pdfs/merge-performance/merge-performance-slides.pdf

Elijah Newren (3):
  diffcore-rename: compute basenames of all source and dest candidates
  diffcore-rename: complete find_basename_matches()
  diffcore-rename: guide inexact rename detection based on basenames
 diffcore-rename.c | 181 +++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 177 insertions(+), 4 deletions(-)
base-commit: 7ae9460d3dba84122c2674b46e4339b9d42bdedd
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-843%2Fnewren%2Fort-perf-batch-7-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-843/newren/ort-perf-batch-7-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/843
-- 
gitgitgadget
Next: Elijah Newren via GitGitGadget
Message 1 of 71 in “Optimization batch 7: use file basenames to guide rename detection”
  1. 0/3 Optimization batch 7: use file basenames to guide rename detectionElijah Newren via GitGitGadget, Feb 6, 2021
  2. 1/3 diffcore-rename: compute basenames of all source and dest candidatesElijah Newren via GitGitGadget, Feb 6, 2021
  3. 2/3 diffcore-rename: complete find_basename_matches()Elijah Newren via GitGitGadget, Feb 6, 2021
  4. 3/3 diffcore-rename: guide inexact rename detection based on basenamesElijah Newren via GitGitGadget, Feb 6, 2021
  5. Derrick StoleeFeb 7, 2021
  6. Junio C HamanoFeb 7, 2021
  7. Elijah NewrenFeb 8, 2021
  8. Derrick StoleeFeb 8, 2021
  9. Elijah NewrenFeb 8, 2021
  10. Junio C HamanoFeb 8, 2021
  11. Elijah NewrenFeb 8, 2021
  12. Junio C HamanoFeb 8, 2021
  13. Elijah NewrenFeb 8, 2021
  14. Elijah NewrenFeb 8, 2021
  15. Derrick StoleeFeb 8, 2021
  16. Elijah NewrenFeb 8, 2021
  17. Junio C HamanoFeb 7, 2021
  18. Elijah NewrenFeb 7, 2021
  19. 0/4 Optimization batch 7: use file basenames to guide rename detectionElijah Newren via GitGitGadget, Feb 9, 2021
  20. 2/4 diffcore-rename: complete find_basename_matches()Elijah Newren via GitGitGadget, Feb 9, 2021
  21. Derrick StoleeFeb 9, 2021
  22. Elijah NewrenFeb 9, 2021
  23. Derrick StoleeFeb 9, 2021
  24. 3/4 diffcore-rename: guide inexact rename detection based on basenamesElijah Newren via GitGitGadget, Feb 9, 2021
  25. Derrick StoleeFeb 9, 2021
  26. Elijah NewrenFeb 9, 2021
  27. Junio C HamanoFeb 9, 2021
  28. 1/4 diffcore-rename: compute basenames of all source and dest candidatesElijah Newren via GitGitGadget, Feb 9, 2021
  29. Derrick StoleeFeb 9, 2021
  30. Elijah NewrenFeb 9, 2021
  31. Derrick StoleeFeb 9, 2021
  32. Elijah NewrenFeb 9, 2021
  33. 4/4 gitdiffcore doc: mention new preliminary step for rename detectionElijah Newren via GitGitGadget, Feb 9, 2021
  34. Derrick StoleeFeb 9, 2021
  35. Junio C HamanoFeb 9, 2021
  36. Elijah NewrenFeb 9, 2021
  37. 0/5 Optimization batch 7: use file basenames to guide rename detectionElijah Newren via GitGitGadget, Feb 10, 2021
  38. 1/5 t4001: add a test comparing basename similarity and content similarityElijah Newren via GitGitGadget, Feb 10, 2021
  39. Junio C HamanoFeb 13, 2021
  40. Elijah NewrenFeb 13, 2021
  41. Junio C HamanoFeb 13, 2021
  42. Elijah NewrenFeb 14, 2021
  43. Junio C HamanoFeb 14, 2021
  44. Elijah NewrenFeb 14, 2021
  45. 2/5 diffcore-rename: compute basenames of all source and dest candidatesElijah Newren via GitGitGadget, Feb 10, 2021
  46. Junio C HamanoFeb 13, 2021
  47. 4/5 diffcore-rename: guide inexact rename detection based on basenamesElijah Newren via GitGitGadget, Feb 10, 2021
  48. Junio C HamanoFeb 13, 2021
  49. 3/5 diffcore-rename: complete find_basename_matches()Elijah Newren via GitGitGadget, Feb 10, 2021
  50. Junio C HamanoFeb 13, 2021
  51. Elijah NewrenFeb 13, 2021
  52. Junio C HamanoFeb 13, 2021
  53. Elijah NewrenFeb 14, 2021
  54. 5/5 gitdiffcore doc: mention new preliminary step for rename detectionElijah Newren via GitGitGadget, Feb 10, 2021
  55. Junio C HamanoFeb 10, 2021
  56. Elijah NewrenFeb 10, 2021
  57. 0/6 Optimization batch 7: use file basenames to guide rename detectionElijah Newren via GitGitGadget, Feb 11, 2021
  58. 1/6 t4001: add a test comparing basename similarity and content similarityElijah Newren via GitGitGadget, Feb 11, 2021
  59. 2/6 diffcore-rename: compute basenames of all source and dest candidatesElijah Newren via GitGitGadget, Feb 11, 2021
  60. 3/6 diffcore-rename: complete find_basename_matches()Elijah Newren via GitGitGadget, Feb 11, 2021
  61. 5/6 gitdiffcore doc: mention new preliminary step for rename detectionElijah Newren via GitGitGadget, Feb 11, 2021
  62. 4/6 diffcore-rename: guide inexact rename detection based on basenamesElijah Newren via GitGitGadget, Feb 11, 2021
  63. 6/6 merge-ort: call diffcore_rename() directlyElijah Newren via GitGitGadget, Feb 11, 2021
  64. Junio C HamanoFeb 13, 2021
  65. 0/6 Optimization batch 7: use file basenames to guide rename detectionElijah Newren via GitGitGadget, Feb 14, 2021
  66. 3/6 diffcore-rename: complete find_basename_matches()Elijah Newren via GitGitGadget, Feb 14, 2021
  67. 5/6 gitdiffcore doc: mention new preliminary step for rename detectionElijah Newren via GitGitGadget, Feb 14, 2021
  68. 1/6 t4001: add a test comparing basename similarity and content similarityElijah Newren via GitGitGadget, Feb 14, 2021
  69. 2/6 diffcore-rename: compute basenames of source and dest candidatesElijah Newren via GitGitGadget, Feb 14, 2021
  70. 4/6 diffcore-rename: guide inexact rename detection based on basenamesElijah Newren via GitGitGadget, Feb 14, 2021
  71. 6/6 merge-ort: call diffcore_rename() directlyElijah Newren via GitGitGadget, Feb 14, 2021

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.