git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2 00/10] Optimization batch 8: use file basenames even more

From
Derrick Stolee <stolee@gmail.com>
Date
Feb 24, 2021, 17:50 UTC
Message-ID
<22419bc6-d35a-7537-d388-d51c24138f5f@gmail.com>
In-Reply-To
<pull.844.v2.git.1614123848.gitgitgadget@gmail.com>
On 2/23/2021 6:43 PM, Elijah Newren via GitGitGadget wrote:
Show 5 quoted lines
> This series depends on en/diffcore-rename (a concatenation of what I was
> calling ort-perf-batch-6 and ort-perf-batch-7).
> 
> There are no changes since v1; it's just a resend a week and a half later to
> bump it so it isn't lost.

Thank you for re-sending. I intended to review it before but got redirected and forgot to pick it up again.

Show 20 quoted lines
> === Results ===
> 
> For the testcases mentioned in commit 557ac0350d ("merge-ort: begin
> performance work; instrument with trace2_region_* calls", 2020-10-28), the
> changes in just this series improves the performance as follows:
> 
>                      Before Series           After Series
> no-renames:       12.775 s ±  0.062 s    12.596 s ±  0.061 s
> mega-renames:    188.754 s ±  0.284 s   130.465 s ±  0.259 s
> just-one-mega:     5.599 s ±  0.019 s     3.958 s ±  0.010 s
> 
> 
> As a reminder, before any merge-ort/diffcore-rename performance work, the
> performance results we started with (as noted in the same commit message)
> were:
> 
> no-renames-am:      6.940 s ±  0.485 s
> no-renames:        18.912 s ±  0.174 s
> mega-renames:    5964.031 s ± 10.459 s
> just-one-mega:    149.583 s ±  0.751 s
These are good results.

I reviewed the patches and believe they do the optimizations claimed. I only found some nits for comments and whitespace things.

You are very careful to create the necessary pieces and connect them from the bottom-up. However, this leads to one big "now everything is done" commit with performance improvements. It seems that there are some smaller performance improvements that could be measured if the logic was instead built from the top-down with stubs for the complicated logic.

For example, the final patch links the rename logic with a call to idx_possible_rename(). But, that could just as well always return -1 and the implementation would be correct. Then, it would be good to see if the performance changes with that non-functional update. It would also help me read the series in patch order and understand the context of the methods a bit better before seeing their implementation.

This is _not_ a recommendation that you rewrite the series. Just food for thought as we continue with similar enhancements in the future.

Thanks, -Stolee

Previous: Derrick StoleeNext: Elijah Newren
Message 33 of 61 in “Optimization batch 8: use file basenames even more”
  1. 00/10 Optimization batch 8: use file basenames even moreElijah Newren via GitGitGadget, Feb 14, 2021
  2. 02/10 diffcore-rename: add functions for clearing dir_rename_countElijah Newren via GitGitGadget, Feb 14, 2021
  3. 01/10 Move computation of dir_rename_count from merge-ort to diffcore-renameElijah Newren via GitGitGadget, Feb 14, 2021
  4. 03/10 diffcore-rename: move dir_rename_counts into a dir_rename_info structElijah Newren via GitGitGadget, Feb 14, 2021
  5. 04/10 diffcore-rename: extend cleanup_dir_rename_info()Elijah Newren via GitGitGadget, Feb 14, 2021
  6. 05/10 diffcore-rename: compute dir_rename_counts in stagesElijah Newren via GitGitGadget, Feb 14, 2021
  7. 06/10 diffcore-rename: add a mapping of destination names to their indicesElijah Newren via GitGitGadget, Feb 14, 2021
  8. 08/10 diffcore-rename: add a new idx_possible_rename functionElijah Newren via GitGitGadget, Feb 14, 2021
  9. 07/10 diffcore-rename: add a dir_rename_guess field to dir_rename_infoElijah Newren via GitGitGadget, Feb 14, 2021
  10. 10/10 diffcore-rename: use directory rename guided basename comparisonsElijah Newren via GitGitGadget, Feb 14, 2021
  11. 09/10 diffcore-rename: limit dir_rename_counts computation to relevant dirsElijah Newren via GitGitGadget, Feb 14, 2021
  12. 00/10 Optimization batch 8: use file basenames even moreElijah Newren via GitGitGadget, Feb 23, 2021
  13. 03/10 diffcore-rename: move dir_rename_counts into a dir_rename_info structElijah Newren via GitGitGadget, Feb 23, 2021
  14. 02/10 diffcore-rename: add functions for clearing dir_rename_countElijah Newren via GitGitGadget, Feb 23, 2021
  15. 01/10 Move computation of dir_rename_count from merge-ort to diffcore-renameElijah Newren via GitGitGadget, Feb 23, 2021
  16. Derrick StoleeFeb 24, 2021
  17. Elijah NewrenFeb 24, 2021
  18. 04/10 diffcore-rename: extend cleanup_dir_rename_info()Elijah Newren via GitGitGadget, Feb 23, 2021
  19. Derrick StoleeFeb 24, 2021
  20. Ævar Arnfjörð BjarmasonFeb 25, 2021
  21. Ævar Arnfjörð BjarmasonFeb 25, 2021
  22. Junio C HamanoFeb 25, 2021
  23. 08/10 diffcore-rename: add a new idx_possible_rename functionElijah Newren via GitGitGadget, Feb 23, 2021
  24. Derrick StoleeFeb 24, 2021
  25. Elijah NewrenFeb 25, 2021
  26. 06/10 diffcore-rename: add a mapping of destination names to their indicesElijah Newren via GitGitGadget, Feb 23, 2021
  27. 10/10 diffcore-rename: use directory rename guided basename comparisonsElijah Newren via GitGitGadget, Feb 23, 2021
  28. Derrick StoleeFeb 24, 2021
  29. 09/10 diffcore-rename: limit dir_rename_counts computation to relevant dirsElijah Newren via GitGitGadget, Feb 23, 2021
  30. 07/10 diffcore-rename: add a dir_rename_guess field to dir_rename_infoElijah Newren via GitGitGadget, Feb 23, 2021
  31. 05/10 diffcore-rename: compute dir_rename_counts in stagesElijah Newren via GitGitGadget, Feb 23, 2021
  32. Derrick StoleeFeb 24, 2021
  33. Derrick StoleeFeb 24, 2021
  34. Elijah NewrenFeb 25, 2021
  35. 00/10 Optimization batch 8: use file basenames even moreElijah Newren via GitGitGadget, Feb 26, 2021
  36. 01/10 diffcore-rename: use directory rename guided basename comparisonsElijah Newren via GitGitGadget, Feb 26, 2021
  37. 02/10 diffcore-rename: add a new idx_possible_rename functionElijah Newren via GitGitGadget, Feb 26, 2021
  38. Derrick StoleeFeb 26, 2021
  39. 03/10 diffcore-rename: add a mapping of destination names to their indicesElijah Newren via GitGitGadget, Feb 26, 2021
  40. 04/10 Move computation of dir_rename_count from merge-ort to diffcore-renameElijah Newren via GitGitGadget, Feb 26, 2021
  41. Derrick StoleeFeb 26, 2021
  42. 05/10 diffcore-rename: add function for clearing dir_rename_countElijah Newren via GitGitGadget, Feb 26, 2021
  43. 06/10 diffcore-rename: move dir_rename_counts into dir_rename_info structElijah Newren via GitGitGadget, Feb 26, 2021
  44. 07/10 diffcore-rename: extend cleanup_dir_rename_info()Elijah Newren via GitGitGadget, Feb 26, 2021
  45. 09/10 diffcore-rename: limit dir_rename_counts computation to relevant dirsElijah Newren via GitGitGadget, Feb 26, 2021
  46. 10/10 diffcore-rename: compute dir_rename_guess from dir_rename_countsElijah Newren via GitGitGadget, Feb 26, 2021
  47. 08/10 diffcore-rename: compute dir_rename_counts in stagesElijah Newren via GitGitGadget, Feb 26, 2021
  48. Derrick StoleeFeb 26, 2021
  49. Elijah NewrenFeb 26, 2021
  50. 00/10 Optimization batch 8: use file basenames even moreElijah Newren via GitGitGadget, Feb 27, 2021
  51. 03/10 diffcore-rename: add a mapping of destination names to their indicesElijah Newren via GitGitGadget, Feb 27, 2021
  52. 02/10 diffcore-rename: provide basic implementation of idx_possible_rename()Elijah Newren via GitGitGadget, Feb 27, 2021
  53. 01/10 diffcore-rename: use directory rename guided basename comparisonsElijah Newren via GitGitGadget, Feb 27, 2021
  54. 04/10 Move computation of dir_rename_count from merge-ort to diffcore-renameElijah Newren via GitGitGadget, Feb 27, 2021
  55. 05/10 diffcore-rename: add function for clearing dir_rename_countElijah Newren via GitGitGadget, Feb 27, 2021
  56. 06/10 diffcore-rename: move dir_rename_counts into dir_rename_info structElijah Newren via GitGitGadget, Feb 27, 2021
  57. 07/10 diffcore-rename: extend cleanup_dir_rename_info()Elijah Newren via GitGitGadget, Feb 27, 2021
  58. 09/10 diffcore-rename: limit dir_rename_counts computation to relevant dirsElijah Newren via GitGitGadget, Feb 27, 2021
  59. 08/10 diffcore-rename: compute dir_rename_counts in stagesElijah Newren via GitGitGadget, Feb 27, 2021
  60. 10/10 diffcore-rename: compute dir_rename_guess from dir_rename_countsElijah Newren via GitGitGadget, Feb 27, 2021
  61. Derrick StoleeMar 9, 2021

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.