git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2 00/10] Optimization batch 8: use file basenames even more

From
Elijah Newren <newren@gmail.com>
Date
Feb 25, 2021, 01:38 UTC
Message-ID
<CABPp-BH0NABbHqY1ZfcatiEryrqXLbv=qQKJKKzsGp-4=tWTEA@mail.gmail.com>
In-Reply-To
<22419bc6-d35a-7537-d388-d51c24138f5f@gmail.com>
On Wed, Feb 24, 2021 at 9:50 AM Derrick Stolee <stolee@gmail.com> wrote:
Show 36 quoted lines
>
> On 2/23/2021 6:43 PM, Elijah Newren via GitGitGadget wrote:
> > This series depends on en/diffcore-rename (a concatenation of what I was
> > calling ort-perf-batch-6 and ort-perf-batch-7).
> >
> > There are no changes since v1; it's just a resend a week and a half later to
> > bump it so it isn't lost.
>
> Thank you for re-sending. I intended to review it before but got redirected
> and forgot to pick it up again.
>
> > === Results ===
> >
> > For the testcases mentioned in commit 557ac0350d ("merge-ort: begin
> > performance work; instrument with trace2_region_* calls", 2020-10-28), the
> > changes in just this series improves the performance as follows:
> >
> >                      Before Series           After Series
> > no-renames:       12.775 s ±  0.062 s    12.596 s ±  0.061 s
> > mega-renames:    188.754 s ±  0.284 s   130.465 s ±  0.259 s
> > just-one-mega:     5.599 s ±  0.019 s     3.958 s ±  0.010 s
> >
> >
> > As a reminder, before any merge-ort/diffcore-rename performance work, the
> > performance results we started with (as noted in the same commit message)
> > were:
> >
> > no-renames-am:      6.940 s ±  0.485 s
> > no-renames:        18.912 s ±  0.174 s
> > mega-renames:    5964.031 s ± 10.459 s
> > just-one-mega:    149.583 s ±  0.751 s
>
> These are good results.
>
> I reviewed the patches and believe they do the optimizations claimed. I
> only found some nits for comments and whitespace things.

Thanks for taking a look; I'll get those fixed up. Also, I think your performance improvement of switching from xstrfmt to a few strbuf calls, even if small, counts as more than "nits for comments and whitespace things". :-)

Show 16 quoted lines
> You are very careful to create the necessary pieces and connect them
> from the bottom-up. However, this leads to one big "now everything is
> done" commit with performance improvements. It seems that there are
> some smaller performance improvements that could be measured if the
> logic was instead built from the top-down with stubs for the complicated
> logic.
>
> For example, the final patch links the rename logic with a call to
> idx_possible_rename(). But, that could just as well always return -1
> and the implementation would be correct. Then, it would be good to see
> if the performance changes with that non-functional update. It would
> also help me read the series in patch order and understand the context
> of the methods a bit better before seeing their implementation.
>
> This is _not_ a recommendation that you rewrite the series. Just food
> for thought as we continue with similar enhancements in the future.

I can give it a shot for future relevant patch series (some of the series this wouldn't be relevant for because they just include a collection of patches implementing separate improvements that are just batched together). A couple of the series are already structured this way, in fact, but the next series after this one has one patch that I think I could reorder to make it more like this.

Previous: Derrick StoleeNext: Elijah Newren via GitGitGadget
Message 34 of 61 in “Optimization batch 8: use file basenames even more”
  1. 00/10 Optimization batch 8: use file basenames even moreElijah Newren via GitGitGadget, Feb 14, 2021
  2. 02/10 diffcore-rename: add functions for clearing dir_rename_countElijah Newren via GitGitGadget, Feb 14, 2021
  3. 01/10 Move computation of dir_rename_count from merge-ort to diffcore-renameElijah Newren via GitGitGadget, Feb 14, 2021
  4. 03/10 diffcore-rename: move dir_rename_counts into a dir_rename_info structElijah Newren via GitGitGadget, Feb 14, 2021
  5. 04/10 diffcore-rename: extend cleanup_dir_rename_info()Elijah Newren via GitGitGadget, Feb 14, 2021
  6. 05/10 diffcore-rename: compute dir_rename_counts in stagesElijah Newren via GitGitGadget, Feb 14, 2021
  7. 06/10 diffcore-rename: add a mapping of destination names to their indicesElijah Newren via GitGitGadget, Feb 14, 2021
  8. 08/10 diffcore-rename: add a new idx_possible_rename functionElijah Newren via GitGitGadget, Feb 14, 2021
  9. 07/10 diffcore-rename: add a dir_rename_guess field to dir_rename_infoElijah Newren via GitGitGadget, Feb 14, 2021
  10. 10/10 diffcore-rename: use directory rename guided basename comparisonsElijah Newren via GitGitGadget, Feb 14, 2021
  11. 09/10 diffcore-rename: limit dir_rename_counts computation to relevant dirsElijah Newren via GitGitGadget, Feb 14, 2021
  12. 00/10 Optimization batch 8: use file basenames even moreElijah Newren via GitGitGadget, Feb 23, 2021
  13. 03/10 diffcore-rename: move dir_rename_counts into a dir_rename_info structElijah Newren via GitGitGadget, Feb 23, 2021
  14. 02/10 diffcore-rename: add functions for clearing dir_rename_countElijah Newren via GitGitGadget, Feb 23, 2021
  15. 01/10 Move computation of dir_rename_count from merge-ort to diffcore-renameElijah Newren via GitGitGadget, Feb 23, 2021
  16. Derrick StoleeFeb 24, 2021
  17. Elijah NewrenFeb 24, 2021
  18. 04/10 diffcore-rename: extend cleanup_dir_rename_info()Elijah Newren via GitGitGadget, Feb 23, 2021
  19. Derrick StoleeFeb 24, 2021
  20. Ævar Arnfjörð BjarmasonFeb 25, 2021
  21. Ævar Arnfjörð BjarmasonFeb 25, 2021
  22. Junio C HamanoFeb 25, 2021
  23. 08/10 diffcore-rename: add a new idx_possible_rename functionElijah Newren via GitGitGadget, Feb 23, 2021
  24. Derrick StoleeFeb 24, 2021
  25. Elijah NewrenFeb 25, 2021
  26. 06/10 diffcore-rename: add a mapping of destination names to their indicesElijah Newren via GitGitGadget, Feb 23, 2021
  27. 10/10 diffcore-rename: use directory rename guided basename comparisonsElijah Newren via GitGitGadget, Feb 23, 2021
  28. Derrick StoleeFeb 24, 2021
  29. 09/10 diffcore-rename: limit dir_rename_counts computation to relevant dirsElijah Newren via GitGitGadget, Feb 23, 2021
  30. 07/10 diffcore-rename: add a dir_rename_guess field to dir_rename_infoElijah Newren via GitGitGadget, Feb 23, 2021
  31. 05/10 diffcore-rename: compute dir_rename_counts in stagesElijah Newren via GitGitGadget, Feb 23, 2021
  32. Derrick StoleeFeb 24, 2021
  33. Derrick StoleeFeb 24, 2021
  34. Elijah NewrenFeb 25, 2021
  35. 00/10 Optimization batch 8: use file basenames even moreElijah Newren via GitGitGadget, Feb 26, 2021
  36. 01/10 diffcore-rename: use directory rename guided basename comparisonsElijah Newren via GitGitGadget, Feb 26, 2021
  37. 02/10 diffcore-rename: add a new idx_possible_rename functionElijah Newren via GitGitGadget, Feb 26, 2021
  38. Derrick StoleeFeb 26, 2021
  39. 03/10 diffcore-rename: add a mapping of destination names to their indicesElijah Newren via GitGitGadget, Feb 26, 2021
  40. 04/10 Move computation of dir_rename_count from merge-ort to diffcore-renameElijah Newren via GitGitGadget, Feb 26, 2021
  41. Derrick StoleeFeb 26, 2021
  42. 05/10 diffcore-rename: add function for clearing dir_rename_countElijah Newren via GitGitGadget, Feb 26, 2021
  43. 06/10 diffcore-rename: move dir_rename_counts into dir_rename_info structElijah Newren via GitGitGadget, Feb 26, 2021
  44. 07/10 diffcore-rename: extend cleanup_dir_rename_info()Elijah Newren via GitGitGadget, Feb 26, 2021
  45. 09/10 diffcore-rename: limit dir_rename_counts computation to relevant dirsElijah Newren via GitGitGadget, Feb 26, 2021
  46. 10/10 diffcore-rename: compute dir_rename_guess from dir_rename_countsElijah Newren via GitGitGadget, Feb 26, 2021
  47. 08/10 diffcore-rename: compute dir_rename_counts in stagesElijah Newren via GitGitGadget, Feb 26, 2021
  48. Derrick StoleeFeb 26, 2021
  49. Elijah NewrenFeb 26, 2021
  50. 00/10 Optimization batch 8: use file basenames even moreElijah Newren via GitGitGadget, Feb 27, 2021
  51. 03/10 diffcore-rename: add a mapping of destination names to their indicesElijah Newren via GitGitGadget, Feb 27, 2021
  52. 02/10 diffcore-rename: provide basic implementation of idx_possible_rename()Elijah Newren via GitGitGadget, Feb 27, 2021
  53. 01/10 diffcore-rename: use directory rename guided basename comparisonsElijah Newren via GitGitGadget, Feb 27, 2021
  54. 04/10 Move computation of dir_rename_count from merge-ort to diffcore-renameElijah Newren via GitGitGadget, Feb 27, 2021
  55. 05/10 diffcore-rename: add function for clearing dir_rename_countElijah Newren via GitGitGadget, Feb 27, 2021
  56. 06/10 diffcore-rename: move dir_rename_counts into dir_rename_info structElijah Newren via GitGitGadget, Feb 27, 2021
  57. 07/10 diffcore-rename: extend cleanup_dir_rename_info()Elijah Newren via GitGitGadget, Feb 27, 2021
  58. 09/10 diffcore-rename: limit dir_rename_counts computation to relevant dirsElijah Newren via GitGitGadget, Feb 27, 2021
  59. 08/10 diffcore-rename: compute dir_rename_counts in stagesElijah Newren via GitGitGadget, Feb 27, 2021
  60. 10/10 diffcore-rename: compute dir_rename_guess from dir_rename_countsElijah Newren via GitGitGadget, Feb 27, 2021
  61. Derrick StoleeMar 9, 2021

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.