git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2 10/13] pack-objects: refactor path-walk delta phase

From
Taylor Blau <me@ttaylorr.com>
Date
May 29, 2025, 00:17 UTC
Message-ID
<aDent08iBbQrFWIB@nand.local>
In-Reply-To
<7ee3884e-b629-4b59-8e9a-a9a56abce786@gmail.com>
On Fri, May 16, 2025 at 12:27:29PM -0400, Derrick Stolee wrote:
Show 31 quoted lines
> > I imagine that "start" here is setup for the following commit which will
> > parallelize this task across multiple threads, with each thread starting
> > at a different position.
> >
> > I wonder if (as an alternative) we could get away with passing in a
> > (struct packing_region *, size_t) pair and drop "start". I think this
> > can work if you pass in the result of adding whatever the value of
> > "start" would be to to_pack.regions here.
> >
> > That somewhat hides the fact that this code is meant to be run across
> > multiple threads, but in a way that I think is worth doing. It lets the
> > function avoid having to do things like:
> >
> >    for (size_t i = start; i < start + nr; i++)
> >
> > and instead do the simpler:
> >
> >    for (size_t i = 0; i < nr; i++)
> >
> > since the caller already adjusted the regions pointer for us. As a
> > side-effect, it also means that the call below in prepare_pack() can
> > avoid passing a literal zero.
>
> I generally avoid this kind of pattern, because it makes it more
> difficult to understand what is going on when debugging. The cost
> of adding 'start' to the index seems low relative to having a
> consistent indexing scheme when accessing from the "global" list
> of items.
>
> This only gets more complicated when the threaded version iterates
> on a "consecutive sublist of regions".

That's fair. After thinking about it some more, I agree that enumerating [start, start + nr) is clearer than enumerating [0, nr) and adjusting the offset at each invocation of the loop body.

Funny enough, I applied that suggestion in a couple of spots in one of the incremental MIDX patches I sent recently.

Show 12 quoted lines
> > This is an extreme nitpick, but I think that "/**" (as opposed to "/*")
> > is preferred, as the former is used for Doxygen-style comments. I feel
> > like I have seen Junio comment on this before, but searching 'f:Junio
> > "/**"' yields a measly 83,000+ results, so I am unlikely to find it ;-).
> >
> > (Not worth rerolling on its own, but if you are rerolling anyway, which
> > is what I gathered from some of your earlier replies, it may be worth
> > picking up along the way.)
>
> I'm confused. Are you saying I should change from "/**" to "/*" (which
> is the opposite of your preferred statement) or did you misread that I
> had used "/**"? I can't determine which statement is the intended one.

Oops, this is a typo on my part. I was saying that we should prefer "/*" to "/**".

The "/**"-style lives on in v4, but this is *extremely* minor and I don't think is worth sending another round when everything (of what I have read so far from v4) looks good to me.

Thanks, Taylor

Previous: Derrick StoleeNext: Derrick Stolee via GitGitGadget
Message 52 of 75 in “PATH WALK II: Add --path-walk option to 'git pack-objects'”
  1. 00/13 PATH WALK II: Add --path-walk option to 'git pack-objects'Derrick Stolee via GitGitGadget, Mar 10, 2025
  2. 01/13 pack-objects: extract should_attempt_deltas()Derrick Stolee via GitGitGadget, Mar 10, 2025
  3. Taylor BlauMar 12, 2025
  4. Derrick StoleeMar 20, 2025
  5. 03/13 pack-objects: update usage to match docsDerrick Stolee via GitGitGadget, Mar 10, 2025
  6. Taylor BlauMar 12, 2025
  7. 02/13 pack-objects: add --path-walk optionDerrick Stolee via GitGitGadget, Mar 10, 2025
  8. Taylor BlauMar 12, 2025
  9. Derrick StoleeMar 20, 2025
  10. 04/13 p5313: add performance tests for --path-walkDerrick Stolee via GitGitGadget, Mar 10, 2025
  11. 05/13 pack-objects: introduce GIT_TEST_PACK_PATH_WALKDerrick Stolee via GitGitGadget, Mar 10, 2025
  12. 06/13 t5538: add tests to confirm deltas in shallow pushesDerrick Stolee via GitGitGadget, Mar 10, 2025
  13. 07/13 repack: add --path-walk optionDerrick Stolee via GitGitGadget, Mar 10, 2025
  14. 08/13 pack-objects: enable --path-walk via configDerrick Stolee via GitGitGadget, Mar 10, 2025
  15. 09/13 scalar: enable path-walk during push via configDerrick Stolee via GitGitGadget, Mar 10, 2025
  16. 10/13 pack-objects: refactor path-walk delta phaseDerrick Stolee via GitGitGadget, Mar 10, 2025
  17. Taylor BlauMar 12, 2025
  18. Derrick StoleeMar 20, 2025
  19. 11/13 pack-objects: thread the path-based compressionDerrick Stolee via GitGitGadget, Mar 10, 2025
  20. 12/13 path-walk: add new 'edge_aggressive' optionDerrick Stolee via GitGitGadget, Mar 10, 2025
  21. 13/13 pack-objects: allow --shallow and --path-walkDerrick Stolee via GitGitGadget, Mar 10, 2025
  22. Junio C HamanoMar 10, 2025
  23. Taylor BlauMar 12, 2025
  24. Derrick StoleeMar 20, 2025
  25. 00/13 PATH WALK II: Add --path-walk option to 'git pack-objects'Derrick Stolee via GitGitGadget, Mar 24, 2025
  26. 01/13 pack-objects: extract should_attempt_deltas()Derrick Stolee via GitGitGadget, Mar 24, 2025
  27. Taylor BlauMay 2, 2025
  28. 02/13 pack-objects: add --path-walk optionDerrick Stolee via GitGitGadget, Mar 24, 2025
  29. Taylor BlauMay 2, 2025
  30. Derrick StoleeMay 6, 2025
  31. Derrick StoleeMay 16, 2025
  32. 03/13 pack-objects: update usage to match docsDerrick Stolee via GitGitGadget, Mar 24, 2025
  33. 04/13 p5313: add performance tests for --path-walkDerrick Stolee via GitGitGadget, Mar 24, 2025
  34. Taylor BlauMay 2, 2025
  35. 06/13 t5538: add tests to confirm deltas in shallow pushesDerrick Stolee via GitGitGadget, Mar 24, 2025
  36. Taylor BlauMay 2, 2025
  37. Derrick StoleeMay 16, 2025
  38. 05/13 pack-objects: introduce GIT_TEST_PACK_PATH_WALKDerrick Stolee via GitGitGadget, Mar 24, 2025
  39. Taylor BlauMay 2, 2025
  40. Derrick StoleeMay 6, 2025
  41. 07/13 repack: add --path-walk optionDerrick Stolee via GitGitGadget, Mar 24, 2025
  42. Taylor BlauMay 2, 2025
  43. 08/13 pack-objects: enable --path-walk via configDerrick Stolee via GitGitGadget, Mar 24, 2025
  44. Taylor BlauMay 2, 2025
  45. Derrick StoleeMay 6, 2025
  46. Derrick StoleeMay 16, 2025
  47. 09/13 scalar: enable path-walk during push via configDerrick Stolee via GitGitGadget, Mar 24, 2025
  48. Taylor BlauMay 7, 2025
  49. 10/13 pack-objects: refactor path-walk delta phaseDerrick Stolee via GitGitGadget, Mar 24, 2025
  50. Taylor BlauMay 7, 2025
  51. Derrick StoleeMay 16, 2025
  52. Taylor BlauMay 29, 2025
  53. 12/13 path-walk: add new 'edge_aggressive' optionDerrick Stolee via GitGitGadget, Mar 24, 2025
  54. 11/13 pack-objects: thread the path-based compressionDerrick Stolee via GitGitGadget, Mar 24, 2025
  55. Taylor BlauMay 7, 2025
  56. 13/13 pack-objects: allow --shallow and --path-walkDerrick Stolee via GitGitGadget, Mar 24, 2025
  57. Junio C HamanoMay 2, 2025
  58. Taylor BlauMay 2, 2025
  59. Taylor BlauMay 2, 2025
  60. Taylor BlauMay 7, 2025
  61. 00/13 PATH WALK II: Add --path-walk option to 'git pack-objects'Derrick Stolee via GitGitGadget, May 16, 2025
  62. 01/13 pack-objects: extract should_attempt_deltas()Derrick Stolee via GitGitGadget, May 16, 2025
  63. 02/13 pack-objects: add --path-walk optionDerrick Stolee via GitGitGadget, May 16, 2025
  64. 03/13 pack-objects: update usage to match docsDerrick Stolee via GitGitGadget, May 16, 2025
  65. 04/13 p5313: add performance tests for --path-walkDerrick Stolee via GitGitGadget, May 16, 2025
  66. 05/13 pack-objects: introduce GIT_TEST_PACK_PATH_WALKDerrick Stolee via GitGitGadget, May 16, 2025
  67. 06/13 t5538: add tests to confirm deltas in shallow pushesDerrick Stolee via GitGitGadget, May 16, 2025
  68. 07/13 repack: add --path-walk optionDerrick Stolee via GitGitGadget, May 16, 2025
  69. 08/13 pack-objects: enable --path-walk via configDerrick Stolee via GitGitGadget, May 16, 2025
  70. 09/13 scalar: enable path-walk during push via configDerrick Stolee via GitGitGadget, May 16, 2025
  71. 10/13 pack-objects: refactor path-walk delta phaseDerrick Stolee via GitGitGadget, May 16, 2025
  72. 11/13 pack-objects: thread the path-based compressionDerrick Stolee via GitGitGadget, May 16, 2025
  73. 12/13 path-walk: add new 'edge_aggressive' optionDerrick Stolee via GitGitGadget, May 16, 2025
  74. 13/13 pack-objects: allow --shallow and --path-walkDerrick Stolee via GitGitGadget, May 16, 2025
  75. Taylor BlauMay 29, 2025

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.