git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 0/3] pack-objects: support bitmaps and delta-islands with `--path-walk`

From
Derrick Stolee <stolee@gmail.com>
Date
May 28, 2026, 15:28 UTC
Message-ID
<a708e23d-e0c2-48c9-86e9-1227f12edd53@gmail.com>
In-Reply-To
<cover.1779923907.git.me@ttaylorr.com>
On 5/27/26 7:18 PM, Taylor Blau wrote:
Show 25 quoted lines
> Here is a trimmed-down reroll of my series to make `--path-walk` work
> with reachability bitmaps and delta-islands. This series was originally
> an RFC that was a companion to Stolee's recent patches to extend
> `--filter` support to `--path-walk` [1].
> 
> Since the previous round, Stolee's series has graduated and incorporated
> the filter-related patches from my earlier RFC [2]. What remains are the
> three patches here that implement support for reachability bitmaps and
> delta-islands under `--path-walk`.
> 
>   * The first patch allows `--path-walk` to use reachability bitmaps when
>     they can answer the request, falling back to path-walk enumeration
>     when they cannot. It also lets bitmap writing see the same commit
>     candidates that the regular traversal would have shown to the bitmap
>     selector.
> 
>   * The second patch is preparatory, and factors the
>     delta-islands-specific tree-depth recording from `show_object()` into
>     a helper.
> 
>   * The final patch teaches the path-walk callback to perform the same
>     delta-islands side effects as the regular traversal: propagating
>     island marks for commits, and recording tree depths for trees. This
>     gives `resolve_tree_islands()` the same input in either enumeration
>     mode, so the existing island checks can be reused unchanged.

I've applied these patches locally and confirmed that each one passes the test suite with GIT_TEST_PACK_PATH_WALK=1, which helps to confirm that the changes are correct (all existing bitmap tests create and use the bitmaps with --path-walk unless explicitly disabled).

Should we add GIT_TEST_PACK_PATH_WALK=1 to the test-var CI build, now that this is going to be more commonly used?

Do you have any end-to-end performance data to demonstrate that these changes are effective at scale? Are we still producing packfiles with the pack-file compression and now with .bitmap files? How does this impact the performance of a clone or fetch when using a bitmap index at read time?

With that in mind, should we update any t/perf/ test to cover some of these scenarios? I'm running a few with GIT_TEST_PACK_PATH_WALK=1 on my laptop as a test, but it's taking a while. If you have stats ready from your local testing, then that would be interesting.

Thanks, -Stolee

Previous: Taylor BlauNext: Derrick Stolee
Message 5 of 33 in “pack-objects: support bitmaps and delta-islands with `--path-walk`”
  1. 0/3 pack-objects: support bitmaps and delta-islands with `--path-walk`Taylor Blau, May 27, 2026
  2. 1/3 pack-objects: support reachability bitmaps with `--path-walk`Taylor Blau, May 27, 2026
  3. 2/3 pack-objects: extract `record_tree_depth()` helperTaylor Blau, May 27, 2026
  4. 3/3 pack-objects: support `--delta-islands` with `--path-walk`Taylor Blau, May 27, 2026
  5. Derrick StoleeMay 28, 2026
  6. Derrick StoleeMay 29, 2026
  7. Taylor BlauMay 29, 2026
  8. Derrick StoleeMay 29, 2026
  9. Taylor BlauMay 29, 2026
  10. 0/4 pack-objects: support bitmaps and delta-islands with `--path-walk`Taylor Blau, Jun 2, 2026
  11. 1/4 t/perf: drop p5311's lookup-table permutationTaylor Blau, Jun 2, 2026
  12. 2/4 pack-objects: support reachability bitmaps with `--path-walk`Taylor Blau, Jun 2, 2026
  13. Derrick StoleeJun 12, 2026
  14. Taylor BlauJun 19, 2026
  15. Derrick StoleeJun 19, 2026
  16. Taylor BlauJun 19, 2026
  17. Derrick StoleeJun 12, 2026
  18. Taylor BlauJun 19, 2026
  19. Derrick StoleeJun 19, 2026
  20. Taylor BlauJun 19, 2026
  21. Derrick StoleeJun 19, 2026
  22. Junio C HamanoJun 15, 2026
  23. Taylor BlauJun 19, 2026
  24. 3/4 pack-objects: extract `record_tree_depth()` helperTaylor Blau, Jun 2, 2026
  25. 4/4 pack-objects: support `--delta-islands` with `--path-walk`Taylor Blau, Jun 2, 2026
  26. 0/4 pack-objects: support bitmaps and delta-islands with `--path-walk`Taylor Blau, Jun 21, 2026
  27. 1/4 t/perf: drop p5311's lookup-table permutationTaylor Blau, Jun 21, 2026
  28. 2/4 pack-objects: support reachability bitmaps with `--path-walk`Taylor Blau, Jun 21, 2026
  29. 3/4 pack-objects: extract `record_tree_depth()` helperTaylor Blau, Jun 21, 2026
  30. 4/4 pack-objects: support `--delta-islands` with `--path-walk`Taylor Blau, Jun 21, 2026
  31. Junio C HamanoJun 22, 2026
  32. Derrick StoleeJun 22, 2026
  33. Junio C HamanoJun 22, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.