git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 08/17] pack-objects: add --path-walk option

From
Taylor Blau <me@ttaylorr.com>
Date
Oct 29, 2024, 18:07 UTC
Message-ID
<ZyEkWyB/C/lGb36b@nand.local>
In-Reply-To
<20241028195404.4119175-1-jonathantanmy@google.com>
On Mon, Oct 28, 2024 at 12:54:04PM -0700, Jonathan Tan wrote:
Show 24 quoted lines
> "Derrick Stolee via GitGitGadget" <gitgitgadget@gmail.com> writes:
> > This new walk is incompatible with some features and is ignored by
> > others:
> >
> >  * Object filters are not currently integrated with the path-walk API,
> >    such as sparse-checkout or tree depth. A blobless packfile could be
> >    integrated easily, but that is deferred for later.
> >
> >  * Server-focused features such as delta islands, shallow packs, and
> >    using a bitmap index are incompatible with the path-walk API.
> >
> >  * The path walk API is only compatible with the --revs option, not
> >    taking object lists or pack lists over stdin. These alternative ways
> >    to specify the objects currently ignores the --path-walk option
> >    without even a warning.
>
> It might be better to declare --path-walk as "internal use only" and
> only supporting what send-pack.c (used by "git push") and "git repack"
> needs. (From this list, it seems that there is a lot of incompatibility,
> some of which can happen without a warning to the user, so it sounds
> better to be up-front and say that we only support what send-pack.c
> needs. This also makes reviewing easier, as we don't have to think about
> the possible interactions with every other rev-list feature - only what
> is used by send-pack.c.)

Is the thinking there that we care mostly about 'git push' and 'git repack' on the client-side?

I don't think it's unreasonable necessarily, but I would add that client-side users definitely do use bitmaps (though not delta islands), either when working in a bare repository (where bitmaps are the default) or when using 'git gc' (and/or through 'git maintenance') when 'repack.writeBitmaps' is enabled.

So I think the approach here would be to limit it to some cases of client side behavior, but we should keep in mind that it will not cover all cases.

My feeling is that it would be nice to pull on the incompatibility string a little more and see if we can't make the two work together without too much effort.

Thanks, Taylor

Previous: Jonathan TanNext: Jonathan Tan
Message 11 of 55 in “pack-objects: add --path-walk option for better deltas”
  1. 00/17 pack-objects: add --path-walk option for better deltasDerrick Stolee via GitGitGadget, Oct 8, 2024
  2. 01/17 path-walk: introduce an object walk by pathDerrick Stolee via GitGitGadget, Oct 8, 2024
  3. 02/17 t6601: add helper for testing path-walk APIDerrick Stolee via GitGitGadget, Oct 8, 2024
  4. 03/17 path-walk: allow consumer to specify object typesDerrick Stolee via GitGitGadget, Oct 8, 2024
  5. 04/17 path-walk: allow visiting tagsDerrick Stolee via GitGitGadget, Oct 8, 2024
  6. 05/17 revision: create mark_trees_uninteresting_dense()Derrick Stolee via GitGitGadget, Oct 8, 2024
  7. 06/17 path-walk: add prune_all_uninteresting optionDerrick Stolee via GitGitGadget, Oct 8, 2024
  8. 07/17 pack-objects: extract should_attempt_deltas()Derrick Stolee via GitGitGadget, Oct 8, 2024
  9. 08/17 pack-objects: add --path-walk optionDerrick Stolee via GitGitGadget, Oct 8, 2024
  10. Jonathan TanOct 28, 2024
  11. Taylor BlauOct 29, 2024
  12. Jonathan TanOct 29, 2024
  13. Taylor BlauOct 29, 2024
  14. Derrick StoleeOct 31, 2024
  15. Derrick StoleeOct 31, 2024
  16. Taylor BlauOct 31, 2024
  17. Derrick StoleeOct 31, 2024
  18. 09/17 pack-objects: update usage to match docsDerrick Stolee via GitGitGadget, Oct 8, 2024
  19. 10/17 p5313: add performance tests for --path-walkDerrick Stolee via GitGitGadget, Oct 8, 2024
  20. 11/17 pack-objects: introduce GIT_TEST_PACK_PATH_WALKDerrick Stolee via GitGitGadget, Oct 8, 2024
  21. 12/17 repack: add --path-walk optionDerrick Stolee via GitGitGadget, Oct 8, 2024
  22. 13/17 repack: update usage to match docsDerrick Stolee via GitGitGadget, Oct 8, 2024
  23. 14/17 pack-objects: enable --path-walk via configDerrick Stolee via GitGitGadget, Oct 8, 2024
  24. 15/17 scalar: enable path-walk during push via configDerrick Stolee via GitGitGadget, Oct 8, 2024
  25. 16/17 pack-objects: refactor path-walk delta phaseDerrick Stolee via GitGitGadget, Oct 8, 2024
  26. 17/17 pack-objects: thread the path-based compressionDerrick Stolee via GitGitGadget, Oct 8, 2024
  27. 00/17 pack-objects: add --path-walk option for better deltasDerrick Stolee via GitGitGadget, Oct 20, 2024
  28. 01/17 path-walk: introduce an object walk by pathDerrick Stolee via GitGitGadget, Oct 20, 2024
  29. 02/17 t6601: add helper for testing path-walk APIDerrick Stolee via GitGitGadget, Oct 20, 2024
  30. 03/17 path-walk: allow consumer to specify object typesDerrick Stolee via GitGitGadget, Oct 20, 2024
  31. 04/17 path-walk: allow visiting tagsDerrick Stolee via GitGitGadget, Oct 20, 2024
  32. 05/17 revision: create mark_trees_uninteresting_dense()Derrick Stolee via GitGitGadget, Oct 20, 2024
  33. 06/17 path-walk: add prune_all_uninteresting optionDerrick Stolee via GitGitGadget, Oct 20, 2024
  34. 07/17 pack-objects: extract should_attempt_deltas()Derrick Stolee via GitGitGadget, Oct 20, 2024
  35. 09/17 pack-objects: update usage to match docsDerrick Stolee via GitGitGadget, Oct 20, 2024
  36. 08/17 pack-objects: add --path-walk optionDerrick Stolee via GitGitGadget, Oct 20, 2024
  37. 10/17 p5313: add performance tests for --path-walkDerrick Stolee via GitGitGadget, Oct 20, 2024
  38. 11/17 pack-objects: introduce GIT_TEST_PACK_PATH_WALKDerrick Stolee via GitGitGadget, Oct 20, 2024
  39. 12/17 repack: add --path-walk optionDerrick Stolee via GitGitGadget, Oct 20, 2024
  40. 13/17 repack: update usage to match docsDerrick Stolee via GitGitGadget, Oct 20, 2024
  41. 14/17 pack-objects: enable --path-walk via configDerrick Stolee via GitGitGadget, Oct 20, 2024
  42. 15/17 scalar: enable path-walk during push via configDerrick Stolee via GitGitGadget, Oct 20, 2024
  43. 16/17 pack-objects: refactor path-walk delta phaseDerrick Stolee via GitGitGadget, Oct 20, 2024
  44. 17/17 pack-objects: thread the path-based compressionDerrick Stolee via GitGitGadget, Oct 20, 2024
  45. Taylor BlauOct 21, 2024
  46. Derrick StoleeOct 24, 2024
  47. Taylor BlauOct 24, 2024
  48. Patrick SteinhardtOct 28, 2024
  49. Taylor BlauOct 28, 2024
  50. Derrick StoleeOct 28, 2024
  51. Taylor BlauOct 28, 2024
  52. Derrick StoleeOct 28, 2024
  53. Taylor BlauOct 29, 2024
  54. Derrick StoleeOct 31, 2024
  55. Taylor BlauOct 31, 2024

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.