git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v4 2/6] list-objects: consume sparse tree walk

From
Junio C Hamano <gitster@pobox.com>
Date
Jan 11, 2019, 23:20 UTC
Message-ID
<xmqqa7k6ye6c.fsf@gitster-ct.c.googlers.com>
In-Reply-To
<39dc89beb91ac12c94d13f7931a4d9ebc602681f.1544822533.git.gitgitgadget@gmail.com>
"Derrick Stolee via GitGitGadget" <gitgitgadget@gmail.com> writes:
Show 41 quoted lines
> From: Derrick Stolee <dstolee@microsoft.com>
>
> When creating a pack-file using 'git pack-objects --revs' we provide
> a list of interesting and uninteresting commits. For example, a push
> operation would make the local topic branch be interesting and the
> known remote refs as uninteresting. We want to discover the set of
> new objects to send to the server as a thin pack.
>
> We walk these commits until we discover a frontier of commits such
> that every commit walk starting at interesting commits ends in a root
> commit or unintersting commit. We then need to discover which
> non-commit objects are reachable from  uninteresting commits. This
> commit walk is not changing during this series.
>
> The mark_edges_uninteresting() method in list-objects.c iterates on
> the commit list and does the following:
>
> * If the commit is UNINTERSTING, then mark its root tree and every
>   object it can reach as UNINTERESTING.
>
> * If the commit is interesting, then mark the root tree of every
>   UNINTERSTING parent (and all objects that tree can reach) as
>   UNINTERSTING.
>
> At the very end, we repeat the process on every commit directly
> given to the revision walk from stdin. This helps ensure we properly
> cover shallow commits that otherwise were not included in the
> frontier.
>
> The logic to recursively follow trees is in the
> mark_tree_uninteresting() method in revision.c. The algorithm avoids
> duplicate work by not recursing into trees that are already marked
> UNINTERSTING.
>
> Add a new 'sparse' option to the mark_edges_uninteresting() method
> that performs this logic in a slightly new way. As we iterate over
> the commits, we add all of the root trees to an oidset. Then, call
> mark_trees_uninteresting_sparse() on that oidset. Note that we
> include interesting trees in this process. The current implementation
> of mark_trees_unintersting_sparse() will walk the same trees as
> the old logic, but this will be replaced in a later change.

It is unclear what "a slightly new way" refers to. The updated code adds the UNINTERSTING edge commits and UNINTERSTING parents of interesting edge commits (the latter is done using the new add-edge-parents() helper function) to an oidset and calls the new helper introduced in [1/6] to ensure all the objects reachable from these UNINTERSTING trees become UNINTERSTING.

Which seems to be doing exactly the same thing as the original.
Puzzled.
Previous: Derrick Stolee via GitGitGadgetNext: Derrick Stolee via GitGitGadget
Message 33 of 51 in “Add a new "sparse" tree walk algorithm”
  1. 0/5 Add a new "sparse" tree walk algorithmDerrick Stolee via GitGitGadget, Nov 28, 2018
  2. 1/5 revision: add mark_tree_uninteresting_sparseDerrick Stolee via GitGitGadget, Nov 28, 2018
  3. 2/5 list-objects: consume sparse tree walkDerrick Stolee via GitGitGadget, Nov 28, 2018
  4. 3/5 pack-objects: add --sparse optionDerrick Stolee via GitGitGadget, Nov 28, 2018
  5. Stefan BellerNov 28, 2018
  6. Derrick StoleeNov 29, 2018
  7. Junio C HamanoNov 30, 2018
  8. Derrick StoleeNov 30, 2018
  9. 5/5 pack-objects: create pack.useSparse settingDerrick Stolee via GitGitGadget, Nov 28, 2018
  10. 4/5 revision: implement sparse algorithmDerrick Stolee via GitGitGadget, Nov 28, 2018
  11. Ævar Arnfjörð BjarmasonNov 28, 2018
  12. Derrick StoleeNov 29, 2018
  13. 0/6 Add a new "sparse" tree walk algorithmDerrick Stolee via GitGitGadget, Nov 29, 2018
  14. 3/6 pack-objects: add --sparse optionDerrick Stolee via GitGitGadget, Nov 29, 2018
  15. 4/6 revision: implement sparse algorithmDerrick Stolee via GitGitGadget, Nov 29, 2018
  16. 6/6 pack-objects: create GIT_TEST_PACK_SPARSEDerrick Stolee via GitGitGadget, Nov 29, 2018
  17. 5/6 pack-objects: create pack.useSparse settingDerrick Stolee via GitGitGadget, Nov 29, 2018
  18. 2/6 list-objects: consume sparse tree walkDerrick Stolee via GitGitGadget, Nov 29, 2018
  19. 1/6 revision: add mark_tree_uninteresting_sparseDerrick Stolee via GitGitGadget, Nov 29, 2018
  20. 0/6 Add a new "sparse" tree walk algorithmDerrick Stolee via GitGitGadget, Dec 10, 2018
  21. 1/6 revision: add mark_tree_uninteresting_sparseDerrick Stolee via GitGitGadget, Dec 10, 2018
  22. 3/6 pack-objects: add --sparse optionDerrick Stolee via GitGitGadget, Dec 10, 2018
  23. 2/6 list-objects: consume sparse tree walkDerrick Stolee via GitGitGadget, Dec 10, 2018
  24. 4/6 revision: implement sparse algorithmDerrick Stolee via GitGitGadget, Dec 10, 2018
  25. 6/6 pack-objects: create GIT_TEST_PACK_SPARSEDerrick Stolee via GitGitGadget, Dec 10, 2018
  26. 5/6 pack-objects: create pack.useSparse settingDerrick Stolee via GitGitGadget, Dec 10, 2018
  27. 0/6 Add a new "sparse" tree walk algorithmDerrick Stolee via GitGitGadget, Dec 14, 2018
  28. 1/6 revision: add mark_tree_uninteresting_sparseDerrick Stolee via GitGitGadget, Dec 14, 2018
  29. Junio C HamanoJan 11, 2019
  30. Junio C HamanoJan 11, 2019
  31. Derrick StoleeJan 11, 2019
  32. 2/6 list-objects: consume sparse tree walkDerrick Stolee via GitGitGadget, Dec 14, 2018
  33. Junio C HamanoJan 11, 2019
  34. 3/6 pack-objects: add --sparse optionDerrick Stolee via GitGitGadget, Dec 14, 2018
  35. Junio C HamanoJan 11, 2019
  36. Derrick StoleeJan 15, 2019
  37. Junio C HamanoJan 15, 2019
  38. 4/6 revision: implement sparse algorithmDerrick Stolee via GitGitGadget, Dec 14, 2018
  39. Ævar Arnfjörð BjarmasonDec 14, 2018
  40. Derrick StoleeDec 17, 2018
  41. Ævar Arnfjörð BjarmasonDec 17, 2018
  42. Derrick StoleeDec 17, 2018
  43. Junio C HamanoJan 11, 2019
  44. 5/6 pack-objects: create pack.useSparse settingDerrick Stolee via GitGitGadget, Dec 14, 2018
  45. 6/6 pack-objects: create GIT_TEST_PACK_SPARSEDerrick Stolee via GitGitGadget, Dec 14, 2018
  46. 0/5 Add a new "sparse" tree walk algorithmDerrick Stolee via GitGitGadget, Jan 16, 2019
  47. 2/5 list-objects: consume sparse tree walkDerrick Stolee via GitGitGadget, Jan 16, 2019
  48. 3/5 revision: implement sparse algorithmDerrick Stolee via GitGitGadget, Jan 16, 2019
  49. 4/5 pack-objects: create pack.useSparse settingDerrick Stolee via GitGitGadget, Jan 16, 2019
  50. 5/5 pack-objects: create GIT_TEST_PACK_SPARSEDerrick Stolee via GitGitGadget, Jan 16, 2019
  51. 1/5 revision: add mark_tree_uninteresting_sparseDerrick Stolee via GitGitGadget, Jan 16, 2019

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.