git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v3] rev-list: exclude promisor objects at walk time

From
Josh Steadmon <steadmon@google.com>
Date
Apr 9, 2019, 23:42 UTC
Message-ID
<20190409234255.GM60888@google.com>
In-Reply-To
<20190409180418.GH8796@szeder.dev>
On 2019.04.09 20:04, SZEDER Gábor wrote:
Show 78 quoted lines
> On Mon, Apr 08, 2019 at 02:06:04PM -0700, Josh Steadmon wrote:
> > For large repositories, enumerating the list of all promisor objects (in
> > order to exclude them from a rev-list walk) can take a significant
> > amount of time).
> > 
> > When --exclude-promisor-objects is passed to rev-list, don't enumerate
> > the promisor objects. Instead, filter them (and any children objects)
> > during the actual graph walk.
> > 
> > Remove the mark_uninteresting() function as it's not used anywhere else.
> > 
> > When testing against a large repo [1], this patch reduces the
> > connectivity check runtime from 3 minutes to ~7 seconds.
> 
> This patch breaks test 'repack -d does not irreversibly delete
> promisor objects' in 't0410-partial-clone.sh' when run with
> GIT_TEST_COMMIT_GRAPH=1.
> 
>   +rm -rf repo
>   +test_create_repo repo
>   +test 1 = 1
>   +repo=repo
>   +mkdir -p repo
>   +cd repo
>   +/home/travis/build/git/git/t/../git init --template=/home/travis/build/git/git/t/../templates/blt/
>   Initialized empty Git repository in /home/travis/build/git/git/t/trash directory.t0410-partial-clone/repo/.git/
>   +mv .git/hooks .git/hooks-disabled
>   +git -C repo config core.repositoryformatversion 1
>   +git -C repo config extensions.partialclone arbitrary string
>   +git -C repo commit --allow-empty -m one
>   [master (root-commit) 71905df] one
>    Author: A U Thor <author@example.com>
>   +git -C repo commit --allow-empty -m two
>   [master 202c4a3] two
>    Author: A U Thor <author@example.com>
>   +git -C repo commit --allow-empty -m three
>   [master 4737577] three
>    Author: A U Thor <author@example.com>
>   +git -C repo commit --allow-empty -m four
>   [master d6ba7e0] four
>    Author: A U Thor <author@example.com>
>   +git -C repo rev-parse HEAD^^^
>   +ONE=71905dfcd543b7cbb0b4b66fbd20379e67220557
>   +git -C repo rev-parse HEAD^^
>   +TWO=202c4a3dd9a2dac927f056abb747cce9ea2eb67b
>   +git -C repo rev-parse HEAD^
>   +THREE=47375779ebcca4b422e3afdd14aa37a358081297
>   +pack_as_from_promisor
>   +printf 202c4a3dd9a2dac927f056abb747cce9ea2eb67b\n
>   +git -C repo pack-objects .git/objects/pack/pack
>   +HASH=2e675cd706e508d6c52a21d28cfcddde5ec02a06
>   +
>   +echo 2e675cd706e508d6c52a21d28cfcddde5ec02a06
>   2e675cd706e508d6c52a21d28cfcddde5ec02a06
>   +printf 47375779ebcca4b422e3afdd14aa37a358081297\n
>   +pack_as_from_promisor
>   +git -C repo pack-objects .git/objects/pack/pack
>   +HASH=31f7d2797549ab9b1c425a9e60eb2030481170e5
>   +
>   +echo 31f7d2797549ab9b1c425a9e60eb2030481170e5
>   31f7d2797549ab9b1c425a9e60eb2030481170e5
>   +delete_object repo 71905dfcd543b7cbb0b4b66fbd20379e67220557
>   +sed -e s|^..|&/|
>   +echo 71905dfcd543b7cbb0b4b66fbd20379e67220557
>   +rm repo/.git/objects/71/905dfcd543b7cbb0b4b66fbd20379e67220557
>   +repack_and_check -a 202c4a3dd9a2dac927f056abb747cce9ea2eb67b 47375779ebcca4b422e3afdd14aa37a358081297
>   +rm -rf repo2
>   +cp -r repo repo2
>   +git -C repo2 repack -a -d
>   warning: reflog of 'HEAD' references pruned commits
>   warning: reflog of 'refs/heads/master' references pruned commits
>   fatal: unable to read 71905dfcd543b7cbb0b4b66fbd20379e67220557
>   error: last command exited with $?=128
>   not ok 23 - repack -d does not irreversibly delete promisor objects
> 
> 
>   https://travis-ci.org/git/git/jobs/517874310#L5822
> 

Thank you for catching this. I haven't yet figured out the cause. I will look into this more tomorrow and then send a V4 once I've fixed it.

Previous: SZEDER GáborNext: Jeff King
Message 21 of 27 in “clone: do faster object check for partial clones”
  1. clone: do faster object check for partial clonesJosh Steadmon, Apr 3, 2019
  2. Jonathan TanApr 3, 2019
  3. Jeff KingApr 3, 2019
  4. Jonathan TanApr 3, 2019
  5. Josh SteadmonApr 4, 2019
  6. Jeff KingApr 4, 2019
  7. rev-list: exclude promisor objects at walk timeJosh Steadmon, Apr 4, 2019
  8. Jeff KingApr 4, 2019
  9. Josh SteadmonApr 4, 2019
  10. Jeff KingApr 5, 2019
  11. Josh SteadmonApr 5, 2019
  12. Josh SteadmonApr 8, 2019
  13. rev-list: exclude promisor objects at walk timeJosh Steadmon, Apr 8, 2019
  14. Christian CouderApr 8, 2019
  15. Josh SteadmonApr 8, 2019
  16. Junio C HamanoApr 9, 2019
  17. Jeff KingApr 9, 2019
  18. Junio C HamanoApr 9, 2019
  19. Josh SteadmonApr 9, 2019
  20. SZEDER GáborApr 9, 2019
  21. Josh SteadmonApr 9, 2019
  22. Jeff KingApr 11, 2019
  23. Josh SteadmonApr 12, 2019
  24. Jeff KingApr 13, 2019
  25. Josh SteadmonApr 19, 2019
  26. clone: do faster object check for partial clonesJosh Steadmon, Apr 19, 2019
  27. Jeff KingApr 22, 2019

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.