[PATCH v2 0/2] prune: save reachable-from-recent objects with bitmaps

From: Jeff King <peff@peff.net>
Date: Apr 28, 2021, 15:41 UTC
Message-ID: <YImCNXC5DUvy5gT8@coredump.intra.peff.net>
In-Reply-To: <YIl7tUJA6sQe2K73@coredump.intra.peff.net>

On Wed, Apr 28, 2021 at 11:13:57AM -0400, Jeff King wrote:

Show 21 quoted lines

> On Wed, Apr 28, 2021 at 01:20:03PM +0100, David Emett wrote:
> 
> > > Here's a fix. Thanks very much for reporting.
> > 
> > Thanks for the quick response! I tried the fix out on the repo I was having
> > trouble with. It's hitting a segfault in traverse_commit_list in the
> > mark_recent block. It looks like the issue is that the bitmap code leaves
> > revs->include_check set, with revs->include_check_data pointing at the stack.
> > Setting revs->include_check to NULL after the traverse_commit_list call in
> > find_objects in pack-bitmap.c fixes the segfault for me. And the original issue
> > appears to be resolved as well, so thanks!
> 
> Oof, good catch. I wondered why my test didn't see this. The answer is
> that the include_check only applies to commits, not trees. And my test
> only has a recent-but-unreachable tree. Beefing it up to include an
> actual commit shows the problem (and ASan nicely pinpoints the issue).
> 
> The solution, as you noted, is to have the bitmap code clean up after
> itself.
> 
> I'll prepare a re-roll of the series.

Here it is.

I did notice one other oddity. The bitmap code doesn't do any sort of progress reporting. So progress is enabled, we'll _just_ count the recent-but-unreachable entries in the progress meter.

That's not entirely wrong (it represents the actual traversal work we're doing), but is a little funny. However, I think the best thing to do for now is to leave it there. The right path forward is for the bitmap code to learn about progress reporting, too (for this case, but also for others). But that's a more involved topic, and I don't want to hold up this fix for it.

  [1/2]: pack-bitmap: clean up include_check after use
  [2/2]: prune: save reachable-from-recent objects with bitmaps

 pack-bitmap.c              |  3 +++
 reachable.c                | 13 ++++---------
 t/t5304-prune.sh           | 16 ++++++++++++++++
 t/t6501-freshen-objects.sh | 21 +++++++++++++++------
 4 files changed, 38 insertions(+), 15 deletions(-)

-:  ---------- > 1:  43f25e88c5 pack-bitmap: clean up include_check after use
1:  a7fb2a7f5b ! 2:  0a60ea597d prune: save reachable-from-recent objects with bitmaps
    @@ t/t5304-prune.sh: test_expect_success 'trivial prune with bitmaps enabled' '
     +	to_save=$(echo bitmap-from-recent-2 | git hash-object -w --stdin) &&
     +	test-tool chmtime -86400 .git/objects/$(test_oid_to_path $to_save) &&
     +	tree=$(printf "100644 blob $to_save\tfile\n" | git mktree) &&
    ++	test-tool chmtime -86400 .git/objects/$(test_oid_to_path $tree) &&
    ++	commit=$(echo foo | git commit-tree $tree) &&
     +	git prune --expire=12.hours.ago &&
    ++	git cat-file -e $commit &&
     +	git cat-file -e $tree &&
     +	git cat-file -e $to_save &&
     +	test_must_fail git cat-file -e $to_drop

-Peff