Re: git fetch --prune fails with "fatal: bad object"

From: Jeff King <peff@peff.net>
Date: Jun 5, 2024, 08:45 UTC
Message-ID: <20240605084517.GC2345232@coredump.intra.peff.net>
In-Reply-To: <xmqqcyowwqah.fsf@gitster.g>

On Tue, Jun 04, 2024 at 10:50:30AM -0700, Junio C Hamano wrote:

Show 15 quoted lines

> Jeff King <peff@peff.net> writes:
> 
> > These are good examples. I was going to suggest fsck, as well, just
> > because I knew it would keep going after seeing bogus results. But more
> > interesting is that it is finding things in your example that other
> > programs would _not_ find, because it's being more thorough than just
> > reading the refs.
> 
> True.
> 
> I wish for-each-ref and friends had an optional mode that lets them
> keep going, but since so many features in them access objects
> pointed at by the refs (e.g., "--format='%(objectname:short)'" and
> "--no-merged HEAD"), it would be very cumbersome to retrofit such a
> mode to the underlying machinery, I suspect.

We try to read as little as possible in for-each-ref for efficiency purposes. The side effect is that if you don't ask for any details of the object, we won't try to read it and notice that it's gone. ;)

So you can do:

  git for-each-ref --format='%(objectname) %(refname)'

which will show even broken refs. Of course you still don't know which ones are broken! For that, I'd rely on something like cat-file, where it will reliably tell you about missing objects and continue (even if we just print the object name, it does an existence check). So naively I'd want something like this to work:

  git for-each-ref --format='%(objectname) %(refname)' |
  git cat-file --batch-check='%(objectname) %(rest)' |
  grep ^missing

But annoyingly, when it encounters a missing object it, cat-file ditches the whole format string and just prints "$oid missing". You can work around it with a little shell magic:

  git for-each-ref --format='%(objectname) %(refname)' >both
  cut -d' ' -f1 <both | git cat-file --batch-check='%(objectname)' >exists
  paste -d' ' both exists | cut -d' ' -f2,4

which yields lines like:

  refs/heads/ok
  refs/heads/broken missing

and so on. Turning that into "update-ref --stdin" input is left as an exercise for the reader.

I suspect it's too late to change the default behavior for a plumbing tool like cat-file. But we could probably teach it an option like "--missing=foo" to expand the format but use "foo" for object-related placeholders that can't be expanded. And then:

  git for-each-ref --format='%(objectname) %(refname)' |
  git cat-file --batch-check='%(objectname) %(rest)' --missing=MISSING |
  grep ^MISSING

would work. I still think it's inferior to fsck, though, as it's a very shallow check of validity.

-Peff