Re: [BUG] `git gc` or `git pack-refs` wipes all notes for `git notes` command

From: Andrew Hlynskyi <ahlincq@gmail.com>
Date: Jan 5, 2023, 16:59 UTC
Message-ID: <CAAYtLE+PWK_v0cc8uqaiKnTHKghrkxuCCgfWyo9bhD23+vxK1g@mail.gmail.com>
In-Reply-To: <Y7Pvqk00sj3R7cZv@coredump.intra.peff.net>

On Tue, Jan 3, 2023 at 11:04 AM Jeff King <peff@peff.net> wrote:

> That would be very surprising. The lookup of the ref in the notes code
> uses the same generic ref code that the rest of Git uses, which
> understands packed-refs and so on.

Thank you Jeff for looking at this also.

I tried to debug the issue by myself and found that the `find_reference_location` function can't find the record for `refs/notes/commits` in a repo. I recall that a year ago I had trouble with a repo and it was damaged, I restored it with the `git fsck` command and took some info from the `git reflog`. But somehow there was an issue in the `.git/packed-refs` file that was there unexplored all past year. The core of the issue I can demonstrate with two `.git/packed-refs` excerpts.

The issue excerpt: ``` dca1abdb5b870d46deae3717e87d0085b62d7242 refs/heads/tmp d5ec01136b79a59a9e39d3ab02d92e1d91558579 refs/heads/tdf c60f96a3f76e99e1b42f07204ad830c1483647b0 refs/heads/trunk 93119b80d4c7676a0ebe1bd6ae46574f96dfc7ed refs/notes/commits 452a23af3cfac6eef3bb1045894521fac8e51e2a refs/heads/cargo-config-toml f5599d73873c451041fde857a09b5f12662357c1 refs/heads/custom 64076392359c4d4989320bde5712a8b389056f49 refs/heads/main 2119ff71ca66bec5117534907a127cee32680788 refs/heads/next 7e0016059ee7a87e3d04f7e59154529a9616a238 refs/recovery/1 de2876aa45022af815e717eb92becd61564efe32 refs/recovery/10 ```

The correct excerpt: ``` dca1abdb5b870d46deae3717e87d0085b62d7242 refs/heads/tmp d5ec01136b79a59a9e39d3ab02d92e1d91558579 refs/heads/tdf c60f96a3f76e99e1b42f07204ad830c1483647b0 refs/heads/trunk 452a23af3cfac6eef3bb1045894521fac8e51e2a refs/heads/cargo-config-toml f5599d73873c451041fde857a09b5f12662357c1 refs/heads/custom 64076392359c4d4989320bde5712a8b389056f49 refs/heads/main 2119ff71ca66bec5117534907a127cee32680788 refs/heads/next 93119b80d4c7676a0ebe1bd6ae46574f96dfc7ed refs/notes/commits 7e0016059ee7a87e3d04f7e59154529a9616a238 refs/recovery/1 de2876aa45022af815e717eb92becd61564efe32 refs/recovery/10 ```

As you may notice that on the first except there is an issue with sorting and after `ref/notes/*` there are several `ref/heads/*`. So I suspect that the issue was introduced by myself a year ago when I was recovering the repo.

> Can you share the .git directory of a repository that exhibits this
> behavior? It's possible there's a bug or something in the packed-refs
> code, though I find it pretty unlikely, as it's fairly well exercised in
> normal use.

The above excerpts completely describe the issue and there is no more special in the repo.

I was surprised that such an issue can remain in the `.git/packed-refs` for more than a year. I thought that commands like `git fsck` would report such unordering problems and `git gc` or `git pack-refs` make ordering checks and make full resorting of the `.git/packed-refs` file in case of issues with ordering. I understand that the `.git/packed-refs` file is for machines and not for humans but sometimes it's the fastest way to make several simple corrections in it manually.

P.S: What was most surprising is that `git pack-refs` was able to correctly update a hash even for an unordered record.

-Andrew