Re: Commit graph chains with no corresponding files?

From: Derrick Stolee <stolee@gmail.com>
Date: Feb 25, 2021, 14:20 UTC
Message-ID: <fc8a2c0f-24b7-5884-b669-bb9700f3ba84@gmail.com>
In-Reply-To: <CAGyf7-G_OdS_0o7j64HA79n9Qv13SxciQSG+gfY7Qj8kNRQS5Q@mail.gmail.com>

On 2/24/2021 11:54 PM, Bryan Turner wrote:

Show 19 quoted lines

> On Mon, Jun 29, 2020 at 6:51 PM Derrick Stolee <stolee@gmail.com> wrote:
>>
>> On 6/29/2020 6:07 PM, Jonathan Tan wrote:
>>> At $DAYJOB, a few people have reported "warning: unable to find all
>>> commit-graph files" warnings. Their commit-graph-chain files have a few
>>> lines, but they only have one commit graph file with very few commits. I
>>> suspected something happening during fetch, because (as far as I know) a
>>> fetch may cause an incremental commit graph to be written, but I ran a
>>> fetch on a large repository myself and didn't run into this problem.
>>>
>>> Has anyone ran into this problem before, and know how to reproduce?
> 
> I don't have any specific reproduction steps, but we've just run into
> our first case of this on Git 2.29. I ended up kicking off a full `git
> commit-graph write` to fix it. That displayed the same warning, but
> commands run after it no longer do. Prior to writing the new graph, I
> had this:
> $ ls
> commit-graph-chain  graph-88f5fe6e0c659e3742e556982263813d528ead81.graph

The contents of the 'commit-graph-chain' file are critical to diagnosing the problems here. Likely it had multiple lines.

> Afterward, the `objects/info/commits-graphs` directory still exists
> but is empty, and there is now an `objects/commit-graph` that didn't
> exist before. `git commit-graph verify` seems happy with the state of
> things.

Yes, a full rewrite without "--split" will get you to this state.

Show 13 quoted lines

>> The incremental commit-graph code deletes any commit-graph files
>> that do not appear in the chain. I believe this is done by comparing
>> the contents of the ".git/objects/info/commit-graphs/" directory to
>> the contents of the chain file.
>>
>> These appear to be case-sensitive, full-path comparisons.
>>
>> It is _possible_ that something like a case switch or a symlink
>> could be causing a problem here. That's where I would look on
>> the affected systems.
> 
> Are commit graphs potentially problematic in repositories that are
> borrowing objects from other repositories via alternates?

This was definitely part of the design, with the intention of working with a common base in the alternate. However, if the alternate collapses layers, then the repo that is borrowing from that alternate may have a broken chain.

It is likely a better setup to have the alternate keep a commit-graph file and leave the dependent repos clear of a commit-graph. _Or_ the dependent repos should use a full commit-graph instead of a chain.

If you have a better idea for how to make this work, then there is room for improvement.

For example, if we ensure during the commit-graph write that all layers of the chain are within our local repo, then these dependency issues go away without breaking any old Git versions that are reading the data.

> Have there
> been important changes to commit graphs since 2.29?

Not in the area of commit-graph chains.

Thanks, -Stolee