Re: [PATCH 3/7] commit-graph: start parsing generation v2 (again)

From: Patrick Steinhardt <ps@pks.im>
Date: Mar 2, 2022, 13:57 UTC
Message-ID: <Yh93vOkt2DkrGPh2@ncase>
In-Reply-To: <1b9912f7-87be-2520-bb53-9e23529ad233@github.com>

On Tue, Mar 01, 2022 at 10:25:46AM -0500, Derrick Stolee wrote:

Show 60 quoted lines

> On 3/1/2022 9:53 AM, Patrick Steinhardt wrote:
> > On Tue, Mar 01, 2022 at 09:06:44AM -0500, Derrick Stolee wrote:
> >> On 3/1/2022 5:35 AM, Patrick Steinhardt wrote:
> >>> On Tue, Mar 01, 2022 at 10:46:14AM +0100, Patrick Steinhardt wrote:
> >>>> On Mon, Feb 28, 2022 at 01:44:01PM -0500, Derrick Stolee wrote:
> >>>>> On 2/28/2022 11:59 AM, Patrick Steinhardt wrote:
> >>>>>> On Mon, Feb 28, 2022 at 11:23:38AM -0500, Derrick Stolee wrote:
> >>>>>>> On 2/28/2022 10:18 AM, Patrick Steinhardt wrote:
> >>>>>>>> [1]: https://gitlab.com/gitlab-com/www-gitlab-com.git
> ...
> >>> So the question is whether this is a change that needs to be rolled out
> >>> over multiple releases. First we'd get in the bug fix such that we write
> >>> correct commit-graphs, and after this fix has been released we can also
> >>> release the fix that starts to actually parse the generation. This
> >>> ensures there's a grace period during which we can hopefully correct the
> >>> data on-disk such that users are not faced with failures.
> >>
> >> You are right that we need to be careful here, but I also think that
> >> previous versions of Git always wrote the correct data. Here is my
> >> thought process:
> >>
> >> 1. To get this bug, we need to have parsed the corrected commit date
> >>    from an existing commit-graph in order to under-count the number
> >>    of overflow values.
> >>
> >> 2. Before this series, Git versions were not parsing the corrected
> >>    commit date, so they recompute the corrected commit date every
> >>    time the commit-graph is written, getting the proper count of
> >>    overflow values.
> >>
> >> For these reasons, data written by previous versions of Git are
> >> correct and can be trusted without a staged release.
> >>
> >> Does this make sense? Or, do you experience a different result when
> >> you build commit-graphs with a released Git version and then when
> >> writing on top with all patches applied?
> > 
> > Just to verify my understanding: you claim that the bug I was hitting
> > shouldn't be encountered in the wild when the release , but
> > only if one were to write a commit-graph with the intermediate stafe
> > until patch 3/4 of your patch series?
> 
> That is my claim. And my testing of the repo at [1] has demonstrated
> that it works correctly in these cases.
>  
> > Hum. I have re-verified, and this indeed seems to play out. So I must've
> > accidentally ran all my testing with the state generated without the
> > final patch which fixes the corruption. I do see lots of the following
> > warnings, but overall I can verify and write the commit-graph just fine:
> > 
> >     commit-graph generation for commit c80a42de8803e2d77818d0c82f88e748d7f9425f is 1623362063 < 1623362139
> 
> But I'm not able to generate these warnings from either version. I
> tried generating different levels of a split commit-graph, but
> could not reproduce it. If you have reproduction steps using current
> 'master' (or any released Git version) and the four patches here,
> then I would love to get a full understanding of your errors.
> 
> Thanks,
> -Stolee

I haven't yet been able to reproduce it with publicly available data, but with the internal references I'm able to evoke the warnings reliably. It only works when I have two repositories connected via alternates, when generating the commit-graph in the linked-to repo first, and then generating the commit-graph in the linking repo.

The following recipe allows me to reproduce, but rely on private data:

    $ git --version
    git version 2.35.1

    # The pool repository is the one we're linked to from the fork.
    $ cd "$pool"
    $ rm -rf objects/info/commit-graph objects/info/commit-graph
    $ git commit-graph write --split

    $ cd "$fork"
    $ rm -rf objects/info/commit-graph objects/info/commit-graph
    $ git commit-graph write --split

    $ git commit-graph verify --no-progress
    $ echo $?
    0

    # This is 715d08a9e51251ad8290b181b6ac3b9e1f9719d7 with your full v2
    # applied on top.
    $ ~/Development/git/bin-wrappers/git --version
    git version 2.35.1.358.g7ede1bea24

    $ ~/Development/git/bin-wrappers/git commit-graph verify --no-progress
    commit-graph generation for commit 06a91bac00ed11128becd48d5ae77eacd8f24c97 is 1623273624 < 1623273710
    commit-graph generation for commit 0ae91029f27238e8f8e109c6bb3907f864dda14f is 1622151146 < 1622151220
    commit-graph generation for commit 0d4582a33d8c8e3eb01adbf564f5e1deeb3b56a2 is 1631045222 < 1631045225
    commit-graph generation for commit 0daf8976439d7e0bb9710c5ee63b570580e0dc03 is 1620347739 < 1620347789
    commit-graph generation for commit 0e0ee8ffb3fa22cee7d28e21cbd6df26454932cf is 1623783297 < 1623783380
    commit-graph generation for commit 0f08ab3de6ec115ea8a956a1996cb9759e640e74 is 1621543278 < 1621543339
    commit-graph generation for commit 133ed0319b5a66ae0c2be76e5a887b880452b111 is 1620949864 < 1620949915
    commit-graph generation for commit 1341b3e6c63343ae94a8a473fa057126ddd4669a is 1637344364 < 1637344384
    commit-graph generation for commit 15bdfc501c2c9f23e9353bf6e6a5facd9c32a07a is 1623348103 < 1623348133
    ...
    $ echo $?
    1

When generating commit-graphs with your patches applied the `verify` step works alright.

I've also by accident stumbled over the original error again:

    fatal: commit-graph requires overflow generation data but has none

This time it's definitely not caused by generating commit-graphs with an in-between state of your patch series because the data comes straight from production with no changes to the commit-graphs performed by myself. There we're running Git v2.33.1 with a couple of backported patches (see [1]). While those patches cause us to make more use of the commit-graph, none modify the way we generate them.

Of note is that the commit-graph contains references to commits which don't exist in the ODB anymore.

Patrick

[1]: https://gitlab.com/gitlab-org/gitlab-git/-/commits/pks-v2.33.1.gl3