threads / discuss / 52224

Split commit graphs and commit-graph read

Subject: Split commit graphs and commit-graph read

## tl;dr

5 messages between Nov 8, 2019 and Nov 12, 2019.

replies: 4people: 4as markdown or json

Bryan Turner· Nov 8, 2019, 23:41 UTC · lore

Just a quick question about a behavior I've noticed with the commit graph. (Amazing feature, by the way!)

If the _very first_ write done is split: git commit-graph write --reachable --split

You end up with something like this: .../objects$ ls -R info info: commit-graphs packs

info/commit-graphs: commit-graph-chain graph-6612fcc8fd04d3af2cc268a6bd9161ae40f5fcbf.graph

info/commit-graph doesn't exist, but I have a 1-graph "chain" in place. (And subsequent write --split calls write additional ones; I've got a few now in this repository, but still no info/commit-graph.)

git commit-graph verify seems happy: .../objects$ git commit-graph verify Verifying commits in commit graph: 100% (98768/98768), done.

But git commit-graph read isn't: .../objects$ git commit-graph read fatal: Could not open commit-graph '/path/to/repository/objects/info/commit-graph': No such file or directory

Running some tests with commands like git for-each-ref and git rev-list shows that the "split" commit graph is being used (setting core.commitGraph=false makes commands noticeably slower), so functionally all seems well. But should git commit-graph read be handling this better?

Best regards, Bryan Turner

Derrick Stolee· Nov 11, 2019, 01:19 UTC · re: Bryan Turner · lore

Re: Split commit graphs and commit-graph read

On 11/8/2019 6:41 PM, Bryan Turner wrote:

Hi Bryan,

> Just a quick question about a behavior I've noticed with the commit
> graph. (Amazing feature, by the way!)

Show 18 quoted lines

> If the _very first_ write done is split:
> git commit-graph write --reachable --split
> 
> You end up with something like this:
> .../objects$ ls -R info
> info:
> commit-graphs  packs
> 
> info/commit-graphs:
> commit-graph-chain  graph-6612fcc8fd04d3af2cc268a6bd9161ae40f5fcbf.graph
> 
> info/commit-graph doesn't exist, but I have a 1-graph "chain" in
> place. (And subsequent write --split calls write additional ones; I've
> got a few now in this repository, but still no info/commit-graph.)
> 
> git commit-graph verify seems happy:
> .../objects$ git commit-graph verify
> Verifying commits in commit graph: 100% (98768/98768), done.

This workflow seems expected.

Show 11 quoted lines

> But git commit-graph read isn't:
> .../objects$ git commit-graph read
> fatal: Could not open commit-graph
> '/path/to/repository/objects/info/commit-graph': No such file or
> directory
> 
> Running some tests with commands like git for-each-ref and git
> rev-list shows that the "split" commit graph is being used (setting
> core.commitGraph=false makes commands noticeably slower), so
> functionally all seems well. But should git commit-graph read be
> handling this better?

Unfortunately, you're running into an issue because I designed the "read" subcommand poorly (and also forgot to update it for incremental commit-graph files). The biggest issue is that "read" is not really meant for end-users. It really should have been built as a test-tool. This point was corrected when I got around to writing the multi-pack-index since it uses "test-tool read-midx" instead of add.

To fix this issue, I would probably go about it by removing the "read" subcommand and creating a "test-tool read-commit-graph" for the tests that need that output.

If others on-list think that the better thing to do is to update the "read" subcommand to provide the same output, but iterate over each layer of an incremental commit-graph, then I can do that work instead.

Thanks, -Stolee

Junio C Hamano· Nov 11, 2019, 02:09 UTC · re: Derrick Stolee · lore

Re: Split commit graphs and commit-graph read

Derrick Stolee <stolee@gmail.com> writes:

Show 7 quoted lines

> To fix this issue, I would probably go about it by removing the "read"
> subcommand and creating a "test-tool read-commit-graph" for the tests
> that need that output.
>
> If others on-list think that the better thing to do is to update the
> "read" subcommand to provide the same output, but iterate over each
> layer of an incremental commit-graph, then I can do that work instead.

I think moving it to test-tool is a good idea, whether the output presents a single result that consolidates all layers together, or allows to name just a single layer and shows its contents. I do not have a strong opinion between the two (both sounds usable debugging aid).

Thanks.

Jeff King· Nov 11, 2019, 03:29 UTC · re: Derrick Stolee · lore

Re: Split commit graphs and commit-graph read

On Sun, Nov 10, 2019 at 08:19:20PM -0500, Derrick Stolee wrote:

Show 21 quoted lines

> > Running some tests with commands like git for-each-ref and git
> > rev-list shows that the "split" commit graph is being used (setting
> > core.commitGraph=false makes commands noticeably slower), so
> > functionally all seems well. But should git commit-graph read be
> > handling this better?
> 
> Unfortunately, you're running into an issue because I designed the
> "read" subcommand poorly (and also forgot to update it for
> incremental commit-graph files). The biggest issue is that "read"
> is not really meant for end-users. It really should have been built
> as a test-tool. This point was corrected when I got around to writing
> the multi-pack-index since it uses "test-tool read-midx" instead of
> add.
> 
> To fix this issue, I would probably go about it by removing the "read"
> subcommand and creating a "test-tool read-commit-graph" for the tests
> that need that output.
> 
> If others on-list think that the better thing to do is to update the
> "read" subcommand to provide the same output, but iterate over each
> layer of an incremental commit-graph, then I can do that work instead.

In theory I suppose one could use it to debug a commit-graph file "in the field" as it were, where somebody does not necessarily have the test-tool programs. But in practice, I have not ever done that (I didn't even know "commit-graph read" was there), and it's not that big a deal to just have a build of git.git handy.

I'd be much more likely to use "commit-graph verify". And perhaps it could grow a "--verbose" flag if somebody really wants that (but I think it would be fine to punt until somebody cares enough to do so).

I guess dropping the sub-command is technically a backwards incompatible change, but since it didn't do anything that normal users would find useful in the first place, I wouldn't be sorry to see it go.

-Peff

Bryan Turner· Nov 12, 2019, 00:28 UTC · re: Derrick Stolee · lore

Re: Split commit graphs and commit-graph read

On Sun, Nov 10, 2019 at 5:19 PM Derrick Stolee <stolee@gmail.com> wrote:

Show 28 quoted lines

>
> > But git commit-graph read isn't:
> > .../objects$ git commit-graph read
> > fatal: Could not open commit-graph
> > '/path/to/repository/objects/info/commit-graph': No such file or
> > directory
> >
> > Running some tests with commands like git for-each-ref and git
> > rev-list shows that the "split" commit graph is being used (setting
> > core.commitGraph=false makes commands noticeably slower), so
> > functionally all seems well. But should git commit-graph read be
> > handling this better?
>
> Unfortunately, you're running into an issue because I designed the
> "read" subcommand poorly (and also forgot to update it for
> incremental commit-graph files). The biggest issue is that "read"
> is not really meant for end-users. It really should have been built
> as a test-tool. This point was corrected when I got around to writing
> the multi-pack-index since it uses "test-tool read-midx" instead of
> add.
>
> To fix this issue, I would probably go about it by removing the "read"
> subcommand and creating a "test-tool read-commit-graph" for the tests
> that need that output.
>
> If others on-list think that the better thing to do is to update the
> "read" subcommand to provide the same output, but iterate over each
> layer of an incremental commit-graph, then I can do that work instead.

I was only running "git commit-graph read" as another way of verifying that everything looked good (to Git, that is). I'd already run "git commit-graph verify" and just found the discrepancy in their behavior surprising/interesting. For what it's worth, I personally don't see anything amiss with removing the "read" subcommand if that's simpler than updating it to handle incremental graphs. (Put differently, it's not a command I'd ever have Bitbucket Server run, or that I'm likely to try and run going forward. In this case I just completed some commit graph-related changes to Bitbucket Server and was looking for every possible avenue I could pursue to try and ensure what I did was producing the "correct" output after my changes got deployed to our internal dogfooding server.)

Thanks for taking the time to look into it and clarify the behavior! I really appreciate it.

Best regards, Bryan Turner

>
> Thanks,
> -Stolee