{"thread":{"id":"52667","subject":"Fwd: Possible git bug in commit-graph: \"invalid commit position\"","startedAt":"2020-01-20T17:33:12Z","lastAt":"2020-06-22T13:45:29Z","messageCount":6,"participants":["Gary Oberbrunner","Derrick Stolee","Alex Riesen","Michael Forney"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"390110","messageId":"CAFChFyi5J-mb+rshtF7U2m=MtPzEPUa+V1_qbEXC=-LdQ218yA@mail.gmail.com","threadId":"52667","inReplyTo":"CAFChFygiaMsUJC5Kfpnk26DLWbY0gPdNJpZ_gLMf4utZ6_oZxA@mail.gmail.com","subject":"Fwd: Possible git bug in commit-graph: \"invalid commit position\"","fromName":"Gary Oberbrunner","fromEmail":"garyo@oberbrunner.com","sentAt":"2020-01-20T17:32:53Z","receivedAt":"2020-01-20T17:33:12Z","isPatch":false,"sender":{"key":"garyo@oberbrunner.com","avatar":null},"body":"I'm running git 2.24.1 on Linux/WSL, and sometimes git fails with this error:\n```\n% git diff --cached\nfatal: invalid commit position. commit-graph is likely corrupt\n```\nUsually that error goes away after I do a few more git operations, but\nI haven't been able to discern a pattern yet. It's not always with\n\"diff\", can be any git command as far as I can tell.\nPerhaps it started after I enabled the commit-graph stuff:\n```\n[core]\n        editor = emacsclient -c -a \\\"\\\"\n        preloadindex = true\n        fscache = true\n        autocrlf = false\n        commitGraph = true\n        writeCommitGraph = true\n```\nIn fact, when I turn off `commitGraph`, the error goes away. But as I\nsay, sometimes it goes away of its own accord.\n\nGoogling turns up almost nothing on this error. I have no idea if I've\ndone something wrong to corrupt the commit graph, or if it's a git\nbug. If there's anything I can do to help debug it, I'd be happy to --\njust let me know.\n\n-- \nGary\n"},{"id":"390142","messageId":"d4313777-ce8c-0b64-997e-17cb719c9ab8@gmail.com","threadId":"52667","inReplyTo":"CAFChFyi5J-mb+rshtF7U2m=MtPzEPUa+V1_qbEXC=-LdQ218yA@mail.gmail.com","subject":"Re: Fwd: Possible git bug in commit-graph: \"invalid commit position\"","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2020-01-21T00:37:40Z","receivedAt":"2020-01-21T00:37:45Z","isPatch":false,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 1/20/2020 12:32 PM, Gary Oberbrunner wrote:\n> I'm running git 2.24.1 on Linux/WSL, and sometimes git fails with this error:\n> ```\n> % git diff --cached\n> fatal: invalid commit position. commit-graph is likely corrupt\n> ```\n> Usually that error goes away after I do a few more git operations, but\n> I haven't been able to discern a pattern yet. It's not always with\n> \"diff\", can be any git command as far as I can tell.\n> Perhaps it started after I enabled the commit-graph stuff:\n> ```\n> [core]\n>         editor = emacsclient -c -a \\\"\\\"\n>         preloadindex = true\n>         fscache = true\n>         autocrlf = false\n>         commitGraph = true\n>         writeCommitGraph = true\n> ```\n> In fact, when I turn off `commitGraph`, the error goes away. But as I\n> say, sometimes it goes away of its own accord.\n\nDisabling the featuere will definitely remove the error. Some commands may\nalso just not \"visit\" the corrupt commit.\n\n> Googling turns up almost nothing on this error. I have no idea if I've\n> done something wrong to corrupt the commit graph, or if it's a git\n> bug. If there's anything I can do to help debug it, I'd be happy to --\n> just let me know.\n\nPlease run \"git commit-graph verify\" to see if that catches the\ncorruption. If not, then the verify feature is broken.\n\nThen, you can delete your .git/objects/info/commit-graph file\nand rebuild it with \"git commit-graph write\". (If you have\nfetch.writeCommitGraph enabled, then you may need to instead\ndelete the .git/objects/info/commit-graphs directory.) If that\nprocess does not fix the problem, then there is definitely a bug\nin the feature. At the moment, it may just be a flipped bit in\nthe file.\n\nThanks,\n-Stolee\n"},{"id":"391133","messageId":"CAFChFyhZYm88cCo6X_XikT=HRQG9Pp0vyveXmAuComFUJYjRcA@mail.gmail.com","threadId":"52667","inReplyTo":"d4313777-ce8c-0b64-997e-17cb719c9ab8@gmail.com","subject":"Re: Fwd: Possible git bug in commit-graph: \"invalid commit position\"","fromName":"Gary Oberbrunner","fromEmail":"garyo@oberbrunner.com","sentAt":"2020-02-04T22:33:42Z","receivedAt":"2020-02-04T22:33:56Z","isPatch":false,"sender":{"key":"garyo@oberbrunner.com","avatar":null},"body":"Sorry for the long reply delay; the bug went away and only just showed\nup again. Here's the info you requested.\nI'm now running git 2.25.0.\n\nShowing the bug:\n% git show --patch df2f\nfatal: invalid commit position. commit-graph is likely corrupt\ncommit df2f311c78b87d16509f352c66d7309957740711 (origin/feature/renderer)\nAuthor: [redacted]\nDate: Mon Jan 20 12:37:01 2020 -0500\n\nWIP: add outline of back-end render web service\n\nSubmodule packages/ngl f713ed5..7de8821:\n%\n\nRun \"git commit-graph verify\": doesn't detect any problem.\n% git commit-graph verify\nVerifying commits in commit graph: 100% (1/1), done.\nVerifying commits in commit graph: 100% (2/2), done.\nVerifying commits in commit graph: 100% (10/10), done.\nVerifying commits in commit graph: 100% (14/14), done.\nVerifying commits in commit graph: 100% (71/71), done.\nVerifying commits in commit graph: 100% (478/478), done\n\nYou asked me to delete .git/objects/info/commit-graph -- I don't have\nsuch a file, but  I do have a subdir \"commit-graphs\":\n% ls .git/objects/info/commit-graphs/\ncommit-graph-chain graph-e96da2872d5c5fc1c4e399f99b7800d08fcdd273.graph\ngraph-4f5807bbdcba22f8fdc081c91205c40d40e99d38.graph\ngraph-e9f543a1389f8f5d6e91c09caa6658a1905d00fe.graph\ngraph-79f0f3bff07f7e7dda9d2438a30ef6483a2b531f.graph tmp_graph_GHI48b\ngraph-83ce268fda98e26938ce702bb261e1719ff816ba.graph tmp_graph_Ypgl2f\ngraph-93a966dc8be3040189367f047f5111e9f29c2d6c.graph\n\nAfter removing that subdir and running \"git commit-graph write\", I\n*do* have a .git/objects/info/commit-graph file.\n\nThe bug is still present though! However, I do have a git submodule\nwith its own .git/objects/info/commit-graphs; let's look into that.\nIn the git submodule dir, \"git commit-graph verify\" succeeds with no\nerrors (just like the parent).\nDeleted the .git/objects/info/commit-graphs subdir, regenerated with\n\"git commit-graph write\".\nRetest original problem:\n- Now the problem is fixed!\n\nSo long story short: the corrupted git commit-graph was in a\nsubmodule. It does appear that \"git commit-graph verify\" failed to\ndetect the corruption. Removing and regenerating fixes it.\n\nI do have the old git submodule's commit-graphs data in case it's\nuseful. Let me know and I can send it along.\n\n\nOn Mon, Jan 20, 2020 at 7:37 PM Derrick Stolee <stolee@gmail.com> wrote:\n>\n> On 1/20/2020 12:32 PM, Gary Oberbrunner wrote:\n> > I'm running git 2.24.1 on Linux/WSL, and sometimes git fails with this error:\n> > ```\n> > % git diff --cached\n> > fatal: invalid commit position. commit-graph is likely corrupt\n> > ```\n> > Usually that error goes away after I do a few more git operations, but\n> > I haven't been able to discern a pattern yet. It's not always with\n> > \"diff\", can be any git command as far as I can tell.\n> > Perhaps it started after I enabled the commit-graph stuff:\n> > ```\n> > [core]\n> >         editor = emacsclient -c -a \\\"\\\"\n> >         preloadindex = true\n> >         fscache = true\n> >         autocrlf = false\n> >         commitGraph = true\n> >         writeCommitGraph = true\n> > ```\n> > In fact, when I turn off `commitGraph`, the error goes away. But as I\n> > say, sometimes it goes away of its own accord.\n>\n> Disabling the featuere will definitely remove the error. Some commands may\n> also just not \"visit\" the corrupt commit.\n>\n> > Googling turns up almost nothing on this error. I have no idea if I've\n> > done something wrong to corrupt the commit graph, or if it's a git\n> > bug. If there's anything I can do to help debug it, I'd be happy to --\n> > just let me know.\n>\n> Please run \"git commit-graph verify\" to see if that catches the\n> corruption. If not, then the verify feature is broken.\n>\n> Then, you can delete your .git/objects/info/commit-graph file\n> and rebuild it with \"git commit-graph write\". (If you have\n> fetch.writeCommitGraph enabled, then you may need to instead\n> delete the .git/objects/info/commit-graphs directory.) If that\n> process does not fix the problem, then there is definitely a bug\n> in the feature. At the moment, it may just be a flipped bit in\n> the file.\n>\n> Thanks,\n> -Stolee\n\n\n\n-- \nGary\n"},{"id":"397872","messageId":"20200515120333.GA4677@pflmari","threadId":"52667","inReplyTo":"CAFChFyhZYm88cCo6X_XikT=HRQG9Pp0vyveXmAuComFUJYjRcA@mail.gmail.com","subject":"Re: Possible git bug in commit-graph: \"invalid commit position\"","fromName":"Alex Riesen","fromEmail":"alexander.riesen@cetitec.com","sentAt":"2020-05-15T12:03:33Z","receivedAt":"2020-05-15T12:03:39Z","isPatch":false,"sender":{"key":"alexander.riesen@cetitec.com","avatar":"https://avatars.githubusercontent.com/u/24452597?v=4"},"body":"Gary Oberbrunner, Tue, Feb 04, 2020 23:33:42 +0100:\n> Sorry for the long reply delay; the bug went away and only just showed\n> up again. Here's the info you requested.\n> I'm now running git 2.25.0.\n\nI hit a very similar problem today with 2.26.0. Also in a submodule.\n\nRemoving and regenerating the commit graph did not help and I did not have the\ncommit-graphs directory (only a file). \"git commit-graph verify\" does not find\nanything. Switching writeCommitGraph on and regenerating the commit graph\nmakes no difference.\n\nI can trigger it reliably by visiting the broken(?) commit in supermodule with:\n\n    git show --submodule=log <commit>\n\nI see nothing special in the commit invovled. It is just a single commit in my\ncase, and the commit is a merge of two branches.\n\nRegards,\nAlex\n"},{"id":"400271","messageId":"CAGw6cBuEshq18O_PrrGYuJi5VZ82XK3T9KuShneUqO2Ju0jtHw@mail.gmail.com","threadId":"52667","inReplyTo":"20200515120333.GA4677@pflmari","subject":"Re: Possible git bug in commit-graph: \"invalid commit position\"","fromName":"Michael Forney","fromEmail":"mforney@mforney.org","sentAt":"2020-06-21T20:45:01Z","receivedAt":"2020-06-21T20:45:06Z","isPatch":false,"sender":{"key":"mforney@mforney.org","avatar":"https://avatars.githubusercontent.com/u/52851?v=4"},"body":"On 2020-05-15, Alex Riesen <alexander.riesen@cetitec.com> wrote:\n> Gary Oberbrunner, Tue, Feb 04, 2020 23:33:42 +0100:\n>> Sorry for the long reply delay; the bug went away and only just showed\n>> up again. Here's the info you requested.\n>> I'm now running git 2.25.0.\n>\n> I hit a very similar problem today with 2.26.0. Also in a submodule.\n>\n> Removing and regenerating the commit graph did not help and I did not have\n> the\n> commit-graphs directory (only a file). \"git commit-graph verify\" does not\n> find\n> anything. Switching writeCommitGraph on and regenerating the commit graph\n> makes no difference.\n>\n> I can trigger it reliably by visiting the broken(?) commit in supermodule\n> with:\n>\n>     git show --submodule=log <commit>\n>\n> I see nothing special in the commit invovled. It is just a single commit in\n> my\n> case, and the commit is a merge of two branches.\n\nI hit this bug a while back, and it went away after I deleted the\ncommit-graph in the submodule and regenerated it (IIRC).\n\nI just ran into it again (on 2.27.0), and this time, I did some digging.\n\nI have a repository containing a number of submodules, and the bug\nappeared after I updated one of the submodules, and then looked at\n`git log -p` with diff.submodule = log. Just like Alex, I can reliably\ntrigger the error with `git show --submodule=log <commit>`.\n\nI rebuilt git with some print statements to try to see what's going\non, and got the following:\n\n\t/src/oasis/.git/modules/pkg/file/src c81d1ccbf4c224af50e6d556419961dba72666c7\n\t\tpos: 4986, num_commits: 6452, num_commits_in_base: 0\n\t/src/oasis/.git/modules/pkg/file/src 9f2f793847c6aeab9501287b6847dc842c84630f\n\t\tpos: 3964, num_commits: 6452, num_commits_in_base: 0\n\t/src/oasis/.git/modules/pkg/file/src fd7eb1f793944635b92bfa56a84a4dc1dbefb119\n\t\tpos: 6383, num_commits: 6452, num_commits_in_base: 0\n\t/src/oasis/.git/modules/pkg/file/src d955cefc956ba537cfc0556023a65fe80bd2d82b\n\t\tpos: 5436, num_commits: 6452, num_commits_in_base: 0\n\t/src/oasis/.git/modules/pkg/file/src 0c79c693d6a86f7ad7ada2a9a1eb3bdf483f77cc\n\t\tpos: 301, num_commits: 6452, num_commits_in_base: 0\n\t.git fa09b87efa9b9664e4e53ab768cfa5f51a6c6fa2\n\t\tpos: 6292, num_commits: 5177, num_commits_in_base: 0\n\tfatal: invalid commit position. commit-graph is likely corrupt\n\nUsing `git commit-graph verify`, I confirmed that the main\nrepository's commit graph contains 5177 commits, and the submodule\nrepository's commit-graph contains 6452 commits. Commit fa09b8 is part\nof the submodule, not the main repository, so it makes sense that it\nis an invalid commit for the main repositories commit-graph.\n\nSo, this seems a little fishy. fill_commit_in_graph is getting called\nwith the main repository and a commit belonging to the submodule.\nLooking through the call stack in gdb, I see that the initial calls to\nfill_commit_in_graph come from show_submodule_header, which computes\nleft, right, and merge_bases. Then, those commits are passed to\nprepare_submodule_summary, but this function does *not* accept a\nsubmodule parameter. prepare_submodule_summary calls\nrepo_init_revisions with the_repository, which seems to be the source\nof the problem. I think it should be using the submodule repository\ninstead.\n\nI changed prepare_submodule_summary to accept a repository and to use\nthat instead, but the issue persisted. Digging deeper, this is because\nrevision.c:process_parents uses parse_commit_gently, which is a\nsynonym for repo_parse_commit_gently(the_repository, ...). I changed\nit to use repo_parse_commit_gently(revs->repo, ...), and this time,\nthe problem went away.\n\nI'm not very familiar with the git codebase, but am I on the right\ntrack here? I also noticed a number of other calls to\nparse_commit_gently in revision.c, and I think those should pass\nrevs->repo as well. Does that sound right? If so, I can send a patch\nto fix these issues.\n\n-Michael\n"},{"id":"400280","messageId":"0f03151c-d8ec-effc-a8f9-c4a3fc1325c7@gmail.com","threadId":"52667","inReplyTo":"CAGw6cBuEshq18O_PrrGYuJi5VZ82XK3T9KuShneUqO2Ju0jtHw@mail.gmail.com","subject":"Re: Possible git bug in commit-graph: \"invalid commit position\"","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2020-06-22T13:45:24Z","receivedAt":"2020-06-22T13:45:29Z","isPatch":false,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 6/21/2020 4:45 PM, Michael Forney wrote:\n> On 2020-05-15, Alex Riesen <alexander.riesen@cetitec.com> wrote:\n>> Gary Oberbrunner, Tue, Feb 04, 2020 23:33:42 +0100:\n>>> Sorry for the long reply delay; the bug went away and only just showed\n>>> up again. Here's the info you requested.\n>>> I'm now running git 2.25.0.\n>>\n>> I hit a very similar problem today with 2.26.0. Also in a submodule.\n>>\n>> Removing and regenerating the commit graph did not help and I did not have\n>> the\n>> commit-graphs directory (only a file). \"git commit-graph verify\" does not\n>> find\n>> anything. Switching writeCommitGraph on and regenerating the commit graph\n>> makes no difference.\n>>\n>> I can trigger it reliably by visiting the broken(?) commit in supermodule\n>> with:\n>>\n>>     git show --submodule=log <commit>\n>>\n>> I see nothing special in the commit invovled. It is just a single commit in\n>> my\n>> case, and the commit is a merge of two branches.\n> \n> I hit this bug a while back, and it went away after I deleted the\n> commit-graph in the submodule and regenerated it (IIRC).\n> \n> I just ran into it again (on 2.27.0), and this time, I did some digging.\n> \n> I have a repository containing a number of submodules, and the bug\n> appeared after I updated one of the submodules, and then looked at\n> `git log -p` with diff.submodule = log. Just like Alex, I can reliably\n> trigger the error with `git show --submodule=log <commit>`.\n> \n> I rebuilt git with some print statements to try to see what's going\n> on, and got the following:\n> \n> \t/src/oasis/.git/modules/pkg/file/src c81d1ccbf4c224af50e6d556419961dba72666c7\n> \t\tpos: 4986, num_commits: 6452, num_commits_in_base: 0\n> \t/src/oasis/.git/modules/pkg/file/src 9f2f793847c6aeab9501287b6847dc842c84630f\n> \t\tpos: 3964, num_commits: 6452, num_commits_in_base: 0\n> \t/src/oasis/.git/modules/pkg/file/src fd7eb1f793944635b92bfa56a84a4dc1dbefb119\n> \t\tpos: 6383, num_commits: 6452, num_commits_in_base: 0\n> \t/src/oasis/.git/modules/pkg/file/src d955cefc956ba537cfc0556023a65fe80bd2d82b\n> \t\tpos: 5436, num_commits: 6452, num_commits_in_base: 0\n> \t/src/oasis/.git/modules/pkg/file/src 0c79c693d6a86f7ad7ada2a9a1eb3bdf483f77cc\n> \t\tpos: 301, num_commits: 6452, num_commits_in_base: 0\n> \t.git fa09b87efa9b9664e4e53ab768cfa5f51a6c6fa2\n> \t\tpos: 6292, num_commits: 5177, num_commits_in_base: 0\n> \tfatal: invalid commit position. commit-graph is likely corrupt\n> \n> Using `git commit-graph verify`, I confirmed that the main\n> repository's commit graph contains 5177 commits, and the submodule\n> repository's commit-graph contains 6452 commits. Commit fa09b8 is part\n> of the submodule, not the main repository, so it makes sense that it\n> is an invalid commit for the main repositories commit-graph.\n> \n> So, this seems a little fishy. fill_commit_in_graph is getting called\n> with the main repository and a commit belonging to the submodule.\n> Looking through the call stack in gdb, I see that the initial calls to\n> fill_commit_in_graph come from show_submodule_header, which computes\n> left, right, and merge_bases. Then, those commits are passed to\n> prepare_submodule_summary, but this function does *not* accept a\n> submodule parameter. prepare_submodule_summary calls\n> repo_init_revisions with the_repository, which seems to be the source\n> of the problem. I think it should be using the submodule repository\n> instead.\n> \n> I changed prepare_submodule_summary to accept a repository and to use\n> that instead, but the issue persisted. Digging deeper, this is because\n> revision.c:process_parents uses parse_commit_gently, which is a\n> synonym for repo_parse_commit_gently(the_repository, ...). I changed\n> it to use repo_parse_commit_gently(revs->repo, ...), and this time,\n> the problem went away.\n> \n> I'm not very familiar with the git codebase, but am I on the right\n> track here? I also noticed a number of other calls to\n> parse_commit_gently in revision.c, and I think those should pass\n> revs->repo as well. Does that sound right? If so, I can send a patch\n> to fix these issues.\n\nThis is some good digging, and I think you are absolutely correct\nwith the root cause. The dependence on the_repository is still\nsomething that is being worked on in the Git codebase (but less\nfrequently lately) and trips up submodule things like this.\n\nI think a simple method swap would be a good patch to send, and\nyou can include many of the details above in the commit message.\nIs that a contribution you have time to make? I'll gladly review\nit.\n\nThanks,\n-Stolee\n\n\n\n"}]}