{"thread":{"id":"47193","subject":"Bug in \"revision.c: --all adds HEAD from all worktrees\" ?","startedAt":"2017-11-13T19:51:31Z","lastAt":"2018-03-13T11:22:58Z","messageCount":9,"participants":["Luke Diamand","Stefan Beller","Junio C Hamano","Jeff King","Ævar Arnfjörð Bjarmason","Duy Nguyen"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"332436","messageId":"CAE5ih78nLL6UhKPObvFEA9xQZUtc1XpPvGJNaYTH9fJ0RyFRvA@mail.gmail.com","threadId":"47193","inReplyTo":null,"subject":"Bug in \"revision.c: --all adds HEAD from all worktrees\" ?","fromName":"Luke Diamand","fromEmail":"luke@diamand.org","sentAt":"2017-11-13T19:51:26Z","receivedAt":"2017-11-13T19:51:31Z","isPatch":false,"sender":{"key":"luke@diamand.org","avatar":"https://avatars.githubusercontent.com/u/5330967?v=4"},"body":"Hi!\n\nI think there may be a regression caused by this change which means\nthat \"git fetch origin\" doesn't work:\n\ncommit d0c39a49ccb5dfe7feba4325c3374d99ab123c59 (refs/bisect/bad)\nAuthor: Nguyễn Thái Ngọc Duy \nDate: Wed Aug 23 19:36:59 2017 +0700\n\n revision.c: --all adds HEAD from all worktrees\n\n$ git fetch origin\nfatal: bad object HEAD\nerror: ssh://my_remote_host/reponame did not send all necessary objects\n\nI used git bisect to find the problem, and it seems pretty consistent.\n\"git fetch\" with the previous revision works fine.\n\nFWIW, I've got a lot of git worktrees associated with this repo, so\nthat may be why it's failing. The remote repo is actually a git-p4\nclone, so HEAD there actually ends up pointing at\nrefs/remote/p4/master.\n\nThanks,\nLuke\n"},{"id":"332454","messageId":"CAE5ih7_uuVVrze9gNr3JMg5HNH8eAcH_wM4wrc2kH6u=Hw0JOg@mail.gmail.com","threadId":"47193","inReplyTo":"CAE5ih78nLL6UhKPObvFEA9xQZUtc1XpPvGJNaYTH9fJ0RyFRvA@mail.gmail.com","subject":"Re: Bug in \"revision.c: --all adds HEAD from all worktrees\" ?","fromName":"Luke Diamand","fromEmail":"luke@diamand.org","sentAt":"2017-11-13T22:03:44Z","receivedAt":"2017-11-13T22:03:51Z","isPatch":false,"sender":{"key":"luke@diamand.org","avatar":"https://avatars.githubusercontent.com/u/5330967?v=4"},"body":"On 13 November 2017 at 19:51, Luke Diamand wrote:\n> Hi!\n>\n> I think there may be a regression caused by this change which means\n> that \"git fetch origin\" doesn't work:\n>\n> commit d0c39a49ccb5dfe7feba4325c3374d99ab123c59 (refs/bisect/bad)\n> Author: Nguyễn Thái Ngọc Duy \n> Date: Wed Aug 23 19:36:59 2017 +0700\n>\n> revision.c: --all adds HEAD from all worktrees\n>\n> $ git fetch origin\n> fatal: bad object HEAD\n> error: ssh://my_remote_host/reponame did not send all necessary objects\n>\n> I used git bisect to find the problem, and it seems pretty consistent.\n> \"git fetch\" with the previous revision works fine.\n>\n> FWIW, I've got a lot of git worktrees associated with this repo, so\n> that may be why it's failing. The remote repo is actually a git-p4\n> clone, so HEAD there actually ends up pointing at\n> refs/remote/p4/master.\n>\n> Thanks,\n> Luke\n\nQuite a few of the worktrees have expired - their head revision has\nbeen GC'd and no longer points to anything sensible\n(gc.worktreePruneExpire). The function other_head_refs() in worktree.c\nbails out if there's an error, which I think is the problem. I wonder\nif it should instead just report something and then keep going.\n"},{"id":"332458","messageId":"CAGZ79kZrZckX=xTgTH4cMper6BPAQ-_LT_x721SBEBB+Ks06jw@mail.gmail.com","threadId":"47193","inReplyTo":"CAE5ih7_uuVVrze9gNr3JMg5HNH8eAcH_wM4wrc2kH6u=Hw0JOg@mail.gmail.com","subject":"Re: Bug in \"revision.c: --all adds HEAD from all worktrees\" ?","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-11-13T22:15:54Z","receivedAt":"2017-11-13T22:16:00Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Mon, Nov 13, 2017 at 2:03 PM, Luke Diamand wrote:\n> On 13 November 2017 at 19:51, Luke Diamand wrote:\n>> Hi!\n>>\n>> I think there may be a regression caused by this change which means\n>> that \"git fetch origin\" doesn't work:\n>>\n>> commit d0c39a49ccb5dfe7feba4325c3374d99ab123c59 (refs/bisect/bad)\n>> Author: Nguyễn Thái Ngọc Duy \n>> Date: Wed Aug 23 19:36:59 2017 +0700\n>>\n>> revision.c: --all adds HEAD from all worktrees\n>>\n>> $ git fetch origin\n>> fatal: bad object HEAD\n>> error: ssh://my_remote_host/reponame did not send all necessary objects\n>>\n>> I used git bisect to find the problem, and it seems pretty consistent.\n>> \"git fetch\" with the previous revision works fine.\n>>\n>> FWIW, I've got a lot of git worktrees associated with this repo, so\n>> that may be why it's failing. The remote repo is actually a git-p4\n>> clone, so HEAD there actually ends up pointing at\n>> refs/remote/p4/master.\n>>\n>> Thanks,\n>> Luke\n>\n> Quite a few of the worktrees have expired - their head revision has\n> been GC'd and no longer points to anything sensible\n> (gc.worktreePruneExpire). The function other_head_refs() in worktree.c\n> bails out if there's an error, which I think is the problem. I wonder\n> if it should instead just report something and then keep going.\n\nAlso see\nhttps://public-inbox.org/git/CAGZ79kYP0z1G_H3nwfmSHraWHMBOcik5LepUXKj0nveeBrihiw@mail.gmail.com/\n"},{"id":"332620","messageId":"CAE5ih79TojvBv4OZnHejm3Huypg=6KVmDqJURGN+uk+izkH0xw@mail.gmail.com","threadId":"47193","inReplyTo":"CAGZ79kZrZckX=xTgTH4cMper6BPAQ-_LT_x721SBEBB+Ks06jw@mail.gmail.com","subject":"Re: Bug in \"revision.c: --all adds HEAD from all worktrees\" ?","fromName":"Luke Diamand","fromEmail":"luke@diamand.org","sentAt":"2017-11-15T21:38:26Z","receivedAt":"2017-11-15T21:40:53Z","isPatch":false,"sender":{"key":"luke@diamand.org","avatar":"https://avatars.githubusercontent.com/u/5330967?v=4"},"body":"+Jeff King\n\nOn 13 November 2017 at 22:15, Stefan Beller wrote:\n> On Mon, Nov 13, 2017 at 2:03 PM, Luke Diamand wrote:\n>> On 13 November 2017 at 19:51, Luke Diamand wrote:\n>>> Hi!\n>>>\n>>> I think there may be a regression caused by this change which means\n>>> that \"git fetch origin\" doesn't work:\n>>>\n>>> commit d0c39a49ccb5dfe7feba4325c3374d99ab123c59 (refs/bisect/bad)\n>>> Author: Nguyễn Thái Ngọc Duy \n>>> Date: Wed Aug 23 19:36:59 2017 +0700\n>>>\n>>> revision.c: --all adds HEAD from all worktrees\n>>>\n>>> $ git fetch origin\n>>> fatal: bad object HEAD\n>>> error: ssh://my_remote_host/reponame did not send all necessary objects\n>>>\n>>> I used git bisect to find the problem, and it seems pretty consistent.\n>>> \"git fetch\" with the previous revision works fine.\n>>>\n>>> FWIW, I've got a lot of git worktrees associated with this repo, so\n>>> that may be why it's failing. The remote repo is actually a git-p4\n>>> clone, so HEAD there actually ends up pointing at\n>>> refs/remote/p4/master.\n>>>\n>>> Thanks,\n>>> Luke\n>>\n>> Quite a few of the worktrees have expired - their head revision has\n>> been GC'd and no longer points to anything sensible\n>> (gc.worktreePruneExpire). The function other_head_refs() in worktree.c\n>> bails out if there's an error, which I think is the problem. I wonder\n>> if it should instead just report something and then keep going.\n>\n> Also see\n> https://public-inbox.org/git/CAGZ79kYP0z1G_H3nwfmSHraWHMBOcik5LepUXKj0nveeBrihiw@mail.gmail.com/\n\nSo is this a bug or user error on my part?\n\nSurely at the very least \"git fetch\" shouldn't give a cryptic error\nmessage just because one of my git worktrees has expired!\n"},{"id":"332622","messageId":"xmqqtvxvyz3m.fsf@gitster.mtv.corp.google.com","threadId":"47193","inReplyTo":"CAE5ih7_uuVVrze9gNr3JMg5HNH8eAcH_wM4wrc2kH6u=Hw0JOg@mail.gmail.com","subject":"Re: Bug in \"revision.c: --all adds HEAD from all worktrees\" ?","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-11-15T22:08:13Z","receivedAt":"2017-11-15T22:08:23Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Luke Diamand writes:\n\n> Quite a few of the worktrees have expired - their head revision has\n> been GC'd and no longer points to anything sensible\n> (gc.worktreePruneExpire). The function other_head_refs() in worktree.c\n> bails out if there's an error, which I think is the problem. I wonder\n> if it should instead just report something and then keep going.\n\nAm I correct to understand that your \"git fsck\" would fail because\nthese HEAD refs used by other stale worktrees are pointing at\nmissing objects?\n\nWhat do you mean by \"expired\"? \"Even though I want to keep using\nthem, Git for some reason decided to destroy them.\" or \"I no longer\nuse them but kept them lying around.\"?\n\nIf the latter, I wonder \"worktree prune\" to remove the\nadmininstrative information for them would unblock you?\n"},{"id":"332633","messageId":"CAE5ih79wG3ws=OyXqvbd+QKyyAmM-D2JVO5r9G5VHtoOfiXdug@mail.gmail.com","threadId":"47193","inReplyTo":"xmqqtvxvyz3m.fsf@gitster.mtv.corp.google.com","subject":"Re: Bug in \"revision.c: --all adds HEAD from all worktrees\" ?","fromName":"Luke Diamand","fromEmail":"luke@diamand.org","sentAt":"2017-11-16T01:06:01Z","receivedAt":"2017-11-16T01:06:10Z","isPatch":false,"sender":{"key":"luke@diamand.org","avatar":"https://avatars.githubusercontent.com/u/5330967?v=4"},"body":"On 15 November 2017 at 22:08, Junio C Hamano wrote:\n> Luke Diamand writes:\n>\n>> Quite a few of the worktrees have expired - their head revision has\n>> been GC'd and no longer points to anything sensible\n>> (gc.worktreePruneExpire). The function other_head_refs() in worktree.c\n>> bails out if there's an error, which I think is the problem. I wonder\n>> if it should instead just report something and then keep going.\n>\n> Am I correct to understand that your \"git fsck\" would fail because\n> these HEAD refs used by other stale worktrees are pointing at\n> missing objects?\n\ngit fsck says:\n\nChecking object directories: 100% (256/256), done.\nChecking objects: 100% (1434634/1434634), done.\nerror: HEAD: invalid reflog entry\n7fa2b7ee4bc0d11529f659db8b13ed1f537d2a98\nerror: HEAD: invalid reflog entry\n7fa2b7ee4bc0d11529f659db8b13ed1f537d2a98\nerror: HEAD: invalid reflog entry\n7e79e09e8a7382f91610f7255a1b99ea59f68c0b\nerror: refs/stash: invalid reflog entry\nfeeb35e7b045d28943c706e761d0a2ac8206af2f\nerror: refs/remotes/origin/master: invalid reflog entry\n7fa2b7ee4bc0d11529f659db8b13ed1f537d2a98\nChecking connectivity: 1419477, done.\nmissing tree 1480c0a7ed2ad59ae701667292399c38d294658e\nmissing tree ca2a01116bfbbd1fcbcf9812b95d8dc6c39e69d5\nmissing tree 5b7c41e547fc5c4c840e5b496da13d3daebc5fbe\n...\n...\n\n>\n> What do you mean by \"expired\"? \"Even though I want to keep using\n> them, Git for some reason decided to destroy them.\" or \"I no longer\n> use them but kept them lying around.\"?\n\ngit worktree automatically prunes work trees:\n\n\"The working tree’s administrative files in the repository (see\n\"DETAILS\" below) will eventually be removed automatically (see\ngc.worktreePruneExpire in git-config(1)),\"\n\nIn my case I didn't actually want them removed, but fortunately\nthere's nothing important in them (there certainly isn't anymore...).\n\n>\n> If the latter, I wonder \"worktree prune\" to remove the\n> admininstrative information for them would unblock you?\n\nIt doesn't seem to help.\n\n$ git worktree prune -n\n\n$ git worktree prune\n$ git fetch\nremote: Counting objects: 35, done.\nremote: Compressing objects: 100% (20/20), done.\nremote: Total 21 (delta 17), reused 5 (delta 1)\nUnpacking objects: 100% (21/21), done.\nfatal: bad object HEAD\nerror: ssh://whatever/myrepol did not send all necessary objects\n$ /usr/bin/git-2.7.3 fetch\n\n"},{"id":"332776","messageId":"20171117220300.5zvyjwxgi2425m7z@sigill.intra.peff.net","threadId":"47193","inReplyTo":"CAE5ih79TojvBv4OZnHejm3Huypg=6KVmDqJURGN+uk+izkH0xw@mail.gmail.com","subject":"Re: Bug in \"revision.c: --all adds HEAD from all worktrees\" ?","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2017-11-17T22:03:00Z","receivedAt":"2017-11-17T22:03:07Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Nov 15, 2017 at 09:38:26PM +0000, Luke Diamand wrote:\n\n> >> Quite a few of the worktrees have expired - their head revision has\n> >> been GC'd and no longer points to anything sensible\n> >> (gc.worktreePruneExpire). The function other_head_refs() in worktree.c\n> >> bails out if there's an error, which I think is the problem. I wonder\n> >> if it should instead just report something and then keep going.\n> >\n> > Also see\n> > https://public-inbox.org/git/CAGZ79kYP0z1G_H3nwfmSHraWHMBOcik5LepUXKj0nveeBrihiw@mail.gmail.com/\n> \n> So is this a bug or user error on my part?\n\nIt's a bug. You didn't do anything wrong, but the worktree feature\ncorrupted your repository.\n\n-Peff\n"},{"id":"341494","messageId":"878taxq8gm.fsf@evledraar.gmail.com","threadId":"47193","inReplyTo":"CAE5ih79wG3ws=OyXqvbd+QKyyAmM-D2JVO5r9G5VHtoOfiXdug@mail.gmail.com","subject":"Re: Bug in \"revision.c: --all adds HEAD from all worktrees\" ?","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2018-03-12T21:39:53Z","receivedAt":"2018-03-12T21:40:02Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Thu, Nov 16 2017, Luke Diamand jotted:\n\n> On 15 November 2017 at 22:08, Junio C Hamano wrote:\n>> Luke Diamand writes:\n>>\n>>> Quite a few of the worktrees have expired - their head revision has\n>>> been GC'd and no longer points to anything sensible\n>>> (gc.worktreePruneExpire). The function other_head_refs() in worktree.c\n>>> bails out if there's an error, which I think is the problem. I wonder\n>>> if it should instead just report something and then keep going.\n>>\n>> Am I correct to understand that your \"git fsck\" would fail because\n>> these HEAD refs used by other stale worktrees are pointing at\n>> missing objects?\n>\n> git fsck says:\n>\n> Checking object directories: 100% (256/256), done.\n> Checking objects: 100% (1434634/1434634), done.\n> error: HEAD: invalid reflog entry\n> 7fa2b7ee4bc0d11529f659db8b13ed1f537d2a98\n> error: HEAD: invalid reflog entry\n> 7fa2b7ee4bc0d11529f659db8b13ed1f537d2a98\n> error: HEAD: invalid reflog entry\n> 7e79e09e8a7382f91610f7255a1b99ea59f68c0b\n> error: refs/stash: invalid reflog entry\n> feeb35e7b045d28943c706e761d0a2ac8206af2f\n> error: refs/remotes/origin/master: invalid reflog entry\n> 7fa2b7ee4bc0d11529f659db8b13ed1f537d2a98\n> Checking connectivity: 1419477, done.\n> missing tree 1480c0a7ed2ad59ae701667292399c38d294658e\n> missing tree ca2a01116bfbbd1fcbcf9812b95d8dc6c39e69d5\n> missing tree 5b7c41e547fc5c4c840e5b496da13d3daebc5fbe\n> ...\n> ...\n>\n>>\n>> What do you mean by \"expired\"? \"Even though I want to keep using\n>> them, Git for some reason decided to destroy them.\" or \"I no longer\n>> use them but kept them lying around.\"?\n>\n> git worktree automatically prunes work trees:\n>\n> \"The working tree’s administrative files in the repository (see\n> \"DETAILS\" below) will eventually be removed automatically (see\n> gc.worktreePruneExpire in git-config(1)),\"\n>\n> In my case I didn't actually want them removed, but fortunately\n> there's nothing important in them (there certainly isn't anymore...).\n>\n>>\n>> If the latter, I wonder \"worktree prune\" to remove the\n>> admininstrative information for them would unblock you?\n>\n> It doesn't seem to help.\n>\n> $ git worktree prune -n\n> \n> $ git worktree prune\n> $ git fetch\n> remote: Counting objects: 35, done.\n> remote: Compressing objects: 100% (20/20), done.\n> remote: Total 21 (delta 17), reused 5 (delta 1)\n> Unpacking objects: 100% (21/21), done.\n> fatal: bad object HEAD\n> error: ssh://whatever/myrepol did not send all necessary objects\n> $ /usr/bin/git-2.7.3 fetch\n> \n\nDigging up this old thread. I've also noticed this bug because I tried\nto \"git repack -A -d\" a repo on a GitLab server and just got \"fatal: bad\nobject HEAD\".\n\nBisect revealed that the reason was that GitLab itself runs 2.14.3,\nwhich is the last release version that doesn't have Duy's d0c39a49cc\n(\"revision.c: --all adds HEAD from all worktrees\", 2017-08-23), and that\nthe HEAD revision of some worktrees was corrupt (GitLab creates squash-*\nworktrees for some reason).\n\nDoing a \"git worktree prune\" beforehand makes it work.\n\nThis can be reproduced with:\n\n (\n rm -rf /tmp/git &&\n git clone --bare https://github.com/git/git.git /tmp/git &&\n cd /tmp/git &&\n git worktree add blah HEAD &&\n echo 1111111111111111111111111111111111111111 > worktrees/blah/HEAD\n )\n\nNow, obviously the root cause is that the repo is corrupt through some\nother bug (since fixed?), but the error message here is really bad, and\nshould at least say \"fatal: bad object HEAD in worktree blah\" or\nsomething like that.\n"},{"id":"341535","messageId":"CACsJy8CQWBy71-TJCZ7aj4Kt8KawCAACLm3hBLnBX5L8jHv7_Q@mail.gmail.com","threadId":"47193","inReplyTo":"878taxq8gm.fsf@evledraar.gmail.com","subject":"Re: Bug in \"revision.c: --all adds HEAD from all worktrees\" ?","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2018-03-13T11:22:23Z","receivedAt":"2018-03-13T11:22:58Z","isPatch":false,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Mon, Mar 12, 2018 at 10:39 PM, Ævar Arnfjörð Bjarmason\n wrote:\n> Now, obviously the root cause is that the repo is corrupt through some\n> other bug (since fixed?), but the error message here is really bad, and\n> should at least say \"fatal: bad object HEAD in worktree blah\" or\n> something like that.\n\nI know. I know. I'm working on it. I hit a wall and had to re-think my\napproach, but I think it should be ok this time. Will be posting a\nseries soonish, which should also fix fsck ignoring HEADs from other\nworktrees.\n-- \nDuy\n"}]}