{"thread":{"id":"61573","subject":"git fetch --prune fails with \"fatal: bad object\"","startedAt":"2024-05-31T22:41:23Z","lastAt":"2024-06-13T03:29:48Z","messageCount":16,"participants":["Curley Joe","Junio C Hamano","rsbecker@nexbridge.com","Jeff King","Fred Long"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"496032","messageId":"16919-1717194882-875013@sneakemail.com","threadId":"61573","inReplyTo":null,"subject":"git fetch --prune fails with \"fatal: bad object\"","fromName":"Curley Joe","fromEmail":"m48cv7wg9w@liamekaens.com","sentAt":"2024-05-31T22:34:42Z","receivedAt":"2024-05-31T22:41:23Z","isPatch":false,"sender":{"key":"m48cv7wg9w@liamekaens.com","avatar":null},"body":"git fetch --prune fails with \"fatal: bad object\" for refs that have an invalid sha1 pointer. It would be nice if \"git fetch --prune\" could prune these refs, because it is a hassle to do it manually. (\"git fetch --prune\" only shows the first invalid ref, and you have to run \"git fsck\" and parse it to find the rest.) If it seems like adding this functionality to --prune is a bad idea, then how about adding an option like \"--prune-invalid\" ?\n\nThanks.\n"},{"id":"496033","messageId":"xmqqplt1d0k0.fsf@gitster.g","threadId":"61573","inReplyTo":"16919-1717194882-875013@sneakemail.com","subject":"Re: git fetch --prune fails with \"fatal: bad object\"","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-05-31T23:27:43Z","receivedAt":"2024-05-31T23:27:47Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Curley Joe\" <m48cv7wg9w@liamekaens.com> writes:\n\n> git fetch --prune fails with \"fatal: bad object\" for refs that\n> have an invalid sha1 pointer. It would be nice if \"git fetch\n> --prune\" could prune these refs, because it is a hassle to do it\n> manually. (\"git fetch --prune\" only shows the first invalid ref,\n> and you have to run \"git fsck\" and parse it to find the rest.) If\n> it seems like adding this functionality to --prune is a bad idea,\n> then how about adding an option like \"--prune-invalid\" ?\n\nA question and a comment.\n\n - Why did the repository got into this state in the first place?\n   It seems that it would be much better solution to prevent refs\n   from having garbage values in them or to prevent objects that are\n   necessary from going away than any \"prune invalid refs\" feature.\n\n - \"fetch\" still feels a wrong place to have the feature, if it is\n   about fixing a local repository corruption.  You should be able\n   to recover from such a broken ref even if you are only working\n   locally without fetching from anybody.\n\nIf you can somehow _enumerate_ such broken refs, you could drive\nupdate-ref to remove them.  Naïvely, an obvious place to add such a\nfeature might be the \"for-each-ref\" command that is used to list\nrefs with various criteria, so it might look like:\n\n   $ git for-each-ref --format='%(refname)' --broken |\n     xargs git update-ref -d\n\nor something?\n"},{"id":"496035","messageId":"000501dab3b3$51779400$f466bc00$@nexbridge.com","threadId":"61573","inReplyTo":"xmqqplt1d0k0.fsf@gitster.g","subject":"RE: git fetch --prune fails with \"fatal: bad object\"","fromName":"","fromEmail":"rsbecker@nexbridge.com","sentAt":"2024-05-31T23:36:04Z","receivedAt":"2024-05-31T23:40:23Z","isPatch":false,"sender":{"key":"randall.becker@nexbridge.ca","avatar":"https://avatars.githubusercontent.com/u/28956764?v=4"},"body":"On Friday, May 31, 2024 7:28 PM, Junio C Hamano wrote:\n>\"Curley Joe\" <m48cv7wg9w@liamekaens.com> writes:\n>\n>> git fetch --prune fails with \"fatal: bad object\" for refs that have an\n>> invalid sha1 pointer. It would be nice if \"git fetch --prune\" could\n>> prune these refs, because it is a hassle to do it manually. (\"git\n>> fetch --prune\" only shows the first invalid ref, and you have to run\n>> \"git fsck\" and parse it to find the rest.) If it seems like adding\n>> this functionality to --prune is a bad idea, then how about adding an\n>> option like \"--prune-invalid\" ?\n>\n>A question and a comment.\n>\n> - Why did the repository got into this state in the first place?\n>   It seems that it would be much better solution to prevent refs\n>   from having garbage values in them or to prevent objects that are\n>   necessary from going away than any \"prune invalid refs\" feature.\n\nI agree. However, there are some configurations where disk write caches are enabled and require a sync or some other flush operation to force a complete write to disk. In such situations, corruptions are always possible despite the best efforts by the application.\n\n> - \"fetch\" still feels a wrong place to have the feature, if it is\n>   about fixing a local repository corruption.  You should be able\n>   to recover from such a broken ref even if you are only working\n>   locally without fetching from anybody.\n\nI think fsck would be a better place for this.\n\n>If you can somehow _enumerate_ such broken refs, you could drive update-ref to\n>remove them.  Naïvely, an obvious place to add such a feature might be the \"for-\n>each-ref\" command that is used to list refs with various criteria, so it might look like:\n>\n>   $ git for-each-ref --format='%(refname)' --broken |\n>     xargs git update-ref -d\n>\n>or something?\n\n"},{"id":"496038","messageId":"xmqqo78kbqwo.fsf@gitster.g","threadId":"61573","inReplyTo":"000501dab3b3$51779400$f466bc00$@nexbridge.com","subject":"Re: git fetch --prune fails with \"fatal: bad object\"","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-06-01T15:53:43Z","receivedAt":"2024-06-01T15:53:46Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"<rsbecker@nexbridge.com> writes:\n\n>> - Why did the repository got into this state in the first place?\n>>   It seems that it would be much better solution to prevent refs\n>>   from having garbage values in them or to prevent objects that are\n>>   necessary from going away than any \"prune invalid refs\" feature.\n>\n> I agree. However, there are some configurations where disk write\n> caches are enabled and require a sync or some other flush\n> operation to force a complete write to disk. In such situations,\n> corruptions are always possible despite the best efforts by the\n> application.\n\nThe question was posed to see where the current \"best efforts\" are\nstill inadequate.\n\n>> - \"fetch\" still feels a wrong place to have the feature, if it is\n>>   about fixing a local repository corruption.  You should be able\n>>   to recover from such a broken ref even if you are only working\n>>   locally without fetching from anybody.\n>\n> I think fsck would be a better place for this.\n>\n>>If you can somehow _enumerate_ such broken refs, you could drive update-ref to\n>>remove them.\n\nInteresting.  \"git fsck\" certainly can be used to help you find out\nabout them.  In a throw-away repository, after manually crafting\nsome \"broken refs\" (because update-ref will refuse to create a ref\npointing at a missing object):\n\n    $ git for-each-ref\n    9e830ad6c4f43159cef50cb1c2205f513c79bc8b commit refs/heads/master\n    $ echo 9e830ad6c4f43159cef50cb1c2205f513c79bc8a >.git/refs/heads/broken-missing\n    $ git rev-parse master: >.git/refs/heads/broken-tree\n    $ git rev-parse \"master:foo /baz\" >.git/refs/heads/broken-blob\n\nrunning \"git fsck\" does tell you about them, ...\n\n    $ git fsck\n    Checking object directories: 100% (256/256), done.\n    error: refs/heads/broken-blob: not a commit\n    error: refs/heads/broken-missing: invalid sha1 pointer 9e830ad6c4f43159cef50cb1c2205f513c79bc8a\n    error: refs/heads/broken-tree: not a commit\n\n... and using the information, you can\n\n    $ for r in refs/heads/broken-{blob,missing,tree}\n      do git update-ref -d \"$r\"\n      done\n\nto unbreak the repository.\n"},{"id":"496240","messageId":"20240604104437.GD1781455@coredump.intra.peff.net","threadId":"61573","inReplyTo":"xmqqo78kbqwo.fsf@gitster.g","subject":"Re: git fetch --prune fails with \"fatal: bad object\"","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2024-06-04T10:44:37Z","receivedAt":"2024-06-04T10:44:39Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Sat, Jun 01, 2024 at 08:53:43AM -0700, Junio C Hamano wrote:\n\n> Interesting.  \"git fsck\" certainly can be used to help you find out\n> about them.  In a throw-away repository, after manually crafting\n> some \"broken refs\" (because update-ref will refuse to create a ref\n> pointing at a missing object):\n> \n>     $ git for-each-ref\n>     9e830ad6c4f43159cef50cb1c2205f513c79bc8b commit refs/heads/master\n>     $ echo 9e830ad6c4f43159cef50cb1c2205f513c79bc8a >.git/refs/heads/broken-missing\n>     $ git rev-parse master: >.git/refs/heads/broken-tree\n>     $ git rev-parse \"master:foo /baz\" >.git/refs/heads/broken-blob\n> \n> running \"git fsck\" does tell you about them, ...\n> \n>     $ git fsck\n>     Checking object directories: 100% (256/256), done.\n>     error: refs/heads/broken-blob: not a commit\n>     error: refs/heads/broken-missing: invalid sha1 pointer 9e830ad6c4f43159cef50cb1c2205f513c79bc8a\n>     error: refs/heads/broken-tree: not a commit\n> \n> ... and using the information, you can\n> \n>     $ for r in refs/heads/broken-{blob,missing,tree}\n>       do git update-ref -d \"$r\"\n>       done\n> \n> to unbreak the repository.\n\nThese are good examples. I was going to suggest fsck, as well, just\nbecause I knew it would keep going after seeing bogus results. But more\ninteresting is that it is finding things in your example that other\nprograms would _not_ find, because it's being more thorough than just\nreading the refs.\n\nHaving to manually convert the human-readable fsck output to a cleanup\ncommand is a minor pain. We could provide \"git fsck --prune-broken-refs\"\nor something, but I'm hesitant. Deleting refs in a corrupted repository\nis a good way to make recovery even harder, as it opens the door to\nremoving whatever objects we do still have.\n\nIn the case of a refs/remotes entry where you happen to know that you\ncould re-clone from the other side, it is relatively low stakes. But I\nthink keeping a human brain in the loop between corruption and deletion\nis a good thing. Corruption should not be happening so often that it's a\nmajor pain point.\n\n-Peff\n"},{"id":"496297","messageId":"xmqqcyowwqah.fsf@gitster.g","threadId":"61573","inReplyTo":"20240604104437.GD1781455@coredump.intra.peff.net","subject":"Re: git fetch --prune fails with \"fatal: bad object\"","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-06-04T17:50:30Z","receivedAt":"2024-06-04T17:50:33Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King <peff@peff.net> writes:\n\n> These are good examples. I was going to suggest fsck, as well, just\n> because I knew it would keep going after seeing bogus results. But more\n> interesting is that it is finding things in your example that other\n> programs would _not_ find, because it's being more thorough than just\n> reading the refs.\n\nTrue.\n\nI wish for-each-ref and friends had an optional mode that lets them\nkeep going, but since so many features in them access objects\npointed at by the refs (e.g., \"--format='%(objectname:short)'\" and\n\"--no-merged HEAD\"), it would be very cumbersome to retrofit such a\nmode to the underlying machinery, I suspect.\n\nThanks.\n"},{"id":"496309","messageId":"876fbb07-210e-128e-5289-57ab01761750@fastmail.fm","threadId":"61573","inReplyTo":"20240604104437.GD1781455@coredump.intra.peff.net","subject":"Re: git fetch --prune fails with \"fatal: bad object\"","fromName":"Fred Long","fromEmail":"fred_long@fastmail.fm","sentAt":"2024-06-04T20:09:10Z","receivedAt":"2024-06-04T20:09:13Z","isPatch":false,"sender":{"key":"fred_long@fastmail.fm","avatar":null},"body":"On 6/4/2024 3:44 AM, Jeff King peff-at-peff.net |git bugs/Example Allow| \nwrote:\n> In the case of a refs/remotes entry where you happen to know that you\n> could re-clone from the other side, it is relatively low stakes. But I\n> think keeping a human brain in the loop between corruption and deletion\n> is a good thing. Corruption should not be happening so often that it's a\n> major pain point.\nIn my case it's not corruption. It's people creating branches, deleting \nthem, and then removing the commits. (Maybe our git server has an option \nto automatically prune commits that are not reachable from a branch or \ntag, I don't know.) But this happens very frequently at my work.\n\n"},{"id":"496347","messageId":"20240605084517.GC2345232@coredump.intra.peff.net","threadId":"61573","inReplyTo":"xmqqcyowwqah.fsf@gitster.g","subject":"Re: git fetch --prune fails with \"fatal: bad object\"","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2024-06-05T08:45:17Z","receivedAt":"2024-06-05T08:45:19Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Jun 04, 2024 at 10:50:30AM -0700, Junio C Hamano wrote:\n\n> Jeff King <peff@peff.net> writes:\n> \n> > These are good examples. I was going to suggest fsck, as well, just\n> > because I knew it would keep going after seeing bogus results. But more\n> > interesting is that it is finding things in your example that other\n> > programs would _not_ find, because it's being more thorough than just\n> > reading the refs.\n> \n> True.\n> \n> I wish for-each-ref and friends had an optional mode that lets them\n> keep going, but since so many features in them access objects\n> pointed at by the refs (e.g., \"--format='%(objectname:short)'\" and\n> \"--no-merged HEAD\"), it would be very cumbersome to retrofit such a\n> mode to the underlying machinery, I suspect.\n\nWe try to read as little as possible in for-each-ref for efficiency\npurposes. The side effect is that if you don't ask for any details of\nthe object, we won't try to read it and notice that it's gone. ;)\n\nSo you can do:\n\n  git for-each-ref --format='%(objectname) %(refname)'\n\nwhich will show even broken refs. Of course you still don't know which\nones are broken! For that, I'd rely on something like cat-file, where it\nwill reliably tell you about missing objects and continue (even if we\njust print the object name, it does an existence check). So naively I'd\nwant something like this to work:\n\n  git for-each-ref --format='%(objectname) %(refname)' |\n  git cat-file --batch-check='%(objectname) %(rest)' |\n  grep ^missing\n\nBut annoyingly, when it encounters a missing object it, cat-file ditches\nthe whole format string and just prints \"$oid missing\". You can work\naround it with a little shell magic:\n\n  git for-each-ref --format='%(objectname) %(refname)' >both\n  cut -d' ' -f1 <both | git cat-file --batch-check='%(objectname)' >exists\n  paste -d' ' both exists | cut -d' ' -f2,4\n\nwhich yields lines like:\n\n  refs/heads/ok\n  refs/heads/broken missing\n\nand so on. Turning that into \"update-ref --stdin\" input is left as an\nexercise for the reader.\n\nI suspect it's too late to change the default behavior for a plumbing\ntool like cat-file. But we could probably teach it an option like\n\"--missing=foo\" to expand the format but use \"foo\" for object-related\nplaceholders that can't be expanded. And then:\n\n  git for-each-ref --format='%(objectname) %(refname)' |\n  git cat-file --batch-check='%(objectname) %(rest)' --missing=MISSING |\n  grep ^MISSING\n\nwould work. I still think it's inferior to fsck, though, as it's a very\nshallow check of validity.\n\n-Peff\n"},{"id":"496348","messageId":"20240605084718.GD2345232@coredump.intra.peff.net","threadId":"61573","inReplyTo":"876fbb07-210e-128e-5289-57ab01761750@fastmail.fm","subject":"Re: git fetch --prune fails with \"fatal: bad object\"","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2024-06-05T08:47:18Z","receivedAt":"2024-06-05T08:47:19Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Jun 04, 2024 at 01:09:10PM -0700, Fred Long wrote:\n\n> On 6/4/2024 3:44 AM, Jeff King peff-at-peff.net |git bugs/Example Allow|\n> wrote:\n> > In the case of a refs/remotes entry where you happen to know that you\n> > could re-clone from the other side, it is relatively low stakes. But I\n> > think keeping a human brain in the loop between corruption and deletion\n> > is a good thing. Corruption should not be happening so often that it's a\n> > major pain point.\n> In my case it's not corruption. It's people creating branches, deleting\n> them, and then removing the commits. (Maybe our git server has an option to\n> automatically prune commits that are not reachable from a branch or tag, I\n> don't know.) But this happens very frequently at my work.\n\nYour local refs should not point to missing objects, though. Each clone\nshould maintain its own consistency. Are you using \"git clone --shared\"\nor another scheme involving alternates?\n\n-Peff\n"},{"id":"496407","messageId":"c2a02982-40a9-6629-05c2-c8de3335f35b@fastmail.fm","threadId":"61573","inReplyTo":"20240605084718.GD2345232@coredump.intra.peff.net","subject":"Re: git fetch --prune fails with \"fatal: bad object\"","fromName":"Fred Long","fromEmail":"fred_long@fastmail.fm","sentAt":"2024-06-05T23:43:45Z","receivedAt":"2024-06-05T23:43:47Z","isPatch":false,"sender":{"key":"fred_long@fastmail.fm","avatar":null},"body":"On 6/5/2024 1:47 AM, Jeff King wrote:\n>\n> Your local refs should not point to missing objects, though. Each clone\n> should maintain its own consistency. Are you using \"git clone --shared\"\n> or another scheme involving alternates?\n>\n> -Peff\nI use Google's \"repo\" script to manage Android repositories. The \"repo\" \nscript runs \"git repack -a -d\", and I think that's what's removing the \ncommits locally. Remotely, our bitbucket server removes commits that are \nno longer referenced.\n"},{"id":"496408","messageId":"20240606011453.GA623737@coredump.intra.peff.net","threadId":"61573","inReplyTo":"c2a02982-40a9-6629-05c2-c8de3335f35b@fastmail.fm","subject":"Re: git fetch --prune fails with \"fatal: bad object\"","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2024-06-06T01:14:53Z","receivedAt":"2024-06-06T01:14:54Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Jun 05, 2024 at 04:43:45PM -0700, Fred Long wrote:\n\n> On 6/5/2024 1:47 AM, Jeff King wrote:\n> > \n> > Your local refs should not point to missing objects, though. Each clone\n> > should maintain its own consistency. Are you using \"git clone --shared\"\n> > or another scheme involving alternates?\n> > \n> I use Google's \"repo\" script to manage Android repositories. The \"repo\"\n> script runs \"git repack -a -d\", and I think that's what's removing the\n> commits locally. Remotely, our bitbucket server removes commits that are no\n> longer referenced.\n\nUsing \"git repack -a -d\" by itself should retain the objects that are\nreachable from your local refs/remotes/ refs. But I'd suspect that\n\"repo\" is doing something clever under the hood. I know there's some\nsupport for alternates, and a quick (hah!) \"repo init && repo sync\" on\nthe android manifest shows that we end up symlinked objects directories.\nSo there is \".repo/projects/foo.git\" whose objects directory is a\nsymlink to \".repo/project-objects/foo.git/objects\".\n\nI don't know how it is all supposed to work, but naively running \"git\nrepack -a -d\" in the \"project-objects\" version risks corrupting the\n\"projects\" version, because it has no idea what refs are in the latter.\n\nCertainly this warning is ominous:\n\n  https://gerrit.googlesource.com/git-repo/+/refs/tags/v2.45/docs/internal-fs-layout.md#project-objects\n\n-Peff\n"},{"id":"496561","messageId":"5469592f-69eb-b1da-1218-abeae88f7303@fastmail.fm","threadId":"61573","inReplyTo":"20240606011453.GA623737@coredump.intra.peff.net","subject":"Re: git fetch --prune fails with \"fatal: bad object\"","fromName":"Fred Long","fromEmail":"fred_long@fastmail.fm","sentAt":"2024-06-06T20:12:31Z","receivedAt":"2024-06-06T20:12:33Z","isPatch":false,"sender":{"key":"fred_long@fastmail.fm","avatar":null},"body":"On 6/5/2024 6:14 PM, Jeff King wrote:\n>\n> Certainly this warning is ominous:\n>\n>    https://gerrit.googlesource.com/git-repo/+/refs/tags/v2.45/docs/internal-fs-layout.md#project-objects\n>\n> -Peff\nYeah, I saw that, and I think I know what's causing the problem. First I \ncreate a bare mirror of AOSP with \"repo init --mirror\" and \"repo sync\", \nthen for all my different source trees I use \"repo init --reference\" to \nshare objects with the mirror using alternates files. Running \"repo \nsync\" on the bare mirror works fine. The problem occurs when I run \"repo \nsync\" in one of my source trees. Commits and refs have been deleted from \nthe mirror, but the refs are not deleted from the repos that point to \nthe mirror. So it all makes sense now. Still, I wish there were a simple \ncommand I could run to remove the dangling refs.\n\n\n"},{"id":"496709","messageId":"20240608112033.GB2966571@coredump.intra.peff.net","threadId":"61573","inReplyTo":"5469592f-69eb-b1da-1218-abeae88f7303@fastmail.fm","subject":"Re: git fetch --prune fails with \"fatal: bad object\"","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2024-06-08T11:20:33Z","receivedAt":"2024-06-08T11:20:34Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Jun 06, 2024 at 01:12:31PM -0700, Fred Long wrote:\n\n> Yeah, I saw that, and I think I know what's causing the problem. First I\n> create a bare mirror of AOSP with \"repo init --mirror\" and \"repo sync\", then\n> for all my different source trees I use \"repo init --reference\" to share\n> objects with the mirror using alternates files. Running \"repo sync\" on the\n> bare mirror works fine. The problem occurs when I run \"repo sync\" in one of\n> my source trees. Commits and refs have been deleted from the mirror, but the\n> refs are not deleted from the repos that point to the mirror. So it all\n> makes sense now. Still, I wish there were a simple command I could run to\n> remove the dangling refs.\n\nYep, that all makes sense.\n\nAs I said before, I don't think auto-deleting corrupted refs is great in\ngeneral, just because it can make a bad situation worse. But if you want\nthe foot-gun, here's a more complete script that you could run. It\n_just_ looks in refs/remotes/, which is perhaps a bit safer.\n\n  git for-each-ref --format='%(refname)' refs/remotes/ |\n  git cat-file --batch-check='%(objectname)' |\n  perl -alne 'print \"delete $F[0]\" if $F[1] eq \"missing\"' |\n  tee /dev/stderr |\n  git update-ref --stdin\n\n-Peff\n"},{"id":"496726","messageId":"5247e0cc-ef0e-558a-7e06-5a7425a52329@fastmail.fm","threadId":"61573","inReplyTo":"20240608112033.GB2966571@coredump.intra.peff.net","subject":"Re: git fetch --prune fails with \"fatal: bad object\"","fromName":"Fred Long","fromEmail":"fred_long@fastmail.fm","sentAt":"2024-06-08T21:02:52Z","receivedAt":"2024-06-08T21:02:57Z","isPatch":false,"sender":{"key":"fred_long@fastmail.fm","avatar":null},"body":"On 6/8/2024 4:20 AM, Jeff King wrote:\n> On Thu, Jun 06, 2024 at 01:12:31PM -0700, Fred Long wrote:\n>\n>> Yeah, I saw that, and I think I know what's causing the problem. First I\n>> create a bare mirror of AOSP with \"repo init --mirror\" and \"repo sync\", then\n>> for all my different source trees I use \"repo init --reference\" to share\n>> objects with the mirror using alternates files. Running \"repo sync\" on the\n>> bare mirror works fine. The problem occurs when I run \"repo sync\" in one of\n>> my source trees. Commits and refs have been deleted from the mirror, but the\n>> refs are not deleted from the repos that point to the mirror. So it all\n>> makes sense now. Still, I wish there were a simple command I could run to\n>> remove the dangling refs.\n> Yep, that all makes sense.\n>\n> As I said before, I don't think auto-deleting corrupted refs is great in\n> general, just because it can make a bad situation worse. But if you want\n> the foot-gun, here's a more complete script that you could run. It\n> _just_ looks in refs/remotes/, which is perhaps a bit safer.\n>\n>    git for-each-ref --format='%(refname)' refs/remotes/ |\n>    git cat-file --batch-check='%(objectname)' |\n>    perl -alne 'print \"delete $F[0]\" if $F[1] eq \"missing\"' |\n>    tee /dev/stderr |\n>    git update-ref --stdin\n>\n> -Peff\nThanks. Here's what I have been using:\n\ngit fsck |& grep \"invalid sha1 pointer\" | (\n     while read err ref rest\n     do\n         ref=${ref%:}\n         echo got $ref, removing .git/$ref\n         rm .git/$ref\n     done\n)\n\n"},{"id":"496837","messageId":"20240611073134.GE3248245@coredump.intra.peff.net","threadId":"61573","inReplyTo":"5247e0cc-ef0e-558a-7e06-5a7425a52329@fastmail.fm","subject":"Re: git fetch --prune fails with \"fatal: bad object\"","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2024-06-11T07:31:34Z","receivedAt":"2024-06-11T07:31:36Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Sat, Jun 08, 2024 at 02:02:52PM -0700, Fred Long wrote:\n\n> >    git for-each-ref --format='%(refname)' refs/remotes/ |\n> >    git cat-file --batch-check='%(objectname)' |\n> >    perl -alne 'print \"delete $F[0]\" if $F[1] eq \"missing\"' |\n> >    tee /dev/stderr |\n> >    git update-ref --stdin\n> > \n> Thanks. Here's what I have been using:\n> \n> git fsck |& grep \"invalid sha1 pointer\" | (\n>     while read err ref rest\n>     do\n>         ref=${ref%:}\n>         echo got $ref, removing .git/$ref\n>         rm .git/$ref\n>     done\n> )\n\nParsing fsck's errors should be OK, I think, but your \"rm\" won't be\nreliable. If the ref is packed, then it needs to be removed from\n.git/packed-refs, too. Calling \"git update-ref -d $ref\" should work,\neven for a broken ref.\n\n-Peff\n"},{"id":"497055","messageId":"fbe6ae6c-a89b-ae71-2474-10ac3afd0766@fastmail.fm","threadId":"61573","inReplyTo":"20240611073134.GE3248245@coredump.intra.peff.net","subject":"Re: git fetch --prune fails with \"fatal: bad object\"","fromName":"Fred Long","fromEmail":"fred_long@fastmail.fm","sentAt":"2024-06-13T03:29:47Z","receivedAt":"2024-06-13T03:29:48Z","isPatch":false,"sender":{"key":"fred_long@fastmail.fm","avatar":null},"body":"On 6/11/2024 12:31 AM, Jeff King wrote:\n> On Sat, Jun 08, 2024 at 02:02:52PM -0700, Fred Long wrote:\n>\n>>>     git for-each-ref --format='%(refname)' refs/remotes/ |\n>>>     git cat-file --batch-check='%(objectname)' |\n>>>     perl -alne 'print \"delete $F[0]\" if $F[1] eq \"missing\"' |\n>>>     tee /dev/stderr |\n>>>     git update-ref --stdin\n>>>\n>> Thanks. Here's what I have been using:\n>>\n>> git fsck |& grep \"invalid sha1 pointer\" | (\n>>      while read err ref rest\n>>      do\n>>          ref=${ref%:}\n>>          echo got $ref, removing .git/$ref\n>>          rm .git/$ref\n>>      done\n>> )\n> Parsing fsck's errors should be OK, I think, but your \"rm\" won't be\n> reliable. If the ref is packed, then it needs to be removed from\n> .git/packed-refs, too. Calling \"git update-ref -d $ref\" should work,\n> even for a broken ref.\n>\n> -Peff\nYes, and your solution is better than a newbie like me could have \nfigured out, which is why I have been arguing for a simple documented \ncommand that people like me can run.\n\n"}]}