{"thread":{"id":"15584","subject":"How to remove a commit object?","startedAt":"2008-09-18T23:41:30Z","lastAt":"2008-10-03T12:03:14Z","messageCount":11,"participants":["Steven Grimm","Michael J Gruber","Klas Lindberg","Jakub Narebski","Johannes Sixt"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"91085","messageId":"D53D127D-97DD-417C-9109-ABD46BBD4E18@midwinter.com","threadId":"15584","inReplyTo":null,"subject":"How to remove a commit object?","fromName":"Steven Grimm","fromEmail":"koreth@midwinter.com","sentAt":"2008-09-18T23:41:30Z","receivedAt":"2008-09-18T23:41:30Z","isPatch":false,"sender":{"key":"koreth@midwinter.com","avatar":"https://gravatar.com/avatar/71b4d2e8b62f168bdc9e9205341159e3567003b4f9e2127c617c5fa0a1f5bad2?d=mp&s=160"},"body":"I maintain a shared repository a bunch of my coworkers push to for \ncode reviews. It has accumulated a lot of packfiles, so I want to \nshrink it down a bit, but there's a problem:\n\n% git repack -A -d\nCounting objects: ...\nerror: Could not read 125bf191b65189aaec7a6aa24ff26460d141d587\nfatal: bad tree object 125bf191b65189aaec7a6aa24ff26460d141d587\n\n\"git fsck\" confirms that the tree object is missing:\n\n% git fsck\nbroken link from commit 1b2f0595bb4a6c2e17ca43a9cc41feec88c72a47\n to tree 125bf191b65189aaec7a6aa24ff26460d141d587\n...\nmissing tree 125bf191b65189aaec7a6aa24ff26460d141d587\n\nThis is a dangling commit, but that's fine; for this particular \nrepository we actually *want* lots of dangling commits since they \nrepresent the history of people's code review requests. (Hence me \nrunning git-repack with -A instead of -a.)\n\nGiven that it's dangling, it seems like it'd be safe to just remove \nentirely (we lose that little bit of code-review history but we've \nlost it already anyway with the tree object missing). But I'm not sure \nhow to do it. Is it possible to delete a commit object, and if so, how?\n\nI don't know how the corruption happened in the first place. There was \na short time at one point where the permissions on the object \ndirectories were inconsistent, so it's possible someone pushed during \nthat period and managed to create the commit object file in .git/ \nobjects but didn't have permission to create the tree object. That's \njust speculation on my part, though. This is the only corrupt object \nin the repository according to git-fsck, so at this point I just want \nto know how to get rid of it so I can do the repack.\n\nThanks!\n\n-Steve\n"},{"id":"91099","messageId":"48D36DF4.3030607@drmicha.warpmail.net","threadId":"15584","inReplyTo":"D53D127D-97DD-417C-9109-ABD46BBD4E18@midwinter.com","subject":"Re: How to remove a commit object?","fromName":"Michael J Gruber","fromEmail":"git@drmicha.warpmail.net","sentAt":"2008-09-19T09:16:36Z","receivedAt":"2008-09-19T09:16:36Z","isPatch":false,"sender":{"key":"git@grubix.eu","avatar":"https://avatars.githubusercontent.com/u/233215?v=4"},"body":"Steven Grimm venit, vidit, dixit 19.09.2008 01:41:\n> I maintain a shared repository a bunch of my coworkers push to for \n> code reviews. It has accumulated a lot of packfiles, so I want to \n> shrink it down a bit, but there's a problem:\n> \n> % git repack -A -d\n> Counting objects: ...\n> error: Could not read 125bf191b65189aaec7a6aa24ff26460d141d587\n> fatal: bad tree object 125bf191b65189aaec7a6aa24ff26460d141d587\n> \n> \"git fsck\" confirms that the tree object is missing:\n> \n> % git fsck\n> broken link from commit 1b2f0595bb4a6c2e17ca43a9cc41feec88c72a47\n> to tree 125bf191b65189aaec7a6aa24ff26460d141d587\n> ...\n> missing tree 125bf191b65189aaec7a6aa24ff26460d141d587\n> \n> This is a dangling commit, but that's fine; for this particular \n> repository we actually *want* lots of dangling commits since they \n> represent the history of people's code review requests. (Hence me \n> running git-repack with -A instead of -a.)\n> \n> Given that it's dangling, it seems like it'd be safe to just remove \n> entirely (we lose that little bit of code-review history but we've \n> lost it already anyway with the tree object missing). But I'm not sure \n> how to do it. Is it possible to delete a commit object, and if so, how?\n> \n> I don't know how the corruption happened in the first place. There was \n> a short time at one point where the permissions on the object \n> directories were inconsistent, so it's possible someone pushed during \n> that period and managed to create the commit object file in .git/ \n> objects but didn't have permission to create the tree object. That's \n> just speculation on my part, though. This is the only corrupt object \n> in the repository according to git-fsck, so at this point I just want \n> to know how to get rid of it so I can do the repack.\n\ngit prune should delete dangling commits. Is that commit already in a\npack? Then the -f option to repack may help.\n\nMichael\n"},{"id":"92150","messageId":"33f4f4d70810020636g77f4180cm7aa6b7de07ae5beb@mail.gmail.com","threadId":"15584","inReplyTo":"48D36DF4.3030607@drmicha.warpmail.net","subject":"Re: How to remove a commit object?","fromName":"Klas Lindberg","fromEmail":"klas.lindberg@gmail.com","sentAt":"2008-10-02T13:36:41Z","receivedAt":"2008-10-02T13:36:41Z","isPatch":false,"sender":{"key":"klas.lindberg@gmail.com","avatar":null},"body":"This doesn't seem to work for me. I will soon be in a situation where\nI need to selectively delete commits in such a way that they become\ncompletely irrecoverable. I.e. it is not enough to revert a commit.\nThe *original* commit must be removed. And of course, the repo history\nis too complex to allow for rebasing followed by garbage collection or\nsomething like that.\n\nThe reason is that we consider opening a repository to external\nparticipants, but some commits contain stuff that we'd really rather\nnot show to anyone else. Making the repository public without loosing\nhistory would then force us to either\n\n 1. Recreate every commit in a new repo, sans the offending commits.\nSeems like hard work.\n 2. ?\n\nWould it be feasible to write a tool that can selectively replace a\nspecific commit in the commit DAG, or would that automatically\ninvalidate every SHA key for every commit that follows the replaced\noriginal?\n\nBR / Klas\n\nOn Fri, Sep 19, 2008 at 11:16 AM, Michael J Gruber\n wrote:\n> Steven Grimm venit, vidit, dixit 19.09.2008 01:41:\n>> I maintain a shared repository a bunch of my coworkers push to for\n>> code reviews. It has accumulated a lot of packfiles, so I want to\n>> shrink it down a bit, but there's a problem:\n>>\n>> % git repack -A -d\n>> Counting objects: ...\n>> error: Could not read 125bf191b65189aaec7a6aa24ff26460d141d587\n>> fatal: bad tree object 125bf191b65189aaec7a6aa24ff26460d141d587\n>>\n>> \"git fsck\" confirms that the tree object is missing:\n>>\n>> % git fsck\n>> broken link from commit 1b2f0595bb4a6c2e17ca43a9cc41feec88c72a47\n>> to tree 125bf191b65189aaec7a6aa24ff26460d141d587\n>> ...\n>> missing tree 125bf191b65189aaec7a6aa24ff26460d141d587\n>>\n>> This is a dangling commit, but that's fine; for this particular\n>> repository we actually *want* lots of dangling commits since they\n>> represent the history of people's code review requests. (Hence me\n>> running git-repack with -A instead of -a.)\n>>\n>> Given that it's dangling, it seems like it'd be safe to just remove\n>> entirely (we lose that little bit of code-review history but we've\n>> lost it already anyway with the tree object missing). But I'm not sure\n>> how to do it. Is it possible to delete a commit object, and if so, how?\n>>\n>> I don't know how the corruption happened in the first place. There was\n>> a short time at one point where the permissions on the object\n>> directories were inconsistent, so it's possible someone pushed during\n>> that period and managed to create the commit object file in .git/\n>> objects but didn't have permission to create the tree object. That's\n>> just speculation on my part, though. This is the only corrupt object\n>> in the repository according to git-fsck, so at this point I just want\n>> to know how to get rid of it so I can do the repack.\n>\n> git prune should delete dangling commits. Is that commit already in a\n> pack? Then the -f option to repack may help.\n>\n> Michael\n> --\n> To unsubscribe from this list: send the line \"unsubscribe git\" in\n> the body of a message to majordomo@vger.kernel.org\n> More majordomo info at http://vger.kernel.org/majordomo-info.html\n>\n"},{"id":"92157","messageId":"48E4D3E9.8020906@drmicha.warpmail.net","threadId":"15584","inReplyTo":"33f4f4d70810020636g77f4180cm7aa6b7de07ae5beb@mail.gmail.com","subject":"Re: How to remove a commit object?","fromName":"Michael J Gruber","fromEmail":"git@drmicha.warpmail.net","sentAt":"2008-10-02T14:00:09Z","receivedAt":"2008-10-02T14:00:09Z","isPatch":false,"sender":{"key":"git@grubix.eu","avatar":"https://avatars.githubusercontent.com/u/233215?v=4"},"body":"Klas Lindberg venit, vidit, dixit 02.10.2008 15:36:\n> This doesn't seem to work for me. I will soon be in a situation where\n> I need to selectively delete commits in such a way that they become\n> completely irrecoverable. I.e. it is not enough to revert a commit.\n> The *original* commit must be removed. And of course, the repo history\n> is too complex to allow for rebasing followed by garbage collection or\n> something like that.\n> \n> The reason is that we consider opening a repository to external\n> participants, but some commits contain stuff that we'd really rather\n> not show to anyone else. Making the repository public without loosing\n> history would then force us to either\n> \n> 1. Recreate every commit in a new repo, sans the offending commits.\n> Seems like hard work.\n> 2. ?\n> \n> Would it be feasible to write a tool that can selectively replace a\n> specific commit in the commit DAG, or would that automatically\n> invalidate every SHA key for every commit that follows the replaced\n> original?\n\nYes, on the or part: If you change a commit then all commits \"after\"\nthat one (in terms of DAG connectedness) will need to be changed: each\ncontains a \"backpointer\" (to the parent commit(s)) which is changed.\n\nI'm a bit confused: You rule out rebasing but don't mind recreating a\nnew repo. So repo size is not a problem, is it?\n\nMichael\n"},{"id":"92159","messageId":"m3ljx7qemk.fsf@localhost.localdomain","threadId":"15584","inReplyTo":"33f4f4d70810020636g77f4180cm7aa6b7de07ae5beb@mail.gmail.com","subject":"Re: How to remove a commit object?","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2008-10-02T14:02:32Z","receivedAt":"2008-10-02T14:02:32Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"\"Klas Lindberg\" writes:\n\n> This doesn't seem to work for me. I will soon be in a situation where\n> I need to selectively delete commits in such a way that they become\n> completely irrecoverable. I.e. it is not enough to revert a commit.\n> The *original* commit must be removed. And of course, the repo history\n> is too complex to allow for rebasing followed by garbage collection or\n> something like that.\n[...]\n\n> Would it be feasible to write a tool that can selectively replace a\n> specific commit in the commit DAG, or would that automatically\n> invalidate every SHA key for every commit that follows the replaced\n> original?\n\nIt would invalidate SHA1 for every commit after first rewritten.\nThere are two tools which you can use to rewrite large parts of\nhistory automatically: git-filter-branch, and git-fast-export +\ngit-fast-import.\n\nHTH\n-- \nJakub Narebski\nPoland\nShadeHawk on #git\n"},{"id":"92165","messageId":"33f4f4d70810020726g71c6f39eq16585269fb268322@mail.gmail.com","threadId":"15584","inReplyTo":"m3ljx7qemk.fsf@localhost.localdomain","subject":"Re: How to remove a commit object?","fromName":"Klas Lindberg","fromEmail":"klas.lindberg@gmail.com","sentAt":"2008-10-02T14:26:29Z","receivedAt":"2008-10-02T14:26:29Z","isPatch":false,"sender":{"key":"klas.lindberg@gmail.com","avatar":null},"body":"Repo size is a problem too, actually.\n\nA solution to both problems seemed to be to use git-filter-branch to\ncreate a new repo by filtering out all the unwanted files. The\nastonishing result was that, for the subdirectory I tried it on, 90%\nor so of the commits on that subdirectory just disappeared. It didn't\nlook right at all. Although I can't say for sure exactly what I did\nwith filter-branch, I would appreciate some guidance for using it. It\nbasically seemed to do exactly what I wanted (recreate the repo, minus\nsome explicit stuff, with history intact otherwise), except the result\nlooked crazy.\n\n/Klas\n\nOn Thu, Oct 2, 2008 at 4:02 PM, Jakub Narebski wrote:\n> \"Klas Lindberg\" writes:\n>\n>> This doesn't seem to work for me. I will soon be in a situation where\n>> I need to selectively delete commits in such a way that they become\n>> completely irrecoverable. I.e. it is not enough to revert a commit.\n>> The *original* commit must be removed. And of course, the repo history\n>> is too complex to allow for rebasing followed by garbage collection or\n>> something like that.\n> [...]\n>\n>> Would it be feasible to write a tool that can selectively replace a\n>> specific commit in the commit DAG, or would that automatically\n>> invalidate every SHA key for every commit that follows the replaced\n>> original?\n>\n> It would invalidate SHA1 for every commit after first rewritten.\n> There are two tools which you can use to rewrite large parts of\n> history automatically: git-filter-branch, and git-fast-export +\n> git-fast-import.\n>\n> HTH\n> --\n> Jakub Narebski\n> Poland\n> ShadeHawk on #git\n>\n"},{"id":"92167","messageId":"48E4DB21.7020304@drmicha.warpmail.net","threadId":"15584","inReplyTo":"33f4f4d70810020726g71c6f39eq16585269fb268322@mail.gmail.com","subject":"Re: How to remove a commit object?","fromName":"Michael J Gruber","fromEmail":"git@drmicha.warpmail.net","sentAt":"2008-10-02T14:30:57Z","receivedAt":"2008-10-02T14:30:57Z","isPatch":false,"sender":{"key":"git@grubix.eu","avatar":"https://avatars.githubusercontent.com/u/233215?v=4"},"body":"Klas Lindberg venit, vidit, dixit 02.10.2008 16:26:\n> Repo size is a problem too, actually.\n> \n> A solution to both problems seemed to be to use git-filter-branch to\n> create a new repo by filtering out all the unwanted files. The\n> astonishing result was that, for the subdirectory I tried it on, 90%\n> or so of the commits on that subdirectory just disappeared. It didn't\n> look right at all. Although I can't say for sure exactly what I did\n> with filter-branch, I would appreciate some guidance for using it. It\n\nI don't know about others, but I would appreciate more info:\nDo you want to remove commits (as stated earlier) or files (as stated here)?\nWhat are the boundary conditions? Rewriting history seems to be OK now.\n\n> basically seemed to do exactly what I wanted (recreate the repo, minus\n> some explicit stuff, with history intact otherwise), except the result\n> looked crazy.\n\nThat may be due to the filter-branch incarnation, i.e. which refs did\nyou rewrite (--all or HEAD)?\n\nMichael\n"},{"id":"92172","messageId":"33f4f4d70810020752g4738fee3p4e99ab417d97106a@mail.gmail.com","threadId":"15584","inReplyTo":"48E4DB21.7020304@drmicha.warpmail.net","subject":"Re: How to remove a commit object?","fromName":"Klas Lindberg","fromEmail":"klas.lindberg@gmail.com","sentAt":"2008-10-02T14:52:57Z","receivedAt":"2008-10-02T14:52:57Z","isPatch":false,"sender":{"key":"klas.lindberg@gmail.com","avatar":null},"body":"What I really want is to remove files, but when filter-branch didn't\nseem to do what I wanted, I turned to the idea of rewriting single\ncommits to not include the files in question.\n\nThis is what I tried with filter-branch: gitk --all shows about 170\ncommits on directory D in the repo. Of these, maybe 10 don't lead to\nHEAD, but dangle off the main track. As a test, I decided to let\nfilter-branch create a new repo that only contained the contents of\nsubdirectory B. So I ran\n\n git-filter-branch --subdirectory-filter B -- --all\n\nand now the resulting repo has just 14 commits. This is clearly not\nwhat I wanted because a lot of the original history for subdirectory B\nis just missing.\n\nActually, in this particular case I get the exact same result with\n\n git-filter-branch --subdirectory-filter B HEAD\n\nBR / Klas\n\nOn Thu, Oct 2, 2008 at 4:30 PM, Michael J Gruber\n wrote:\n> Klas Lindberg venit, vidit, dixit 02.10.2008 16:26:\n>> Repo size is a problem too, actually.\n>>\n>> A solution to both problems seemed to be to use git-filter-branch to\n>> create a new repo by filtering out all the unwanted files. The\n>> astonishing result was that, for the subdirectory I tried it on, 90%\n>> or so of the commits on that subdirectory just disappeared. It didn't\n>> look right at all. Although I can't say for sure exactly what I did\n>> with filter-branch, I would appreciate some guidance for using it. It\n>\n> I don't know about others, but I would appreciate more info:\n> Do you want to remove commits (as stated earlier) or files (as stated here)?\n> What are the boundary conditions? Rewriting history seems to be OK now.\n>\n>> basically seemed to do exactly what I wanted (recreate the repo, minus\n>> some explicit stuff, with history intact otherwise), except the result\n>> looked crazy.\n>\n> That may be due to the filter-branch incarnation, i.e. which refs did\n> you rewrite (--all or HEAD)?\n>\n> Michael\n>\n"},{"id":"92175","messageId":"48E4E27E.7030308@viscovery.net","threadId":"15584","inReplyTo":"33f4f4d70810020726g71c6f39eq16585269fb268322@mail.gmail.com","subject":"Re: How to remove a commit object?","fromName":"Johannes Sixt","fromEmail":"j.sixt@viscovery.net","sentAt":"2008-10-02T15:02:22Z","receivedAt":"2008-10-02T15:02:22Z","isPatch":false,"sender":{"key":"j6t@kdbg.org","avatar":"https://avatars.githubusercontent.com/u/14810926?v=4"},"body":"Klas Lindberg schrieb:\n> A solution to both problems seemed to be to use git-filter-branch to\n> create a new repo by filtering out all the unwanted files. The\n> astonishing result was that, for the subdirectory I tried it on, 90%\n> or so of the commits on that subdirectory just disappeared. It didn't\n> look right at all. Although I can't say for sure exactly what I did\n> with filter-branch, I would appreciate some guidance for using it. It\n> basically seemed to do exactly what I wanted (recreate the repo, minus\n> some explicit stuff, with history intact otherwise), except the result\n> looked crazy.\n\nAnd your definition of 'crazy' is...?\n\nI assume that you used --subdirectory-filter. This has issues that will be\nfixed in 1.6.1. You need a current 'master' git (at least b805ef08).\n\n-- Hannes\n"},{"id":"92240","messageId":"33f4f4d70810030442l5042fdbfw18f97336c5a331cc@mail.gmail.com","threadId":"15584","inReplyTo":"48E4E27E.7030308@viscovery.net","subject":"Re: How to remove a commit object?","fromName":"Klas Lindberg","fromEmail":"klas.lindberg@gmail.com","sentAt":"2008-10-03T11:42:53Z","receivedAt":"2008-10-03T11:42:53Z","isPatch":false,"sender":{"key":"klas.lindberg@gmail.com","avatar":null},"body":"On Thu, Oct 2, 2008 at 5:02 PM, Johannes Sixt wrote:\n>> with filter-branch, I would appreciate some guidance for using it. It\n>> basically seemed to do exactly what I wanted (recreate the repo, minus\n>> some explicit stuff, with history intact otherwise), except the result\n>> looked crazy.\n>\n> And your definition of 'crazy' is...?\n\nRight... :-)\nCrazy == Obviously incorrect behaviour that I didn't analyze. Out of\n167 commits on subdirectory B, only 14 survived the filtering.\n\nI tried \"git filter-branch --tree-filter 'rm -rf ' HEAD\" instead, but I can't use that. The change history for\nall the non-B paths are still in the repo afterwards, and thus you can\neasily recreate any file outside subdirectory B.\n\nIs there some way to do what I need with git-filter-branch today, or\nmust I wait until 1.6.1 is released?\n\nBR / Klas\n\n\n> I assume that you used --subdirectory-filter. This has issues that will be\n> fixed in 1.6.1. You need a current 'master' git (at least b805ef08).\n>\n> -- Hannes\n>\n"},{"id":"92241","messageId":"48E60A02.7080608@viscovery.net","threadId":"15584","inReplyTo":"33f4f4d70810030442l5042fdbfw18f97336c5a331cc@mail.gmail.com","subject":"Re: How to remove a commit object?","fromName":"Johannes Sixt","fromEmail":"j.sixt@viscovery.net","sentAt":"2008-10-03T12:03:14Z","receivedAt":"2008-10-03T12:03:14Z","isPatch":false,"sender":{"key":"j6t@kdbg.org","avatar":"https://avatars.githubusercontent.com/u/14810926?v=4"},"body":"Klas Lindberg schrieb:\n> On Thu, Oct 2, 2008 at 5:02 PM, Johannes Sixt wrote:\n>> I assume that you used --subdirectory-filter. This has issues that will be\n>> fixed in 1.6.1. You need a current 'master' git (at least b805ef08).\n>\n> Is there some way to do what I need with git-filter-branch today, or\n> must I wait until 1.6.1 is released?\n\nYou can remove all occurences of the \"--full-history\" flag from your\n/usr/libexec/git-core/git-filter-branch script. This is sufficient for\nsome repositories because this triggers the bug less often. This means\nthat the resulting history may still be incorrect, but chances are higher\nthat it is correct.\n\nOther than that, you can just clone git.git and compile it yourself. It's\na simple matter of \"make prefix=$HOME/mytempgit install\".\n\n-- Hannes\n"}]}