# How to remove a commit object? 11 messages from 2008-09-18 to 2008-10-03. Participants: Steven Grimm, Michael J Gruber, Klas Lindberg, Jakub Narebski, Johannes Sixt. Thread: https://gitlist.dev/t/15584 ## Steven Grimm, 2008-09-18 23:41 Subject: How to remove a commit object? Message-ID: URL: https://gitlist.dev/e/D53D127D-97DD-417C-9109-ABD46BBD4E18%40midwinter.com ``` I maintain a shared repository a bunch of my coworkers push to for code reviews. It has accumulated a lot of packfiles, so I want to shrink it down a bit, but there's a problem: % git repack -A -d Counting objects: ... error: Could not read 125bf191b65189aaec7a6aa24ff26460d141d587 fatal: bad tree object 125bf191b65189aaec7a6aa24ff26460d141d587 "git fsck" confirms that the tree object is missing: % git fsck broken link from commit 1b2f0595bb4a6c2e17ca43a9cc41feec88c72a47 to tree 125bf191b65189aaec7a6aa24ff26460d141d587 ... missing tree 125bf191b65189aaec7a6aa24ff26460d141d587 This is a dangling commit, but that's fine; for this particular repository we actually *want* lots of dangling commits since they represent the history of people's code review requests. (Hence me running git-repack with -A instead of -a.) Given that it's dangling, it seems like it'd be safe to just remove entirely (we lose that little bit of code-review history but we've lost it already anyway with the tree object missing). But I'm not sure how to do it. Is it possible to delete a commit object, and if so, how? I don't know how the corruption happened in the first place. There was a short time at one point where the permissions on the object directories were inconsistent, so it's possible someone pushed during that period and managed to create the commit object file in .git/ objects but didn't have permission to create the tree object. That's just speculation on my part, though. This is the only corrupt object in the repository according to git-fsck, so at this point I just want to know how to get rid of it so I can do the repack. Thanks! -Steve ``` ## Michael J Gruber, 2008-09-19 09:16 Subject: Re: How to remove a commit object? Message-ID: <48D36DF4.3030607@drmicha.warpmail.net> URL: https://gitlist.dev/e/48D36DF4.3030607%40drmicha.warpmail.net In-Reply-To: ``` Steven Grimm venit, vidit, dixit 19.09.2008 01:41: > I maintain a shared repository a bunch of my coworkers push to for > code reviews. It has accumulated a lot of packfiles, so I want to > shrink it down a bit, but there's a problem: > > % git repack -A -d > Counting objects: ... > error: Could not read 125bf191b65189aaec7a6aa24ff26460d141d587 > fatal: bad tree object 125bf191b65189aaec7a6aa24ff26460d141d587 > > "git fsck" confirms that the tree object is missing: > > % git fsck > broken link from commit 1b2f0595bb4a6c2e17ca43a9cc41feec88c72a47 > to tree 125bf191b65189aaec7a6aa24ff26460d141d587 > ... > missing tree 125bf191b65189aaec7a6aa24ff26460d141d587 > > This is a dangling commit, but that's fine; for this particular > repository we actually *want* lots of dangling commits since they > represent the history of people's code review requests. (Hence me > running git-repack with -A instead of -a.) > > Given that it's dangling, it seems like it'd be safe to just remove > entirely (we lose that little bit of code-review history but we've > lost it already anyway with the tree object missing). But I'm not sure > how to do it. Is it possible to delete a commit object, and if so, how? > > I don't know how the corruption happened in the first place. There was > a short time at one point where the permissions on the object > directories were inconsistent, so it's possible someone pushed during > that period and managed to create the commit object file in .git/ > objects but didn't have permission to create the tree object. That's > just speculation on my part, though. This is the only corrupt object > in the repository according to git-fsck, so at this point I just want > to know how to get rid of it so I can do the repack. git prune should delete dangling commits. Is that commit already in a pack? Then the -f option to repack may help. Michael ``` ## Klas Lindberg, 2008-10-02 13:36 Subject: Re: How to remove a commit object? Message-ID: <33f4f4d70810020636g77f4180cm7aa6b7de07ae5beb@mail.gmail.com> URL: https://gitlist.dev/e/33f4f4d70810020636g77f4180cm7aa6b7de07ae5beb%40mail.gmail.com In-Reply-To: <48D36DF4.3030607@drmicha.warpmail.net> ``` This doesn't seem to work for me. I will soon be in a situation where I need to selectively delete commits in such a way that they become completely irrecoverable. I.e. it is not enough to revert a commit. The *original* commit must be removed. And of course, the repo history is too complex to allow for rebasing followed by garbage collection or something like that. The reason is that we consider opening a repository to external participants, but some commits contain stuff that we'd really rather not show to anyone else. Making the repository public without loosing history would then force us to either 1. Recreate every commit in a new repo, sans the offending commits. Seems like hard work. 2. ? Would it be feasible to write a tool that can selectively replace a specific commit in the commit DAG, or would that automatically invalidate every SHA key for every commit that follows the replaced original? BR / Klas On Fri, Sep 19, 2008 at 11:16 AM, Michael J Gruber wrote: > Steven Grimm venit, vidit, dixit 19.09.2008 01:41: >> I maintain a shared repository a bunch of my coworkers push to for >> code reviews. It has accumulated a lot of packfiles, so I want to >> shrink it down a bit, but there's a problem: >> >> % git repack -A -d >> Counting objects: ... >> error: Could not read 125bf191b65189aaec7a6aa24ff26460d141d587 >> fatal: bad tree object 125bf191b65189aaec7a6aa24ff26460d141d587 >> >> "git fsck" confirms that the tree object is missing: >> >> % git fsck >> broken link from commit 1b2f0595bb4a6c2e17ca43a9cc41feec88c72a47 >> to tree 125bf191b65189aaec7a6aa24ff26460d141d587 >> ... >> missing tree 125bf191b65189aaec7a6aa24ff26460d141d587 >> >> This is a dangling commit, but that's fine; for this particular >> repository we actually *want* lots of dangling commits since they >> represent the history of people's code review requests. (Hence me >> running git-repack with -A instead of -a.) >> >> Given that it's dangling, it seems like it'd be safe to just remove >> entirely (we lose that little bit of code-review history but we've >> lost it already anyway with the tree object missing). But I'm not sure >> how to do it. Is it possible to delete a commit object, and if so, how? >> >> I don't know how the corruption happened in the first place. There was >> a short time at one point where the permissions on the object >> directories were inconsistent, so it's possible someone pushed during >> that period and managed to create the commit object file in .git/ >> objects but didn't have permission to create the tree object. That's >> just speculation on my part, though. This is the only corrupt object >> in the repository according to git-fsck, so at this point I just want >> to know how to get rid of it so I can do the repack. > > git prune should delete dangling commits. Is that commit already in a > pack? Then the -f option to repack may help. > > Michael > -- > To unsubscribe from this list: send the line "unsubscribe git" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > ``` ## Michael J Gruber, 2008-10-02 14:00 Subject: Re: How to remove a commit object? Message-ID: <48E4D3E9.8020906@drmicha.warpmail.net> URL: https://gitlist.dev/e/48E4D3E9.8020906%40drmicha.warpmail.net In-Reply-To: <33f4f4d70810020636g77f4180cm7aa6b7de07ae5beb@mail.gmail.com> ``` Klas Lindberg venit, vidit, dixit 02.10.2008 15:36: > This doesn't seem to work for me. I will soon be in a situation where > I need to selectively delete commits in such a way that they become > completely irrecoverable. I.e. it is not enough to revert a commit. > The *original* commit must be removed. And of course, the repo history > is too complex to allow for rebasing followed by garbage collection or > something like that. > > The reason is that we consider opening a repository to external > participants, but some commits contain stuff that we'd really rather > not show to anyone else. Making the repository public without loosing > history would then force us to either > > 1. Recreate every commit in a new repo, sans the offending commits. > Seems like hard work. > 2. ? > > Would it be feasible to write a tool that can selectively replace a > specific commit in the commit DAG, or would that automatically > invalidate every SHA key for every commit that follows the replaced > original? Yes, on the or part: If you change a commit then all commits "after" that one (in terms of DAG connectedness) will need to be changed: each contains a "backpointer" (to the parent commit(s)) which is changed. I'm a bit confused: You rule out rebasing but don't mind recreating a new repo. So repo size is not a problem, is it? Michael ``` ## Jakub Narebski, 2008-10-02 14:02 Subject: Re: How to remove a commit object? Message-ID: URL: https://gitlist.dev/e/m3ljx7qemk.fsf%40localhost.localdomain In-Reply-To: <33f4f4d70810020636g77f4180cm7aa6b7de07ae5beb@mail.gmail.com> ``` "Klas Lindberg" writes: > This doesn't seem to work for me. I will soon be in a situation where > I need to selectively delete commits in such a way that they become > completely irrecoverable. I.e. it is not enough to revert a commit. > The *original* commit must be removed. And of course, the repo history > is too complex to allow for rebasing followed by garbage collection or > something like that. [...] > Would it be feasible to write a tool that can selectively replace a > specific commit in the commit DAG, or would that automatically > invalidate every SHA key for every commit that follows the replaced > original? It would invalidate SHA1 for every commit after first rewritten. There are two tools which you can use to rewrite large parts of history automatically: git-filter-branch, and git-fast-export + git-fast-import. HTH -- Jakub Narebski Poland ShadeHawk on #git ``` ## Klas Lindberg, 2008-10-02 14:26 Subject: Re: How to remove a commit object? Message-ID: <33f4f4d70810020726g71c6f39eq16585269fb268322@mail.gmail.com> URL: https://gitlist.dev/e/33f4f4d70810020726g71c6f39eq16585269fb268322%40mail.gmail.com In-Reply-To: ``` Repo size is a problem too, actually. A solution to both problems seemed to be to use git-filter-branch to create a new repo by filtering out all the unwanted files. The astonishing result was that, for the subdirectory I tried it on, 90% or so of the commits on that subdirectory just disappeared. It didn't look right at all. Although I can't say for sure exactly what I did with filter-branch, I would appreciate some guidance for using it. It basically seemed to do exactly what I wanted (recreate the repo, minus some explicit stuff, with history intact otherwise), except the result looked crazy. /Klas On Thu, Oct 2, 2008 at 4:02 PM, Jakub Narebski wrote: > "Klas Lindberg" writes: > >> This doesn't seem to work for me. I will soon be in a situation where >> I need to selectively delete commits in such a way that they become >> completely irrecoverable. I.e. it is not enough to revert a commit. >> The *original* commit must be removed. And of course, the repo history >> is too complex to allow for rebasing followed by garbage collection or >> something like that. > [...] > >> Would it be feasible to write a tool that can selectively replace a >> specific commit in the commit DAG, or would that automatically >> invalidate every SHA key for every commit that follows the replaced >> original? > > It would invalidate SHA1 for every commit after first rewritten. > There are two tools which you can use to rewrite large parts of > history automatically: git-filter-branch, and git-fast-export + > git-fast-import. > > HTH > -- > Jakub Narebski > Poland > ShadeHawk on #git > ``` ## Michael J Gruber, 2008-10-02 14:30 Subject: Re: How to remove a commit object? Message-ID: <48E4DB21.7020304@drmicha.warpmail.net> URL: https://gitlist.dev/e/48E4DB21.7020304%40drmicha.warpmail.net In-Reply-To: <33f4f4d70810020726g71c6f39eq16585269fb268322@mail.gmail.com> ``` Klas Lindberg venit, vidit, dixit 02.10.2008 16:26: > Repo size is a problem too, actually. > > A solution to both problems seemed to be to use git-filter-branch to > create a new repo by filtering out all the unwanted files. The > astonishing result was that, for the subdirectory I tried it on, 90% > or so of the commits on that subdirectory just disappeared. It didn't > look right at all. Although I can't say for sure exactly what I did > with filter-branch, I would appreciate some guidance for using it. It I don't know about others, but I would appreciate more info: Do you want to remove commits (as stated earlier) or files (as stated here)? What are the boundary conditions? Rewriting history seems to be OK now. > basically seemed to do exactly what I wanted (recreate the repo, minus > some explicit stuff, with history intact otherwise), except the result > looked crazy. That may be due to the filter-branch incarnation, i.e. which refs did you rewrite (--all or HEAD)? Michael ``` ## Klas Lindberg, 2008-10-02 14:52 Subject: Re: How to remove a commit object? Message-ID: <33f4f4d70810020752g4738fee3p4e99ab417d97106a@mail.gmail.com> URL: https://gitlist.dev/e/33f4f4d70810020752g4738fee3p4e99ab417d97106a%40mail.gmail.com In-Reply-To: <48E4DB21.7020304@drmicha.warpmail.net> ``` What I really want is to remove files, but when filter-branch didn't seem to do what I wanted, I turned to the idea of rewriting single commits to not include the files in question. This is what I tried with filter-branch: gitk --all shows about 170 commits on directory D in the repo. Of these, maybe 10 don't lead to HEAD, but dangle off the main track. As a test, I decided to let filter-branch create a new repo that only contained the contents of subdirectory B. So I ran git-filter-branch --subdirectory-filter B -- --all and now the resulting repo has just 14 commits. This is clearly not what I wanted because a lot of the original history for subdirectory B is just missing. Actually, in this particular case I get the exact same result with git-filter-branch --subdirectory-filter B HEAD BR / Klas On Thu, Oct 2, 2008 at 4:30 PM, Michael J Gruber wrote: > Klas Lindberg venit, vidit, dixit 02.10.2008 16:26: >> Repo size is a problem too, actually. >> >> A solution to both problems seemed to be to use git-filter-branch to >> create a new repo by filtering out all the unwanted files. The >> astonishing result was that, for the subdirectory I tried it on, 90% >> or so of the commits on that subdirectory just disappeared. It didn't >> look right at all. Although I can't say for sure exactly what I did >> with filter-branch, I would appreciate some guidance for using it. It > > I don't know about others, but I would appreciate more info: > Do you want to remove commits (as stated earlier) or files (as stated here)? > What are the boundary conditions? Rewriting history seems to be OK now. > >> basically seemed to do exactly what I wanted (recreate the repo, minus >> some explicit stuff, with history intact otherwise), except the result >> looked crazy. > > That may be due to the filter-branch incarnation, i.e. which refs did > you rewrite (--all or HEAD)? > > Michael > ``` ## Johannes Sixt, 2008-10-02 15:02 Subject: Re: How to remove a commit object? Message-ID: <48E4E27E.7030308@viscovery.net> URL: https://gitlist.dev/e/48E4E27E.7030308%40viscovery.net In-Reply-To: <33f4f4d70810020726g71c6f39eq16585269fb268322@mail.gmail.com> ``` Klas Lindberg schrieb: > A solution to both problems seemed to be to use git-filter-branch to > create a new repo by filtering out all the unwanted files. The > astonishing result was that, for the subdirectory I tried it on, 90% > or so of the commits on that subdirectory just disappeared. It didn't > look right at all. Although I can't say for sure exactly what I did > with filter-branch, I would appreciate some guidance for using it. It > basically seemed to do exactly what I wanted (recreate the repo, minus > some explicit stuff, with history intact otherwise), except the result > looked crazy. And your definition of 'crazy' is...? I assume that you used --subdirectory-filter. This has issues that will be fixed in 1.6.1. You need a current 'master' git (at least b805ef08). -- Hannes ``` ## Klas Lindberg, 2008-10-03 11:42 Subject: Re: How to remove a commit object? Message-ID: <33f4f4d70810030442l5042fdbfw18f97336c5a331cc@mail.gmail.com> URL: https://gitlist.dev/e/33f4f4d70810030442l5042fdbfw18f97336c5a331cc%40mail.gmail.com In-Reply-To: <48E4E27E.7030308@viscovery.net> ``` On Thu, Oct 2, 2008 at 5:02 PM, Johannes Sixt wrote: >> with filter-branch, I would appreciate some guidance for using it. It >> basically seemed to do exactly what I wanted (recreate the repo, minus >> some explicit stuff, with history intact otherwise), except the result >> looked crazy. > > And your definition of 'crazy' is...? Right... :-) Crazy == Obviously incorrect behaviour that I didn't analyze. Out of 167 commits on subdirectory B, only 14 survived the filtering. I tried "git filter-branch --tree-filter 'rm -rf ' HEAD" instead, but I can't use that. The change history for all the non-B paths are still in the repo afterwards, and thus you can easily recreate any file outside subdirectory B. Is there some way to do what I need with git-filter-branch today, or must I wait until 1.6.1 is released? BR / Klas > I assume that you used --subdirectory-filter. This has issues that will be > fixed in 1.6.1. You need a current 'master' git (at least b805ef08). > > -- Hannes > ``` ## Johannes Sixt, 2008-10-03 12:03 Subject: Re: How to remove a commit object? Message-ID: <48E60A02.7080608@viscovery.net> URL: https://gitlist.dev/e/48E60A02.7080608%40viscovery.net In-Reply-To: <33f4f4d70810030442l5042fdbfw18f97336c5a331cc@mail.gmail.com> ``` Klas Lindberg schrieb: > On Thu, Oct 2, 2008 at 5:02 PM, Johannes Sixt wrote: >> I assume that you used --subdirectory-filter. This has issues that will be >> fixed in 1.6.1. You need a current 'master' git (at least b805ef08). > > Is there some way to do what I need with git-filter-branch today, or > must I wait until 1.6.1 is released? You can remove all occurences of the "--full-history" flag from your /usr/libexec/git-core/git-filter-branch script. This is sufficient for some repositories because this triggers the bug less often. This means that the resulting history may still be incorrect, but chances are higher that it is correct. Other than that, you can just clone git.git and compile it yourself. It's a simple matter of "make prefix=$HOME/mytempgit install". -- Hannes ```