{"thread":{"id":"7486","subject":"Pruning objects from history?","startedAt":"2007-03-31T02:01:27Z","lastAt":"2007-03-31T16:18:32Z","messageCount":4,"participants":["Steven Grimm","Shawn O. Pearce","Theodore Tso","Linus Torvalds"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"38405","messageId":"460DC0F7.1070607@midwinter.com","threadId":"7486","inReplyTo":null,"subject":"Pruning objects from history?","fromName":"Steven Grimm","fromEmail":"koreth@midwinter.com","sentAt":"2007-03-31T02:01:27Z","receivedAt":"2007-03-31T02:01:27Z","isPatch":false,"sender":{"key":"koreth@midwinter.com","avatar":"https://gravatar.com/avatar/71b4d2e8b62f168bdc9e9205341159e3567003b4f9e2127c617c5fa0a1f5bad2?d=mp&s=160"},"body":"I've imported the full history of a large project from Subversion using \nthe latest git-svn. The resulting repo is huge, and I believe it's due \nin large part to a series of big tar.gz files that got checked into the \nSubversion repository by mistake early in the project's history. They \nwere subsequently removed from svn, but of course git-svn grabs them and \nputs them in my local history.\n\nIs there any way to excise those files? They are of no interest to us \nnow -- they were data files for a third-party application we ended up \nnot using -- and they're making git look bad in the disk usage department.\n\nI believe this has been asked before in the context of removing \ncopyrighted content from public repositories. However, I have a twist \nthat may make it easier: nobody else has cloned this repository yet. I \nam free to rewrite history with no risk of messing up any downstream \nrepositories, and I don't have to worry about propagating the deletions \nout to anyone. I just don't know how to do it (assuming it's doable at all.)\n\nThanks!\n\n-Steve\n"},{"id":"38407","messageId":"20070331020806.GB15922@spearce.org","threadId":"7486","inReplyTo":"460DC0F7.1070607@midwinter.com","subject":"Re: Pruning objects from history?","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2007-03-31T02:08:06Z","receivedAt":"2007-03-31T02:08:06Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Steven Grimm <koreth@midwinter.com> wrote:\n> Is there any way to excise those files? They are of no interest to us \n> now -- they were data files for a third-party application we ended up \n> not using -- and they're making git look bad in the disk usage department.\n\nTry cg-admin-rewritehistory (in Cogito)?\n\n-- \nShawn.\n"},{"id":"38433","messageId":"20070331131134.GC25539@thunk.org","threadId":"7486","inReplyTo":"460DC0F7.1070607@midwinter.com","subject":"Re: Pruning objects from history?","fromName":"Theodore Tso","fromEmail":"tytso@mit.edu","sentAt":"2007-03-31T13:11:34Z","receivedAt":"2007-03-31T13:11:34Z","isPatch":false,"sender":{"key":"tytso@mit.edu","avatar":"https://avatars.githubusercontent.com/u/51416?v=4"},"body":"On Fri, Mar 30, 2007 at 07:01:27PM -0700, Steven Grimm wrote:\n> I've imported the full history of a large project from Subversion using \n> the latest git-svn. The resulting repo is huge, and I believe it's due \n> in large part to a series of big tar.gz files that got checked into the \n> Subversion repository by mistake early in the project's history. They \n> were subsequently removed from svn, but of course git-svn grabs them and \n> puts them in my local history.\n> \n> Is there any way to excise those files? They are of no interest to us \n> now -- they were data files for a third-party application we ended up \n> not using -- and they're making git look bad in the disk usage department.\n> \n> I believe this has been asked before in the context of removing \n> copyrighted content from public repositories. However, I have a twist \n> that may make it easier: nobody else has cloned this repository yet. I \n> am free to rewrite history with no risk of messing up any downstream \n> repositories, and I don't have to worry about propagating the deletions \n> out to anyone. I just don't know how to do it (assuming it's doable at all.)\n\nIt's painful to rewrite history, since you end up needing to rewrite\nevery single commit after the point where you've tampered with time to\nfix up the parent commit ID.\n\nAre you planning on doing a one-shot import, or are you hoping to be\nable to do bidirectional gatewaying between svn and git?  If you want\nto do the latter, rewriting history is going to be very painful if you\nwant the bidirectional gateway to work afterwards.\n\nIf you just want to do a one-way import, it's probably going to be\nmuch easier to modify whatever importer you use to not import the big\nfiles in the first place.\n\n\t\t\t\t\t\t- Ted\n"},{"id":"38441","messageId":"Pine.LNX.4.64.0703310913350.6730@woody.linux-foundation.org","threadId":"7486","inReplyTo":"20070331131134.GC25539@thunk.org","subject":"Re: Pruning objects from history?","fromName":"Linus Torvalds","fromEmail":"torvalds@linux-foundation.org","sentAt":"2007-03-31T16:18:32Z","receivedAt":"2007-03-31T16:18:32Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Sat, 31 Mar 2007, Theodore Tso wrote:\n> \n> It's painful to rewrite history, since you end up needing to rewrite\n> every single commit after the point where you've tampered with time to\n> fix up the parent commit ID.\n\nWell, if you don't ever need to actually rewrite blob objects, and many of \nthe trees end up being the same, history rewriting is actually pretty \ncheap. If it mostly just has to rewrite the commits and a few trees, you \nend up having (even for big projects) just a few hundred thousand easy \nobjects to rewrite. It's going to take a minute or two at most.\n\n\"git-convert-objects\" has most of the logic, so some trivial added code to \n\"convert\" tree objects by removing certain entries should just do it. But \nyeah, it would involve some real changes. cg-admin-rewritehist should be \nable to do it already, although I suspect it would be slower (but for a \none-shot thing, nobody probably cares, and changing convert-objects is \nprobably going to take more time than just running the rewritehist \nscripts).\n\n> If you just want to do a one-way import, it's probably going to be\n> much easier to modify whatever importer you use to not import the big\n> files in the first place.\n\nI'd actually like cg-admin-rewritehist to be merged into git. I think it's \none of the few things that cogito does that native git doesn't do.\n\n\t\tLinus\n"}]}