Pruning objects from history?

I've imported the full history of a large project from Subversion using the latest git-svn. The resulting repo is huge, and I believe it's due in large part to a series of big tar.gz files that got checked into the Subversion repository by mistake early in the project's history. They were subsequently removed from svn, but of course git-svn grabs them and puts them in my local history.

Is there any way to excise those files? They are of no interest to us now -- they were data files for a third-party application we ended up not using -- and they're making git look bad in the disk usage department.

I believe this has been asked before in the context of removing copyrighted content from public repositories. However, I have a twist that may make it easier: nobody else has cloned this repository yet. I am free to rewrite history with no risk of messing up any downstream repositories, and I don't have to worry about propagating the deletions out to anyone. I just don't know how to do it (assuming it's doable at all.)

Thanks!

-Steve

Steven Grimm <koreth@midwinter.com> wrote:

Try cg-admin-rewritehistory (in Cogito)?

On Fri, Mar 30, 2007 at 07:01:27PM -0700, Steven Grimm wrote:

It's painful to rewrite history, since you end up needing to rewrite every single commit after the point where you've tampered with time to fix up the parent commit ID.

Are you planning on doing a one-shot import, or are you hoping to be able to do bidirectional gatewaying between svn and git? If you want to do the latter, rewriting history is going to be very painful if you want the bidirectional gateway to work afterwards.

If you just want to do a one-way import, it's probably going to be much easier to modify whatever importer you use to not import the big files in the first place.

						- Ted

On Sat, 31 Mar 2007, Theodore Tso wrote:

Well, if you don't ever need to actually rewrite blob objects, and many of the trees end up being the same, history rewriting is actually pretty cheap. If it mostly just has to rewrite the commits and a few trees, you end up having (even for big projects) just a few hundred thousand easy objects to rewrite. It's going to take a minute or two at most.

"git-convert-objects" has most of the logic, so some trivial added code to "convert" tree objects by removing certain entries should just do it. But yeah, it would involve some real changes. cg-admin-rewritehist should be able to do it already, although I suspect it would be slower (but for a one-shot thing, nobody probably cares, and changing convert-objects is probably going to take more time than just running the rewritehist scripts).

I'd actually like cg-admin-rewritehist to be merged into git. I think it's one of the few things that cogito does that native git doesn't do.

		Linus