Re: Update on SoC proposal: git-remote-svn

From: David Michael Barr <david.barr@cordelta.com>
Date: Apr 14, 2010, 12:52 UTC
Message-ID: <E7EBCCD4-418B-4573-A70A-B7E06D6C6D50@cordelta.com>
In-Reply-To: <F0CD5B83-2D28-47E1-A336-5C88E2803CBE@gmail.com>

Hi Steve,

Show 6 quoted lines

> In reading this I wondered how a svn dump of one of the
> repositories monitor would size.  If I were to check out the svn
> root of that repository, I would use well over 3TB of disk space
> to have that checked out, I filled my 750GB drive with about a
> third of it checked out.  About 256MB of code with thousands
> of tags and hundreds of branches.

I encountered this issue with my first attempt to validate the output of my dump conversion tool. My case wasn't as dire, 350GB would have sufficed but I was working in a 160GB partition. Checking out tags side by side is a sure way to fill your disk.

> It looks like svnadmin dump defaults to dumping all data.
> Fortunately it has a delta option, which looks like it would be
> needed to dump this repository I am speaking of without filling
> up many hard drives.

The svn dump format is not quite that silly, even without deltification it doesn't output blobs that are just an unaltered copy from a previous revision. Handling deltified dumps will greatly increase the complexity of the import process. Blob content would have be computed from existing blobs rather than simply passed through.

> This might also be helped if the dumps are chunked into ranges
> for many thousands of commits as well, this would keep the files
> more manageable

Being able to handle a dump stream reassembled from such piecewise dumps is an important feature which I haven't finished implementing yet.

> Just food for thought.

Thanks for the feed.

-- David Barr