Re: [PATCH] RFC: git lazy clone proof-of-concept

From: Mike Hommey <mh@glandium.org>
Date: Feb 8, 2008, 18:49 UTC
Message-ID: <20080208184902.GA19404@glandium.org>
In-Reply-To: <200802081828.43849.kendy@suse.cz>

On Fri, Feb 08, 2008 at 06:28:43PM +0100, Jan Holesovsky wrote:

Show 27 quoted lines

> Hi,
> 
> This is my attempt to implement the 'lazy clone' I've read about a bit in the
> git mailing list archive, but did not see implemented anywhere - the clone
> that fetches a minimal amount of data with the possibility to download the
> rest later (transparently!) when necessary.  I am sorry to send it as a huge
> patch, not as a series of patches, but as I don't know if I chose a way that is
> acceptable for you [I'm new to the git code ;-)], I'd like to hear some
> feedback first, and then I'll split it into smaller pieces for easier
> integration - if OK.
> 
> Background:
> 
> Currently we are evaluating the usage of git for OpenOffice.org as one of the
> candidates (SVN is the other one), see
> 
>   http://wiki.services.openoffice.org/wiki/SCM_Migration
> 
> I've provided a git import of OOo with the entire history; the problem is that
> the pack has 2.5G, so it's not too convenient to download for casual
> developers that just want to try it.  Shallow clone is not a possibility - we
> don't get patches through mailing lists, so we need the pull/push, and also
> thanks to the OOo development cycle, we have too many living heads which
> causes the shallow clone to download about 1.5G even with --depth 1.  Lazy
> clone sounded like the right idea to me.  With this proof-of-concept
> implementation, just about 550M from the 2.5G is downloaded, which is still
> about twice as much in comparison with downloading a tarball, but bearable.

<snip>

There are 2 things, here:
- Probably, you can make your pack smaller with proper window sizing.
Try taking a look at the "Git and GCC" that crossed borders between
the gcc and the git mailing lists.
- There are tricks to do roughly what you want without modifying git.
For example, you can prepare several "shared" clones of your repo (git
clone -s) and leave in each only a few branches. Cloning from these will
only pull the needed data.

Mike