Lazy clone ideas

From: Jakub Narebski <jnareb@gmail.com>
Date: Jun 10, 2006, 08:58 UTC
Message-ID: <e6e1jm$tes$1@sea.gmane.org>

I've started new thread for lazy clone ideas, splitting from "Figured out how to get Mozilla into git"

Rogan Dawes wrote:

Show 9 quoted lines

> Here's an idea. How about separating trees and commits from the actual 
> blobs (e.g. in separate packs)? My reasoning is that the commits and 
> trees should only be a small portion of the overall repository size, and 
> should not be that expensive to transfer. (Of course, this is only a 
> guess, and needs some numbers to back it up.)
> 
> So, a shallow clone would receive all of the tree objects, and all of 
> the commit objects, and could then request a pack containing the blobs 
> represented by the current HEAD.

That would be _lazy_ clone (with on-demand pack downloading from "master" full history repository), rather than shallow clone.

I had an idea for having all the commit objects (without all the tree
objects) below the soft-grafts line (beyond the line we cut-off full
history and start being lazy).

Show 15 quoted lines

> In this way, the user has a history that will show all of the commit 
> messages, and would be able to see _which_ files have changed over time 
> e.g. gitk would still work - except for the actual file level diff, "git 
> log" should also still work, etc
> 
> This would also enable other optimisations.
> 
> For example, documentation people would only need to get the objects 
> under the doc/ tree, and would not need to actually check out the 
> source. Git could detect any actual changes by checking whether it has 
> the previous blob in its local repository, and whether the file exists 
> locally. Creating a patch would obviously require that the person checks 
> out the previous version, but one could theoretically commit a new blob 
> to a repo without having the previous one (not saying that this would be 
> a good idea, of course)

Something akin to CVS's modules, or rather to how CVS modules can be abused? Something called, I think, partial checkout?

This is a separate idea and I think worth implementing even for full repository.

> This would probably require Eric Biederman's "direct access to blob" 
> patches, I guess, in order to be feasible.

And it would need place to store URI from where to doenload objects on-demand: perhaps 'remote alternatives'?

-- 
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git