Lazy clone ideas

Jakub Narebski· Jun 10, 2006, 08:58 UTC · lore

I've started new thread for lazy clone ideas, splitting from "Figured out how to get Mozilla into git"

Rogan Dawes wrote:

Show 9 quoted lines

> Here's an idea. How about separating trees and commits from the actual 
> blobs (e.g. in separate packs)? My reasoning is that the commits and 
> trees should only be a small portion of the overall repository size, and 
> should not be that expensive to transfer. (Of course, this is only a 
> guess, and needs some numbers to back it up.)
> 
> So, a shallow clone would receive all of the tree objects, and all of 
> the commit objects, and could then request a pack containing the blobs 
> represented by the current HEAD.

That would be _lazy_ clone (with on-demand pack downloading from "master" full history repository), rather than shallow clone.

I had an idea for having all the commit objects (without all the tree
objects) below the soft-grafts line (beyond the line we cut-off full
history and start being lazy).

Show 15 quoted lines

> In this way, the user has a history that will show all of the commit 
> messages, and would be able to see _which_ files have changed over time 
> e.g. gitk would still work - except for the actual file level diff, "git 
> log" should also still work, etc
> 
> This would also enable other optimisations.
> 
> For example, documentation people would only need to get the objects 
> under the doc/ tree, and would not need to actually check out the 
> source. Git could detect any actual changes by checking whether it has 
> the previous blob in its local repository, and whether the file exists 
> locally. Creating a patch would obviously require that the person checks 
> out the previous version, but one could theoretically commit a new blob 
> to a repo without having the previous one (not saying that this would be 
> a good idea, of course)

Something akin to CVS's modules, or rather to how CVS modules can be abused? Something called, I think, partial checkout?

This is a separate idea and I think worth implementing even for full repository.

> This would probably require Eric Biederman's "direct access to blob" 
> patches, I guess, in order to be feasible.

And it would need place to store URI from where to doenload objects on-demand: perhaps 'remote alternatives'?

-- 
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git

Elrond· Jun 16, 2006, 22:59 UTC · re: Jakub Narebski · lore

Re: Lazy clone ideas

Jakub Narebski <jnareb <at> gmail.com> writes:

> 
> I've started new thread for lazy clone ideas,
> splitting from "Figured out how to get Mozilla into git"

[...]

I like the lazy clone idea, I think, I said that earlier.

> > This would probably require Eric Biederman's "direct access to blob" 
> > patches, I guess, in order to be feasible.

Are those patches allowing the git: protocol to request a list of objects directly? (Like my "remote git-cat-file" request?)

What's the status of the patch?

> And it would need place to store URI from where to doenload objects
> on-demand: perhaps 'remote alternatives'?

Yep, that would be the next step. Having direct access to blobs would be needed first though.

    Elrond