Re: Sparse clones (Was: Re: [PATCH 1/2] upload-pack: support subtree packing)

From: Elijah Newren <newren@gmail.com>
Date: Jul 28, 2010, 03:59 UTC
Message-ID: <AANLkTikoOL0jYi4kD0L1uLU=UCmG9p0Rc3VLOU-djQnY@mail.gmail.com>
In-Reply-To: <AANLkTikvhi-Bm-eV9ZfjoZdugRpiCwg8n-TQ4oOVvC=G@mail.gmail.com>

On Tue, Jul 27, 2010 at 9:36 PM, Nguyen Thai Ngoc Duy <pclouds@gmail.com> wrote:

> 2010/7/28 Elijah Newren <newren@gmail.com>:
>> 1) A user controls sparseness by passing rev-list arguments to clone.

<snip>

>> For example,
>>  $ git clone ssh://repo.git dst -- Documentation/
>
> Does pathspec is supported to in addition to prefix?

Basically, whatever git log or git rev-list accepts. I think I saw some other discussion about making those adopt some of the code/capability of git grep, which would automatically benefit sparse clones. But until then, no, because I need to be able to take these arguments and automatically pass them on to log, rev-list, etc.

> So you basically kill off shallow clone too, with "master~6..master".

Yes, that was part of the plan...extend the capabilities of shallow clones in two ways: allowing the user to specify a cutoff via a revision identifier as well as a number of commits, and allow people to clone (and fetch-from/push-to) other "shallow" clones.

> I wonder what happens if user does "git clone ... master~6..master~3"?

Currently, that'd break -- just like it similarly does for fast-export (see t/t9350-fast-export.sh, 'no exact-ref revisions included'). I had been thinking of trying to get that fixed for both cases by making it result in a "master" branch that is "three commits behind" what you clone/fast-export from. You'd have to look for and disallow other special cases like "git fast-export ... master^1 master^2" or "git clone ... :/searchstring".

I'm not sure how this interacts with Avery's suggestion to just ignore branch/tag limiting.

> Revision walking is not the only gate to access objects. Others like
> diff machinery needs also be taught about rev-list limits.

Right, good point. Are there others than the diff machinery (and the fsck special case) that you know of?

> What information would you send to the server to request new pack in
> sparse clone? Currently we send all commit tips. rev-list has a notion
> to subtract commit trees. I don't know if it can "add" or "subtract"
> tree prefix though.

When "densifying" a sparse clone, I was (initially at least) just going to treat it like an initial clone and re-download _everything_ (even if sparsifying rather than densifying). I assumed it'd be rare to want to do such an operation, but yeah, in the future someone might want a smarter way to handle it.