Re: New Feature wanted: Is it possible to let git clone continue last break point?

From: Clemens Buchacher <drizzd@aon.at>
Date: Nov 4, 2011, 08:56 UTC
Message-ID: <20111104085633.GA13924@ecki>
In-Reply-To: <CAJo=hJt2kU10r5rq23qgimtW8dmzu-N92vjO_hNBbVVsKSpDmg@mail.gmail.com>

On Wed, Nov 02, 2011 at 09:19:03PM -0700, Shawn Pearce wrote:

> 
> [...] But they can use sendfile() on the server they have and get
> a lot of improvement in clone speed due to lower system load,
> plus resumable clone for the relatively stable history part.

Setting aside the system load issue for now, couldn't we simply do the following?

1. Figure out HAVE's and WANT's [1], based on which an ad-hoc pack
   will be made and sent to the client.
2. Cache the information on disk (not the pack but the information
   to re-create it), and give the client a 'ticket number' which
   corresponds to that ad-hoc pack.
3. Start downloading the packfile

When the connection drops, we can resume like this:

1. Send the previously received 'ticket number', and the amount of
   previously received data.
2. Re-generate the pack from the HAVE's and WANT's cached under
   'ticket number'. (This may fail if the repo state has changed
   such that previously accessible refs are now inaccessible.)
3. Resume download of that pack.

The upside of this approach is that it would work automatically, without any manual setup by the server admin. All the previously discussed ideas skip the step where we figure out the HAVE's and WANT's. And to me that implies that we manually prepare a packfile somewhere on disk, which contains what the user usually WANT's and is allowed to have (think per-branch access control). Even if we disregard access control, wouldn't that at least require the server to create a "clean" pack which does not contain any objects from the reflog?

The whole mirror thing could be pursued independently of the resume capability, and if each git repo is capable of resuming the mirrors can be plain git clones as well.

Just my 2 cents, Clemens