Re: RFC: Resumable clone based on hybrid "smart" and "dumb" HTTP

On Wed, Feb 10, 2016 at 10:59 AM, Shawn Pearce <spearce@spearce.org> wrote:

Several of us at $DAY_JOB talked about this more today and thought a variation makes more sense:

1. Clients attempting clone ask for /info/refs?service=git-upload-pack
like they do today.

2. Servers that support resumable clone include a "resumable"
capability in the advertisement.

3. Updated clients on clone request GET /info/refs?service=git-resumable-clone.

4. The server may return a 302 Redirect to its current "mostly whole"
pack file. This can be more flexible than "refs/heads/*", it just
needs to be a mostly complete pack file that contains a complete graph
from any arbitrary roots.

5. Clients fetch the file using standard HTTP GET, possibly with
byte-ranges to resume.

6. Once stored and indexed with .idx, clients run `git fsck
--lost-found` to discover the roots of the pack it downloaded. These
are saved as temporary references.

7. Client runs incremental fetch, and then deletes the temporary
references from 6.

An advantage to this process is its much more flexible for the server. There is no additional pack-*.info file required. GC can organize packs anyway it wants, etc.

To make step 4 really resume well, clients may need to save the first Location header it gets back from /info/refs?service=git-resumable-clone and use that on resume. Servers are likely to embed the pack SHA-1 in the Location header, and the client wants to use this on subsequent GET attempts to abort early if the server has deleted the pack the client is trying to obtain.