Re: Multi-threaded 'git clone'

From: Junio C Hamano <gitster@pobox.com>
Date: Feb 18, 2015, 03:14 UTC
Message-ID: <CAPc5daWK-GbjL=YEyW55bcBELoq05dy4umh11upG=YxKQuFMpQ@mail.gmail.com>
In-Reply-To: <xmqqegpoxgf3.fsf@gitster.dls.corp.google.com>

On Tue, Feb 17, 2015 at 3:32 PM, Junio C Hamano <gitster@pobox.com> wrote:

A few typofixes and clarifications.

> *4* The scheme in *3* can be extended to bring the fetcher
>     step-wise.  If the server's state was X when the fetcher last

"bring the fetcher up-to-date step-wise", or "update the fetcher step-wise".

Show 22 quoted lines

>     contacted it, and since then the server received multiple pushes
>     and has two snapshots of states, Y and Z, then the exchange may
>     go like this:
>
>     fetcher: I am interested in refs/heads/* and refs/tags/* and I
>              have your state X.
>
>     server:  Here is the incremental difference to the refs and the
>              end result should hash to Y.  Here comes the pack data
>              to bring you up to date.
>
>     fetcher: (after receiving, unpacking and updating the
>              remote-tracking refs) Thanks.  Do you have more?
>
>     server:  Yes, here is the incremental difference to the refs and the
>              end result should hash to Z.  Here comes the pack data
>              to bring you up to date.
>
>     fetcher: (after receiving, unpacking and updating the
>              remote-tracking refs) Thanks.  Do you have more?
>
>     server:  No, you are now fully up to date with me.  Bye.

The initial part of this exchange may go like this, if the state the fetcher grabbed the last time from this server is even older than X:

  fetcher: I am interested in refs/heads/* and refs/tags/* and I have
        your state W (or "I know I am too old that I do not know what
        you call that state").

  server: Sorry, I do not know what W is. Please be more specific.

  fetcher: Here are the refs and their objects: refs/heads/master points
        at commit A, refs/heads/maint points at commit B, ...

  server: (goes and checks and finds out that fetcher is behind X).
        OK, I'll compute a custom pack to bring you up to date with
        one of my states. Here is the incremental difference to the refs,
        and the end result should hash to X. Here comes the pack data.

  fetcher: (after receiving, unpacking and updating the
        remote-tracking refs) Thanks. Do you have more?

After that, the server would update this client to state Y and then state Z as above.

Needless to say, this would naturally extend to a case where you follow only a single branch (you would say "I am interested in your refs/heads/dev" with a wildcard that matches exactly that branch). Of course, depending on the access pattern by common project participants, the server side may choose what set of refs to prepare such snapshots and uncommon requests may always be served by the traditional object enumeration codepath.