Re: Git-aware HTTP transport

Tarmigan <tarmigan+git@gmail.com> wrote:

Correct. Today _none_ of the transport protocols allow the server to force the client to use some sort of reference repository for an initial clone. There are likely two reasons for this:

 *) Its a lot simpler to program to just get everything from
    one location.

 *) If you really are forking an open source project then in
    some cases you may need to distribute the full source,
	not your delta.  You may just as well distribute the full
	source and call it a day.

The dumb http:// currently supports getting packs from a remote HTTP server via its objects/info/http-alternates. But the native and rsync protocols don't support that. The logic behind http-alternates isn't to allow moving load onto a different server, but to make the locally available alternate repository available through the same web server. The path on the UNIX filesystem that is used in objects/info/alternates may not be the same path used in the web server's namespace.

I think this isn't a bad idea, but I'd rather have the server say "In order to talk to me you need at least these objects in common with me: ...". If you don't have those then the user should go find it on their own, rather than forcing them to a particular URL and automatically following it.

I'm a little concerned about a US user putting a US mirror site
of kernel.org into the server and forcing a user in India to do a
full clone over the Atlantic links when they could have just used
a more local mirror for that initial "linus-2.6" clone.
 

When the server receives a "give-up" it needs to create a pack based on "git rev-list --objects-boundary $WANT --not $COMMON". If the set $COMMON is non-empty then its a partial pack; if $COMMON is empty then its a full clone. This is what the native protocol does when the client gives up.

You are thinking the wrong way. If you have more than 256 commits that the other side doesn't have you may give up too early. For that to be true you need to create 256 commits locally that aren't on the remote peer and whose timestamps are all ahead of the commits you last fetched from the remote peer.

Yes, it can happen. But its less likely than you think because we're talking about you doing 256 commits worth of development and not picking up any new commits from remote peers in the middle of that time period. Get just one and it resets the counter back to 0 and allows it to try another 256 commits before giving up.

I should amend this section to talk about what giving up here really means. If we have nothing sent in common yet or maybe very little sent in common we may have existing remote refs tied to this URL in .git/config that can send, and we may have one or more annotated tags that we know for a fact are in common as both peers have the same tag name pointing to the same tag object.

A smart(er) client might try to toss some recently dated annotated tags at the server before throwing a give-up if it would otherwise throw a give-up. Its likely to narrow the result set, and doesn't hurt if it doesn't.

Oh, this is a live-lock condition. If the client grabs the list of refs from the server, then has to wait 100 ms to get back to the server and start upload-pack (due to latency) and in that 100ms window Linus shoves a 1001 commit merge into his tree then yes, the server may abort and tell the client "error invalid want".

At which point the client may try to restart from the beginning, or just plain give up and tell the end user try again later.

This condition of 1000 is just some aribtrary limit to allow the
client to still continue with an in-progress download if right in
the middle of the client's RPCs the remote was modified by its owner.
 

Yes, thanks.  I will make the correction.  Damn copy and paste.