Re: Clone hangs when done over http with --reference

From: Jeff King <peff@peff.net>
Date: May 14, 2015, 19:52 UTC
Message-ID: <20150514195226.GA10205@peff.net>
In-Reply-To: <5554D69E.20307@linuxfoundation.org>

On Thu, May 14, 2015 at 01:08:46PM -0400, Konstantin Ryabitsev wrote:

Show 6 quoted lines

> Two questions:
> 
> 1. Does this mean there are potential problems with other git operations
> that involve ref negotiation, not just when doing git clone --reference?
> Is there a chance to run in to this deadlock by doing an operation like
> "git remote update"?

Yes. From the server's perspective, a "clone --reference" is really no different than a fetch in which the client happened to have all of those refs already. I didn't try it, but you should be able to reproduce the problem with:

  cd first-repo.git
  git fetch https://.../second-repo.git refs/*:refs/remotes/foo/*

which should have to do the exact same ref negotiation ("I have these commits, I want these other commits").

Show 9 quoted lines

> 2. If we configure the webserver to serve some files directly, without
> passing them to http-backend, e.g. doing the recommended apache magic:
> 
> > AliasMatch ^/git/(.*/objects/[0-9a-f]{2}/[0-9a-f]{38})$          /var/lib/git/$1
> > AliasMatch ^/git/(.*/objects/pack/pack-[0-9a-f]{40}.(pack|idx))$ /var/lib/git/$1
> > AliasMatch ^/git/(.*/refs/heads/.*)$                             /var/lib/git/$1
> 
> Will that make the spooling less of a problem, since it won't involve
> the super-huge files?

No, that won't help. Once git is doing the smart protocol, it will never ask for arbitrary files. So you would have to disable smart-http entirely, which I don't recommend.

Besides which, it's not the size of the objects or packs that is an issue here. It's the relationship of the tips in the second repo to the tips in the first. That is, the "big" data here is the client and server finding the common commits between the two (and it's not even _that_ big; it's just big by "stuffing into a pipe buffer" standards; as you noticed, the git protocol handles it just fine).

-Peff