GSS-Negotiate authentication requires that all data fit into postbuffer

One thing I just noticed is that if git is using GSS-Negotiate authentication, the entire POST contents have to fit into however much memory is specified by http.postbuffer:

  vauxhall ok % git push https://bmc@git.crustytoothpaste.net/git/bmc/test.git development
  Counting objects: 37994, done.
  Delta compression using up to 4 threads.
  Compressing objects: 100% (10683/10683), done.
  Writing objects: 100% (37994/37994), 9.15 MiB | 4.45 MiB/s, done.
  Total 37994 (delta 26760), reused 37633 (delta 26467)
  Unable to rewind rpc post data - try increasing http.postBuffer
  Password for 'https://bmc@git.crustytoothpaste.net': 

GSS-Negotiate authentication always requires a rewind with CURL.

The remote in question only supports Negotiate authentication, so prompting for a password in this case isn't going to help. I'm probably going to look into this over the next couple of days, but two things need to be done here: 1) do not prompt for a password if only Negotiate authentication is requested, since it just won't work, and 2) recreate the data as needed if we have to rewind, since otherwise pushing a fresh copy of the Linux kernel repo simply isn't going to work as the buffer will have to be too large. An alternative is to send a small amount of data, smaller than the postbuffer, in the first chunk and only fail to rewind if the second or subsequent chunks need rewinding.

On Sun, Oct 06, 2013 at 12:42:36AM +0000, brian m. carlson wrote:

Isn't 'Expect: 100-Continue' meant for stuff like this (not that it is always supported properly)?

-Ilari

On Sun, 6 Oct 2013, Ilari Liusvaara wrote:

Yes it is and libcurl uses 100-Continue by default for that purpose. But the harsh reality is that lots of (most?) servers just don't care and aren't setup to respond properly and instead we end up having to send data multiple times in vain.

On Sun, Oct 06, 2013 at 05:00:02PM +0200, Daniel Stenberg wrote:

If there's a way to make Apache with mod_auth_kerb do that with curl, then it doesn't require a change to git, and I'm happy to make it on my end. But using the curl command line client, I don't see an Expect: 100-continue anywhere during the connection using Debian's curl 7.32.0-1. Do I need to send a certain amount of data to see that behavior?

The command line I used was

  curl -v -d '0000' -H'Transfer-Encoding: chunked' -H'Content-Type: application/x-git-receive-pack-request' --negotiate -u bmc: https://git.crustytoothpaste.net/git/bmc/test.git/git-receive-pack

On Sun, 6 Oct 2013, brian m. carlson wrote:

Correct, curl will enable "Expect: 100-continue" if the post size is > 1024 bytes.

On Sun, Oct 06, 2013 at 05:38:24PM +0200, Daniel Stenberg wrote:

I've been able to reproduce this behavior with the curl command line client, but it looks like we disable Expect: 100-continue in git since some proxy servers are too stupid to understand it, according to commit 959dfcf.

On Sun, Oct 06, 2013 at 05:50:00PM +0000, brian m. carlson wrote:

Yeah, instead we try to make two separate requests, and assume that the first one clears the path for any further requests. Of course that doesn't work for auth methods that actually negotiate for each request.

We should probably make the "Expect" suppression optional for people who know they have working systems. It would be nice to trigger it automatically when people are using something like GSS, but that decision happens at the curl layer.

-Peff

On Mon, Oct 07, 2013 at 08:02:41AM -0400, Jeff King wrote:

Yeah, that's what I ultimately determined would be the best choice. Technically, we could respawn the process, but that would be a logistical nightmare and extremely inefficient for large transfers.