[PATCH 0/4] Report rejections over HTTP when the remote rejects during the transfer

Hello git list,

While investigating a difference between HTTP and SSH when rejecting a push due to it being too large, I noticed that rejecting a push without receiving the entire packfile causes git to print out the error message "pack exceeds maximum allowed size" but it also shows "Everything up-to-date" instead of the rejection of every ref update like the server has specified.

This is the result of two issues in git, of which I aim to fix one here, namely

  1) when the server sends the response and closes the connection, remote-curl
  sees that as an error and stops processing the send-pack output, combined with

  2) git does not remember what it asked the remote helper to push so it cannot
  distinguish whether an empty report means "I had an error and did nothing" or
  "everything was up to date and I didn't have to do anything".

The latter issue is more complex so here I'm concentrating on the former, which has a simple solution but a complex test. The solution is to read in to the end of what send-pack is telling us (it's creating the whole packfile that we're throwing away anyway) so we can report back to the user.

The testing however proved a bit complicated as this bug requires the server to cut off the connection while git is uploading the packfile. The existing HTTP tests use CGI and as far as I've been able to test, httpd buffers too much for us to be able to replicate the situation.

This is why there's a python Git server in this patch series that doesn't rely on CGI but streams the data both ways so it can close the stream as soon as receive-pack exits. There's already some python tooling in the project and I'm much more familiar with it than e.g. perl, so I hope that's fine. I tried to make it as simple as possible while still being able to stream bidirectionally.

Cheers,
   cmn

Carlos Martín Nieto (4):
  t/lib-http: add serve-git.py
  t/lib-http.sh: add functions related to serve-git.py
  t5541: add test for rejecting a push due to packfile size
  remote-curl: read in the push report even if we fail to finish sending
    data

 remote-curl.c             |  24 ++-
 t/lib-httpd.sh            |  20 +++
 t/lib-httpd/serve-git.py  | 353 ++++++++++++++++++++++++++++++++++++++
 t/t5546-receive-limits.sh |  24 +++
 4 files changed, 414 insertions(+), 7 deletions(-)
 create mode 100755 t/lib-httpd/serve-git.py