git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v3] http.postbuffer: allow full range of ssize_t values

From
Christian Couder <christian.couder@gmail.com>
Date
Apr 6, 2017, 17:24 UTC
Message-ID
<CAP8UFD3r7C_OcQMmtOju636okqrTB-af6CDo2jw5vGsiWcLVrg@mail.gmail.com>
In-Reply-To
<20170404204031.geh72k6yuiky4wsw@sigill.intra.peff.net>
On Tue, Apr 4, 2017 at 10:40 PM, Jeff King <peff@peff.net> wrote:
Show 18 quoted lines
> On Tue, Apr 04, 2017 at 06:42:23PM +0000, David Turner wrote:
>
>> > What does it look like when it fails? What does GIT_TRACE_CURL look like (or
>> > GIT_CURL_VERBOSE if your client is older, but remember to sanitize any auth
>> > lines)?
>>
>> Unfortunately, we've already worked around the problem by pushing over SSH,
>> so I no longer have a failing case to examine. Last time I tried, I actually did some
>> hackery to create a push smaller than 2GB, but it still failed (this time, with
>> "502 Bad Gateway").  So, something is clearly weird in GitLab land.
>>
>> I did see "Transfer-Encoding: chunked" in one of the responses from the server,
>>  but not in the request (not sure if that's normal). The smaller push had:
>> Content-Length: 1048908476
>
> The 502 makes me think it's a problem in the GitLab reverse-proxy layer
> (and also my experience debugging Git-over-HTTP weirdness on GitHub's reverse
> proxy layer, which had a number of pitfalls ;) ).
Yeah, maybe.
Show 34 quoted lines
> You should be able to do a synthetic test like:
>
>   git init
>   dd if=/dev/urandom of=foo.rand bs=1k count=1024
>   git add .
>   git commit -m 'random megabyte'
>   GIT_TRACE_CURL=/tmp/foo.out \
>     git -c http.postbuffer=0 push https://...
>
> You should see two POSTs to /git-receive-pack, like this:
>
>   Send header: POST /peff/test.git/git-receive-pack HTTP/1.1
>   Send header: Host: github.com
>   Send header: Authorization: Basic <redacted>
>   Send header: User-Agent: git/2.12.2.952.g759391acc
>   Send header: Content-Type: application/x-git-receive-pack-request
>   Send header: Accept: application/x-git-receive-pack-result
>   Send header: Content-Length: 4
>
>   Send header: POST /peff/test.git/git-receive-pack HTTP/1.1
>   Send header: Host: github.com
>   Send header: Authorization: Basic <redacted>
>   Send header: User-Agent: git/2.12.2.952.g759391acc
>   Send header: Accept-Encoding: gzip
>   Send header: Content-Type: application/x-git-receive-pack-request
>   Send header: Accept: application/x-git-receive-pack-result
>   Send header: Transfer-Encoding: chunked
>
> The first is a probe to make sure we can hit the endpoint without
> sending the whole payload. And the second should pass up the 1MB
> packfile in chunks.
>
> That would at least tell you if the problem is the chunked encoding, or
> if it's related to the size.
The above commands work for me using gitlab.com and the log shows:

Send header, 0000000309 bytes (0x00000135) Send header: POST /chriscool/yet-another-test-project.git/git-receive-pack HTTP/1.1 Send header: Authorization: Basic <redacted> Send header: User-Agent: git/2.12.2.625.g14da1346c9.dirty Send header: Host: gitlab.com Send header: Content-Type: application/x-git-receive-pack-request Send header: Accept: application/x-git-receive-pack-result Send header: Content-Length: 4

Send header, 0000000341 bytes (0x00000155) Send header: POST /chriscool/yet-another-test-project.git/git-receive-pack HTTP/1.1 Send header: Authorization: Basic <redacted> Send header: User-Agent: git/2.12.2.625.g14da1346c9.dirty Send header: Host: gitlab.com Send header: Accept-Encoding: gzip Send header: Content-Type: application/x-git-receive-pack-request Send header: Accept: application/x-git-receive-pack-result Send header: Transfer-Encoding: chunked

Maybe the reverse proxy doesn't like it when the push is really big.
Previous: Jeff KingNext: Jeff King
Message 9 of 10 in “http.postbuffer: allow full range of ssize_t values”
  1. http.postbuffer: allow full range of ssize_t valuesDavid Turner, Mar 31, 2017
  2. Junio C HamanoMar 31, 2017
  3. Jeff KingApr 1, 2017
  4. Junio C HamanoApr 1, 2017
  5. David TurnerApr 3, 2017
  6. Jeff KingApr 4, 2017
  7. David TurnerApr 4, 2017
  8. Jeff KingApr 4, 2017
  9. Christian CouderApr 6, 2017
  10. Jeff KingApr 7, 2017

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.