From: Paweł Bogusławski Date: Thu, 23 May 2024 11:15:03 GMT Subject: Re: Cloning does not work on available download bandwidth changes Message-ID: <011b01e4-eee7-4300-8498-3af436c15a81@ib.pl> In-Reply-To: <20240523090419.GB1306938@coredump.intra.peff.net> W dniu 23.05.2024 o 11:04, Jeff King pisze: > I did run it a few times. It usually succeeds, but sometimes fails. When > it does fail, it does so in "slow" mode, without speeding back up. And > I see one message from before your "error:" line, which is a report that > curl reported that the connection dropped (curl is shuttling data back > from the server in the form of "packet lines", each of which is prefixed > with its expected size; the "N bytes of body are still expected" is just > that parser seeing a truncated response from curl). > > But I don't think any of this is due to what Git is doing on the client > side. From the client view, the server sends a TCP RST (which I observed > with wireshark), and the connection drops. I'd suspect the drop is not > even from the application level on the server side, but rather that > something in the network path was mad about the delayed packets (or > possibly the spam of duplicated TCP acks that I sometimes saw). > > So I really think this is a network issue, and not a Git issue in > particular. It's possible that the networking stack that github.com uses > is more picky about the things I mentioned above. But again, not a Git > issue. Checked once again with... $ export GIT_TRACE_PACKET=1 $ export GIT_TRACE=1 $ export GIT_CURL_VERBOSE=1 ...and git dies with... [...] remote: Counting objects: 100% (6960/6960), done. remote: Compressing objects: 100% (2926/2926), done. 12:55:29.776311 http.c:756 == Info: Connection #0 to host github.com left intact error: 3805 bytes of body are still expected fetch-pack: unexpected disconnect while reading sideband packet fatal: early EOF fatal: index-pack failed ...a few secs after interface speed is restored (git transfer increases a few secs). In tcpdump of above test I can see FIN sent by client (192.168.100.100) first: # tcpdump -i eth0 -nnvvXSs 1514 -w test.pcap 'tcp and src 140.82.121.4 or dst 140.82.121.4' [...] 21346 2024-05-23 12:55:29,776138 140.82.121.4 192.168.100.100 TLSv1.3 959 Continuation Data 21347 2024-05-23 12:55:29,776151 192.168.100.100 140.82.121.4 TCP 54 40624 → 443 [ACK] Seq=6192 Ack=19611541 Win=65535 Len=0 21348 2024-05-23 12:55:29,776702 192.168.100.100 140.82.121.4 TLSv1.3 78 Application Data >>> here git client sends FIN but transfer is not completed <<<< 21349 2024-05-23 12:55:29,778374 192.168.100.100 140.82.121.4 TCP 54 40624 → 443 [FIN, ACK] Seq=6216 Ack=19611541 Win=65535 Len=0 21350 2024-05-23 12:55:29,836364 140.82.121.4 192.168.100.100 TLSv1.3 78 Application Data 21351 2024-05-23 12:55:29,836364 140.82.121.4 192.168.100.100 TCP 54 443 → 40624 [FIN, ACK] Seq=19611565 Ack=6216 Win=65535 Len=0 21352 2024-05-23 12:55:29,836421 192.168.100.100 140.82.121.4 TCP 54 40624 → 443 [RST] Seq=6216 Win=0 Len=0 21353 2024-05-23 12:55:29,836441 192.168.100.100 140.82.121.4 TCP 54 40624 → 443 [RST] Seq=6216 Win=0 Len=0 21354 2024-05-23 12:55:29,838428 140.82.121.4 192.168.100.100 TCP 54 443 → 40624 [ACK] Seq=19611566 Ack=6217 Win=65535 Len=0 21355 2024-05-23 12:55:29,838448 192.168.100.100 140.82.121.4 TCP 54 40624 → 443 [RST] Seq=6217 Win=0 Len=0 Cloning $ git clone https://gitlab.com/gitlab-org/gitlab $ git clone https://codeberg.org/forgejo/forgejo does not generate such problem after interface speed changes so maybe it's something specific for git <---> github.com communication.