Re: Cloning does not work on available download bandwidth changes
- From
- Paweł Bogusławski <pawel.boguslawski@ib.pl>
- Date
- May 22, 2024, 22:58 UTC
- Message-ID
- <293afb32-99b1-4562-b339-7862698ef00f@ib.pl>
- In-Reply-To
- <Zk5pDZ1gTcyrGfUk@tapette.crustytoothpaste.net>
W dniu 22.05.2024 o 23:52, brian m. carlson pisze:
Show 61 quoted lines
> On 2024-05-22 at 15:02:51, Paweł Bogusławski wrote: >> Started to clone big git repo on not-too-fast, _stable_ VDSL link (up to >> 20mbps down)... >> >> $ git clone https://github.com/googleapis/google-api-go-client >> Cloning into 'google-api-go-client'... >> remote: Enumerating objects: 644446, done. >> remote: Counting objects: 100% (6922/6922), done. >> remote: Compressing objects: 100% (2904/2904), done. >> Receiving objects: 0% (3859/644446), 20.82 MiB | 1.01 MiB/s >> >> ...and then started to watch a VOD movie on same link; when VOD buffers >> data, eats almost all available down bandwidth and leaves only about 100 >> kB/s for git... >> >> Receiving objects: 1% (7111/644446), 44.49 MiB | 130.00 KiB/s >> >> ...and when VOD stops buffering and whole bandwith is available for git >> again, git transfer starts to grow... >> >> Receiving objects: 1% (7660/644446), 50.56 MiB | 575.00 KiB/s >> >> ...but finally git throws an error >> >> error: 181 bytes of body are still expected5 MiB | 1015.00 KiB/s >> fetch-pack: unexpected disconnect while reading sideband packet >> fatal: early EOF >> fatal: index-pack failed >> >> or sometimes: >> >> error: RPC failed; curl 92 HTTP/2 stream 5 was not closed cleanly: >> CANCEL (err 8) >> error: 6109 bytes of body are still expected >> fetch-pack: unexpected disconnect while reading sideband packet >> fatal: early EOF >> fatal: fetch-pack: invalid index-pack output >> >> No such problems when downloading bigger file (i.e. linux kernel source) >> with wget or curl instead of git clone (wget/curl transfer drops to about >> 100 kB/s when VOD buffers and increases to full speed when VOD is not >> transferring and transfer finishes successfully). >> >> Sounds like a bug in git; should not throw an error on available download >> bandwidth changes as wget and curl do and should not require any params >> tuning (to stop users flooding bugtrackers). > I don't believe this is a bug in Git. The problem here is a network > issue of some sort that's causing the connection to be interrupted or > dropped. This is very common, but the possible causes are many. > > A lot of times we see this it's some sort of proxy, and that can be a > non-default antivirus or firewall or TLS MITM device, but that's usually > on Windows, and you're using Debian. It can also be just a bad > connection, poor traffic management by your ISP, or a flaky wireless or > wired network card. > > My _guess_ about what's happening here is poor traffic management: > because video is typically flagged as low-latency in DSCP, some device > on the path is prioritizing it to the exclusion of all other traffic, > which is causing the Git connection to be dropped. I have no proof of > this, though.
Try to run
git clone https://github.com/googleapis/google-api-go-client
and limit interface speed somehow i.e.
tc qdisc add dev eth0 root handle 1: htb default 12 tc class add dev eth0 parent 1:1 classid 1:12 htb rate 20kbps ceil 20kbps tc qdisc add dev eth0 parent 1:12 netem delay 1000ms
and after a while restore speed with i.e.
tc qdisc del dev eth0 parent 1:12 netem delay 1000ms
I tried this in Debian 11 and git 2.30.2 died with:
error: 5723 bytes of body are still expected MiB | 920.00 KiB/s fetch-pack: unexpected disconnect while reading sideband packet fatal: early EOF fatal: index-pack failed