Cloning does not work on available download bandwidth changes

> Started to clone big git repo on not-too-fast, _stable_ VDSL link (up to
> 20mbps down)...
> 
>     $ git clone https://github.com/googleapis/google-api-go-client
>     Cloning into 'google-api-go-client'...
>     remote: Enumerating objects: 644446, done.
>     remote: Counting objects: 100% (6922/6922), done.
>     remote: Compressing objects: 100% (2904/2904), done.
>     Receiving objects:   0% (3859/644446), 20.82 MiB | 1.01 MiB/s
> 
> ...and then started to watch a VOD movie on same link; when VOD buffers
> data, eats almost all available down bandwidth and leaves only about 100
> kB/s for git...
> 
>     Receiving objects:   1% (7111/644446), 44.49 MiB | 130.00 KiB/s
> 
> ...and when VOD stops buffering and whole bandwith is available for git
> again, git transfer starts to grow...
> 
>     Receiving objects:   1% (7660/644446), 50.56 MiB | 575.00 KiB/s
> 
> ...but finally git throws an error
> 
>     error: 181 bytes of body are still expected5 MiB | 1015.00 KiB/s
>     fetch-pack: unexpected disconnect while reading sideband packet
>     fatal: early EOF
>     fatal: index-pack failed
> 
> or sometimes:
> 
>     error: RPC failed; curl 92 HTTP/2 stream 5 was not closed cleanly:
> CANCEL (err 8)
>     error: 6109 bytes of body are still expected
>     fetch-pack: unexpected disconnect while reading sideband packet
>     fatal: early EOF
>     fatal: fetch-pack: invalid index-pack output
> 
> No such problems when downloading bigger file (i.e. linux kernel source)
> with wget or curl instead of git clone (wget/curl transfer drops to about
> 100 kB/s when VOD buffers and increases to full speed when VOD is not
> transferring and transfer finishes successfully).
> 
> Sounds like a bug in git; should not throw an error on available download
> bandwidth changes as wget and curl do and should not require any params
> tuning (to stop users flooding bugtrackers).

> On 2024-05-22 at 15:02:51, Paweł Bogusławski wrote:
>> Started to clone big git repo on not-too-fast, _stable_ VDSL link (up to
>> 20mbps down)...
>>
>>      $ git clone https://github.com/googleapis/google-api-go-client
>>      Cloning into 'google-api-go-client'...
>>      remote: Enumerating objects: 644446, done.
>>      remote: Counting objects: 100% (6922/6922), done.
>>      remote: Compressing objects: 100% (2904/2904), done.
>>      Receiving objects:   0% (3859/644446), 20.82 MiB | 1.01 MiB/s
>>
>> ...and then started to watch a VOD movie on same link; when VOD buffers
>> data, eats almost all available down bandwidth and leaves only about 100
>> kB/s for git...
>>
>>      Receiving objects:   1% (7111/644446), 44.49 MiB | 130.00 KiB/s
>>
>> ...and when VOD stops buffering and whole bandwith is available for git
>> again, git transfer starts to grow...
>>
>>      Receiving objects:   1% (7660/644446), 50.56 MiB | 575.00 KiB/s
>>
>> ...but finally git throws an error
>>
>>      error: 181 bytes of body are still expected5 MiB | 1015.00 KiB/s
>>      fetch-pack: unexpected disconnect while reading sideband packet
>>      fatal: early EOF
>>      fatal: index-pack failed
>>
>> or sometimes:
>>
>>      error: RPC failed; curl 92 HTTP/2 stream 5 was not closed cleanly:
>> CANCEL (err 8)
>>      error: 6109 bytes of body are still expected
>>      fetch-pack: unexpected disconnect while reading sideband packet
>>      fatal: early EOF
>>      fatal: fetch-pack: invalid index-pack output
>>
>> No such problems when downloading bigger file (i.e. linux kernel source)
>> with wget or curl instead of git clone (wget/curl transfer drops to about
>> 100 kB/s when VOD buffers and increases to full speed when VOD is not
>> transferring and transfer finishes successfully).
>>
>> Sounds like a bug in git; should not throw an error on available download
>> bandwidth changes as wget and curl do and should not require any params
>> tuning (to stop users flooding bugtrackers).
> I don't believe this is a bug in Git.  The problem here is a network
> issue of some sort that's causing the connection to be interrupted or
> dropped.  This is very common, but the possible causes are many.
>
> A lot of times we see this it's some sort of proxy, and that can be a
> non-default antivirus or firewall or TLS MITM device, but that's usually
> on Windows, and you're using Debian.  It can also be just a bad
> connection, poor traffic management by your ISP, or a flaky wireless or
> wired network card.
>
> My _guess_ about what's happening here is poor traffic management:
> because video is typically flagged as low-latency in DSCP, some device
> on the path is prioritizing it to the exclusion of all other traffic,
> which is causing the Git connection to be dropped.  I have no proof of
> this, though.

> Try to run
> 
> git clone https://github.com/googleapis/google-api-go-client
> 
> and limit interface speed somehow i.e.
> 
> tc qdisc add dev eth0 root handle 1: htb default 12
> tc class add dev eth0 parent 1:1 classid 1:12 htb rate 20kbps ceil 20kbps
> tc qdisc add dev eth0 parent 1:12 netem delay 1000ms
> 
> and after a while restore speed with i.e.
> 
> tc qdisc del dev eth0 parent 1:12 netem delay 1000ms
> 
> I tried this in Debian 11 and  git 2.30.2 died with:
> 
> error: 5723 bytes of body are still expected MiB | 920.00 KiB/s
> fetch-pack: unexpected disconnect while reading sideband packet
> fatal: early EOF
> fatal: index-pack failed

> I did run it a few times. It usually succeeds, but sometimes fails. When
> it does fail, it does so in "slow" mode, without speeding back up. And
> I see one message from before your "error:" line, which is a report that
> curl reported that the connection dropped (curl is shuttling data back
> from the server in the form of "packet lines", each of which is prefixed
> with its expected size; the "N bytes of body are still expected" is just
> that parser seeing a truncated response from curl).
>
> But I don't think any of this is due to what Git is doing on the client
> side. From the client view, the server sends a TCP RST (which I observed
> with wireshark), and the connection drops. I'd suspect the drop is not
> even from the application level on the server side, but rather that
> something in the network path was mad about the delayed packets (or
> possibly the spam of duplicated TCP acks that I sometimes saw).
>
> So I really think this is a network issue, and not a Git issue in
> particular. It's possible that the networking stack that github.com uses
> is more picky about the things I mentioned above. But again, not a Git
> issue.

>W dniu 22.05.2024 o 23:52, brian m. carlson pisze:
>
>> On 2024-05-22 at 15:02:51, Paweł Bogusławski wrote:
>>> Started to clone big git repo on not-too-fast, _stable_ VDSL link (up
>>> to 20mbps down)...
>>>
>>>      $ git clone https://github.com/googleapis/google-api-go-client
>>>      Cloning into 'google-api-go-client'...
>>>      remote: Enumerating objects: 644446, done.
>>>      remote: Counting objects: 100% (6922/6922), done.
>>>      remote: Compressing objects: 100% (2904/2904), done.
>>>      Receiving objects:   0% (3859/644446), 20.82 MiB | 1.01 MiB/s
>>>
>>> ...and then started to watch a VOD movie on same link; when VOD
>>> buffers data, eats almost all available down bandwidth and leaves
>>> only about 100 kB/s for git...
>>>
>>>      Receiving objects:   1% (7111/644446), 44.49 MiB | 130.00 KiB/s
>>>
>>> ...and when VOD stops buffering and whole bandwith is available for
>>> git again, git transfer starts to grow...
>>>
>>>      Receiving objects:   1% (7660/644446), 50.56 MiB | 575.00 KiB/s
>>>
>>> ...but finally git throws an error
>>>
>>>      error: 181 bytes of body are still expected5 MiB | 1015.00 KiB/s
>>>      fetch-pack: unexpected disconnect while reading sideband packet
>>>      fatal: early EOF
>>>      fatal: index-pack failed
>>>
>>> or sometimes:
>>>
>>>      error: RPC failed; curl 92 HTTP/2 stream 5 was not closed cleanly:
>>> CANCEL (err 8)
>>>      error: 6109 bytes of body are still expected
>>>      fetch-pack: unexpected disconnect while reading sideband packet
>>>      fatal: early EOF
>>>      fatal: fetch-pack: invalid index-pack output
>>>
>>> No such problems when downloading bigger file (i.e. linux kernel
>>> source) with wget or curl instead of git clone (wget/curl transfer
>>> drops to about
>>> 100 kB/s when VOD buffers and increases to full speed when VOD is not
>>> transferring and transfer finishes successfully).
>>>
>>> Sounds like a bug in git; should not throw an error on available
>>> download bandwidth changes as wget and curl do and should not require
>>> any params tuning (to stop users flooding bugtrackers).
>> I don't believe this is a bug in Git.  The problem here is a network
>> issue of some sort that's causing the connection to be interrupted or
>> dropped.  This is very common, but the possible causes are many.
>>
>> A lot of times we see this it's some sort of proxy, and that can be a
>> non-default antivirus or firewall or TLS MITM device, but that's
>> usually on Windows, and you're using Debian.  It can also be just a
>> bad connection, poor traffic management by your ISP, or a flaky
>> wireless or wired network card.
>>
>> My _guess_ about what's happening here is poor traffic management:
>> because video is typically flagged as low-latency in DSCP, some device
>> on the path is prioritizing it to the exclusion of all other traffic,
>> which is causing the Git connection to be dropped.  I have no proof of
>> this, though.
>
>Try to run
>
>git clone https://github.com/googleapis/google-api-go-client
>
>and limit interface speed somehow i.e.
>
>tc qdisc add dev eth0 root handle 1: htb default 12 tc class add dev eth0 parent 1:1
>classid 1:12 htb rate 20kbps ceil 20kbps tc qdisc add dev eth0 parent 1:12 netem
>delay 1000ms
>
>and after a while restore speed with i.e.
>
>tc qdisc del dev eth0 parent 1:12 netem delay 1000ms
>
>I tried this in Debian 11 and  git 2.30.2 died with:
>
>error: 5723 bytes of body are still expected MiB | 920.00 KiB/s
>fetch-pack: unexpected disconnect while reading sideband packet
>fatal: early EOF
>fatal: index-pack failed