Unable to get pack file - HTTP 416 [was: error: Unable to get pack file]

From: Tim Düsterhus <tim@bastelstu.be>
Date: Oct 14, 2020, 18:25 UTC
Message-ID: <419fd4ad-f9c2-927c-7ae0-c6083f70dcba@bastelstu.be>
In-Reply-To: <CAGP6POKvZHHWDcV4-4P6Y4yy6gCujm7vpEqKxWW0whT2FyqYEw@mail.gmail.com>

Hi List, Hongyi, Willy,

[please leave me in Cc, I am not subscribed]

Am 20.06.20 um 16:58 schrieb Hongyi Zhao:

Show 16 quoted lines

> I have a local repo mirror for https://source.winehq.org/git/wine.git
> which was updated some time ago. When I try to update the local repo
> to its latest version, I meet the following error message:
> 
> $ git pull
> error: Unable to get pack file
> https://source.winehq.org/git/wine.git/objects/pack/pack-3776ee1d93e316ed56507e9062d8fc8e5c07feed.pack
> The requested URL returned error: 416 Requested Range Not Satisfiable
> error: Unable to find 6ed1d04b1325d4c2480907b76b8b1d5529caebab under
> https://source.winehq.org/git/wine.git
> Cannot obtain needed object 6ed1d04b1325d4c2480907b76b8b1d5529caebab
> while processing commit bf454cc39428fc5299e5c26d9c0ddc6a9277c7ae.
> error: fetch failed.
> 
> Any suggestions for this problem?
>

I experienced the same issue yesterday with the canonical git repository of HAProxy [0]. The git repository uses the dumb HTTP transport.

I am using git 2.28.0 from the `git-core` PPA for Ubuntu: https://launchpad.net/~git-core/+archive/ubuntu/ppa

I attempted to update the repository using `git fetch` as usual, but interrupted the fetch with Ctrl+C, because it appeared stuck. It did not return in tens of seconds, while it usually updates in single-digit seconds.

Afterwards I retried the operation, but it failed with the same error message Hongyi experienced in June.

I could track the issue down to the following:

1. git already downloaded all the bytes of the pack file in question.
2. When retrying the `git fetch` git attempted to resume the download by
sending a `Range` request.
3. Because the file was complete git was sending an invalid `Range`.
4. The nginx at haproxy.org returned the HTTP 416 in response, causing
git to fail.

I could fix the issue by truncating the temporary pack file to be 1 Byte smaller than before. I suspect deleting the temporary file would also have worked. Even after truncation the operation took quite a while. I am not sure what git was doing under the hood. The network connection was not the limiting factor there.

Neither running `git fsck` nor `git gc` was able to fix the issue. I had to edit around in the `.git/objects/` folder.

I would claim that this behavior is a bug in git, because I was unable to fix my repository with regular git commands. The next repack on the server might or might not have fixed the issue.

I believe one of the following suggestions should prevent this issue from happening:

1. Retry without the `Range` header when receiving a 416 in response.
2. Checking the returned `Content-Range` to check whether the full pack
is already downloaded and then ignoring the error status.
3. Deleting the temporary packfile in progress when resuming the
download fails. Then the next retry would have worked from my understanding.

Please see the following link to mail-archive.com for my initial thread on the HAProxy mailing list. It might contain additional details:

https://www.mail-archive.com/haproxy@formilux.org/msg38650.html

Best regards Tim Düsterhus

[0] https://git.haproxy.org/git/haproxy.git/