Re: About connection resuming

From: Stefan Beller <sbeller@google.com>
Date: Feb 20, 2018, 19:43 UTC
Message-ID: <CAGZ79kYytJOFhzdpWnWNs574KeLQ2f1AtkdAQ1-o6P5=VO=r_Q@mail.gmail.com>
In-Reply-To: <b85f9654-3552-30c0-79bb-84fe10309d8b@netease.com>

On Tue, Feb 20, 2018 at 4:49 AM, chenzero <chenzero@netease.com> wrote:

> Hello,
> First, I am a user of git for about 2 years, I really appreciated you all to
> create this great useful software!

Hi, welcome to the mailing list!

Show 17 quoted lines

> My encountered problem is:
> sometimes, the repo is big and because my networking is not stable(or my
> network proxy has
> some limitations), so, the clone will always fail.
> I tried following ways to solve this, however, not much success.
> 1. clone depth 1
> git clone --depth=1 https://..../repo.git
> however, some repo even depth=1 might fail.(It seemed that my network proxy
> limit tcp connection.
> if transfer over 50M, it will break)
> 2. download bundle file.
> but no all git repo provide bundle files to download.
>
> After some investment on the code, I think, perhaps,
> if enhance the git-http-backend to support http header: Range, or
> Content-Range,
> maybe it will enable connection resuming.

This thought has come up before, for example https://public-inbox.org/git/1473984742-12516-1-git-send-email-kevin.m.wern@gmail.com/ (Or you can search for "resumable clone" in that mailing list archive, there are more interesting discussions)

But the current problem is that the bytes of a clone are not stable, as the packs are ordered racily IIUC: So if you clone the same repository (with the same branch values), the observed communication over the wire may differ in the part when the pack file is transferred as the packfile is packed up in a multi threaded fashion, that has no clear order defined.

Show 6 quoted lines

> the problem of this way is: it needs to upgrade the current deployed
> "git-http-backend",
> and maybe much code need to change including git-remote-http etc.
>
> This is the very basic thought, and whether I should try other way ?
> Thanks a lot!

I think a more promising approach is to have support for references inside the packfile for transfer, this could become one of the features of the new protocol that is being worked on https://public-inbox.org/git/20180207011312.189834-1-bmwill@google.com/ I do not find a reference for the idea, but Jonathan (cc'd) laid it out well once upon a time. Jonathan do you have a mailing list reference for references in packfiles?

Thanks, Stefan