# [feature request] resume capability for users in enemies of Internet countries 6 messages from 2022-10-06 to 2022-10-07. Participants: m, Bagas Sanjaya, Michal Suchánek, Eric Wong, Philip Oakley, Derrick Stolee. Thread: https://gitlist.dev/t/58575 ## m, 2022-10-06 18:01 Subject: [feature request] resume capability for users in enemies of Internet countries Message-ID: <6e4a08d2-5d72-29fd-6917-11f0a74e9314@gmail.com> URL: https://gitlist.dev/e/6e4a08d2-5d72-29fd-6917-11f0a74e9314%40gmail.com ``` In my country government make connections unstable on purpose. Please add resume capability for commands like git clone ``` ## Bagas Sanjaya, 2022-10-07 01:44 Subject: Re: [feature request] resume capability for users in enemies of Internet countries Message-ID: <83814bcb-1d72-07bf-32d1-d05db6cc6481@gmail.com> URL: https://gitlist.dev/e/83814bcb-1d72-07bf-32d1-d05db6cc6481%40gmail.com In-Reply-To: <6e4a08d2-5d72-29fd-6917-11f0a74e9314@gmail.com> ``` On 10/7/22 01:01, m wrote: > In my country government make connections unstable on purpose. Please add resume capability for commands like git clone > Bandwidth issue? -- An old man doll... just what I always wanted! - Clara ``` ## Michal Suchánek, 2022-10-07 07:14 Subject: Re: [feature request] resume capability for users in enemies of Internet countries Message-ID: <20221007071459.GM28810@kitsune.suse.cz> URL: https://gitlist.dev/e/20221007071459.GM28810%40kitsune.suse.cz In-Reply-To: <83814bcb-1d72-07bf-32d1-d05db6cc6481@gmail.com> ``` On Fri, Oct 07, 2022 at 08:44:09AM +0700, Bagas Sanjaya wrote: > On 10/7/22 01:01, m wrote: > > In my country government make connections unstable on purpose. Please add resume capability for commands like git clone > > > > Bandwidth issue? > Bandwidth is one thing but the other thing is that git network operations require that the whole operation succeeds in one go. If your connectivity is bad to the point that the TCP connection breaks you have downloaded a bunch of data that is AFAIK just thrown away when you retry. It is difficult to know if that data would be useful in the future, and you cannot meaningfully 'resume' because the remote state might have changed in the meantine as well. Further, this whole fetch operation is using a heuristic to fetch some data in the hope that it will be enough to reconstruct the history that is requested, and this has been wrong in some cases, too. Not very precise and reproducible hence hard to 'resume' as well. Let's say that the git networking has been developed at and tuned for the 'first world' Internet, and may be problematic to use in net-wise backwater areas. And it would require non-trivial effort to change. Thanks Michal ``` ## Eric Wong, 2022-10-07 07:26 Subject: Re: [feature request] resume capability for users in enemies of Internet countries Message-ID: <20221007072606.M704368@dcvr> URL: https://gitlist.dev/e/20221007072606.M704368%40dcvr In-Reply-To: <20221007071459.GM28810@kitsune.suse.cz> ``` Michal Suchánek wrote: > On Fri, Oct 07, 2022 at 08:44:09AM +0700, Bagas Sanjaya wrote: > > On 10/7/22 01:01, m wrote: > > > In my country government make connections unstable on purpose. Please add resume capability for commands like git clone > > > > > > > Bandwidth issue? > > > Bandwidth is one thing but the other thing is that git network > operations require that the whole operation succeeds in one go. > > If your connectivity is bad to the point that the TCP connection breaks > you have downloaded a bunch of data that is AFAIK just thrown away when > you retry. > > It is difficult to know if that data would be useful in the future, and > you cannot meaningfully 'resume' because the remote state might have > changed in the meantine as well. > > Further, this whole fetch operation is using a heuristic to fetch some > data in the hope that it will be enough to reconstruct the history that > is requested, and this has been wrong in some cases, too. Not very > precise and reproducible hence hard to 'resume' as well. > > Let's say that the git networking has been developed at and tuned for > the 'first world' Internet, and may be problematic to use in net-wise > backwater areas. And it would require non-trivial effort to change. Increased adoption of bundles would help, since `wget -c' and such would work nicely, but that puts the burden on hosts for extra storage. Perhaps GIT_SMART_HTTP=0 and having dumb clones not throwaway incomplete xfers would be more transparent to hosters, but dumb HTTP tends to be slow even on good connections. ``` ## Philip Oakley, 2022-10-07 10:03 Subject: Re: [feature request] resume capability for users in enemies of Internet countries Message-ID: <8e11ecf0-735a-a104-a3d1-8324fbd76862@iee.email> URL: https://gitlist.dev/e/8e11ecf0-735a-a104-a3d1-8324fbd76862%40iee.email In-Reply-To: <20221007072606.M704368@dcvr> ``` On 07/10/2022 08:26, Eric Wong wrote: > Michal Suchánek wrote: >> On Fri, Oct 07, 2022 at 08:44:09AM +0700, Bagas Sanjaya wrote: >>> On 10/7/22 01:01, m wrote: >>>> In my country government make connections unstable on purpose. Please add resume capability for commands like git clone >>>> >>> Bandwidth issue? >>> >> Bandwidth is one thing but the other thing is that git network >> operations require that the whole operation succeeds in one go. >> >> If your connectivity is bad to the point that the TCP connection breaks >> you have downloaded a bunch of data that is AFAIK just thrown away when >> you retry. >> >> It is difficult to know if that data would be useful in the future, and >> you cannot meaningfully 'resume' because the remote state might have >> changed in the meantine as well. >> >> Further, this whole fetch operation is using a heuristic to fetch some >> data in the hope that it will be enough to reconstruct the history that >> is requested, and this has been wrong in some cases, too. Not very >> precise and reproducible hence hard to 'resume' as well. >> >> Let's say that the git networking has been developed at and tuned for >> the 'first world' Internet, and may be problematic to use in net-wise >> backwater areas. And it would require non-trivial effort to change. > Increased adoption of bundles would help, since `wget -c' and such > would work nicely, but that puts the burden on hosts for extra storage. > > Perhaps GIT_SMART_HTTP=0 and having dumb clones not throwaway > incomplete xfers would be more transparent to hosters, but dumb > HTTP tends to be slow even on good connections. There is work going on by Stollee (cc'd) on Bundle-URIs that look to split up repository serving into bite sized chunk, so may be worth looking at. https://lore.kernel.org/git/pull.1248.v4.git.1660050761.gitgitgadget@gmail.com/ : [PATCH v4 0/2] bundle URIs: design doc Also discussed at the recent Contributor's Summit https://lore.kernel.org/git/YzXvwv%2FzK5AjhVvV@nand.local/ : [TOPIC 1/8] Bundle URIs -- Philip ``` ## Derrick Stolee, 2022-10-07 14:24 Subject: Re: [feature request] resume capability for users in enemies of Internet countries Message-ID: URL: https://gitlist.dev/e/ae9de8dd-d1e6-2898-90cb-2a6593aa8f3a%40github.com In-Reply-To: <8e11ecf0-735a-a104-a3d1-8324fbd76862@iee.email> ``` On 10/7/22 6:03 AM, Philip Oakley wrote: > On 07/10/2022 08:26, Eric Wong wrote: >> Perhaps GIT_SMART_HTTP=0 and having dumb clones not throwaway >> incomplete xfers would be more transparent to hosters, but dumb >> HTTP tends to be slow even on good connections. > > There is work going on by Stollee (cc'd) on Bundle-URIs that look to > split up repository serving into bite sized chunk, so may be worth > looking at. > https://lore.kernel.org/git/pull.1248.v4.git.1660050761.gitgitgadget@gmail.com/ > : [PATCH v4 0/2] bundle URIs: design doc > > Also discussed at the recent Contributor's Summit > https://lore.kernel.org/git/YzXvwv%2FzK5AjhVvV@nand.local/ : [TOPIC 1/8] > Bundle URIs Thanks, Philip. The bundle URIs feature should make it possible to help here. One is that we could break the clone into smaller pieces, but I think that most bundle providers will still have at least one bundle with "most" of the repo. The real payoff is that we should be able to have resumable downloads for those bundles (pending some implementation details, but I'm waiting for the rest of the series to land first). Of course, this all depends on the host providing bundles and advertising them, but we will get to that when we can. Thanks, -Stolee ```