Re: With big repos and slower connections, git clone can be hard to work with

From: Patrick Steinhardt <ps@pks.im>
Date: Jun 10, 2024, 06:46 UTC
Message-ID: <ZmahU1Gv_abt5RGn@tanuki>
In-Reply-To: <20240608084323.GB2390433@coredump.intra.peff.net>

On Sat, Jun 08, 2024 at 04:43:23AM -0400, Jeff King wrote:

Show 16 quoted lines

> On Sat, Jun 08, 2024 at 02:46:38AM +0200, ellie wrote:
> 
> > The deepening worked perfectly, thank you so much! I hope a resume will
> > still be considered however, if even just to help out newcomers.
> 
> Because the packfile to send the user is created on the fly, making a
> clone fully resumable is tricky (a second clone may get an equivalent
> but slightly different pack due to new objects entering the repo, or
> even raciness between threads).
> 
> One strategy people have worked on is for servers to point clients at
> static packfiles (which _do_ remain byte-for-byte identical, and can be
> resumed) to get some of the objects. But it requires some scheme on the
> server side to decide when and how to create those packfiles. So while
> there is support inside Git itself for this idea (both on the server and
> client side), I don't know of any servers where it is in active use.

At GitLab, we have started to roll out use of bundle URIs so that we can pregenerate them and thus reduce load. The next step to evaluate in this context is whether we can easily reuse that infrastructure to eventually enable resumable clones via such bundle URIs. I assume that it cannot be that hard to make this work.

That of course wouldn't be a perfect solution, as the clone can only be resumed as long as such a pregenerated bundle continues to exist on the server. But it should still be way better compared to the status quo.

Patrick