Re: [GSoC23] Working on project Idea from SOC 2011

From: Derrick Stolee <derrickstolee@github.com>
Date: Mar 9, 2023, 14:26 UTC
Message-ID: <7097d1d6-00a1-2a82-1923-610d41f4053f@github.com>
In-Reply-To: <CAABMjtGXGZtUnU+8KgEccNeLXRJmWnE5f24BMG8ysbZKfT-ctQ@mail.gmail.com>

On 3/9/23 7:07 AM, Khalid Masum wrote:

Show 8 quoted lines

> There is this SOC 2011 idea named "Resumable clone" here:
> 
> https://archive.kernel.org/oldwiki/git.wiki.kernel.org/index.php/SoC2011Ideas.html
> 
> ...
> Currently cloning a remote repository has to be done in one session.
> If the process fails or is aborted for any reason any already downloaded
> data is lost and one has to start from scratch.

> Goal: Allow Git to resume a cloning process that
> has been aborted for any reason.
> Languages: C

"for any reason" is going to be pretty difficult.
 
One direction that is relatively new in the Git project
(much newer than that project idea) is the bundle URI
standard, allowed by "git clone --bundle-uri=<X>". It
helps bootstrap clones by fetching bundle files and using
them to populate the object directory before finishing
the clone with an incremental fetch to the origin server.

Since the bundles are expected to be precomputed files, it is much easier to use standard HTTP range queries to download only the "missing" portion of the file from the bundle server.

I think one thing that would need to change on the Git client is the location of the temporary file being used to store the bundle as it is downloaded. It currently uses a random name, but if the name was a hash of the URL, then it would be predictable and could restart the download if the 'git clone' process was halted for any reason. (Resuming a download due to a network error noticed in-process is possibly simpler.)

This might be a more focused approach that is more likely to have progress in a GSoC project.

That said, I don't have the capacity to be a mentor, but I thought it worth mentioning this variant of the project.

Thanks, -Stolee