git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [git-users] How hard would it be to implement sparse fetching/pulling?

From
JHJeff Hostetler <git@jeffhostetler.com>
Date
Nov 30, 2017, 15:48 UTC
Message-ID
<11be7135-29a4-5f4a-2d61-ffe16a965cd5@jeffhostetler.com>
In-Reply-To
<20171130081232.7hk73s5d62gmpqdz@tigra>
On 11/30/2017 3:12 AM, Konstantin Khomoutov wrote:
Show 41 quoted lines
> On Wed, Nov 29, 2017 at 06:42:54PM -0800, vit via Git for human beings wrote:
> 
>> I'm looking for ways to improve fetch/pull/clone time for large git
>> (mono)repositories with unrelated source trees (that span across multiple
>> services).
>> I've found sparse checkout approach appealing and helpful for most of
>> client-side operations (e.g. status, reset, commit, add etc)
>> The problem is that there is no feature like sparse fetch/pull in git, this
>> means that ALL objects in unrelated trees are always fetched.
>> It takes a lot of time for large repositories and results in some practical
>> scalability limits for git.
>> This forced some large companies like Facebook and Google to move to
>> Mercurial as they were unable to improve client-side experience with git
>> and Microsoft has developed GVFS which seems to be a step back to CVCS
>> world.
> [...]
> 
> (To anyone interested, there's a cross-post to the main Git list which
> Vitaly failed to mention: [1]. I think it could spark some interesting
> discussion.)
> 
> As to the essence of the question, I think you blame GVFS for no real
> reason. While Microsoft is being Microsoft — their implementation of
> GVFS is written in .NET and *requires* Windows 10 (this one is beyond
> me), it's based on an open protocol [2] which basically assumes the
> presence of a RESTful HTTP endpoint at the "Git server side" and
> apparently designed to work well with the repository format the current
> stock Git uses which makes it implementable on both sides by anyone
> interested.
> 
> The second hint I have is that the idea of fetching data lazily
> is being circulated among the Git developers for some time already, and
> something is really being done in this venue so you could check and see
> what's there [3, 4] and maybe trial it and help out those who works on this
> stuff.
> 
> 1. https://public-inbox.org/git/CANxXvsMbpBOSRKaAi8iVUikfxtQp=kofZ60N0pHXs+R+q1k3_Q@mail.gmail.com/
> 2. https://github.com/Microsoft/GVFS/blob/master/Protocol.md
> 3. https://public-inbox.org/git/?q=lazy+fetch
> 4. https://public-inbox.org/git/?q=partial+clone
> 

For completeness with the git-users mailing list. Here is info on the work-in-progress for this feature.

https://public-inbox.org/git/e2d5470b-9252-07b4-f3cf-57076d103a17@jeffhostetler.com/
Jeff
Previous: Konstantin Khomoutov
Message 2 of 2 in “Re: [git-users] How hard would it be to implement sparse fetching/pulling?”
  1. Konstantin KhomoutovNov 30, 2017
  2. Jeff HostetlerNov 30, 2017

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.