Re: 'git sparse-checkout add' hangs if local repo is using alternative objects store

From: Derrick Stolee <derrickstolee@github.com>
Date: Jul 15, 2022, 03:06 UTC
Message-ID: <89310a11-b8cd-b643-5d46-c6d273f66b49@github.com>
In-Reply-To: <CAKSRnEzYYQ1EOmbUvafQvzJUNW8kojLuD+BbakxU09JSm=niHA@mail.gmail.com>

On 7/12/2022 9:58 AM, Dian Xu wrote:

Show 5 quoted lines

> Reproduction steps:
>     1. Local repo setup:
>         git init
>         git remote add origin <repo_url>
>         echo <alternative_objects_store> >! .git/objects/info/alternates

Is this alternative object store on the local disk? Or is it across a network connection?

Show 8 quoted lines

>         git config --unset-all remote.origin.fetch
>         git config --unset-all remote.origin.tagopt
>         git config --add remote.origin.tagopt --no-tags
>         git config --add remote.origin.fetch
> +refs/heads/main:refs/remotes/origin/main
>         git config core.sparsecheckout true
>         git config core.sparsecheckoutcone true
>         git fetch

This is interesting that you are fetching without having any existing refs, only the alternate store. Are you using something like the core.alternateRefsCommand config option?

Otherwise, I would expect this "git fetch" to act as if it was a normal clone (unless of course the alternate already has the current tip of 'main'). Maybe fetch negotiation helps find some common commits that exist in the alternate, but the client is useless in that discovery without refs.

Show 8 quoted lines

>     2. Sparse checkout <dir1>:
>         git sparse-checkout set <dir1>
>         git checkout -b tmp1 origin/main
> 
>     3. Add another <dir2> via sparse-checkout, which hangs:
>         git sparse-checkout add <dir2>
>             09:39:11.169116 read-cache.c:2451       performance:
> 6.326754679 s:  read cache .git/index

Could you re-run this with GIT_TRACE2_PERF=1 so we can see even more details about how this works?

Show 10 quoted lines

> Repo and dir info:
>     <repo_url> is a large repo, containing large number of directories
> and 2 million files;
>     Local repo is using <alternative_objects_store>, which is working
> by verifying: git count-objects -vH;
>     <dir1> used here is one level deep, <dir2> used here is 5 level deep
> 
> Observations:
>     1. Even 'sparse-checkout add' hangs, scm code from <dir2> was
> actually downloaded pretty fast

I'm interested to learn what you mean by "downloaded" here.

>     2. If waited out 'sparse-checkout add', git seems start to
> download large number of objects from alternative objects store into
> local .git/objects

Hm. Adding the objects into .git/objects is strange. It should recognize that the objects are in the alternate and not write them to the local repo (this is independent of the sparse-checkout builtin).

Thanks, -Stolee