Re: [PATCH] connected.c: reprepare packs for corner cases

From: Junio C Hamano <gitster@pobox.com>
Date: Mar 12, 2020, 20:42 UTC
Message-ID: <xmqqzhcle23x.fsf@gitster.c.googlers.com>
In-Reply-To: <pull.579.git.1584027403779.gitgitgadget@gmail.com>

"Derrick Stolee via GitGitGadget" <gitgitgadget@gmail.com> writes:

Show 32 quoted lines

> From: Derrick Stolee <dstolee@microsoft.com>
>
> While updating the microsoft/git fork on top of v2.26.0-rc0 and
> consuming that build into Scalar, I noticed a corner case bug around
> partial clone.
>
> The "scalar clone" command can create a Git repository with the
> proper config for using partial clone with the "blob:none" filter.
> Instead of calling "git clone", it runs "git init" then sets a few
> more config values before running "git fetch".
>
> In our builds on v2.26.0-rc0, we noticed that our "git fetch"
> command was failing with
>
>   error: https://github.com/microsoft/scalar did not send all necessary objects
>
> This does not happen if you copy the config file from a repository
> created by "git clone --filter=blob:none <url>", but it does happen
> when adding the config option "core.logAllRefUpdates = true".
>
> By debugging, I was able to see that the loop inside
> check_connnected() that checks if all refs are contained in
> promisor packs actually did not have any packfiles in the packed_git
> list.
> I'm not sure what corner-case issues caused this config option to
> prevent the reprepare_packed_git() from being called at the proper
> spot during the fetch operation. Even worse, I have failed to create
> a test case to prevent a regression.
>
> Placing a reprepare_packed_git() call inside chck_connected() before
> looping through the packed_git list seems like the safest way to
> avoid this issue in the future.

Hmmm. I am not sure if I am convinced that check_connected() is the best place to do this. Do we know the place that adds a new pack to the repository, yet forgets to add it to the packed-git list, that caused the failure you were observing? Doing this change, without describing the answer to the question in the log message, makes it smell rather like a random hack than a designed solution to me.

If lazy fetching of objects happen in multiple fetches before a single check_connected() sweeps them to check for connectivity, then perhaps the lazy fetching codepath needs to remember the fact that it added a new pack that is still not known to the packed-git list (or just add it immediately, without having to scan at all), and check_connected() would need to rescan only when there is at least one such new pack? That way, you do not have to penalize normal callers of check_connected() that do not use lazy fetches at all, right?

Thanks.