Re: [PATCH 00/27] [RFC] Sparse Index

From: Derrick Stolee <stolee@gmail.com>
Date: Jan 25, 2021, 21:18 UTC
Message-ID: <ef05a917-e3ec-7099-bc44-e53e31d63a8e@gmail.com>
In-Reply-To: <xmqqh7n4ydyw.fsf@gitster.c.googlers.com>

On 1/25/21 3:10 PM, Junio C Hamano wrote:

Show 28 quoted lines

> "Derrick Stolee via GitGitGadget" <gitgitgadget@gmail.com> writes:
> 
>> This RFC proposes an update to the index formats to allow "sparse directory
>> entries". These entries correspond to directories that are completely
>> excluded from the sparse checkout definition. We can detect that a directory
>> is excluded when using "cone mode" patterns.
> 
> Yay.
> 
>> Since having directory entries is a radical departure from the existing
>> index format, a new extension "extensions.sparseIndex" is added. Using a
>> sparse index should cause incompatible tools to fail because they do not
>> understand this extension.
> 
> Safety is good, but because the index is purely a local matter, we
> do not have to be so careful as updating the network protocols or
> pack/object formats.
> 
> I think the use of "extensions.*" mechanism to render the repository
> that uses the new feature unusable by older Git is safe enough, but
> it may be too draconian.  For example, when things go wrong, don't
> you want to "fetch"/"clone" from it into another repository to first
> save the objects and refs?  You do not need a version of the index
> file you understand in order to do that.
> 
> The index format has a mechanism to make older versions of Git bail
> when it encounters a file that uses newer feature that they do not
> understand.  Perhaps using it is sufficient instead?

There are interesting subtleties with the differences between index formats 2, 3, and 4 that are worth keeping around. Perhaps the extension could be a mechanism for allowing sparse directories in those versions, but then a future "index version 5" includes sparse directories without the extension.

I could spend time working on such an index v5 in parallel with the updates to Git commands to make them sparse aware. The logic from patches 1-14 in this series will be required before that could begin.

Thanks, -Stolee