Re: [PATCHv8 0/5] pathspec magic extension to search for attributes

From: Duy Nguyen <pclouds@gmail.com>
Date: May 24, 2016, 02:00 UTC
Message-ID: <CACsJy8CqxYVJt=311xmm3+js_1vx01by4=SH5_0fNw0t2bEzvw@mail.gmail.com>
In-Reply-To: <CAGZ79kZPPdUK1=GB6fnHP0cJyoM_VYmDj3wizApN5iHdCQCUyg@mail.gmail.com>

On Tue, May 24, 2016 at 1:49 AM, Stefan Beller <sbeller@google.com> wrote:

Show 14 quoted lines

> On Sun, May 22, 2016 at 4:45 AM, Duy Nguyen <pclouds@gmail.com> wrote:
>> On Fri, May 20, 2016 at 4:25 AM, Stefan Beller <sbeller@google.com> wrote:
>>>> My take is to pretend sparse checkout does not exist at all and then
>>>> go from there ;-)
>>
>> Hehe.. shameless plug, narrow checkout [1] should be its great
>> successor where everything is done right (famous last words). Maybe I
>> can convince Stefan to finish that off then I'll finally bring narrow
>> clone!
>
> I started reviewing that (see separate email).
>
> So conceptually the narrow checkout is "sparse checkout just for
> directories" IIUC.

Yep.

> A submodule is treated as a files (sometimes, e.g. diff) and sometimes
> as a directory (e.g. in the working tree)

It's the same here. Submodules have a special mode in the index (S_IFDIR + S_IF<something>) while dirs just have S_IFDIR. It's up to the index user to treat it as a "file" or a directory (maybe only unpack-trees)

Show 24 quoted lines

>>> Using these pathspec attrs are a good example for why we would
>>> want to make it more generic. Imagine a future, where more attributes
>>> can be codified into pathspecs and this is one of the possibilities:
>>>
>>>     git clone --sparse=":(exclude,size>5MB,binary)
>>>
>>> which would not checkout files that are large binary files. We could
>>> also extend the protocol (v2 with the capabilities in client speaks first)
>>> to transmit such a requirement to the server.
>>
>> I think you need narrow clone there ;-) It's the first step to have a
>> clone with missing directories. I think then we can extend it further
>> to exclude (large) files.
>
> But if we only exclude some files we are not having a binary
> decision for a directory, so narrow checkout doesn't work here?
>
>     git clone --sparse=":(exclude,size>5MB,binary)
>
> would maybe lead to have
>   dir/path.c #included
>   dir/binary-asset.img # excluded
>
> so it is more a sparse thing than a narrow thing?

I wrote two paragraphs, only to realized that you were right. In the end we just need a way to exchange "sparse" (not narrow) information about this. Narrow clone may need to exchange more side information (for git-replace) but I don't see how sparse files fit exactly in this whole exchange yet.

Show 19 quoted lines

>>> Why is sparseness considered bad?
>>
>> It does not scale well when the worktree gets bigger. It can be slow
>> (but this is just a technical problem I haven't spent time on fixing).
>> And it does not enable narrow clone (not with lots of hacks, I think I
>> did just that in some early iterations).
>>
>>> If I wanted to just do the submodule only thing, this would be a smaller
>>> series, I would guess.
>>
>> No no no. Do more. Start with narrow checkout. I'll help ;-)
>
> Thanks for the encouragement!
> How is the interface going to look like for the narrow checkout/clone ?
> If the UI is supposed to be very similar, we can merge the two concepts
> and make it one thing.
>
> But if the UI is different enough, we may want to keep it separate as
> it is not as confusing to the user as we'd think?

For narrow clone, I imagine you need to specify what directories, say with --narrow-path option or simply --path, to clone, and that's it. In future you may add --path-filter option that does "exclude,size>5MB,binary" above.

For narrow checkout, we have two more operations: fold and unfold directories. At plumbing level, maybe update-index can support these operations explicitly. At porcelain level, checkout probably learns about --narrow-path and submodule can make use of it if it wants.

Show 9 quoted lines

> What I imagine UI-wise, see
> http://thread.gmane.org/gmane.comp.version-control.git/295221
>
>
> Thanks,
> Stefan
>
>> --
>> Duy

-- 
Duy