Re: [PATCHv8 0/5] pathspec magic extension to search for attributes

From: Stefan Beller <sbeller@google.com>
Date: May 20, 2016, 18:12 UTC
Message-ID: <CAGZ79kaBjHXGNxR_6WNWMtmBm0H_QCpHGciM7B1yY1yJ3S23wA@mail.gmail.com>
In-Reply-To: <xmqq1t4wfyv4.fsf@gitster.mtv.corp.google.com>

On Fri, May 20, 2016 at 10:00 AM, Junio C Hamano <gitster@pobox.com> wrote:

Show 11 quoted lines

> Stefan Beller <sbeller@google.com> writes:
>
>> Right. But upon finding the new name for clone, I wondered why
>> this has to be submodule specific. The attr pathspecs are also working
>> with any other files. So if you don't use submodules, I think it would be
>> pretty cool to have a
>>
>>     git clone --sparse-checkout=Documentation/ ...
>
> It would be cool, but arent' "sparse" and the various existing
> status "submodule" has very different things?

Yes they are. In one of the various "submodule groups" series I proposed a "defaultGroup" which allows commands to ignore some submodules. That was conceptually the very same as a "sparse checkout, just for submodules", i.e. the submodule is initialized and has a directory as a place holder, but most commands ignore its existence.

We decided that was a bad thing, so now I think of a light weight "submodule.updateGroup" which holds a pathspec and is only used for "submodule update" commands that have no explicit pathspec given. (That setting would be set via "git clone --submodule-pathspec <pathspec>")

Show 6 quoted lines

>
>  - A submodule can be uninitialized, in which case you do get an empty
>    directory but you do not see .git in it.
>
>  - A path can be excluded by the sparse checkout mechanism, in which
>    case you do not get _anything_ in the filesystem.

Yes, but isn't that one of the minor issues?

Show 17 quoted lines

>
> So "git clone --sparse-exclude=Documentation/" that does not waste
> diskspace for Documentation/ directory may be an interesting thing
> to have, and "git clone --sparse-exclude=submodule-dir/" that does
> not even create submodule-dir/ directory may also be, but the latter
> is quite different from a submodule that is not initialied in a
> superproject that does not use any "sparse" mechanism.
>
> Besides, I think (improved) submodule mechanism would be a good way
> forward for scalability, and "sparse" hack is not (primarily because
> it still populates the index fully with 5 million entries even when
> your attention is narrowed only to a handful of directories with
> 2000 leaf entries; this misdesign requires other ugly hacks to be
> piled on, like untracked cache and split index).
>
> I do not think we want "submodule" to be tied to and dependent on
> the latter.

Ok I just wanted to probe how much resistance I get here as an indicator of how much more work that would be.

Besides I think (improved) sparse mechanism would be a good way to not confuse users between submodule scalability and single repo scalability. ;)

We don't have to keep 5 million things in the index there, but we can stop on the tree/directory level, i.e. if a whole directory is excluded That's all we'd need to keep a record of, no?

As a user I'd prefer to be exposed to as few concepts as possible, and adding yet another concept of sparseness is not a good thing IMHO, so I'll try to keep it simple there.

Thanks, Stefan