Re: [PATCH v3 2/3] sparse-checkout: custom tab completion

From: Elijah Newren <newren@gmail.com>
Date: Jan 18, 2022, 23:30 UTC
Message-ID: <CABPp-BEwMAPHGt5xD9jDU58grbrAqCdqNY9Nh8UJGLKuLbArXQ@mail.gmail.com>
In-Reply-To: <20220118222234.GB2632@szeder.dev>

On Tue, Jan 18, 2022 at 2:22 PM SZEDER Gábor <szeder.dev@gmail.com> wrote:

Show 5 quoted lines

>
> On Sat, Jan 15, 2022 at 05:03:24PM -0800, Elijah Newren wrote:
> > On Sat, Jan 15, 2022 at 1:57 AM SZEDER Gábor <szeder.dev@gmail.com> wrote:
> > >
> > > On Mon, Jan 10, 2022 at 06:59:51PM +0000, Lessley Dennington via GitGitGadget wrote:

...

Show 11 quoted lines

> > > > 4. A list of directories (but not files) is provided when users enter git
> > > > sparse-checkout add <TAB> or git sparse-checkout set <TAB>.
> > >
> > > Why limit completion only to directories?  Both of those subcommands
> > > accept files,
> >
> > Discussed in part at [1], but let me give a more detailed answer.
>
> It was a semi-rhetorical question.  Whether the reasons for expluding
> files are sound or not, it should be convincingly justified in the
> commit message.

Fair enough.

...

Show 13 quoted lines

> I used the following command to create a sparse-checkout from
> linux.git to build and play with 'perf':
>
>   git sparse-checkout set tools/perf/ tools/scripts/ tools/build/ tools/include/ tools/arch/x86/ tools/lib/ /.gitignore /.gitattributes
>
> Including the top-level '.gitignore' and '.gitattributes' was
> important, becase those ignore object files and specify userdiff.
> Now, I wouldn't mind having other files present in the top-level
> directory, because there are only a handful of files there.  However,
> being able to specify just those two files to 'git sparse-checkout'
> was great, because I didn't even have to think about what wildcard
> pattern to use, and what negative pattern to use to exclude anything
> that might have been included recursively.

Yeah, we should really make cone mode the default, because then this simplifies to:

    git sparse-checkout set tools/perf/ tools/scripts/ tools/build/
tools/include/ tools/arch/x86/ tools/lib/

In cone-mode you automatically get the directories specified PLUS:

  * all files in the toplevel directory (you always get these in cone mode)
  * all files directly within a parent (or ancestor) of one of the
directories you specified (thus all files directly under tools/ and
tools/arch/).

> I don't remember having any performance issues with it, on the
> contrary, I do remember that Git suddenly became much faster that in
> the full worktree.

Cool, thanks for testing it out and reporting.

If I can be permitted to share a bit of my experience...

In this specific case, you did keep it to 8 patterns and the linux kernel "only" has ~60k files. So O(N*M) isn't too bad here...but experience at $DAYJOB is that even with only 60k files, the pattern list tends to grow and it's not clear to users why there are so many ugly pauses when "Git used to be fast". Users tend to not make the connection either between the sparsity patterns and the slowness for whatever reason, and when I'm asked questions about the slowness, I may have to check a few other possible causes before finally realizing that it's just "too many sparsity patterns".

At least, that's the way it was at $DAYJOB before I switched our internal "sparsify" tool to just use cone mode. Then that type of problem went away.

> So I'm fairly convinced that specifying files to sparse-checkout is a
> feature that can make users' life easier.  It certainly made my life
> easier.

Or we can make it even easier and faster by using cone mode. That really, really should be the default (as we've been talking about for a couple cycles now), and I think your email reinforces my belief in it. I've got a series ready after some other things merge down to next.

Show 7 quoted lines

> On a related note: I just noticed the leading slashes in '/.gitignore'
> and '/.gitattributes'.  __git_complete_index_file() is not ready for
> that, I'm afraid; but I don't think the proposed patches could deal
> with that, either (but I didn't actually try).
>
> It would be great if completion could cope with patterns starting with
> '/' and/or '!'.

Ah, very good point. The leading '/' is kind of critical here, otherwise you get all .gitignore files in every directory at every depth, so completing on files (as currently done) wouldn't really help at all in your case.

Since you brought up a new good point here, can I also mention two others not previously covered in this thread (i.e. besides scaling)?

* If we complete file and directory names in non-cone mode, and those
file and directory names contain a leading '#' or '!' or contain
anywhere in their names a '*', '?', '\', '[', or ']', then the users
may be in for a very nasty surprise.  (Which perhaps suggests that in
non-cone mode, we shouldn't provide file OR directory completions for
set/add?)

* sparse-checkout currently ignores prefix[1], causing file and
directory completions to be wrong if the user is running from a
subdirectory.  This is just a bug in cone mode, though I'm not sure if
it's bug or feature in non-cone mode[2].  A decision about what it is
might have a bearing on what kinds of completions make sense in
non-cone mode (and might reinforce files/directories, or suggest
something else.).

[1] https://lore.kernel.org/git/29f0410e-6dfa-2e86-394d-b1fb735e7608@gmail.com/ [2] https://lore.kernel.org/git/CABPp-BH5woi6KY7OBpnsS-M2EmgLHii9zs8rSwrgcPFkOAvn_A@mail.gmail.com/