Re: [PATCH v4] sparse-checkout.txt: new document with sparse-checkout directions

> > The fact that files have been "vivified" is slightly ambiguous and
> > thus a bad way to define this set.  When git vivifies files, it'll
> > clear the SKIP_WORKTREE bit.  If an editor, or script external to git,
> > or something else restores the file, it will likely overlook that
> > detail.  We want vivified files to be part of the sparse specification
> > when interacting with the working tree regardless of whether the
> > SKIP_WORKTREE bit was correctly updated, so I defined it the way I did
> > to remove such ambiguity.  (I guess I should note that as per
> > af6a51875a ("repo_read_index: clear SKIP_WORKTREE bit from files
> > present in worktree", 2022-01-14), git will clear the SKIP_WORKTREE
> > bit for files present in the working copy as one of the first things
> > it does, but that could leave people wondering whether I meant the
> > SKIP_WORKTREE bit was set as of the time of git invocation.  So, I
> > explicitly call out files present in the working copy for which the
> > index entry has the SKIP_WORKTREE bit set, so folks know these files
> > are definitely included in the sparse specification.)
> >
>
> You are right: the definition of vivifying explicitly clears the
> SKIP_WORKTREE bit from the index, So the behavior described
> here is not vivifying, but very much like vivifying: clear the
> SKIP_WORKTREE bit from index_entry in memory instead of actually
> clearing it from the index on disk.

> > > I think we can add some examples to describe these terms.
> > >
> > > #!/bin/sh
> > >
> > > set -x
> > >
> > > rm -rf mono-repo
> > > git init mono-repo -b main
> > > (
> > >   cd mono-repo &&
> > >   mkdir p1 p2 &&
> > >   echo a >p1/a &&
> > >   echo b >p1/b &&
> > >   echo a >p2/a &&
> > >   echo b >p2/b &&
> > >   git add . &&
> > >   git commit -m ok &&
> > >   git sparse-checkout set p1 &&
> > >   git ls-files -t &&
> > >   echo a >>p1/a &&
> > >   echo b >>p1/b &&
> > >   mkdir p2 p3 &&
> > >   echo next >>p2/a &&
> > >   echo next >>p3/c &&
> > >   git add p3/c &&
>
> Here I forget a "--sparse"
>
> > >   # p2/a and p3/c vivify
> > >   git ls-files -t &&
> > >   # compare wortree/commit
> > >   git --no-pager diff HEAD --name-only
> > > )
> >
> > You've added a bunch of code with this example, but you have not said
> > what the output should be, so how exactly does this help describe the
> > terms?
> >
>
> We create a repo with two sub projects p1/ p2/, then set
> sparsity directory p1.
>
> First git ls-files -t outputs:
>
> H p1/a
> H p1/b
> S p2/a
> S p2/b
>
> It shows that index entries in p2 are skip-worktree.
> Then we restore p2/a in the working tree and create a
> new file p3/c and add it to the index.
>
> The second git ls-files -t output:
>
> H p1/a
> H p1/b
> H p2/a
> S p2/b
> H p3/c
>
> p2/a and p3/c are not in sparse patterns, but they are in
> sparse specification. It's like a special "vivifying".

> > > > +       * When modifying or showing results from the index, the sparse
> > > > +         specification is the set of files with a clear SKIP_WORKTREE bit
> > > > +         or that differ in the index from HEAD.
> > >
> > > #!/bin/sh
> > >
> > > set -x
> > >
> > > rm -rf mono-repo
> > > git init mono-repo -b main
> > > (
> > >   cd mono-repo &&
> > >   mkdir p1 p2 &&
> > >   echo a >p1/a &&
> > >   echo b >p1/b &&
> > >   echo a >p2/a &&
> > >   echo b >p2/b &&
> > >   git add . &&
> > >   git commit -m ok &&
> > >   git sparse-checkout set p1 &&
> > >   git update-index --chmod=+x p2/a &&
> > >   # compare commit/index
> > >   git --no-pager diff --cached --name-only
> > > )
> >
> > Same issue here; you haven't stated the expected output of these
> > commands, so I don't see how they help with the description at all.
> >
>
> Here only output p2/a:
>
> p2/a is out of sparse patterns, but this index entry mode has been
> changed compared to HEAD. So we should consider it as a part of
> sparse specification.

> > Perhaps it's worth noting why I think the sparse specification should
> > be extended when dealing with the index:
> >
> >   * "mergy" commands (merge, rebase, cherry-pick, am, revert) can
> > modify the index outside the sparsity patterns, without creating a
> > commit.
> >   * `git commit` (or `rebase --continue`, or whatever) will create a
> > commit from whatever staged versions of files there are
> >   => `git status` should show what is about to be committed
> >   => `git diff --cached --name-only` ought to be usable to show what
> > is to be committed
> >   => `git grep --cached ...` ought to be usable to search through what
> > is about to be committed
> >
> > See also https://lore.kernel.org/git/CABPp-BESkb=04vVnqTvZyeCa+7cymX7rosUW3rhtA02khMJKHA@mail.gmail.com/
> > (starting with the paragraph with "leery" in it), and the thread
> > starting there.  If the sparse specification is not expanded, users
> > will get some nasty surprises, and the only other alternative I can
> > think of to avoid such surprises would be making several commands
> > always run full tree.  Running full-tree with a non-default option to
> > run sparse forces behavior A folks into a "pick your poison"
> > situation, which is not nice.  Extending the sparse specification to
> > include files whose index entries do not match HEAD for index-related
> > operations provides the nice middle ground that avoids such usability
> > problems while also allowing users to avoid operating on a full tree.
> >
>
> I can understand the reason why we need to extend sparse specification:
> index often needs to handle files that are not in the sparse pattern.

> > > A** : Some users are _only_ interested in the sparse portion of the repo,
> > > but they want to download all the blobs in it to avoid some unnecessary
> > > network connections afterwards.
> >
> > Here you just repeated `A*` but relabelled it as `A**`.  Yes, this one
> > is explicitly tied to partial clone behavior.
>
> Ah,  `A*` part say “so things like `git log -S${SEARCH_TERM} -p`
> or `git grep ${SEARCH_TERM} OLDREV ` would need to be prepared to provide
> partial results that depend on what happens to have been downloaded."
>
> So I think it's probably a lot like the behavior after a shallow
> clone: git log -p or other
> git commands returning partial results.

> >
> > [...]
> > > > +    The fact that files can move between the 'tracked' and 'untracked'
> > > > +    categories means some commands will have to treat untracked files
> > > > +    differently.  But if we have to treat untracked files differently,
> > > > +    then additional commands may also need changes:
> > > > +
> > > > +      * status
> > > > +      * clean
> > > > +
> > >
> > > I'm a bit worried about git status, because it's used in many shells
> > > (e.g. zsh) i
> > > in the git prompt function. Its default behavior is restricted, otherwise users
> > > may get blocked when they use zsh to cd to that directory. I don't know how
> > > to reproduce this problem (since the scenario is built on checkout to a local
> > > unborn branch).
> >
> > Could you elaborate?  I'm not sure if you are talking about an
> > existing problem that you are worried about being exacerbated, or a
> > hypothetical problem that could occur with changes.  Further, your
> > wording is so vague about the problem, that I have no idea what its
> > nature is or whether any changes to status would even possibly have
> > any bearing on it.  But the suggested changes to git status are
> > simply:
> >
>
> I just might have caused this in one particular case. So it's not very
> important at the moment. But it's worth noting that many shells, IDEs’
> git plugins
> may also need to understand sparse-checkout properly, otherwise it can
>  cause some usability problems.