threads / patch / 55201

v2, 7 partsadd: include magic part of pathspec on --refresh error

Subject: [PATCH v2 1/7] add: include magic part of pathspec on --refresh error

## tl;dr

19 messages between Feb 24, 2021 and Mar 4, 2021. Diffs are folded; open one to read it.

replies: 18people: 3as markdown or json

Matheus Tavares· Feb 24, 2021, 04:05 UTC · lore

[PATCH v2 0/7] add/rm: honor sparse checkout and warn on sparse paths

Make `rm` honor sparse checkouts, and make both `rm` and `add` warn about pathspecs that only match sparse entries.

This is now based on mt/add-chmod-fixes.
Main changes since RFC:

Ejected "add --chmod: don't update index when --dry-run is used", which is now part of mt/add-chmod-fixes.

Patch 2:
- Fixed style problems.
- Fixed comment on setup_sparse_entry() about argument being passed to
  printf.
- Standardized on 'sparse entries' on test names.
- Added test for 'git add .', and set up a .gitignore to run this and
  the 'git add -A' test, to avoid smudging the index.
- Added test for --dry-run.
- Modified `git add --refresh` test to use `git ls-files --debug`
  instead of diff-files.
Added patch 5
Patch 6:
- Improved commit message.
- Used the flag added in patch 5 instead of directly changing
  refresh_index(), so that new users of seen[] can still have the
  complete match array.
- Added test for --ignore-missing.
- Added test where the pathspec matches both sparse and dense entries.
- Changed 'git add .' behavior where . only contains sparse entries. [1]
Patch 7:
- Moved rm-sparse-checkout tests to their own file. They need a repo
  with a different setup than the one t3600 has, and the subshells (or
  git -C ...) were making the tests too hard to read.
- Added tests for -f, --dry-run and --ignore-unmatch.
- Added test where the pathspec matches both sparse and dense entries.
- Also test if the index entry is really not removed (besides testing
  `git rm` exit code and stderr).
- Passed "a" instead of "/a" to git sparse-checkout to avoid MSYS path
  conversions on Windows that would make some of the new tests fail.
[1]: Although the sparse entries warning is based on the ignored files
     warning, they have some differences. One of them is that running
     'git add dir' where 'dir/somefile' is ignored does not trigger a
     warning, whereas if 'somefile' is a sparse entry, we warn. In
     this sense, I think it's more coherent to also warn on 'git add .'
     when . only has sparse entries. This is also consistent with what
     'git rm -r .' does in the same scenario, after this series.
Matheus Tavares (7):
  add: include magic part of pathspec on --refresh error
  t3705: add tests for `git add` in sparse checkouts
  add: make --chmod and --renormalize honor sparse checkouts
  pathspec: allow to ignore SKIP_WORKTREE entries on index matching
  refresh_index(): add REFRESH_DONT_MARK_SPARSE_MATCHES flag
  add: warn when pathspec only matches SKIP_WORKTREE entries
  rm: honor sparse checkout patterns
 Documentation/config/advice.txt  |   4 +
 Documentation/git-rm.txt         |   4 +-
 advice.c                         |  19 ++++
 advice.h                         |   4 +
 builtin/add.c                    |  75 ++++++++++++---
 builtin/check-ignore.c           |   2 +-
 builtin/rm.c                     |  35 ++++---
 cache.h                          |  15 +--
 pathspec.c                       |  25 ++++-
 pathspec.h                       |  13 ++-
 read-cache.c                     |   5 +-
 t/t3602-rm-sparse-checkout.sh    |  76 +++++++++++++++
 t/t3700-add.sh                   |   6 ++
 t/t3705-add-sparse-checkout.sh   | 155 +++++++++++++++++++++++++++++++
 t/t7011-skip-worktree-reading.sh |   5 -
 t/t7012-skip-worktree-writing.sh |  19 ----
 16 files changed, 398 insertions(+), 64 deletions(-)
 create mode 100755 t/t3602-rm-sparse-checkout.sh
 create mode 100755 t/t3705-add-sparse-checkout.sh
Range-diff against v1:
1:  5612c57977 < -:  ---------- add --chmod: don't update index when --dry-run is used
2:  5a06223007 = 1:  2831fd5744 add: include magic part of pathspec on --refresh error
3:  4fc81a83b1 ! 2:  72b8787018 t3705: add tests for `git add` in sparse checkouts
    @@ t/t3705-add-sparse-checkout.sh (new)
     +
     +SPARSE_ENTRY_BLOB=""
     +
    -+# Optionally take a string for the entry's contents
    -+setup_sparse_entry()
    -+{
    -+	if test -f sparse_entry
    -+	then
    -+		rm sparse_entry
    -+	fi &&
    ++# Optionally take a printf format string to write to the sparse_entry file
    ++setup_sparse_entry () {
    ++	rm -f sparse_entry &&
     +	git update-index --force-remove sparse_entry &&
     +
    -+	if test "$#" -eq 1
    ++	if test $# -eq 1
     +	then
     +		printf "$1" >sparse_entry
     +	else
    -+		printf "" >sparse_entry
    ++		>sparse_entry
     +	fi &&
     +	git add sparse_entry &&
     +	git update-index --skip-worktree sparse_entry &&
     +	SPARSE_ENTRY_BLOB=$(git rev-parse :sparse_entry)
     +}
     +
    -+test_sparse_entry_unchanged() {
    ++test_sparse_entry_unchanged () {
     +	echo "100644 $SPARSE_ENTRY_BLOB 0	sparse_entry" >expected &&
     +	git ls-files --stage sparse_entry >actual &&
     +	test_cmp expected actual
     +}
     +
    -+test_expect_success "git add does not remove SKIP_WORKTREE entries" '
    ++setup_gitignore () {
    ++	test_when_finished rm -f .gitignore &&
    ++	cat >.gitignore <<-EOF
    ++	*
    ++	!/sparse_entry
    ++	EOF
    ++}
    ++
    ++test_expect_success 'git add does not remove sparse entries' '
     +	setup_sparse_entry &&
     +	rm sparse_entry &&
     +	git add sparse_entry &&
     +	test_sparse_entry_unchanged
     +'
     +
    -+test_expect_success "git add -A does not remove SKIP_WORKTREE entries" '
    ++test_expect_success 'git add -A does not remove sparse entries' '
     +	setup_sparse_entry &&
     +	rm sparse_entry &&
    ++	setup_gitignore &&
     +	git add -A &&
     +	test_sparse_entry_unchanged
     +'
     +
    -+for opt in "" -f -u --ignore-removal
    -+do
    -+	if test -n "$opt"
    -+	then
    -+		opt=" $opt"
    -+	fi
    ++test_expect_success 'git add . does not remove sparse entries' '
    ++	setup_sparse_entry &&
    ++	rm sparse_entry &&
    ++	setup_gitignore &&
    ++	git add . &&
    ++	test_sparse_entry_unchanged
    ++'
     +
    -+	test_expect_success "git add$opt does not update SKIP_WORKTREE entries" '
    ++for opt in "" -f -u --ignore-removal --dry-run
    ++do
    ++	test_expect_success "git add${opt:+ $opt} does not update sparse entries" '
     +		setup_sparse_entry &&
     +		echo modified >sparse_entry &&
     +		git add $opt sparse_entry &&
    @@ t/t3705-add-sparse-checkout.sh (new)
     +	'
     +done
     +
    -+test_expect_success 'git add --refresh does not update SKIP_WORKTREE entries' '
    ++test_expect_success 'git add --refresh does not update sparse entries' '
     +	setup_sparse_entry &&
    ++	git ls-files --debug sparse_entry | grep mtime >before &&
     +	test-tool chmtime -60 sparse_entry &&
     +	git add --refresh sparse_entry &&
    -+
    -+	# We must unset the SKIP_WORKTREE bit, otherwise
    -+	# git diff-files would skip examining the file
    -+	git update-index --no-skip-worktree sparse_entry &&
    -+
    -+	echo sparse_entry >expected &&
    -+	git diff-files --name-only sparse_entry >actual &&
    -+	test_cmp actual expected
    ++	git ls-files --debug sparse_entry | grep mtime >after &&
    ++	test_cmp before after
     +'
     +
    -+test_expect_failure 'git add --chmod does not update SKIP_WORKTREE entries' '
    ++test_expect_failure 'git add --chmod does not update sparse entries' '
     +	setup_sparse_entry &&
     +	git add --chmod=+x sparse_entry &&
    -+	test_sparse_entry_unchanged
    ++	test_sparse_entry_unchanged &&
    ++	! test -x sparse_entry
     +'
     +
    -+test_expect_failure 'git add --renormalize does not update SKIP_WORKTREE entries' '
    ++test_expect_failure 'git add --renormalize does not update sparse entries' '
     +	test_config core.autocrlf false &&
     +	setup_sparse_entry "LINEONE\r\nLINETWO\r\n" &&
     +	echo "sparse_entry text=auto" >.gitattributes &&
4:  ccb05cc3d7 ! 3:  0f03adf241 add: make --chmod and --renormalize honor sparse checkouts
    @@ Commit message
         Signed-off-by: Matheus Tavares <matheus.bernardino@usp.br>
     
      ## builtin/add.c ##
    -@@ builtin/add.c: static void chmod_pathspec(struct pathspec *pathspec, char flip, int show_only)
    - 	for (i = 0; i < active_nr; i++) {
    +@@ builtin/add.c: static int chmod_pathspec(struct pathspec *pathspec, char flip, int show_only)
      		struct cache_entry *ce = active_cache[i];
    + 		int err;
      
     +		if (ce_skip_worktree(ce))
     +			continue;
    @@ builtin/add.c: static int renormalize_tracked_files(const struct pathspec *paths
      		if (!S_ISREG(ce->ce_mode) && !S_ISLNK(ce->ce_mode))
     
      ## t/t3705-add-sparse-checkout.sh ##
    -@@ t/t3705-add-sparse-checkout.sh: test_expect_success 'git add --refresh does not update SKIP_WORKTREE entries' '
    - 	test_cmp actual expected
    +@@ t/t3705-add-sparse-checkout.sh: test_expect_success 'git add --refresh does not update sparse entries' '
    + 	test_cmp before after
      '
      
    --test_expect_failure 'git add --chmod does not update SKIP_WORKTREE entries' '
    -+test_expect_success 'git add --chmod does not update SKIP_WORKTREE entries' '
    +-test_expect_failure 'git add --chmod does not update sparse entries' '
    ++test_expect_success 'git add --chmod does not update sparse entries' '
      	setup_sparse_entry &&
      	git add --chmod=+x sparse_entry &&
    - 	test_sparse_entry_unchanged
    + 	test_sparse_entry_unchanged &&
    + 	! test -x sparse_entry
      '
      
    --test_expect_failure 'git add --renormalize does not update SKIP_WORKTREE entries' '
    -+test_expect_success 'git add --renormalize does not update SKIP_WORKTREE entries' '
    +-test_expect_failure 'git add --renormalize does not update sparse entries' '
    ++test_expect_success 'git add --renormalize does not update sparse entries' '
      	test_config core.autocrlf false &&
      	setup_sparse_entry "LINEONE\r\nLINETWO\r\n" &&
      	echo "sparse_entry text=auto" >.gitattributes &&
5:  00786cba82 = 4:  a8a8af22a0 pathspec: allow to ignore SKIP_WORKTREE entries on index matching
-:  ---------- > 5:  d65b214dd1 refresh_index(): add REFRESH_DONT_MARK_SPARSE_MATCHES flag
6:  ce74a60e0d ! 6:  24e889ca9b add: warn when pathspec only matches SKIP_WORKTREE entries
    @@ Commit message
         add: warn when pathspec only matches SKIP_WORKTREE entries
     
         `git add` already refrains from updating SKIP_WORKTREE entries, but it
    -    silently succeeds when a pathspec only matches these entries. Instead,
    -    let's warn the user and display a hint on how to update these entries.
    +    silently exits with zero code when a pathspec only matches these
    +    entries. Instead, let's warn the user and display a hint on how to
    +    update these entries.
     
         Note that the warning is only shown if the pathspec matches no untracked
         paths in the working tree and only matches index entries with the
    -    SKIP_WORKTREE bit set. Performance-wise, this patch doesn't change the
    -    number of ce_path_match() calls in the worst case scenario (because we
    -    still need to check the sparse entries for the warning). But in the
    -    general case, it avoids unnecessarily calling this function for each
    -    SKIP_WORKTREE entry.
    -
    -    A warning message was chosen over erroring out right away to reproduce
    -    the same behavior `add` already exhibits with ignored files. This also
    -    allow users to continue their workflow without having to invoke `add`
    -    again with only the matching pathspecs, as the matched files will have
    -    already been added.
    -
    -    Note: refresh_index() was changed to only mark matches with
    -    no-SKIP-WORKTREE entries in the `seen` output parameter. This is exactly
    -    the behavior we want for `add`, and only `add` calls this function with
    -    a non-NULL `seen` pointer. So the change brings no side effect on
    -    other callers.
    +    SKIP_WORKTREE bit set. A warning message was chosen over erroring out
    +    right away to reproduce the same behavior `add` already exhibits with
    +    ignored files. This also allow users to continue their workflow without
    +    having to invoke `add` again with only the matching pathspecs, as the
    +    matched files will have already been added.
     
         Signed-off-by: Matheus Tavares <matheus.bernardino@usp.br>
     
    @@ builtin/add.c: static char *prune_directory(struct dir_struct *dir, struct paths
     +	int i, ret = 0;
     +	char *skip_worktree_seen = NULL;
     +	struct string_list only_match_skip_worktree = STRING_LIST_INIT_NODUP;
    ++	int flags = REFRESH_DONT_MARK_SPARSE_MATCHES |
    ++		    (verbose ? REFRESH_IN_PORCELAIN : REFRESH_QUIET);
      
      	seen = xcalloc(pathspec->nr, 1);
    - 	refresh_index(&the_index, verbose ? REFRESH_IN_PORCELAIN : REFRESH_QUIET,
    - 		      pathspec, seen, _("Unstaged changes after refreshing the index:"));
    +-	refresh_index(&the_index, verbose ? REFRESH_IN_PORCELAIN : REFRESH_QUIET,
    +-		      pathspec, seen, _("Unstaged changes after refreshing the index:"));
    ++	refresh_index(&the_index, flags, pathspec, seen,
    ++		      _("Unstaged changes after refreshing the index:"));
      	for (i = 0; i < pathspec->nr; i++) {
     -		if (!seen[i])
     -			die(_("pathspec '%s' did not match any files"),
    @@ builtin/add.c: int cmd_add(int argc, const char **argv, const char *prefix)
     -			    ((pathspec.items[i].magic &
     -			      (PATHSPEC_GLOB | PATHSPEC_ICASE)) ||
     -			     !file_exists(path))) {
    -+			if (seen[i] || !path[0])
    ++			if (seen[i])
     +				continue;
     +
     +			if (matches_skip_worktree(&pathspec, i, &skip_worktree_seen)) {
    @@ builtin/add.c: int cmd_add(int argc, const char **argv, const char *prefix)
     +				continue;
     +			}
     +
    ++			/* Don't complain at 'git add .' inside empty repo. */
    ++			if (!path[0])
    ++				continue;
    ++
     +			if ((pathspec.items[i].magic & (PATHSPEC_GLOB | PATHSPEC_ICASE)) ||
     +			    !file_exists(path)) {
      				if (ignore_missing) {
    @@ pathspec.h: void add_pathspec_matches_against_index(const struct pathspec *paths
      			 const char *name, int namelen,
      			 const struct pathspec_item *item);
     
    - ## read-cache.c ##
    -@@ read-cache.c: int refresh_index(struct index_state *istate, unsigned int flags,
    - 		if (ignore_submodules && S_ISGITLINK(ce->ce_mode))
    - 			continue;
    - 
    --		if (pathspec && !ce_path_match(istate, ce, pathspec, seen))
    -+		if (pathspec && !ce_path_match(istate, ce, pathspec,
    -+					       ce_skip_worktree(ce) ? NULL : seen))
    - 			filtered = 1;
    - 
    - 		if (ce_stage(ce)) {
    -
      ## t/t3705-add-sparse-checkout.sh ##
    -@@ t/t3705-add-sparse-checkout.sh: test_sparse_entry_unchanged() {
    - 	test_cmp expected actual
    +@@ t/t3705-add-sparse-checkout.sh: setup_gitignore () {
    + 	EOF
      }
      
    -+cat >sparse_entry_error <<-EOF
    -+The following pathspecs only matched index entries outside the current
    -+sparse checkout:
    -+sparse_entry
    -+EOF
    ++test_expect_success 'setup' '
    ++	cat >sparse_error_header <<-EOF &&
    ++	The following pathspecs only matched index entries outside the current
    ++	sparse checkout:
    ++	EOF
    ++
    ++	cat >sparse_hint <<-EOF &&
    ++	hint: Disable or modify the sparsity rules if you intend to update such entries.
    ++	hint: Disable this message with "git config advice.updateSparsePath false"
    ++	EOF
     +
    -+cat >error_and_hint sparse_entry_error - <<-EOF
    -+hint: Disable or modify the sparsity rules if you intend to update such entries.
    -+hint: Disable this message with "git config advice.updateSparsePath false"
    -+EOF
    ++	echo sparse_entry | cat sparse_error_header - >sparse_entry_error &&
    ++	cat sparse_entry_error sparse_hint >error_and_hint
    ++'
     +
    - test_expect_success "git add does not remove SKIP_WORKTREE entries" '
    + test_expect_success 'git add does not remove sparse entries' '
      	setup_sparse_entry &&
      	rm sparse_entry &&
     -	git add sparse_entry &&
    @@ t/t3705-add-sparse-checkout.sh: test_sparse_entry_unchanged() {
      	test_sparse_entry_unchanged
      '
      
    +@@ t/t3705-add-sparse-checkout.sh: test_expect_success 'git add -A does not remove sparse entries' '
    + 	setup_sparse_entry &&
    + 	rm sparse_entry &&
    + 	setup_gitignore &&
    +-	git add -A &&
    ++	git add -A 2>stderr &&
    ++	test_must_be_empty stderr &&
    + 	test_sparse_entry_unchanged
    + '
    + 
    +@@ t/t3705-add-sparse-checkout.sh: test_expect_success 'git add . does not remove sparse entries' '
    + 	setup_sparse_entry &&
    + 	rm sparse_entry &&
    + 	setup_gitignore &&
    +-	git add . &&
    ++	test_must_fail git add . 2>stderr &&
    ++
    ++	cat sparse_error_header >expect &&
    ++	echo . >>expect &&
    ++	cat sparse_hint >>expect &&
    ++
    ++	test_i18ncmp expect stderr &&
    + 	test_sparse_entry_unchanged
    + '
    + 
     @@ t/t3705-add-sparse-checkout.sh: do
    - 	test_expect_success "git add$opt does not update SKIP_WORKTREE entries" '
    + 	test_expect_success "git add${opt:+ $opt} does not update sparse entries" '
      		setup_sparse_entry &&
      		echo modified >sparse_entry &&
     -		git add $opt sparse_entry &&
    @@ t/t3705-add-sparse-checkout.sh: do
      		test_sparse_entry_unchanged
      	'
      done
    -@@ t/t3705-add-sparse-checkout.sh: done
    - test_expect_success 'git add --refresh does not update SKIP_WORKTREE entries' '
    +@@ t/t3705-add-sparse-checkout.sh: test_expect_success 'git add --refresh does not update sparse entries' '
      	setup_sparse_entry &&
    + 	git ls-files --debug sparse_entry | grep mtime >before &&
      	test-tool chmtime -60 sparse_entry &&
     -	git add --refresh sparse_entry &&
     +	test_must_fail git add --refresh sparse_entry 2>stderr &&
     +	test_i18ncmp error_and_hint stderr &&
    + 	git ls-files --debug sparse_entry | grep mtime >after &&
    + 	test_cmp before after
    + '
      
    - 	# We must unset the SKIP_WORKTREE bit, otherwise
    - 	# git diff-files would skip examining the file
    -@@ t/t3705-add-sparse-checkout.sh: test_expect_success 'git add --refresh does not update SKIP_WORKTREE entries' '
    - 
    - test_expect_success 'git add --chmod does not update SKIP_WORKTREE entries' '
    + test_expect_success 'git add --chmod does not update sparse entries' '
      	setup_sparse_entry &&
     -	git add --chmod=+x sparse_entry &&
     +	test_must_fail git add --chmod=+x sparse_entry 2>stderr &&
     +	test_i18ncmp error_and_hint stderr &&
    - 	test_sparse_entry_unchanged
    + 	test_sparse_entry_unchanged &&
    + 	! test -x sparse_entry
      '
    - 
    -@@ t/t3705-add-sparse-checkout.sh: test_expect_success 'git add --renormalize does not update SKIP_WORKTREE entries
    +@@ t/t3705-add-sparse-checkout.sh: test_expect_success 'git add --renormalize does not update sparse entries' '
      	test_config core.autocrlf false &&
      	setup_sparse_entry "LINEONE\r\nLINETWO\r\n" &&
      	echo "sparse_entry text=auto" >.gitattributes &&
    @@ t/t3705-add-sparse-checkout.sh: test_expect_success 'git add --renormalize does
      	test_sparse_entry_unchanged
      '
      
    ++test_expect_success 'git add --dry-run --ignore-missing warn on sparse path' '
    ++	setup_sparse_entry &&
    ++	rm sparse_entry &&
    ++	test_must_fail git add --dry-run --ignore-missing sparse_entry 2>stderr &&
    ++	test_i18ncmp error_and_hint stderr &&
    ++	test_sparse_entry_unchanged
    ++'
    ++
     +test_expect_success 'do not advice about sparse entries when they do not match the pathspec' '
     +	setup_sparse_entry &&
    -+	test_must_fail git add nonexistent sp 2>stderr &&
    ++	test_must_fail git add nonexistent 2>stderr &&
     +	test_i18ngrep "fatal: pathspec .nonexistent. did not match any files" stderr &&
     +	test_i18ngrep ! "The following pathspecs only matched index entries" stderr
     +'
     +
    ++test_expect_success 'do not warn when pathspec matches dense entries' '
    ++	setup_sparse_entry &&
    ++	echo modified >sparse_entry &&
    ++	>dense_entry &&
    ++	git add "*_entry" 2>stderr &&
    ++	test_must_be_empty stderr &&
    ++	test_sparse_entry_unchanged &&
    ++	git ls-files --error-unmatch dense_entry
    ++'
    ++
     +test_expect_success 'add obeys advice.updateSparsePath' '
     +	setup_sparse_entry &&
     +	test_must_fail git -c advice.updateSparsePath=false add sparse_entry 2>stderr &&
7:  e76c7c6999 ! 7:  08f0c32bfc rm: honor sparse checkout patterns
    @@ Commit message
         rules, but `git rm` doesn't follow the same restrictions. This is
         somewhat counter-intuitive and inconsistent. So make `rm` honor the
         sparse checkout and advise on how to remove SKIP_WORKTREE entries, just
    -    like `add` does. Also add a few tests for the new behavior.
    +    like `add` does. Also add some tests for the new behavior.
     
         Suggested-by: Elijah Newren <newren@gmail.com>
         Signed-off-by: Matheus Tavares <matheus.bernardino@usp.br>
    @@ builtin/rm.c: int cmd_rm(int argc, const char **argv, const char *prefix)
     +	return ret;
      }
     
    - ## t/t3600-rm.sh ##
    -@@ t/t3600-rm.sh: test_expect_success 'rm empty string should fail' '
    - 	test_must_fail git rm -rf ""
    - '
    - 
    -+test_expect_success 'setup repo for tests with sparse-checkout' '
    -+	git init sparse &&
    -+	(
    -+		cd sparse &&
    -+		mkdir -p sub/dir &&
    -+		touch a b c sub/d sub/dir/e &&
    -+		git add -A &&
    -+		git commit -m files
    -+	) &&
    + ## t/t3602-rm-sparse-checkout.sh (new) ##
    +@@
    ++#!/bin/sh
    ++
    ++test_description='git rm in sparse checked out working trees'
    ++
    ++. ./test-lib.sh
    ++
    ++test_expect_success 'setup' '
    ++	mkdir -p sub/dir &&
    ++	touch a b c sub/d sub/dir/e &&
    ++	git add -A &&
    ++	git commit -m files &&
     +
     +	cat >sparse_entry_b_error <<-EOF &&
     +	The following pathspecs only matched index entries outside the current
    @@ t/t3600-rm.sh: test_expect_success 'rm empty string should fail' '
     +	EOF
     +'
     +
    -+test_expect_success 'rm should respect sparse-checkout' '
    -+	git -C sparse sparse-checkout set "/a" &&
    -+	test_must_fail git -C sparse rm b 2>stderr &&
    -+	test_i18ncmp b_error_and_hint stderr
    ++for opt in "" -f --dry-run
    ++do
    ++	test_expect_success "rm${opt:+ $opt} does not remove sparse entries" '
    ++		git sparse-checkout set a &&
    ++		test_must_fail git rm $opt b 2>stderr &&
    ++		test_i18ncmp b_error_and_hint stderr &&
    ++		git ls-files --error-unmatch b
    ++	'
    ++done
    ++
    ++test_expect_success 'recursive rm does not remove sparse entries' '
    ++	git reset --hard &&
    ++	git sparse-checkout set sub/dir &&
    ++	git rm -r sub &&
    ++	git status --porcelain -uno >actual &&
    ++	echo "D  sub/dir/e" >expected &&
    ++	test_cmp expected actual
     +'
     +
     +test_expect_success 'rm obeys advice.updateSparsePath' '
    -+	git -C sparse reset --hard &&
    -+	git -C sparse sparse-checkout set "/a" &&
    -+	test_must_fail git -C sparse -c advice.updateSparsePath=false rm b 2>stderr &&
    ++	git reset --hard &&
    ++	git sparse-checkout set a &&
    ++	test_must_fail git -c advice.updateSparsePath=false rm b 2>stderr &&
     +	test_i18ncmp sparse_entry_b_error stderr
    -+
    -+'
    -+
    -+test_expect_success 'recursive rm should respect sparse-checkout' '
    -+	(
    -+		cd sparse &&
    -+		git reset --hard &&
    -+		git sparse-checkout set "sub/dir" &&
    -+		git rm -r sub &&
    -+		git status --porcelain -uno >../actual
    -+	) &&
    -+	echo "D  sub/dir/e" >expected &&
    -+	test_cmp expected actual
     +'
     +
     +test_expect_success 'do not advice about sparse entries when they do not match the pathspec' '
    -+	test_must_fail git -C sparse rm nonexistent 2>stderr &&
    ++	git reset --hard &&
    ++	git sparse-checkout set a &&
    ++	test_must_fail git rm nonexistent 2>stderr &&
     +	test_i18ngrep "fatal: pathspec .nonexistent. did not match any files" stderr &&
     +	test_i18ngrep ! "The following pathspecs only matched index entries" stderr
     +'
     +
    - test_done
    ++test_expect_success 'do not warn about sparse entries when pathspec matches dense entries' '
    ++	git reset --hard &&
    ++	git sparse-checkout set a &&
    ++	git rm "[ba]" 2>stderr &&
    ++	test_must_be_empty stderr &&
    ++	git ls-files --error-unmatch b &&
    ++	test_must_fail git ls-files --error-unmatch a
    ++'
    ++
    ++test_expect_success 'do not warn about sparse entries with --ignore-unmatch' '
    ++	git reset --hard &&
    ++	git sparse-checkout set a &&
    ++	git rm --ignore-unmatch b 2>stderr &&
    ++	test_must_be_empty stderr &&
    ++	git ls-files --error-unmatch b
    ++'
    ++
    ++test_done
     
      ## t/t7011-skip-worktree-reading.sh ##
     @@ t/t7011-skip-worktree-reading.sh: test_expect_success 'diff-files does not examine skip-worktree dirty entries' '
-- 
2.30.1
Matheus Tavares· Feb 24, 2021, 04:05 UTC · re: Matheus Tavares · lore

When `git add --refresh <pathspec>` doesn't find any matches for the given pathspec, it prints an error message using the `match` field of the `struct pathspec_item`. However, this field doesn't contain the magic part of the pathspec. Instead, let's use the `original` field.

Signed-off-by: Matheus Tavares <matheus.bernardino@usp.br>
---
 builtin/add.c  | 2 +-
 t/t3700-add.sh | 6 ++++++
 2 files changed, 7 insertions(+), 1 deletion(-)
Show changes to 2 files +7 −1

builtin/add.c, t/t3700-add.sh

diff --git a/builtin/add.c b/builtin/add.c
index ea762a41e3..24ed7e25f3 100644
--- a/builtin/add.c
+++ b/builtin/add.c
@@ -187,7 +187,7 @@ static void refresh(int verbose, const struct pathspec *pathspec)
 	for (i = 0; i < pathspec->nr; i++) {
 		if (!seen[i])
 			die(_("pathspec '%s' did not match any files"),
-			    pathspec->items[i].match);
+			    pathspec->items[i].original);
 	}
 	free(seen);
 }
diff --git a/t/t3700-add.sh b/t/t3700-add.sh
index d402c775c0..15ede17804 100755
--- a/t/t3700-add.sh
+++ b/t/t3700-add.sh
@@ -196,6 +196,12 @@ test_expect_success 'git add --refresh with pathspec' '
 	grep baz actual
 '
 
+test_expect_success 'git add --refresh correctly reports no match error' "
+	echo \"fatal: pathspec ':(icase)nonexistent' did not match any files\" >expect &&
+	test_must_fail git add --refresh ':(icase)nonexistent' 2>actual &&
+	test_i18ncmp expect actual
+"
+
 test_expect_success POSIXPERM,SANITY 'git add should fail atomically upon an unreadable file' '
 	git reset --hard &&
 	date >foo1 &&
-- 
2.30.1
Matheus Tavares· Feb 24, 2021, 04:05 UTC · re: Matheus Tavares · lore

[PATCH v2 2/7] t3705: add tests for `git add` in sparse checkouts

We already have a couple tests for `add` with SKIP_WORKTREE entries in t7012, but these only cover the most basic scenarios. As we will be changing how `add` deals with sparse paths in the subsequent commits, let's move these two tests to their own file and add more test cases for different `add` options and situations. This also demonstrates two options that don't currently respect SKIP_WORKTREE entries: `--chmod` and `--renormalize`.

Signed-off-by: Matheus Tavares <matheus.bernardino@usp.br>
---
 t/t3705-add-sparse-checkout.sh   | 96 ++++++++++++++++++++++++++++++++
 t/t7012-skip-worktree-writing.sh | 19 -------
 2 files changed, 96 insertions(+), 19 deletions(-)
 create mode 100755 t/t3705-add-sparse-checkout.sh
Show changes to 2 files +96 −19

t/t3705-add-sparse-checkout.sh, t/t7012-skip-worktree-writing.sh

diff --git a/t/t3705-add-sparse-checkout.sh b/t/t3705-add-sparse-checkout.sh
new file mode 100755
index 0000000000..9bb5dc2389
--- /dev/null
+++ b/t/t3705-add-sparse-checkout.sh
@@ -0,0 +1,96 @@
+#!/bin/sh
+
+test_description='git add in sparse checked out working trees'
+
+. ./test-lib.sh
+
+SPARSE_ENTRY_BLOB=""
+
+# Optionally take a printf format string to write to the sparse_entry file
+setup_sparse_entry () {
+	rm -f sparse_entry &&
+	git update-index --force-remove sparse_entry &&
+
+	if test $# -eq 1
+	then
+		printf "$1" >sparse_entry
+	else
+		>sparse_entry
+	fi &&
+	git add sparse_entry &&
+	git update-index --skip-worktree sparse_entry &&
+	SPARSE_ENTRY_BLOB=$(git rev-parse :sparse_entry)
+}
+
+test_sparse_entry_unchanged () {
+	echo "100644 $SPARSE_ENTRY_BLOB 0	sparse_entry" >expected &&
+	git ls-files --stage sparse_entry >actual &&
+	test_cmp expected actual
+}
+
+setup_gitignore () {
+	test_when_finished rm -f .gitignore &&
+	cat >.gitignore <<-EOF
+	*
+	!/sparse_entry
+	EOF
+}
+
+test_expect_success 'git add does not remove sparse entries' '
+	setup_sparse_entry &&
+	rm sparse_entry &&
+	git add sparse_entry &&
+	test_sparse_entry_unchanged
+'
+
+test_expect_success 'git add -A does not remove sparse entries' '
+	setup_sparse_entry &&
+	rm sparse_entry &&
+	setup_gitignore &&
+	git add -A &&
+	test_sparse_entry_unchanged
+'
+
+test_expect_success 'git add . does not remove sparse entries' '
+	setup_sparse_entry &&
+	rm sparse_entry &&
+	setup_gitignore &&
+	git add . &&
+	test_sparse_entry_unchanged
+'
+
+for opt in "" -f -u --ignore-removal --dry-run
+do
+	test_expect_success "git add${opt:+ $opt} does not update sparse entries" '
+		setup_sparse_entry &&
+		echo modified >sparse_entry &&
+		git add $opt sparse_entry &&
+		test_sparse_entry_unchanged
+	'
+done
+
+test_expect_success 'git add --refresh does not update sparse entries' '
+	setup_sparse_entry &&
+	git ls-files --debug sparse_entry | grep mtime >before &&
+	test-tool chmtime -60 sparse_entry &&
+	git add --refresh sparse_entry &&
+	git ls-files --debug sparse_entry | grep mtime >after &&
+	test_cmp before after
+'
+
+test_expect_failure 'git add --chmod does not update sparse entries' '
+	setup_sparse_entry &&
+	git add --chmod=+x sparse_entry &&
+	test_sparse_entry_unchanged &&
+	! test -x sparse_entry
+'
+
+test_expect_failure 'git add --renormalize does not update sparse entries' '
+	test_config core.autocrlf false &&
+	setup_sparse_entry "LINEONE\r\nLINETWO\r\n" &&
+	echo "sparse_entry text=auto" >.gitattributes &&
+	git add --renormalize sparse_entry &&
+	test_sparse_entry_unchanged
+'
+
+test_done
diff --git a/t/t7012-skip-worktree-writing.sh b/t/t7012-skip-worktree-writing.sh
index e5c6a038fb..217207c1ce 100755
--- a/t/t7012-skip-worktree-writing.sh
+++ b/t/t7012-skip-worktree-writing.sh
@@ -60,13 +60,6 @@ setup_absent() {
 	git update-index --skip-worktree 1
 }
 
-test_absent() {
-	echo "100644 $EMPTY_BLOB 0	1" > expected &&
-	git ls-files --stage 1 > result &&
-	test_cmp expected result &&
-	test ! -f 1
-}
-
 setup_dirty() {
 	git update-index --force-remove 1 &&
 	echo dirty > 1 &&
@@ -100,18 +93,6 @@ test_expect_success 'index setup' '
 	test_cmp expected result
 '
 
-test_expect_success 'git-add ignores worktree content' '
-	setup_absent &&
-	git add 1 &&
-	test_absent
-'
-
-test_expect_success 'git-add ignores worktree content' '
-	setup_dirty &&
-	git add 1 &&
-	test_dirty
-'
-
 test_expect_success 'git-rm fails if worktree is dirty' '
 	setup_dirty &&
 	test_must_fail git rm 1 &&
-- 
2.30.1
Elijah Newren· Feb 24, 2021, 05:15 UTC · re: Matheus Tavares · lore

Re: [PATCH v2 2/7] t3705: add tests for `git add` in sparse checkouts

On Tue, Feb 23, 2021 at 8:05 PM Matheus Tavares <matheus.bernardino@usp.br> wrote:

Show 33 quoted lines
>
> We already have a couple tests for `add` with SKIP_WORKTREE entries in
> t7012, but these only cover the most basic scenarios. As we will be
> changing how `add` deals with sparse paths in the subsequent commits,
> let's move these two tests to their own file and add more test cases
> for different `add` options and situations. This also demonstrates two
> options that don't currently respect SKIP_WORKTREE entries: `--chmod`
> and `--renormalize`.
>
> Signed-off-by: Matheus Tavares <matheus.bernardino@usp.br>
> ---
>  t/t3705-add-sparse-checkout.sh   | 96 ++++++++++++++++++++++++++++++++
>  t/t7012-skip-worktree-writing.sh | 19 -------
>  2 files changed, 96 insertions(+), 19 deletions(-)
>  create mode 100755 t/t3705-add-sparse-checkout.sh
>
> diff --git a/t/t3705-add-sparse-checkout.sh b/t/t3705-add-sparse-checkout.sh
> new file mode 100755
> index 0000000000..9bb5dc2389
> --- /dev/null
> +++ b/t/t3705-add-sparse-checkout.sh
> @@ -0,0 +1,96 @@
> +#!/bin/sh
> +
> +test_description='git add in sparse checked out working trees'
> +
> +. ./test-lib.sh
> +
> +SPARSE_ENTRY_BLOB=""
> +
> +# Optionally take a printf format string to write to the sparse_entry file
> +setup_sparse_entry () {
> +       rm -f sparse_entry &&
I think this is unnecessary.
> +       git update-index --force-remove sparse_entry &&

It might be worth adding a comment above this line that it is necessary when sparse_entry starts out in the SKIP_WORKTREE state, otherwise the subsequent git add below will ignore it.

Show 123 quoted lines
> +
> +       if test $# -eq 1
> +       then
> +               printf "$1" >sparse_entry
> +       else
> +               >sparse_entry
> +       fi &&
> +       git add sparse_entry &&
> +       git update-index --skip-worktree sparse_entry &&
> +       SPARSE_ENTRY_BLOB=$(git rev-parse :sparse_entry)
> +}
> +
> +test_sparse_entry_unchanged () {
> +       echo "100644 $SPARSE_ENTRY_BLOB 0       sparse_entry" >expected &&
> +       git ls-files --stage sparse_entry >actual &&
> +       test_cmp expected actual
> +}
> +
> +setup_gitignore () {
> +       test_when_finished rm -f .gitignore &&
> +       cat >.gitignore <<-EOF
> +       *
> +       !/sparse_entry
> +       EOF
> +}
> +
> +test_expect_success 'git add does not remove sparse entries' '
> +       setup_sparse_entry &&
> +       rm sparse_entry &&
> +       git add sparse_entry &&
> +       test_sparse_entry_unchanged
> +'
> +
> +test_expect_success 'git add -A does not remove sparse entries' '
> +       setup_sparse_entry &&
> +       rm sparse_entry &&
> +       setup_gitignore &&
> +       git add -A &&
> +       test_sparse_entry_unchanged
> +'
> +
> +test_expect_success 'git add . does not remove sparse entries' '
> +       setup_sparse_entry &&
> +       rm sparse_entry &&
> +       setup_gitignore &&
> +       git add . &&
> +       test_sparse_entry_unchanged
> +'
> +
> +for opt in "" -f -u --ignore-removal --dry-run
> +do
> +       test_expect_success "git add${opt:+ $opt} does not update sparse entries" '
> +               setup_sparse_entry &&
> +               echo modified >sparse_entry &&
> +               git add $opt sparse_entry &&
> +               test_sparse_entry_unchanged
> +       '
> +done
> +
> +test_expect_success 'git add --refresh does not update sparse entries' '
> +       setup_sparse_entry &&
> +       git ls-files --debug sparse_entry | grep mtime >before &&
> +       test-tool chmtime -60 sparse_entry &&
> +       git add --refresh sparse_entry &&
> +       git ls-files --debug sparse_entry | grep mtime >after &&
> +       test_cmp before after
> +'
> +
> +test_expect_failure 'git add --chmod does not update sparse entries' '
> +       setup_sparse_entry &&
> +       git add --chmod=+x sparse_entry &&
> +       test_sparse_entry_unchanged &&
> +       ! test -x sparse_entry
> +'
> +
> +test_expect_failure 'git add --renormalize does not update sparse entries' '
> +       test_config core.autocrlf false &&
> +       setup_sparse_entry "LINEONE\r\nLINETWO\r\n" &&
> +       echo "sparse_entry text=auto" >.gitattributes &&
> +       git add --renormalize sparse_entry &&
> +       test_sparse_entry_unchanged
> +'
> +
> +test_done
> diff --git a/t/t7012-skip-worktree-writing.sh b/t/t7012-skip-worktree-writing.sh
> index e5c6a038fb..217207c1ce 100755
> --- a/t/t7012-skip-worktree-writing.sh
> +++ b/t/t7012-skip-worktree-writing.sh
> @@ -60,13 +60,6 @@ setup_absent() {
>         git update-index --skip-worktree 1
>  }
>
> -test_absent() {
> -       echo "100644 $EMPTY_BLOB 0      1" > expected &&
> -       git ls-files --stage 1 > result &&
> -       test_cmp expected result &&
> -       test ! -f 1
> -}
> -
>  setup_dirty() {
>         git update-index --force-remove 1 &&
>         echo dirty > 1 &&
> @@ -100,18 +93,6 @@ test_expect_success 'index setup' '
>         test_cmp expected result
>  '
>
> -test_expect_success 'git-add ignores worktree content' '
> -       setup_absent &&
> -       git add 1 &&
> -       test_absent
> -'
> -
> -test_expect_success 'git-add ignores worktree content' '
> -       setup_dirty &&
> -       git add 1 &&
> -       test_dirty
> -'
> -
>  test_expect_success 'git-rm fails if worktree is dirty' '
>         setup_dirty &&
>         test_must_fail git rm 1 &&
> --
> 2.30.1
The rest looks good.
Matheus Tavares· Feb 24, 2021, 04:05 UTC · re: Matheus Tavares · lore

[PATCH v2 3/7] add: make --chmod and --renormalize honor sparse checkouts

Signed-off-by: Matheus Tavares <matheus.bernardino@usp.br>
---
 builtin/add.c                  | 5 +++++
 t/t3705-add-sparse-checkout.sh | 4 ++--
 2 files changed, 7 insertions(+), 2 deletions(-)
Show changes to 2 files +7 −2

builtin/add.c, t/t3705-add-sparse-checkout.sh

diff --git a/builtin/add.c b/builtin/add.c
index 24ed7e25f3..5fec21a792 100644
--- a/builtin/add.c
+++ b/builtin/add.c
@@ -46,6 +46,9 @@ static int chmod_pathspec(struct pathspec *pathspec, char flip, int show_only)
 		struct cache_entry *ce = active_cache[i];
 		int err;
 
+		if (ce_skip_worktree(ce))
+			continue;
+
 		if (pathspec && !ce_path_match(&the_index, ce, pathspec, NULL))
 			continue;
 
@@ -144,6 +147,8 @@ static int renormalize_tracked_files(const struct pathspec *pathspec, int flags)
 	for (i = 0; i < active_nr; i++) {
 		struct cache_entry *ce = active_cache[i];
 
+		if (ce_skip_worktree(ce))
+			continue;
 		if (ce_stage(ce))
 			continue; /* do not touch unmerged paths */
 		if (!S_ISREG(ce->ce_mode) && !S_ISLNK(ce->ce_mode))
diff --git a/t/t3705-add-sparse-checkout.sh b/t/t3705-add-sparse-checkout.sh
index 9bb5dc2389..6781620297 100755
--- a/t/t3705-add-sparse-checkout.sh
+++ b/t/t3705-add-sparse-checkout.sh
@@ -78,14 +78,14 @@ test_expect_success 'git add --refresh does not update sparse entries' '
 	test_cmp before after
 '
 
-test_expect_failure 'git add --chmod does not update sparse entries' '
+test_expect_success 'git add --chmod does not update sparse entries' '
 	setup_sparse_entry &&
 	git add --chmod=+x sparse_entry &&
 	test_sparse_entry_unchanged &&
 	! test -x sparse_entry
 '
 
-test_expect_failure 'git add --renormalize does not update sparse entries' '
+test_expect_success 'git add --renormalize does not update sparse entries' '
 	test_config core.autocrlf false &&
 	setup_sparse_entry "LINEONE\r\nLINETWO\r\n" &&
 	echo "sparse_entry text=auto" >.gitattributes &&
-- 
2.30.1
Matheus Tavares· Feb 24, 2021, 04:05 UTC · re: Matheus Tavares · lore

[PATCH v2 4/7] pathspec: allow to ignore SKIP_WORKTREE entries on index matching

Add the 'ignore_skip_worktree' boolean parameter to both add_pathspec_matches_against_index() and find_pathspecs_matching_against_index(). When true, these functions will not try to match the given pathspec with SKIP_WORKTREE entries. This will be used in a future patch to make `git add` display a hint when the pathspec matches only sparse paths.

Signed-off-by: Matheus Tavares <matheus.bernardino@usp.br>
---
 builtin/add.c          |  4 ++--
 builtin/check-ignore.c |  2 +-
 pathspec.c             | 10 +++++++---
 pathspec.h             |  5 +++--
 4 files changed, 13 insertions(+), 8 deletions(-)
Show changes to 4 files +13 −8

builtin/add.c, builtin/check-ignore.c, pathspec.c, pathspec.h

diff --git a/builtin/add.c b/builtin/add.c
index 5fec21a792..e15b25a623 100644
--- a/builtin/add.c
+++ b/builtin/add.c
@@ -177,7 +177,7 @@ static char *prune_directory(struct dir_struct *dir, struct pathspec *pathspec,
 			*dst++ = entry;
 	}
 	dir->nr = dst - dir->entries;
-	add_pathspec_matches_against_index(pathspec, &the_index, seen);
+	add_pathspec_matches_against_index(pathspec, &the_index, seen, 0);
 	return seen;
 }
 
@@ -578,7 +578,7 @@ int cmd_add(int argc, const char **argv, const char *prefix)
 		int i;
 
 		if (!seen)
-			seen = find_pathspecs_matching_against_index(&pathspec, &the_index);
+			seen = find_pathspecs_matching_against_index(&pathspec, &the_index, 0);
 
 		/*
 		 * file_exists() assumes exact match
diff --git a/builtin/check-ignore.c b/builtin/check-ignore.c
index 3c652748d5..235b7fc905 100644
--- a/builtin/check-ignore.c
+++ b/builtin/check-ignore.c
@@ -100,7 +100,7 @@ static int check_ignore(struct dir_struct *dir,
 	 * should not be ignored, in order to be consistent with
 	 * 'git status', 'git add' etc.
 	 */
-	seen = find_pathspecs_matching_against_index(&pathspec, &the_index);
+	seen = find_pathspecs_matching_against_index(&pathspec, &the_index, 0);
 	for (i = 0; i < pathspec.nr; i++) {
 		full_path = pathspec.items[i].match;
 		pattern = NULL;
diff --git a/pathspec.c b/pathspec.c
index 7a229d8d22..e5e6b7458d 100644
--- a/pathspec.c
+++ b/pathspec.c
@@ -21,7 +21,7 @@
  */
 void add_pathspec_matches_against_index(const struct pathspec *pathspec,
 					const struct index_state *istate,
-					char *seen)
+					char *seen, int ignore_skip_worktree)
 {
 	int num_unmatched = 0, i;
 
@@ -38,6 +38,8 @@ void add_pathspec_matches_against_index(const struct pathspec *pathspec,
 		return;
 	for (i = 0; i < istate->cache_nr; i++) {
 		const struct cache_entry *ce = istate->cache[i];
+		if (ignore_skip_worktree && ce_skip_worktree(ce))
+			continue;
 		ce_path_match(istate, ce, pathspec, seen);
 	}
 }
@@ -51,10 +53,12 @@ void add_pathspec_matches_against_index(const struct pathspec *pathspec,
  * given pathspecs achieves against all items in the index.
  */
 char *find_pathspecs_matching_against_index(const struct pathspec *pathspec,
-					    const struct index_state *istate)
+					    const struct index_state *istate,
+					    int ignore_skip_worktree)
 {
 	char *seen = xcalloc(pathspec->nr, 1);
-	add_pathspec_matches_against_index(pathspec, istate, seen);
+	add_pathspec_matches_against_index(pathspec, istate, seen,
+					   ignore_skip_worktree);
 	return seen;
 }
 
diff --git a/pathspec.h b/pathspec.h
index 454ce364fa..8202882ecd 100644
--- a/pathspec.h
+++ b/pathspec.h
@@ -151,9 +151,10 @@ static inline int ps_strcmp(const struct pathspec_item *item,
 
 void add_pathspec_matches_against_index(const struct pathspec *pathspec,
 					const struct index_state *istate,
-					char *seen);
+					char *seen, int ignore_skip_worktree);
 char *find_pathspecs_matching_against_index(const struct pathspec *pathspec,
-					    const struct index_state *istate);
+					    const struct index_state *istate,
+					    int ignore_skip_worktree);
 int match_pathspec_attrs(const struct index_state *istate,
 			 const char *name, int namelen,
 			 const struct pathspec_item *item);
-- 
2.30.1
Elijah Newren· Feb 24, 2021, 05:23 UTC · re: Matheus Tavares · lore

Re: [PATCH v2 4/7] pathspec: allow to ignore SKIP_WORKTREE entries on index matching

On Tue, Feb 23, 2021 at 8:05 PM Matheus Tavares <matheus.bernardino@usp.br> wrote:

Show 26 quoted lines
>
> Add the 'ignore_skip_worktree' boolean parameter to both
> add_pathspec_matches_against_index() and
> find_pathspecs_matching_against_index(). When true, these functions will
> not try to match the given pathspec with SKIP_WORKTREE entries. This
> will be used in a future patch to make `git add` display a hint
> when the pathspec matches only sparse paths.
>
> Signed-off-by: Matheus Tavares <matheus.bernardino@usp.br>
> ---
>  builtin/add.c          |  4 ++--
>  builtin/check-ignore.c |  2 +-
>  pathspec.c             | 10 +++++++---
>  pathspec.h             |  5 +++--
>  4 files changed, 13 insertions(+), 8 deletions(-)
>
> diff --git a/builtin/add.c b/builtin/add.c
> index 5fec21a792..e15b25a623 100644
> --- a/builtin/add.c
> +++ b/builtin/add.c
> @@ -177,7 +177,7 @@ static char *prune_directory(struct dir_struct *dir, struct pathspec *pathspec,
>                         *dst++ = entry;
>         }
>         dir->nr = dst - dir->entries;
> -       add_pathspec_matches_against_index(pathspec, &the_index, seen);
> +       add_pathspec_matches_against_index(pathspec, &the_index, seen, 0);

One thing to consider here is something Stolee has suggested to me multiple times -- introducing an enum with self-documenting values. For example:

enum ignore_skip_worktree_values {
  HEED_SKIP_WORKTREE = 0,
  IGNORE_SKIP_WORKTREE = 1
};

This would allow all the function callers to pass HEED_SKIP_WORKTREE instead of a bare 0, and allows code readers to avoid looking up function signatures to find the meaning behind the 0. It's a minor point, though.

Show 81 quoted lines
>         return seen;
>  }
>
> @@ -578,7 +578,7 @@ int cmd_add(int argc, const char **argv, const char *prefix)
>                 int i;
>
>                 if (!seen)
> -                       seen = find_pathspecs_matching_against_index(&pathspec, &the_index);
> +                       seen = find_pathspecs_matching_against_index(&pathspec, &the_index, 0);
>
>                 /*
>                  * file_exists() assumes exact match
> diff --git a/builtin/check-ignore.c b/builtin/check-ignore.c
> index 3c652748d5..235b7fc905 100644
> --- a/builtin/check-ignore.c
> +++ b/builtin/check-ignore.c
> @@ -100,7 +100,7 @@ static int check_ignore(struct dir_struct *dir,
>          * should not be ignored, in order to be consistent with
>          * 'git status', 'git add' etc.
>          */
> -       seen = find_pathspecs_matching_against_index(&pathspec, &the_index);
> +       seen = find_pathspecs_matching_against_index(&pathspec, &the_index, 0);
>         for (i = 0; i < pathspec.nr; i++) {
>                 full_path = pathspec.items[i].match;
>                 pattern = NULL;
> diff --git a/pathspec.c b/pathspec.c
> index 7a229d8d22..e5e6b7458d 100644
> --- a/pathspec.c
> +++ b/pathspec.c
> @@ -21,7 +21,7 @@
>   */
>  void add_pathspec_matches_against_index(const struct pathspec *pathspec,
>                                         const struct index_state *istate,
> -                                       char *seen)
> +                                       char *seen, int ignore_skip_worktree)
>  {
>         int num_unmatched = 0, i;
>
> @@ -38,6 +38,8 @@ void add_pathspec_matches_against_index(const struct pathspec *pathspec,
>                 return;
>         for (i = 0; i < istate->cache_nr; i++) {
>                 const struct cache_entry *ce = istate->cache[i];
> +               if (ignore_skip_worktree && ce_skip_worktree(ce))
> +                       continue;
>                 ce_path_match(istate, ce, pathspec, seen);
>         }
>  }
> @@ -51,10 +53,12 @@ void add_pathspec_matches_against_index(const struct pathspec *pathspec,
>   * given pathspecs achieves against all items in the index.
>   */
>  char *find_pathspecs_matching_against_index(const struct pathspec *pathspec,
> -                                           const struct index_state *istate)
> +                                           const struct index_state *istate,
> +                                           int ignore_skip_worktree)
>  {
>         char *seen = xcalloc(pathspec->nr, 1);
> -       add_pathspec_matches_against_index(pathspec, istate, seen);
> +       add_pathspec_matches_against_index(pathspec, istate, seen,
> +                                          ignore_skip_worktree);
>         return seen;
>  }
>
> diff --git a/pathspec.h b/pathspec.h
> index 454ce364fa..8202882ecd 100644
> --- a/pathspec.h
> +++ b/pathspec.h
> @@ -151,9 +151,10 @@ static inline int ps_strcmp(const struct pathspec_item *item,
>
>  void add_pathspec_matches_against_index(const struct pathspec *pathspec,
>                                         const struct index_state *istate,
> -                                       char *seen);
> +                                       char *seen, int ignore_skip_worktree);
>  char *find_pathspecs_matching_against_index(const struct pathspec *pathspec,
> -                                           const struct index_state *istate);
> +                                           const struct index_state *istate,
> +                                           int ignore_skip_worktree);
>  int match_pathspec_attrs(const struct index_state *istate,
>                          const char *name, int namelen,
>                          const struct pathspec_item *item);
> --
> 2.30.1
Looks good.
Matheus Tavares· Feb 24, 2021, 04:05 UTC · re: Matheus Tavares · lore

[PATCH v2 5/7] refresh_index(): add REFRESH_DONT_MARK_SPARSE_MATCHES flag

refresh_index() optionally takes a seen[] array to mark the pathspec items that had matches in the index. This is used by `git add --refresh` to find out if there were any pathspec without matches, and display an error accordingly.

In the following patch, `git add` will also learn to warn about pathspecs that only match sparse entries (which are not updated). But for that, we will need a seen[] array marked exclusively with matches from dense entries. To avoid having to call ce_path_match() again for these entries after refresh_index() returns, add a flag that implements this restriction inside the function itself.

Note that refresh_index() does not update sparse entries, regardless of passing the flag or not. The flag only controls whether matches with these entries should appear in the seen[] array.

While we are here, also realign the REFRESH_* flags and convert the hex values to the more natural bit shift format, which makes it easier to spot holes.

Signed-off-by: Matheus Tavares <matheus.bernardino@usp.br>
---
 cache.h      | 15 ++++++++-------
 read-cache.c |  5 ++++-
 2 files changed, 12 insertions(+), 8 deletions(-)
Show changes to 2 files +12 −8

cache.h, read-cache.c

diff --git a/cache.h b/cache.h
index d928149614..eec864921a 100644
--- a/cache.h
+++ b/cache.h
@@ -879,13 +879,14 @@ int match_stat_data_racy(const struct index_state *istate,
 
 void fill_stat_cache_info(struct index_state *istate, struct cache_entry *ce, struct stat *st);
 
-#define REFRESH_REALLY		0x0001	/* ignore_valid */
-#define REFRESH_UNMERGED	0x0002	/* allow unmerged */
-#define REFRESH_QUIET		0x0004	/* be quiet about it */
-#define REFRESH_IGNORE_MISSING	0x0008	/* ignore non-existent */
-#define REFRESH_IGNORE_SUBMODULES	0x0010	/* ignore submodules */
-#define REFRESH_IN_PORCELAIN	0x0020	/* user friendly output, not "needs update" */
-#define REFRESH_PROGRESS	0x0040  /* show progress bar if stderr is tty */
+#define REFRESH_REALLY                   (1 << 0) /* ignore_valid */
+#define REFRESH_UNMERGED                 (1 << 1) /* allow unmerged */
+#define REFRESH_QUIET                    (1 << 2) /* be quiet about it */
+#define REFRESH_IGNORE_MISSING           (1 << 3) /* ignore non-existent */
+#define REFRESH_IGNORE_SUBMODULES        (1 << 4) /* ignore submodules */
+#define REFRESH_IN_PORCELAIN             (1 << 5) /* user friendly output, not "needs update" */
+#define REFRESH_PROGRESS                 (1 << 6) /* show progress bar if stderr is tty */
+#define REFRESH_DONT_MARK_SPARSE_MATCHES (1 << 7) /* don't mark sparse entries' matches on seen[] */
 int refresh_index(struct index_state *, unsigned int flags, const struct pathspec *pathspec, char *seen, const char *header_msg);
 /*
  * Refresh the index and write it to disk.
diff --git a/read-cache.c b/read-cache.c
index 29144cf879..485510845c 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -1508,6 +1508,7 @@ int refresh_index(struct index_state *istate, unsigned int flags,
 	int quiet = (flags & REFRESH_QUIET) != 0;
 	int not_new = (flags & REFRESH_IGNORE_MISSING) != 0;
 	int ignore_submodules = (flags & REFRESH_IGNORE_SUBMODULES) != 0;
+	int no_sparse_on_seen = (flags & REFRESH_DONT_MARK_SPARSE_MATCHES) != 0;
 	int first = 1;
 	int in_porcelain = (flags & REFRESH_IN_PORCELAIN);
 	unsigned int options = (CE_MATCH_REFRESH |
@@ -1541,12 +1542,14 @@ int refresh_index(struct index_state *istate, unsigned int flags,
 		int cache_errno = 0;
 		int changed = 0;
 		int filtered = 0;
+		char *cur_seen;
 
 		ce = istate->cache[i];
 		if (ignore_submodules && S_ISGITLINK(ce->ce_mode))
 			continue;
 
-		if (pathspec && !ce_path_match(istate, ce, pathspec, seen))
+		cur_seen = no_sparse_on_seen && ce_skip_worktree(ce) ? NULL : seen;
+		if (pathspec && !ce_path_match(istate, ce, pathspec, cur_seen))
 			filtered = 1;
 
 		if (ce_stage(ce)) {
-- 
2.30.1
Matheus Tavares· Feb 24, 2021, 04:05 UTC · re: Matheus Tavares · lore

[PATCH v2 6/7] add: warn when pathspec only matches SKIP_WORKTREE entries

`git add` already refrains from updating SKIP_WORKTREE entries, but it silently exits with zero code when a pathspec only matches these entries. Instead, let's warn the user and display a hint on how to update these entries.

Note that the warning is only shown if the pathspec matches no untracked paths in the working tree and only matches index entries with the SKIP_WORKTREE bit set. A warning message was chosen over erroring out right away to reproduce the same behavior `add` already exhibits with ignored files. This also allow users to continue their workflow without having to invoke `add` again with only the matching pathspecs, as the matched files will have already been added.

Signed-off-by: Matheus Tavares <matheus.bernardino@usp.br>
---
 Documentation/config/advice.txt |  3 ++
 advice.c                        | 19 +++++++++
 advice.h                        |  4 ++
 builtin/add.c                   | 70 ++++++++++++++++++++++++-------
 pathspec.c                      | 15 +++++++
 pathspec.h                      |  8 ++++
 t/t3705-add-sparse-checkout.sh  | 73 +++++++++++++++++++++++++++++----
 7 files changed, 171 insertions(+), 21 deletions(-)
Show changes to 7 files +171 −21

Documentation/config/advice.txt, advice.c, advice.h, builtin/add.c, pathspec.c, pathspec.h, t/t3705-add-sparse-checkout.sh

diff --git a/Documentation/config/advice.txt b/Documentation/config/advice.txt
index acbd0c09aa..d53eafa00b 100644
--- a/Documentation/config/advice.txt
+++ b/Documentation/config/advice.txt
@@ -119,4 +119,7 @@ advice.*::
 	addEmptyPathspec::
 		Advice shown if a user runs the add command without providing
 		the pathspec parameter.
+	updateSparsePath::
+		Advice shown if the pathspec given to linkgit:git-add[1] only
+		matches index entries outside the current sparse-checkout.
 --
diff --git a/advice.c b/advice.c
index 164742305f..cf22c1a6e5 100644
--- a/advice.c
+++ b/advice.c
@@ -2,6 +2,7 @@
 #include "config.h"
 #include "color.h"
 #include "help.h"
+#include "string-list.h"
 
 int advice_fetch_show_forced_updates = 1;
 int advice_push_update_rejected = 1;
@@ -136,6 +137,7 @@ static struct {
 	[ADVICE_STATUS_HINTS]				= { "statusHints", 1 },
 	[ADVICE_STATUS_U_OPTION]			= { "statusUoption", 1 },
 	[ADVICE_SUBMODULE_ALTERNATE_ERROR_STRATEGY_DIE] = { "submoduleAlternateErrorStrategyDie", 1 },
+	[ADVICE_UPDATE_SPARSE_PATH]			= { "updateSparsePath", 1 },
 	[ADVICE_WAITING_FOR_EDITOR]			= { "waitingForEditor", 1 },
 };
 
@@ -284,6 +286,23 @@ void NORETURN die_conclude_merge(void)
 	die(_("Exiting because of unfinished merge."));
 }
 
+void advise_on_updating_sparse_paths(struct string_list *pathspec_list)
+{
+	struct string_list_item *item;
+
+	if (!pathspec_list->nr)
+		return;
+
+	fprintf(stderr, _("The following pathspecs only matched index entries outside the current\n"
+			  "sparse checkout:\n"));
+	for_each_string_list_item(item, pathspec_list)
+		fprintf(stderr, "%s\n", item->string);
+
+	advise_if_enabled(ADVICE_UPDATE_SPARSE_PATH,
+			  _("Disable or modify the sparsity rules if you intend to update such entries."));
+
+}
+
 void detach_advice(const char *new_name)
 {
 	const char *fmt =
diff --git a/advice.h b/advice.h
index bc2432980a..bd26c385d0 100644
--- a/advice.h
+++ b/advice.h
@@ -3,6 +3,8 @@
 
 #include "git-compat-util.h"
 
+struct string_list;
+
 extern int advice_fetch_show_forced_updates;
 extern int advice_push_update_rejected;
 extern int advice_push_non_ff_current;
@@ -71,6 +73,7 @@ extern int advice_add_empty_pathspec;
 	ADVICE_STATUS_HINTS,
 	ADVICE_STATUS_U_OPTION,
 	ADVICE_SUBMODULE_ALTERNATE_ERROR_STRATEGY_DIE,
+	ADVICE_UPDATE_SPARSE_PATH,
 	ADVICE_WAITING_FOR_EDITOR,
 };
 
@@ -92,6 +95,7 @@ void advise_if_enabled(enum advice_type type, const char *advice, ...);
 int error_resolve_conflict(const char *me);
 void NORETURN die_resolve_conflict(const char *me);
 void NORETURN die_conclude_merge(void);
+void advise_on_updating_sparse_paths(struct string_list *pathspec_list);
 void detach_advice(const char *new_name);
 
 #endif /* ADVICE_H */
diff --git a/builtin/add.c b/builtin/add.c
index e15b25a623..fde6462850 100644
--- a/builtin/add.c
+++ b/builtin/add.c
@@ -177,24 +177,43 @@ static char *prune_directory(struct dir_struct *dir, struct pathspec *pathspec,
 			*dst++ = entry;
 	}
 	dir->nr = dst - dir->entries;
-	add_pathspec_matches_against_index(pathspec, &the_index, seen, 0);
+	add_pathspec_matches_against_index(pathspec, &the_index, seen, 1);
 	return seen;
 }
 
-static void refresh(int verbose, const struct pathspec *pathspec)
+static int refresh(int verbose, const struct pathspec *pathspec)
 {
 	char *seen;
-	int i;
+	int i, ret = 0;
+	char *skip_worktree_seen = NULL;
+	struct string_list only_match_skip_worktree = STRING_LIST_INIT_NODUP;
+	int flags = REFRESH_DONT_MARK_SPARSE_MATCHES |
+		    (verbose ? REFRESH_IN_PORCELAIN : REFRESH_QUIET);
 
 	seen = xcalloc(pathspec->nr, 1);
-	refresh_index(&the_index, verbose ? REFRESH_IN_PORCELAIN : REFRESH_QUIET,
-		      pathspec, seen, _("Unstaged changes after refreshing the index:"));
+	refresh_index(&the_index, flags, pathspec, seen,
+		      _("Unstaged changes after refreshing the index:"));
 	for (i = 0; i < pathspec->nr; i++) {
-		if (!seen[i])
-			die(_("pathspec '%s' did not match any files"),
-			    pathspec->items[i].original);
+		if (!seen[i]) {
+			if (matches_skip_worktree(pathspec, i, &skip_worktree_seen)) {
+				string_list_append(&only_match_skip_worktree,
+						   pathspec->items[i].original);
+			} else {
+				die(_("pathspec '%s' did not match any files"),
+				    pathspec->items[i].original);
+			}
+		}
+	}
+
+	if (only_match_skip_worktree.nr) {
+		advise_on_updating_sparse_paths(&only_match_skip_worktree);
+		ret = 1;
 	}
+
 	free(seen);
+	free(skip_worktree_seen);
+	string_list_clear(&only_match_skip_worktree, 0);
+	return ret;
 }
 
 int run_add_interactive(const char *revision, const char *patch_mode,
@@ -570,15 +589,17 @@ int cmd_add(int argc, const char **argv, const char *prefix)
 	}
 
 	if (refresh_only) {
-		refresh(verbose, &pathspec);
+		exit_status |= refresh(verbose, &pathspec);
 		goto finish;
 	}
 
 	if (pathspec.nr) {
 		int i;
+		char *skip_worktree_seen = NULL;
+		struct string_list only_match_skip_worktree = STRING_LIST_INIT_NODUP;
 
 		if (!seen)
-			seen = find_pathspecs_matching_against_index(&pathspec, &the_index, 0);
+			seen = find_pathspecs_matching_against_index(&pathspec, &the_index, 1);
 
 		/*
 		 * file_exists() assumes exact match
@@ -592,12 +613,24 @@ int cmd_add(int argc, const char **argv, const char *prefix)
 
 		for (i = 0; i < pathspec.nr; i++) {
 			const char *path = pathspec.items[i].match;
+
 			if (pathspec.items[i].magic & PATHSPEC_EXCLUDE)
 				continue;
-			if (!seen[i] && path[0] &&
-			    ((pathspec.items[i].magic &
-			      (PATHSPEC_GLOB | PATHSPEC_ICASE)) ||
-			     !file_exists(path))) {
+			if (seen[i])
+				continue;
+
+			if (matches_skip_worktree(&pathspec, i, &skip_worktree_seen)) {
+				string_list_append(&only_match_skip_worktree,
+						   pathspec.items[i].original);
+				continue;
+			}
+
+			/* Don't complain at 'git add .' inside empty repo. */
+			if (!path[0])
+				continue;
+
+			if ((pathspec.items[i].magic & (PATHSPEC_GLOB | PATHSPEC_ICASE)) ||
+			    !file_exists(path)) {
 				if (ignore_missing) {
 					int dtype = DT_UNKNOWN;
 					if (is_excluded(&dir, &the_index, path, &dtype))
@@ -608,7 +641,16 @@ int cmd_add(int argc, const char **argv, const char *prefix)
 					    pathspec.items[i].original);
 			}
 		}
+
+
+		if (only_match_skip_worktree.nr) {
+			advise_on_updating_sparse_paths(&only_match_skip_worktree);
+			exit_status = 1;
+		}
+
 		free(seen);
+		free(skip_worktree_seen);
+		string_list_clear(&only_match_skip_worktree, 0);
 	}
 
 	plug_bulk_checkin();
diff --git a/pathspec.c b/pathspec.c
index e5e6b7458d..61f294fed5 100644
--- a/pathspec.c
+++ b/pathspec.c
@@ -62,6 +62,21 @@ char *find_pathspecs_matching_against_index(const struct pathspec *pathspec,
 	return seen;
 }
 
+char *find_pathspecs_matching_skip_worktree(const struct pathspec *pathspec)
+{
+	struct index_state *istate = the_repository->index;
+	char *seen = xcalloc(pathspec->nr, 1);
+	int i;
+
+	for (i = 0; i < istate->cache_nr; i++) {
+		struct cache_entry *ce = istate->cache[i];
+		if (ce_skip_worktree(ce))
+		    ce_path_match(istate, ce, pathspec, seen);
+	}
+
+	return seen;
+}
+
 /*
  * Magic pathspec
  *
diff --git a/pathspec.h b/pathspec.h
index 8202882ecd..f591ba625c 100644
--- a/pathspec.h
+++ b/pathspec.h
@@ -155,6 +155,14 @@ void add_pathspec_matches_against_index(const struct pathspec *pathspec,
 char *find_pathspecs_matching_against_index(const struct pathspec *pathspec,
 					    const struct index_state *istate,
 					    int ignore_skip_worktree);
+char *find_pathspecs_matching_skip_worktree(const struct pathspec *pathspec);
+static inline int matches_skip_worktree(const struct pathspec *pathspec,
+					int item, char **seen_ptr)
+{
+	if (!*seen_ptr)
+		*seen_ptr = find_pathspecs_matching_skip_worktree(pathspec);
+	return (*seen_ptr)[item];
+}
 int match_pathspec_attrs(const struct index_state *istate,
 			 const char *name, int namelen,
 			 const struct pathspec_item *item);
diff --git a/t/t3705-add-sparse-checkout.sh b/t/t3705-add-sparse-checkout.sh
index 6781620297..fdfd8b085e 100755
--- a/t/t3705-add-sparse-checkout.sh
+++ b/t/t3705-add-sparse-checkout.sh
@@ -36,10 +36,26 @@ setup_gitignore () {
 	EOF
 }
 
+test_expect_success 'setup' '
+	cat >sparse_error_header <<-EOF &&
+	The following pathspecs only matched index entries outside the current
+	sparse checkout:
+	EOF
+
+	cat >sparse_hint <<-EOF &&
+	hint: Disable or modify the sparsity rules if you intend to update such entries.
+	hint: Disable this message with "git config advice.updateSparsePath false"
+	EOF
+
+	echo sparse_entry | cat sparse_error_header - >sparse_entry_error &&
+	cat sparse_entry_error sparse_hint >error_and_hint
+'
+
 test_expect_success 'git add does not remove sparse entries' '
 	setup_sparse_entry &&
 	rm sparse_entry &&
-	git add sparse_entry &&
+	test_must_fail git add sparse_entry 2>stderr &&
+	test_i18ncmp error_and_hint stderr &&
 	test_sparse_entry_unchanged
 '
 
@@ -47,7 +63,8 @@ test_expect_success 'git add -A does not remove sparse entries' '
 	setup_sparse_entry &&
 	rm sparse_entry &&
 	setup_gitignore &&
-	git add -A &&
+	git add -A 2>stderr &&
+	test_must_be_empty stderr &&
 	test_sparse_entry_unchanged
 '
 
@@ -55,7 +72,13 @@ test_expect_success 'git add . does not remove sparse entries' '
 	setup_sparse_entry &&
 	rm sparse_entry &&
 	setup_gitignore &&
-	git add . &&
+	test_must_fail git add . 2>stderr &&
+
+	cat sparse_error_header >expect &&
+	echo . >>expect &&
+	cat sparse_hint >>expect &&
+
+	test_i18ncmp expect stderr &&
 	test_sparse_entry_unchanged
 '
 
@@ -64,7 +87,8 @@ do
 	test_expect_success "git add${opt:+ $opt} does not update sparse entries" '
 		setup_sparse_entry &&
 		echo modified >sparse_entry &&
-		git add $opt sparse_entry &&
+		test_must_fail git add $opt sparse_entry 2>stderr &&
+		test_i18ncmp error_and_hint stderr &&
 		test_sparse_entry_unchanged
 	'
 done
@@ -73,14 +97,16 @@ test_expect_success 'git add --refresh does not update sparse entries' '
 	setup_sparse_entry &&
 	git ls-files --debug sparse_entry | grep mtime >before &&
 	test-tool chmtime -60 sparse_entry &&
-	git add --refresh sparse_entry &&
+	test_must_fail git add --refresh sparse_entry 2>stderr &&
+	test_i18ncmp error_and_hint stderr &&
 	git ls-files --debug sparse_entry | grep mtime >after &&
 	test_cmp before after
 '
 
 test_expect_success 'git add --chmod does not update sparse entries' '
 	setup_sparse_entry &&
-	git add --chmod=+x sparse_entry &&
+	test_must_fail git add --chmod=+x sparse_entry 2>stderr &&
+	test_i18ncmp error_and_hint stderr &&
 	test_sparse_entry_unchanged &&
 	! test -x sparse_entry
 '
@@ -89,8 +115,41 @@ test_expect_success 'git add --renormalize does not update sparse entries' '
 	test_config core.autocrlf false &&
 	setup_sparse_entry "LINEONE\r\nLINETWO\r\n" &&
 	echo "sparse_entry text=auto" >.gitattributes &&
-	git add --renormalize sparse_entry &&
+	test_must_fail git add --renormalize sparse_entry 2>stderr &&
+	test_i18ncmp error_and_hint stderr &&
 	test_sparse_entry_unchanged
 '
 
+test_expect_success 'git add --dry-run --ignore-missing warn on sparse path' '
+	setup_sparse_entry &&
+	rm sparse_entry &&
+	test_must_fail git add --dry-run --ignore-missing sparse_entry 2>stderr &&
+	test_i18ncmp error_and_hint stderr &&
+	test_sparse_entry_unchanged
+'
+
+test_expect_success 'do not advice about sparse entries when they do not match the pathspec' '
+	setup_sparse_entry &&
+	test_must_fail git add nonexistent 2>stderr &&
+	test_i18ngrep "fatal: pathspec .nonexistent. did not match any files" stderr &&
+	test_i18ngrep ! "The following pathspecs only matched index entries" stderr
+'
+
+test_expect_success 'do not warn when pathspec matches dense entries' '
+	setup_sparse_entry &&
+	echo modified >sparse_entry &&
+	>dense_entry &&
+	git add "*_entry" 2>stderr &&
+	test_must_be_empty stderr &&
+	test_sparse_entry_unchanged &&
+	git ls-files --error-unmatch dense_entry
+'
+
+test_expect_success 'add obeys advice.updateSparsePath' '
+	setup_sparse_entry &&
+	test_must_fail git -c advice.updateSparsePath=false add sparse_entry 2>stderr &&
+	test_i18ncmp sparse_entry_error stderr
+
+'
+
 test_done
-- 
2.30.1
Elijah Newren· Feb 24, 2021, 06:50 UTC · re: Matheus Tavares · lore

Re: [PATCH v2 6/7] add: warn when pathspec only matches SKIP_WORKTREE entries

On Tue, Feb 23, 2021 at 8:05 PM Matheus Tavares <matheus.bernardino@usp.br> wrote:

Show 72 quoted lines
>
> `git add` already refrains from updating SKIP_WORKTREE entries, but it
> silently exits with zero code when a pathspec only matches these
> entries. Instead, let's warn the user and display a hint on how to
> update these entries.
>
> Note that the warning is only shown if the pathspec matches no untracked
> paths in the working tree and only matches index entries with the
> SKIP_WORKTREE bit set. A warning message was chosen over erroring out
> right away to reproduce the same behavior `add` already exhibits with
> ignored files. This also allow users to continue their workflow without
> having to invoke `add` again with only the matching pathspecs, as the
> matched files will have already been added.
>
> Signed-off-by: Matheus Tavares <matheus.bernardino@usp.br>
> ---
>  Documentation/config/advice.txt |  3 ++
>  advice.c                        | 19 +++++++++
>  advice.h                        |  4 ++
>  builtin/add.c                   | 70 ++++++++++++++++++++++++-------
>  pathspec.c                      | 15 +++++++
>  pathspec.h                      |  8 ++++
>  t/t3705-add-sparse-checkout.sh  | 73 +++++++++++++++++++++++++++++----
>  7 files changed, 171 insertions(+), 21 deletions(-)
>
> diff --git a/Documentation/config/advice.txt b/Documentation/config/advice.txt
> index acbd0c09aa..d53eafa00b 100644
> --- a/Documentation/config/advice.txt
> +++ b/Documentation/config/advice.txt
> @@ -119,4 +119,7 @@ advice.*::
>         addEmptyPathspec::
>                 Advice shown if a user runs the add command without providing
>                 the pathspec parameter.
> +       updateSparsePath::
> +               Advice shown if the pathspec given to linkgit:git-add[1] only
> +               matches index entries outside the current sparse-checkout.
>  --
> diff --git a/advice.c b/advice.c
> index 164742305f..cf22c1a6e5 100644
> --- a/advice.c
> +++ b/advice.c
> @@ -2,6 +2,7 @@
>  #include "config.h"
>  #include "color.h"
>  #include "help.h"
> +#include "string-list.h"
>
>  int advice_fetch_show_forced_updates = 1;
>  int advice_push_update_rejected = 1;
> @@ -136,6 +137,7 @@ static struct {
>         [ADVICE_STATUS_HINTS]                           = { "statusHints", 1 },
>         [ADVICE_STATUS_U_OPTION]                        = { "statusUoption", 1 },
>         [ADVICE_SUBMODULE_ALTERNATE_ERROR_STRATEGY_DIE] = { "submoduleAlternateErrorStrategyDie", 1 },
> +       [ADVICE_UPDATE_SPARSE_PATH]                     = { "updateSparsePath", 1 },
>         [ADVICE_WAITING_FOR_EDITOR]                     = { "waitingForEditor", 1 },
>  };
>
> @@ -284,6 +286,23 @@ void NORETURN die_conclude_merge(void)
>         die(_("Exiting because of unfinished merge."));
>  }
>
> +void advise_on_updating_sparse_paths(struct string_list *pathspec_list)
> +{
> +       struct string_list_item *item;
> +
> +       if (!pathspec_list->nr)
> +               return;
> +
> +       fprintf(stderr, _("The following pathspecs only matched index entries outside the current\n"
> +                         "sparse checkout:\n"));
> +       for_each_string_list_item(item, pathspec_list)
> +               fprintf(stderr, "%s\n", item->string);

Was the use of fprintf(stderr, ...) because of the fact that you want to do multiple print statements? I'm just curious if that was the reason for avoiding the warning() function, or if there was another consideration at play as well.

Show 48 quoted lines
> +
> +       advise_if_enabled(ADVICE_UPDATE_SPARSE_PATH,
> +                         _("Disable or modify the sparsity rules if you intend to update such entries."));
> +
> +}
> +
>  void detach_advice(const char *new_name)
>  {
>         const char *fmt =
> diff --git a/advice.h b/advice.h
> index bc2432980a..bd26c385d0 100644
> --- a/advice.h
> +++ b/advice.h
> @@ -3,6 +3,8 @@
>
>  #include "git-compat-util.h"
>
> +struct string_list;
> +
>  extern int advice_fetch_show_forced_updates;
>  extern int advice_push_update_rejected;
>  extern int advice_push_non_ff_current;
> @@ -71,6 +73,7 @@ extern int advice_add_empty_pathspec;
>         ADVICE_STATUS_HINTS,
>         ADVICE_STATUS_U_OPTION,
>         ADVICE_SUBMODULE_ALTERNATE_ERROR_STRATEGY_DIE,
> +       ADVICE_UPDATE_SPARSE_PATH,
>         ADVICE_WAITING_FOR_EDITOR,
>  };
>
> @@ -92,6 +95,7 @@ void advise_if_enabled(enum advice_type type, const char *advice, ...);
>  int error_resolve_conflict(const char *me);
>  void NORETURN die_resolve_conflict(const char *me);
>  void NORETURN die_conclude_merge(void);
> +void advise_on_updating_sparse_paths(struct string_list *pathspec_list);
>  void detach_advice(const char *new_name);
>
>  #endif /* ADVICE_H */
> diff --git a/builtin/add.c b/builtin/add.c
> index e15b25a623..fde6462850 100644
> --- a/builtin/add.c
> +++ b/builtin/add.c
> @@ -177,24 +177,43 @@ static char *prune_directory(struct dir_struct *dir, struct pathspec *pathspec,
>                         *dst++ = entry;
>         }
>         dir->nr = dst - dir->entries;
> -       add_pathspec_matches_against_index(pathspec, &the_index, seen, 0);
> +       add_pathspec_matches_against_index(pathspec, &the_index, seen, 1);

_If_ you add the enum as mentioned earlier, this 1 would become whatever the other enum value was. I'll omit making similar comments for other call sites like this one, though there are a few more in the patch.

Show 38 quoted lines
>         return seen;
>  }
>
> -static void refresh(int verbose, const struct pathspec *pathspec)
> +static int refresh(int verbose, const struct pathspec *pathspec)
>  {
>         char *seen;
> -       int i;
> +       int i, ret = 0;
> +       char *skip_worktree_seen = NULL;
> +       struct string_list only_match_skip_worktree = STRING_LIST_INIT_NODUP;
> +       int flags = REFRESH_DONT_MARK_SPARSE_MATCHES |
> +                   (verbose ? REFRESH_IN_PORCELAIN : REFRESH_QUIET);
>
>         seen = xcalloc(pathspec->nr, 1);
> -       refresh_index(&the_index, verbose ? REFRESH_IN_PORCELAIN : REFRESH_QUIET,
> -                     pathspec, seen, _("Unstaged changes after refreshing the index:"));
> +       refresh_index(&the_index, flags, pathspec, seen,
> +                     _("Unstaged changes after refreshing the index:"));
>         for (i = 0; i < pathspec->nr; i++) {
> -               if (!seen[i])
> -                       die(_("pathspec '%s' did not match any files"),
> -                           pathspec->items[i].original);
> +               if (!seen[i]) {
> +                       if (matches_skip_worktree(pathspec, i, &skip_worktree_seen)) {
> +                               string_list_append(&only_match_skip_worktree,
> +                                                  pathspec->items[i].original);
> +                       } else {
> +                               die(_("pathspec '%s' did not match any files"),
> +                                   pathspec->items[i].original);
> +                       }
> +               }
> +       }
> +
> +       if (only_match_skip_worktree.nr) {
> +               advise_on_updating_sparse_paths(&only_match_skip_worktree);
> +               ret = 1;
>         }

On first reading, I missed that the code die()s if there are any non-SKIP_WORKTREE entries matched, and that is the reason you know that only SKIP_WORKTREE entries could have been matched for this last if-statement. Perhaps a short comment for the reader like me who is prone to miss the obvious?

Show 54 quoted lines
> +
>         free(seen);
> +       free(skip_worktree_seen);
> +       string_list_clear(&only_match_skip_worktree, 0);
> +       return ret;
>  }
>
>  int run_add_interactive(const char *revision, const char *patch_mode,
> @@ -570,15 +589,17 @@ int cmd_add(int argc, const char **argv, const char *prefix)
>         }
>
>         if (refresh_only) {
> -               refresh(verbose, &pathspec);
> +               exit_status |= refresh(verbose, &pathspec);
>                 goto finish;
>         }
>
>         if (pathspec.nr) {
>                 int i;
> +               char *skip_worktree_seen = NULL;
> +               struct string_list only_match_skip_worktree = STRING_LIST_INIT_NODUP;
>
>                 if (!seen)
> -                       seen = find_pathspecs_matching_against_index(&pathspec, &the_index, 0);
> +                       seen = find_pathspecs_matching_against_index(&pathspec, &the_index, 1);
>
>                 /*
>                  * file_exists() assumes exact match
> @@ -592,12 +613,24 @@ int cmd_add(int argc, const char **argv, const char *prefix)
>
>                 for (i = 0; i < pathspec.nr; i++) {
>                         const char *path = pathspec.items[i].match;
> +
>                         if (pathspec.items[i].magic & PATHSPEC_EXCLUDE)
>                                 continue;
> -                       if (!seen[i] && path[0] &&
> -                           ((pathspec.items[i].magic &
> -                             (PATHSPEC_GLOB | PATHSPEC_ICASE)) ||
> -                            !file_exists(path))) {
> +                       if (seen[i])
> +                               continue;
> +
> +                       if (matches_skip_worktree(&pathspec, i, &skip_worktree_seen)) {
> +                               string_list_append(&only_match_skip_worktree,
> +                                                  pathspec.items[i].original);
> +                               continue;
> +                       }
> +
> +                       /* Don't complain at 'git add .' inside empty repo. */
> +                       if (!path[0])
> +                               continue;
> +
> +                       if ((pathspec.items[i].magic & (PATHSPEC_GLOB | PATHSPEC_ICASE)) ||
> +                           !file_exists(path)) {

Breaking up that if-statement into several with some continues seems to make it a lot easier for me to read; thanks. It also makes it easier to add your new condition.

Show 13 quoted lines
>                                 if (ignore_missing) {
>                                         int dtype = DT_UNKNOWN;
>                                         if (is_excluded(&dir, &the_index, path, &dtype))
> @@ -608,7 +641,16 @@ int cmd_add(int argc, const char **argv, const char *prefix)
>                                             pathspec.items[i].original);
>                         }
>                 }
> +
> +
> +               if (only_match_skip_worktree.nr) {
> +                       advise_on_updating_sparse_paths(&only_match_skip_worktree);
> +                       exit_status = 1;
> +               }
Hmm...here's an interesting command sequence:

git init lame cd lame mkdir baz touch baz/tracked git add baz/tracked git update-index --skip-worktree baz/tracked rm baz/tracked. # But leave the empty directory! echo baz >.gitignore git add --ignore-missing --dry-run baz

Reports the following: """ The following pathspecs only matched index entries outside the current sparse checkout: baz hint: Disable or modify the sparsity rules if you intend to update such entries. hint: Disable this message with "git config advice.updateSparsePath false" The following paths are ignored by one of your .gitignore files: baz hint: Use -f if you really want to add them. hint: Turn this message off by running hint: "git config advice.addIgnoredFile false" """

That's probably okay because it does match both, but the "only matched" in the first message followed by saying it matched something else seems a little surprising at first. It's not wrong, just slightly surprising. But then again, this setup is super weird...so maybe it's all okay?

Show 154 quoted lines
> +
>                 free(seen);
> +               free(skip_worktree_seen);
> +               string_list_clear(&only_match_skip_worktree, 0);
>         }
>
>         plug_bulk_checkin();
> diff --git a/pathspec.c b/pathspec.c
> index e5e6b7458d..61f294fed5 100644
> --- a/pathspec.c
> +++ b/pathspec.c
> @@ -62,6 +62,21 @@ char *find_pathspecs_matching_against_index(const struct pathspec *pathspec,
>         return seen;
>  }
>
> +char *find_pathspecs_matching_skip_worktree(const struct pathspec *pathspec)
> +{
> +       struct index_state *istate = the_repository->index;
> +       char *seen = xcalloc(pathspec->nr, 1);
> +       int i;
> +
> +       for (i = 0; i < istate->cache_nr; i++) {
> +               struct cache_entry *ce = istate->cache[i];
> +               if (ce_skip_worktree(ce))
> +                   ce_path_match(istate, ce, pathspec, seen);
> +       }
> +
> +       return seen;
> +}
> +
>  /*
>   * Magic pathspec
>   *
> diff --git a/pathspec.h b/pathspec.h
> index 8202882ecd..f591ba625c 100644
> --- a/pathspec.h
> +++ b/pathspec.h
> @@ -155,6 +155,14 @@ void add_pathspec_matches_against_index(const struct pathspec *pathspec,
>  char *find_pathspecs_matching_against_index(const struct pathspec *pathspec,
>                                             const struct index_state *istate,
>                                             int ignore_skip_worktree);
> +char *find_pathspecs_matching_skip_worktree(const struct pathspec *pathspec);
> +static inline int matches_skip_worktree(const struct pathspec *pathspec,
> +                                       int item, char **seen_ptr)
> +{
> +       if (!*seen_ptr)
> +               *seen_ptr = find_pathspecs_matching_skip_worktree(pathspec);
> +       return (*seen_ptr)[item];
> +}
>  int match_pathspec_attrs(const struct index_state *istate,
>                          const char *name, int namelen,
>                          const struct pathspec_item *item);
> diff --git a/t/t3705-add-sparse-checkout.sh b/t/t3705-add-sparse-checkout.sh
> index 6781620297..fdfd8b085e 100755
> --- a/t/t3705-add-sparse-checkout.sh
> +++ b/t/t3705-add-sparse-checkout.sh
> @@ -36,10 +36,26 @@ setup_gitignore () {
>         EOF
>  }
>
> +test_expect_success 'setup' '
> +       cat >sparse_error_header <<-EOF &&
> +       The following pathspecs only matched index entries outside the current
> +       sparse checkout:
> +       EOF
> +
> +       cat >sparse_hint <<-EOF &&
> +       hint: Disable or modify the sparsity rules if you intend to update such entries.
> +       hint: Disable this message with "git config advice.updateSparsePath false"
> +       EOF
> +
> +       echo sparse_entry | cat sparse_error_header - >sparse_entry_error &&
> +       cat sparse_entry_error sparse_hint >error_and_hint
> +'
> +
>  test_expect_success 'git add does not remove sparse entries' '
>         setup_sparse_entry &&
>         rm sparse_entry &&
> -       git add sparse_entry &&
> +       test_must_fail git add sparse_entry 2>stderr &&
> +       test_i18ncmp error_and_hint stderr &&
>         test_sparse_entry_unchanged
>  '
>
> @@ -47,7 +63,8 @@ test_expect_success 'git add -A does not remove sparse entries' '
>         setup_sparse_entry &&
>         rm sparse_entry &&
>         setup_gitignore &&
> -       git add -A &&
> +       git add -A 2>stderr &&
> +       test_must_be_empty stderr &&
>         test_sparse_entry_unchanged
>  '
>
> @@ -55,7 +72,13 @@ test_expect_success 'git add . does not remove sparse entries' '
>         setup_sparse_entry &&
>         rm sparse_entry &&
>         setup_gitignore &&
> -       git add . &&
> +       test_must_fail git add . 2>stderr &&
> +
> +       cat sparse_error_header >expect &&
> +       echo . >>expect &&
> +       cat sparse_hint >>expect &&
> +
> +       test_i18ncmp expect stderr &&
>         test_sparse_entry_unchanged
>  '
>
> @@ -64,7 +87,8 @@ do
>         test_expect_success "git add${opt:+ $opt} does not update sparse entries" '
>                 setup_sparse_entry &&
>                 echo modified >sparse_entry &&
> -               git add $opt sparse_entry &&
> +               test_must_fail git add $opt sparse_entry 2>stderr &&
> +               test_i18ncmp error_and_hint stderr &&
>                 test_sparse_entry_unchanged
>         '
>  done
> @@ -73,14 +97,16 @@ test_expect_success 'git add --refresh does not update sparse entries' '
>         setup_sparse_entry &&
>         git ls-files --debug sparse_entry | grep mtime >before &&
>         test-tool chmtime -60 sparse_entry &&
> -       git add --refresh sparse_entry &&
> +       test_must_fail git add --refresh sparse_entry 2>stderr &&
> +       test_i18ncmp error_and_hint stderr &&
>         git ls-files --debug sparse_entry | grep mtime >after &&
>         test_cmp before after
>  '
>
>  test_expect_success 'git add --chmod does not update sparse entries' '
>         setup_sparse_entry &&
> -       git add --chmod=+x sparse_entry &&
> +       test_must_fail git add --chmod=+x sparse_entry 2>stderr &&
> +       test_i18ncmp error_and_hint stderr &&
>         test_sparse_entry_unchanged &&
>         ! test -x sparse_entry
>  '
> @@ -89,8 +115,41 @@ test_expect_success 'git add --renormalize does not update sparse entries' '
>         test_config core.autocrlf false &&
>         setup_sparse_entry "LINEONE\r\nLINETWO\r\n" &&
>         echo "sparse_entry text=auto" >.gitattributes &&
> -       git add --renormalize sparse_entry &&
> +       test_must_fail git add --renormalize sparse_entry 2>stderr &&
> +       test_i18ncmp error_and_hint stderr &&
>         test_sparse_entry_unchanged
>  '
>
> +test_expect_success 'git add --dry-run --ignore-missing warn on sparse path' '
> +       setup_sparse_entry &&
> +       rm sparse_entry &&
> +       test_must_fail git add --dry-run --ignore-missing sparse_entry 2>stderr &&
> +       test_i18ncmp error_and_hint stderr &&
> +       test_sparse_entry_unchanged

See also the slightly convoluted setup I responded above with. I'm not sure if we need to make it a test, in part because it's crazy enough that I'm not quite convinced that the current behavior is right -- or wrong.

Show 29 quoted lines
> +'
> +
> +test_expect_success 'do not advice about sparse entries when they do not match the pathspec' '
> +       setup_sparse_entry &&
> +       test_must_fail git add nonexistent 2>stderr &&
> +       test_i18ngrep "fatal: pathspec .nonexistent. did not match any files" stderr &&
> +       test_i18ngrep ! "The following pathspecs only matched index entries" stderr
> +'
> +
> +test_expect_success 'do not warn when pathspec matches dense entries' '
> +       setup_sparse_entry &&
> +       echo modified >sparse_entry &&
> +       >dense_entry &&
> +       git add "*_entry" 2>stderr &&
> +       test_must_be_empty stderr &&
> +       test_sparse_entry_unchanged &&
> +       git ls-files --error-unmatch dense_entry
> +'
> +
> +test_expect_success 'add obeys advice.updateSparsePath' '
> +       setup_sparse_entry &&
> +       test_must_fail git -c advice.updateSparsePath=false add sparse_entry 2>stderr &&
> +       test_i18ncmp sparse_entry_error stderr
> +
> +'
> +
>  test_done
> --
> 2.30.1

Overall, a nice read. Didn't spot any other issues, just the minor points I highlighted above. I'm curious if others have thoughts on my really weird setup and the dual warnings, though.

Matheus Tavares· Feb 24, 2021, 15:33 UTC · re: Elijah Newren · lore

Re: [PATCH v2 6/7] add: warn when pathspec only matches SKIP_WORKTREE entries

On Wed, Feb 24, 2021 at 3:50 AM Elijah Newren <newren@gmail.com> wrote:
Show 20 quoted lines
>
> On Tue, Feb 23, 2021 at 8:05 PM Matheus Tavares
> <matheus.bernardino@usp.br> wrote:
> >
> > +void advise_on_updating_sparse_paths(struct string_list *pathspec_list)
> > +{
> > +       struct string_list_item *item;
> > +
> > +       if (!pathspec_list->nr)
> > +               return;
> > +
> > +       fprintf(stderr, _("The following pathspecs only matched index entries outside the current\n"
> > +                         "sparse checkout:\n"));
> > +       for_each_string_list_item(item, pathspec_list)
> > +               fprintf(stderr, "%s\n", item->string);
>
> Was the use of fprintf(stderr, ...) because of the fact that you want
> to do multiple print statements?  I'm just curious if that was the
> reason for avoiding the warning() function, or if there was another
> consideration at play as well.

Yes, that was one of the reasons. The other was to use the same style as the ignored files message, which doesn't print the "warning:" prefix. But I don't have any strong preference here, I'd be OK with using warning() too.

Show 40 quoted lines
> > -static void refresh(int verbose, const struct pathspec *pathspec)
> > +static int refresh(int verbose, const struct pathspec *pathspec)
> >  {
> >         char *seen;
> > -       int i;
> > +       int i, ret = 0;
> > +       char *skip_worktree_seen = NULL;
> > +       struct string_list only_match_skip_worktree = STRING_LIST_INIT_NODUP;
> > +       int flags = REFRESH_DONT_MARK_SPARSE_MATCHES |
> > +                   (verbose ? REFRESH_IN_PORCELAIN : REFRESH_QUIET);
> >
> >         seen = xcalloc(pathspec->nr, 1);
> > -       refresh_index(&the_index, verbose ? REFRESH_IN_PORCELAIN : REFRESH_QUIET,
> > -                     pathspec, seen, _("Unstaged changes after refreshing the index:"));
> > +       refresh_index(&the_index, flags, pathspec, seen,
> > +                     _("Unstaged changes after refreshing the index:"));
> >         for (i = 0; i < pathspec->nr; i++) {
> > -               if (!seen[i])
> > -                       die(_("pathspec '%s' did not match any files"),
> > -                           pathspec->items[i].original);
> > +               if (!seen[i]) {
> > +                       if (matches_skip_worktree(pathspec, i, &skip_worktree_seen)) {
> > +                               string_list_append(&only_match_skip_worktree,
> > +                                                  pathspec->items[i].original);
> > +                       } else {
> > +                               die(_("pathspec '%s' did not match any files"),
> > +                                   pathspec->items[i].original);
> > +                       }
> > +               }
> > +       }
> > +
> > +       if (only_match_skip_worktree.nr) {
> > +               advise_on_updating_sparse_paths(&only_match_skip_worktree);
> > +               ret = 1;
> >         }
>
> On first reading, I missed that the code die()s if there are any
> non-SKIP_WORKTREE entries matched, and that is the reason you know
> that only SKIP_WORKTREE entries could have been matched for this last
> if-statement.

Hmm, I may be misinterpreting your explanation, but I think the reasoning is slightly different. The code die()s if there are _no_ matches either among sparse or dense entries. The reason why we know that only sparse entries matched the pathspecs in this last if-statement is because the `only_match_skip_worktree` list is only appended when a pathspec is not marked on seen[] (dense entries only), but it is marked on skip_worktree_seen[] (sparse entries only).

Show 26 quoted lines
> Hmm...here's an interesting command sequence:
>
> git init lame
> cd lame
> mkdir baz
> touch baz/tracked
> git add baz/tracked
> git update-index --skip-worktree baz/tracked
> rm baz/tracked.  # But leave the empty directory!
> echo baz >.gitignore
> git add --ignore-missing --dry-run baz
>
>
> Reports the following:
> """
> The following pathspecs only matched index entries outside the current
> sparse checkout:
> baz
> hint: Disable or modify the sparsity rules if you intend to update such entries.
> hint: Disable this message with "git config advice.updateSparsePath false"
> The following paths are ignored by one of your .gitignore files:
> baz
> hint: Use -f if you really want to add them.
> hint: Turn this message off by running
> hint: "git config advice.addIgnoredFile false"
> """

That's interesting. You can also trigger this behavior with a plain add (i.e. without "--ignore-missing --dry-run").

Since we only get the list of ignored paths from fill_directory(), we can't really tell whether a specific pathspec item had matches among ignored files or not. If we had this information, we could conditionally skip the sparse warning.

I.e. something like this (WARNING: hacky and just briefly tested):
Show changes to 3 files +15 −3

builtin/add.c, dir.c, dir.h

diff --git a/builtin/add.c b/builtin/add.c
index fde6462850..90614e7e76 100644
--- a/builtin/add.c
+++ b/builtin/add.c
@@ -597,3 +597,3 @@ int cmd_add(int argc, const char **argv, const char *prefix)
 		int i;
-		char *skip_worktree_seen = NULL;
+		char *skip_worktree_seen = NULL, *ignored_seen = NULL;
 		struct string_list only_match_skip_worktree = STRING_LIST_INIT_NODUP;
@@ -621,3 +621,14 @@ int cmd_add(int argc, const char **argv, const char *prefix)

-			if (matches_skip_worktree(&pathspec, i, &skip_worktree_seen)) {
+			if (dir.ignored_nr) {
+				int j, prefix_len = common_prefix_len(&pathspec);
+				ignored_seen = xcalloc(pathspec.nr, 1);
+				for (j = 0; j < dir.ignored_nr; j++) {
+					dir_path_match(&the_index, dir.ignored[j],
+						       &pathspec, prefix_len,
+						       ignored_seen);
+				}
+			}
+
+			if (ignored_seen && !ignored_seen[i] &&
+			    matches_skip_worktree(&pathspec, i, &skip_worktree_seen)) {
 				string_list_append(&only_match_skip_worktree,
diff --git a/dir.c b/dir.c
index d153a63bbd..a19bc7aa0b 100644
--- a/dir.c
+++ b/dir.c
@@ -136,3 +136,3 @@ static int fnmatch_icase_mem(const char *pattern, int patternlen,

-static size_t common_prefix_len(const struct pathspec *pathspec)
+size_t common_prefix_len(const struct pathspec *pathspec)
 {
diff --git a/dir.h b/dir.h
index facfae4740..aa2d4aa71b 100644
--- a/dir.h
+++ b/dir.h
@@ -355,2 +355,3 @@ int simple_length(const char *match);
 int no_wildcard(const char *string);
+size_t common_prefix_len(const struct pathspec *pathspec);
 char *common_prefix(const struct pathspec *pathspec);


Now `git add baz` would only produce:

The following paths are ignored by one of your .gitignore files:
baz
hint: Use -f if you really want to add them.
hint: Turn this message off by running
hint: "git config advice.addIgnoredFile false"
Matheus Tavares· Mar 4, 2021, 15:23 UTC · re: Elijah Newren · lore

Re: [PATCH v2 6/7] add: warn when pathspec only matches SKIP_WORKTREE entries

Hi, Elijah

I was going to send v3 a couple days ago, but there is still one issue that I'm having a bit of trouble with:

On Wed, Feb 24, 2021 at 3:50 AM Elijah Newren <newren@gmail.com> wrote:
Show 31 quoted lines
>
> Hmm...here's an interesting command sequence:
>
> git init lame
> cd lame
> mkdir baz
> touch baz/tracked
> git add baz/tracked
> git update-index --skip-worktree baz/tracked
> rm baz/tracked.  # But leave the empty directory!
> echo baz >.gitignore
> git add --ignore-missing --dry-run baz
>
>
> Reports the following:
> """
> The following pathspecs only matched index entries outside the current
> sparse checkout:
> baz
> hint: Disable or modify the sparsity rules if you intend to update such entries.
> hint: Disable this message with "git config advice.updateSparsePath false"
> The following paths are ignored by one of your .gitignore files:
> baz
> hint: Use -f if you really want to add them.
> hint: Turn this message off by running
> hint: "git config advice.addIgnoredFile false"
> """
>
> That's probably okay because it does match both, but the "only
> matched" in the first message followed by saying it matched something
> else seems a little surprising at first.

After giving this a little more thought, I got to this idea [1]. With this patch, git-add doesn't warn about sparse paths that are inside ignored dirs (it only warns about the ignored dirs themselves). If --force is used, then git-add recurses into the ignored directories, and warns about any sparse path inside them. I think this is more consistent with the way in which ignored dirs (and their contents) are handled with and without --force. And we avoid the double warning.

But there is a performance penalty. The relevant code is:
Show changes to diff +12 −8
@@ -2029,19 +2033,23 @@ static int exclude_matches_pathspec(const char *path, int pathlen,
 		       PATHSPEC_ICASE |
 		       PATHSPEC_EXCLUDE);
 
+	assert(matched_ignored);
+
 	for (i = 0; i < pathspec->nr; i++) {
 		const struct pathspec_item *item = &pathspec->items[i];
 		int len = item->nowildcard_len;
 
-		if (len == pathlen &&
-		    !ps_strncmp(item, item->match, path, pathlen))
-			return 1;
-		if (len > pathlen &&
-		    item->match[pathlen] == '/' &&
-		    !ps_strncmp(item, item->match, path, pathlen))
-			return 1;
+		if (matched && matched_ignored[i])
+			continue;
+
+		if ((len == pathlen ||
+		    (len > pathlen && item->match[pathlen] == '/')) &&
+		    !ps_strncmp(item, item->match, path, pathlen)) {
+			matched = 1;
+			matched_ignored[i] = 1;
+		}
 	}
-	return 0;
+	return matched;
 }

This function is called for each ignored path (not recursing into
completely ignored dirs, though). Before the patch, the internal loop
stops at the first pathspec match. But with the patch, the loop must
continue as we need to know about all pathspecs that had matches with
the ignored paths...

And there is also one odd case:
$ mkdir d
$ touch d/s d/i
$ git add d/s
$ git update-index --skip-worktree d/s
$ echo d/i >.gitinore
$ git add 'd/[is]'

This outputs:
The following pathspecs only matched index entries outside the current
sparse checkout:
d/[is]
(and no warning on ignored paths).

I don't think the warning here is 100% correct, as 'd/[is]' *does* match
other files, only they are ignored. And git-add only warns about an
ignored path when the pathspec is an exact match with it (no glob
patterns) or inside it. 

So, I don't know... Perhaps should we just rephrase the sparse warning
to remove the "only" part (and don't try to avoid the double warnings)?
I'm open to any suggestions on alternative wordings or ideas :)

[1]: https://github.com/matheustavares/git/commit/5d4be321e1f69cb08b0666ffd3f8fa658e9f6953
Elijah Newren· Mar 4, 2021, 17:21 UTC · re: Matheus Tavares · lore

Re: [PATCH v2 6/7] add: warn when pathspec only matches SKIP_WORKTREE entries

On Thu, Mar 4, 2021 at 7:23 AM Matheus Tavares <matheus.bernardino@usp.br> wrote:

Show 110 quoted lines
>
> Hi, Elijah
>
> I was going to send v3 a couple days ago, but there is still one issue
> that I'm having a bit of trouble with:
>
> On Wed, Feb 24, 2021 at 3:50 AM Elijah Newren <newren@gmail.com> wrote:
> >
> > Hmm...here's an interesting command sequence:
> >
> > git init lame
> > cd lame
> > mkdir baz
> > touch baz/tracked
> > git add baz/tracked
> > git update-index --skip-worktree baz/tracked
> > rm baz/tracked.  # But leave the empty directory!
> > echo baz >.gitignore
> > git add --ignore-missing --dry-run baz
> >
> >
> > Reports the following:
> > """
> > The following pathspecs only matched index entries outside the current
> > sparse checkout:
> > baz
> > hint: Disable or modify the sparsity rules if you intend to update such entries.
> > hint: Disable this message with "git config advice.updateSparsePath false"
> > The following paths are ignored by one of your .gitignore files:
> > baz
> > hint: Use -f if you really want to add them.
> > hint: Turn this message off by running
> > hint: "git config advice.addIgnoredFile false"
> > """
> >
> > That's probably okay because it does match both, but the "only
> > matched" in the first message followed by saying it matched something
> > else seems a little surprising at first.
>
> After giving this a little more thought, I got to this idea [1]. With
> this patch, git-add doesn't warn about sparse paths that are inside
> ignored dirs (it only warns about the ignored dirs themselves). If
> --force is used, then git-add recurses into the ignored directories, and
> warns about any sparse path inside them. I think this is more consistent
> with the way in which ignored dirs (and their contents) are handled with
> and without --force. And we avoid the double warning.
>
> But there is a performance penalty. The relevant code is:
>
> @@ -2029,19 +2033,23 @@ static int exclude_matches_pathspec(const char *path, int pathlen,
>                        PATHSPEC_ICASE |
>                        PATHSPEC_EXCLUDE);
>
> +       assert(matched_ignored);
> +
>         for (i = 0; i < pathspec->nr; i++) {
>                 const struct pathspec_item *item = &pathspec->items[i];
>                 int len = item->nowildcard_len;
>
> -               if (len == pathlen &&
> -                   !ps_strncmp(item, item->match, path, pathlen))
> -                       return 1;
> -               if (len > pathlen &&
> -                   item->match[pathlen] == '/' &&
> -                   !ps_strncmp(item, item->match, path, pathlen))
> -                       return 1;
> +               if (matched && matched_ignored[i])
> +                       continue;
> +
> +               if ((len == pathlen ||
> +                   (len > pathlen && item->match[pathlen] == '/')) &&
> +                   !ps_strncmp(item, item->match, path, pathlen)) {
> +                       matched = 1;
> +                       matched_ignored[i] = 1;
> +               }
>         }
> -       return 0;
> +       return matched;
>  }
>
> This function is called for each ignored path (not recursing into
> completely ignored dirs, though). Before the patch, the internal loop
> stops at the first pathspec match. But with the patch, the loop must
> continue as we need to know about all pathspecs that had matches with
> the ignored paths...
>
> And there is also one odd case:
> $ mkdir d
> $ touch d/s d/i
> $ git add d/s
> $ git update-index --skip-worktree d/s
> $ echo d/i >.gitinore
> $ git add 'd/[is]'
>
> This outputs:
> The following pathspecs only matched index entries outside the current
> sparse checkout:
> d/[is]
> (and no warning on ignored paths).
>
> I don't think the warning here is 100% correct, as 'd/[is]' *does* match
> other files, only they are ignored. And git-add only warns about an
> ignored path when the pathspec is an exact match with it (no glob
> patterns) or inside it.
>
> So, I don't know... Perhaps should we just rephrase the sparse warning
> to remove the "only" part (and don't try to avoid the double warnings)?
> I'm open to any suggestions on alternative wordings or ideas :)
>
> [1]: https://github.com/matheustavares/git/commit/5d4be321e1f69cb08b0666ffd3f8fa658e9f6953

The only thing that I think was problematic about the double warning was the contradiction between them due to the use of "only" in the first; if we remove that, I think printing two warnings is perfectly fine. So, I'm in favor of just rephrasing as you suggest.

I think it's kind of lame that git-add won't provide a warning for ignored files when the match is via glob/pattern; if git-add didn't end up adding anything, but an ignored file was matched, I think a warning is appropriate. However, I don't think your patch series needs to address that; your series addressing sparse behavior is long enough, and fixing the handling with ignored files could be done separately.

Junio C Hamano· Mar 4, 2021, 21:03 UTC · re: Elijah Newren · lore

Re: [PATCH v2 6/7] add: warn when pathspec only matches SKIP_WORKTREE entries

Elijah Newren <newren@gmail.com> writes:
> I think it's kind of lame that git-add won't provide a warning for
> ignored files when the match is via glob/pattern; if git-add didn't
> end up adding anything, but an ignored file was matched, I think a
> warning is appropriate.
I am not so sure.  When I have this in my exclude list:
	tmp-*
and have this file in the working tree:
	tmp-000.txt
I am not sure if I want to see any such warning when I did
	$ git add "*.txt"

After all, I said that I never would want to worry about anything that begins with "tmp-". It would be a totally different issue if I did

	$ git add tmp-000.txt

as I am contermanding my previous wish and saying that I care about this particular file, even if its name begins with "tmp-".

So far, the flow of logic is simple, clear and straight-forward.
BUT.
What makes the whole thing tricky is that people don't do
	$ git add "*.txt"
The instead do
	$ git add *.txt

and "git add" cannot tell if the tmp-000.txt it sees is what the user meant, or what the shell expanded.

If there were no shell glob expansion, then "warn only if a glob/pattern did not match any" would have been an attractive option, especially when we had another file in the working tree, say, hello.txt, next to the tmp-000.txt. Then

	$ git add "*.txt"

would add only hello.txt, and we won't see a warning about tmp-000.txt. But end-users will use shell expansion, i.e.

	$ git add *.txt
(1) warning against tmp-000.txt because the command line named it
    explicitly (from "git add"'s point of view, but never from the
    user's point of view) would be a disaster.  I think that is why
    your suggestion was "if git-add did not add anything then warn".
    but ...
(2) not warning, because the command line named hello.txt as well,
    i.e. "git add" added something, would make it inconsistent.
    Whether there is another file whose name ends with .txt, what
    the user typed (i.e. *.txt) to the command line is the same, and
    the exclude pattern (i.e. tmp-*) is the same, but the presence
    of an unrelated hello.txt affects if a warning is given for
    tmp-000.txt.
So, I dunno.
Elijah Newren· Mar 4, 2021, 22:48 UTC · re: Junio C Hamano · lore

Re: [PATCH v2 6/7] add: warn when pathspec only matches SKIP_WORKTREE entries

On Thu, Mar 4, 2021 at 1:03 PM Junio C Hamano <gitster@pobox.com> wrote:
Show 71 quoted lines
>
> Elijah Newren <newren@gmail.com> writes:
>
> > I think it's kind of lame that git-add won't provide a warning for
> > ignored files when the match is via glob/pattern; if git-add didn't
> > end up adding anything, but an ignored file was matched, I think a
> > warning is appropriate.
>
> I am not so sure.  When I have this in my exclude list:
>
>         tmp-*
>
> and have this file in the working tree:
>
>         tmp-000.txt
>
> I am not sure if I want to see any such warning when I did
>
>         $ git add "*.txt"
>
> After all, I said that I never would want to worry about anything
> that begins with "tmp-".  It would be a totally different issue if I
> did
>
>         $ git add tmp-000.txt
>
> as I am contermanding my previous wish and saying that I care about
> this particular file, even if its name begins with "tmp-".
>
> So far, the flow of logic is simple, clear and straight-forward.
>
> BUT.
>
> What makes the whole thing tricky is that people don't do
>
>         $ git add "*.txt"
>
> The instead do
>
>         $ git add *.txt
>
> and "git add" cannot tell if the tmp-000.txt it sees is what the
> user meant, or what the shell expanded.
>
> If there were no shell glob expansion, then "warn only if a
> glob/pattern did not match any" would have been an attractive
> option, especially when we had another file in the working tree,
> say, hello.txt, next to the tmp-000.txt.  Then
>
>         $ git add "*.txt"
>
> would add only hello.txt, and we won't see a warning about
> tmp-000.txt.  But end-users will use shell expansion, i.e.
>
>         $ git add *.txt
>
> (1) warning against tmp-000.txt because the command line named it
>     explicitly (from "git add"'s point of view, but never from the
>     user's point of view) would be a disaster.  I think that is why
>     your suggestion was "if git-add did not add anything then warn".
>     but ...
>
> (2) not warning, because the command line named hello.txt as well,
>     i.e. "git add" added something, would make it inconsistent.
>     Whether there is another file whose name ends with .txt, what
>     the user typed (i.e. *.txt) to the command line is the same, and
>     the exclude pattern (i.e. tmp-*) is the same, but the presence
>     of an unrelated hello.txt affects if a warning is given for
>     tmp-000.txt.
>
> So, I dunno.

Fair enough. All the more reason to leave it out of Matheus' patch series, then. :-)

Matheus Tavares Bernardino· Mar 4, 2021, 21:26 UTC · re: Elijah Newren · lore

Re: [PATCH v2 6/7] add: warn when pathspec only matches SKIP_WORKTREE entries

On Thu, Mar 4, 2021 at 2:22 PM Elijah Newren <newren@gmail.com> wrote:
Show 41 quoted lines
>
> On Thu, Mar 4, 2021 at 7:23 AM Matheus Tavares
> <matheus.bernardino@usp.br> wrote:
> >
> > On Wed, Feb 24, 2021 at 3:50 AM Elijah Newren <newren@gmail.com> wrote:
> > >
> > > Hmm...here's an interesting command sequence:
> > >
> > > git init lame
> > > cd lame
> > > mkdir baz
> > > touch baz/tracked
> > > git add baz/tracked
> > > git update-index --skip-worktree baz/tracked
> > > rm baz/tracked.  # But leave the empty directory!
> > > echo baz >.gitignore
> > > git add --ignore-missing --dry-run baz
> > >
> > >
> > > Reports the following:
> > > """
> > > The following pathspecs only matched index entries outside the current
> > > sparse checkout:
> > > baz
> > > hint: Disable or modify the sparsity rules if you intend to update such entries.
> > > hint: Disable this message with "git config advice.updateSparsePath false"
> > > The following paths are ignored by one of your .gitignore files:
> > > baz
> > > hint: Use -f if you really want to add them.
> > > hint: Turn this message off by running
> > > hint: "git config advice.addIgnoredFile false"
> > > """
> >
> >[...] Perhaps should we just rephrase the sparse warning
> > to remove the "only" part (and don't try to avoid the double warnings)?
> > I'm open to any suggestions on alternative wordings or ideas :)
>
> The only thing that I think was problematic about the double warning
> was the contradiction between them due to the use of "only" in the
> first; if we remove that, I think printing two warnings is perfectly
> fine.  So, I'm in favor of just rephrasing as you suggest.

Ok, let's do that! I may be overthinking this, but would it be enough to say "The following pathspecs match index entries outside the current sparse checkout"?

This is not wrong per-se, but I think it doesn't sound as informative. I mean, another add/rm execution could also have pathspecs that match sparse entries, and yet no warning is displayed because they also match non-sparse entries.

I haven't thought of a better alternative yet, but I was considering something along "The following pathspecs won't update any paths, but they match index entries outside the current sparse checkout". (The idea is that "won't update any paths" include both pathspecs that only match sparse paths and the ones that match both sparse and ignored paths, as neither will be updated.)

Matheus Tavares· Feb 24, 2021, 04:05 UTC · re: Matheus Tavares · lore

[PATCH v2 7/7] rm: honor sparse checkout patterns

`git add` refrains from adding or updating paths outside the sparsity rules, but `git rm` doesn't follow the same restrictions. This is somewhat counter-intuitive and inconsistent. So make `rm` honor the sparse checkout and advise on how to remove SKIP_WORKTREE entries, just like `add` does. Also add some tests for the new behavior.

Suggested-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Matheus Tavares <matheus.bernardino@usp.br>
---
 Documentation/config/advice.txt  |  5 ++-
 Documentation/git-rm.txt         |  4 +-
 builtin/rm.c                     | 35 ++++++++++-----
 t/t3602-rm-sparse-checkout.sh    | 76 ++++++++++++++++++++++++++++++++
 t/t7011-skip-worktree-reading.sh |  5 ---
 5 files changed, 106 insertions(+), 19 deletions(-)
 create mode 100755 t/t3602-rm-sparse-checkout.sh
Show changes to 5 files +106 −19

Documentation/config/advice.txt, Documentation/git-rm.txt, builtin/rm.c, t/t3602-rm-sparse-checkout.sh, t/t7011-skip-worktree-reading.sh

diff --git a/Documentation/config/advice.txt b/Documentation/config/advice.txt
index d53eafa00b..bdd423ade4 100644
--- a/Documentation/config/advice.txt
+++ b/Documentation/config/advice.txt
@@ -120,6 +120,7 @@ advice.*::
 		Advice shown if a user runs the add command without providing
 		the pathspec parameter.
 	updateSparsePath::
-		Advice shown if the pathspec given to linkgit:git-add[1] only
-		matches index entries outside the current sparse-checkout.
+		Advice shown if the pathspec given to linkgit:git-add[1] or
+		linkgit:git-rm[1] only matches index entries outside the
+		current sparse-checkout.
 --
diff --git a/Documentation/git-rm.txt b/Documentation/git-rm.txt
index ab750367fd..26e9b28470 100644
--- a/Documentation/git-rm.txt
+++ b/Documentation/git-rm.txt
@@ -23,7 +23,9 @@ branch, and no updates to their contents can be staged in the index,
 though that default behavior can be overridden with the `-f` option.
 When `--cached` is given, the staged content has to
 match either the tip of the branch or the file on disk,
-allowing the file to be removed from just the index.
+allowing the file to be removed from just the index. When
+sparse-checkouts are in use (see linkgit:git-sparse-checkout[1]),
+`git rm` will only remove paths within the sparse-checkout patterns.
 
 
 OPTIONS
diff --git a/builtin/rm.c b/builtin/rm.c
index 4858631e0f..d23a3b2164 100644
--- a/builtin/rm.c
+++ b/builtin/rm.c
@@ -5,6 +5,7 @@
  */
 #define USE_THE_INDEX_COMPATIBILITY_MACROS
 #include "builtin.h"
+#include "advice.h"
 #include "config.h"
 #include "lockfile.h"
 #include "dir.h"
@@ -254,7 +255,7 @@ static struct option builtin_rm_options[] = {
 int cmd_rm(int argc, const char **argv, const char *prefix)
 {
 	struct lock_file lock_file = LOCK_INIT;
-	int i;
+	int i, ret = 0;
 	struct pathspec pathspec;
 	char *seen;
 
@@ -295,6 +296,8 @@ int cmd_rm(int argc, const char **argv, const char *prefix)
 
 	for (i = 0; i < active_nr; i++) {
 		const struct cache_entry *ce = active_cache[i];
+		if (ce_skip_worktree(ce))
+			continue;
 		if (!ce_path_match(&the_index, ce, &pathspec, seen))
 			continue;
 		ALLOC_GROW(list.entry, list.nr + 1, list.alloc);
@@ -308,24 +311,34 @@ int cmd_rm(int argc, const char **argv, const char *prefix)
 	if (pathspec.nr) {
 		const char *original;
 		int seen_any = 0;
+		char *skip_worktree_seen = NULL;
+		struct string_list only_match_skip_worktree = STRING_LIST_INIT_NODUP;
+
 		for (i = 0; i < pathspec.nr; i++) {
 			original = pathspec.items[i].original;
-			if (!seen[i]) {
-				if (!ignore_unmatch) {
-					die(_("pathspec '%s' did not match any files"),
-					    original);
-				}
-			}
-			else {
+			if (seen[i])
 				seen_any = 1;
-			}
+			else if (ignore_unmatch)
+				continue;
+			else if (matches_skip_worktree(&pathspec, i, &skip_worktree_seen))
+				string_list_append(&only_match_skip_worktree, original);
+			else
+				die(_("pathspec '%s' did not match any files"), original);
+
 			if (!recursive && seen[i] == MATCHED_RECURSIVELY)
 				die(_("not removing '%s' recursively without -r"),
 				    *original ? original : ".");
 		}
 
+		if (only_match_skip_worktree.nr) {
+			advise_on_updating_sparse_paths(&only_match_skip_worktree);
+			ret = 1;
+		}
+		free(skip_worktree_seen);
+		string_list_clear(&only_match_skip_worktree, 0);
+
 		if (!seen_any)
-			exit(0);
+			exit(ret);
 	}
 
 	if (!index_only)
@@ -405,5 +418,5 @@ int cmd_rm(int argc, const char **argv, const char *prefix)
 			       COMMIT_LOCK | SKIP_IF_UNCHANGED))
 		die(_("Unable to write new index file"));
 
-	return 0;
+	return ret;
 }
diff --git a/t/t3602-rm-sparse-checkout.sh b/t/t3602-rm-sparse-checkout.sh
new file mode 100755
index 0000000000..34f4debacf
--- /dev/null
+++ b/t/t3602-rm-sparse-checkout.sh
@@ -0,0 +1,76 @@
+#!/bin/sh
+
+test_description='git rm in sparse checked out working trees'
+
+. ./test-lib.sh
+
+test_expect_success 'setup' '
+	mkdir -p sub/dir &&
+	touch a b c sub/d sub/dir/e &&
+	git add -A &&
+	git commit -m files &&
+
+	cat >sparse_entry_b_error <<-EOF &&
+	The following pathspecs only matched index entries outside the current
+	sparse checkout:
+	b
+	EOF
+
+	cat >b_error_and_hint sparse_entry_b_error - <<-EOF
+	hint: Disable or modify the sparsity rules if you intend to update such entries.
+	hint: Disable this message with "git config advice.updateSparsePath false"
+	EOF
+'
+
+for opt in "" -f --dry-run
+do
+	test_expect_success "rm${opt:+ $opt} does not remove sparse entries" '
+		git sparse-checkout set a &&
+		test_must_fail git rm $opt b 2>stderr &&
+		test_i18ncmp b_error_and_hint stderr &&
+		git ls-files --error-unmatch b
+	'
+done
+
+test_expect_success 'recursive rm does not remove sparse entries' '
+	git reset --hard &&
+	git sparse-checkout set sub/dir &&
+	git rm -r sub &&
+	git status --porcelain -uno >actual &&
+	echo "D  sub/dir/e" >expected &&
+	test_cmp expected actual
+'
+
+test_expect_success 'rm obeys advice.updateSparsePath' '
+	git reset --hard &&
+	git sparse-checkout set a &&
+	test_must_fail git -c advice.updateSparsePath=false rm b 2>stderr &&
+	test_i18ncmp sparse_entry_b_error stderr
+'
+
+test_expect_success 'do not advice about sparse entries when they do not match the pathspec' '
+	git reset --hard &&
+	git sparse-checkout set a &&
+	test_must_fail git rm nonexistent 2>stderr &&
+	test_i18ngrep "fatal: pathspec .nonexistent. did not match any files" stderr &&
+	test_i18ngrep ! "The following pathspecs only matched index entries" stderr
+'
+
+test_expect_success 'do not warn about sparse entries when pathspec matches dense entries' '
+	git reset --hard &&
+	git sparse-checkout set a &&
+	git rm "[ba]" 2>stderr &&
+	test_must_be_empty stderr &&
+	git ls-files --error-unmatch b &&
+	test_must_fail git ls-files --error-unmatch a
+'
+
+test_expect_success 'do not warn about sparse entries with --ignore-unmatch' '
+	git reset --hard &&
+	git sparse-checkout set a &&
+	git rm --ignore-unmatch b 2>stderr &&
+	test_must_be_empty stderr &&
+	git ls-files --error-unmatch b
+'
+
+test_done
diff --git a/t/t7011-skip-worktree-reading.sh b/t/t7011-skip-worktree-reading.sh
index 37525cae3a..f87749951f 100755
--- a/t/t7011-skip-worktree-reading.sh
+++ b/t/t7011-skip-worktree-reading.sh
@@ -141,11 +141,6 @@ test_expect_success 'diff-files does not examine skip-worktree dirty entries' '
 	test -z "$(git diff-files -- one)"
 '
 
-test_expect_success 'git-rm succeeds on skip-worktree absent entries' '
-	setup_absent &&
-	git rm 1
-'
-
 test_expect_success 'commit on skip-worktree absent entries' '
 	git reset &&
 	setup_absent &&
-- 
2.30.1
Elijah Newren· Feb 24, 2021, 06:59 UTC · re: Matheus Tavares · lore

Re: [PATCH v2 7/7] rm: honor sparse checkout patterns

On Tue, Feb 23, 2021 at 8:05 PM Matheus Tavares <matheus.bernardino@usp.br> wrote:

Show 228 quoted lines
>
> `git add` refrains from adding or updating paths outside the sparsity
> rules, but `git rm` doesn't follow the same restrictions. This is
> somewhat counter-intuitive and inconsistent. So make `rm` honor the
> sparse checkout and advise on how to remove SKIP_WORKTREE entries, just
> like `add` does. Also add some tests for the new behavior.
>
> Suggested-by: Elijah Newren <newren@gmail.com>
> Signed-off-by: Matheus Tavares <matheus.bernardino@usp.br>
> ---
>  Documentation/config/advice.txt  |  5 ++-
>  Documentation/git-rm.txt         |  4 +-
>  builtin/rm.c                     | 35 ++++++++++-----
>  t/t3602-rm-sparse-checkout.sh    | 76 ++++++++++++++++++++++++++++++++
>  t/t7011-skip-worktree-reading.sh |  5 ---
>  5 files changed, 106 insertions(+), 19 deletions(-)
>  create mode 100755 t/t3602-rm-sparse-checkout.sh
>
> diff --git a/Documentation/config/advice.txt b/Documentation/config/advice.txt
> index d53eafa00b..bdd423ade4 100644
> --- a/Documentation/config/advice.txt
> +++ b/Documentation/config/advice.txt
> @@ -120,6 +120,7 @@ advice.*::
>                 Advice shown if a user runs the add command without providing
>                 the pathspec parameter.
>         updateSparsePath::
> -               Advice shown if the pathspec given to linkgit:git-add[1] only
> -               matches index entries outside the current sparse-checkout.
> +               Advice shown if the pathspec given to linkgit:git-add[1] or
> +               linkgit:git-rm[1] only matches index entries outside the
> +               current sparse-checkout.
>  --
> diff --git a/Documentation/git-rm.txt b/Documentation/git-rm.txt
> index ab750367fd..26e9b28470 100644
> --- a/Documentation/git-rm.txt
> +++ b/Documentation/git-rm.txt
> @@ -23,7 +23,9 @@ branch, and no updates to their contents can be staged in the index,
>  though that default behavior can be overridden with the `-f` option.
>  When `--cached` is given, the staged content has to
>  match either the tip of the branch or the file on disk,
> -allowing the file to be removed from just the index.
> +allowing the file to be removed from just the index. When
> +sparse-checkouts are in use (see linkgit:git-sparse-checkout[1]),
> +`git rm` will only remove paths within the sparse-checkout patterns.
>
>
>  OPTIONS
> diff --git a/builtin/rm.c b/builtin/rm.c
> index 4858631e0f..d23a3b2164 100644
> --- a/builtin/rm.c
> +++ b/builtin/rm.c
> @@ -5,6 +5,7 @@
>   */
>  #define USE_THE_INDEX_COMPATIBILITY_MACROS
>  #include "builtin.h"
> +#include "advice.h"
>  #include "config.h"
>  #include "lockfile.h"
>  #include "dir.h"
> @@ -254,7 +255,7 @@ static struct option builtin_rm_options[] = {
>  int cmd_rm(int argc, const char **argv, const char *prefix)
>  {
>         struct lock_file lock_file = LOCK_INIT;
> -       int i;
> +       int i, ret = 0;
>         struct pathspec pathspec;
>         char *seen;
>
> @@ -295,6 +296,8 @@ int cmd_rm(int argc, const char **argv, const char *prefix)
>
>         for (i = 0; i < active_nr; i++) {
>                 const struct cache_entry *ce = active_cache[i];
> +               if (ce_skip_worktree(ce))
> +                       continue;
>                 if (!ce_path_match(&the_index, ce, &pathspec, seen))
>                         continue;
>                 ALLOC_GROW(list.entry, list.nr + 1, list.alloc);
> @@ -308,24 +311,34 @@ int cmd_rm(int argc, const char **argv, const char *prefix)
>         if (pathspec.nr) {
>                 const char *original;
>                 int seen_any = 0;
> +               char *skip_worktree_seen = NULL;
> +               struct string_list only_match_skip_worktree = STRING_LIST_INIT_NODUP;
> +
>                 for (i = 0; i < pathspec.nr; i++) {
>                         original = pathspec.items[i].original;
> -                       if (!seen[i]) {
> -                               if (!ignore_unmatch) {
> -                                       die(_("pathspec '%s' did not match any files"),
> -                                           original);
> -                               }
> -                       }
> -                       else {
> +                       if (seen[i])
>                                 seen_any = 1;
> -                       }
> +                       else if (ignore_unmatch)
> +                               continue;
> +                       else if (matches_skip_worktree(&pathspec, i, &skip_worktree_seen))
> +                               string_list_append(&only_match_skip_worktree, original);
> +                       else
> +                               die(_("pathspec '%s' did not match any files"), original);
> +
>                         if (!recursive && seen[i] == MATCHED_RECURSIVELY)
>                                 die(_("not removing '%s' recursively without -r"),
>                                     *original ? original : ".");
>                 }
>
> +               if (only_match_skip_worktree.nr) {
> +                       advise_on_updating_sparse_paths(&only_match_skip_worktree);
> +                       ret = 1;
> +               }
> +               free(skip_worktree_seen);
> +               string_list_clear(&only_match_skip_worktree, 0);
> +
>                 if (!seen_any)
> -                       exit(0);
> +                       exit(ret);
>         }
>
>         if (!index_only)
> @@ -405,5 +418,5 @@ int cmd_rm(int argc, const char **argv, const char *prefix)
>                                COMMIT_LOCK | SKIP_IF_UNCHANGED))
>                 die(_("Unable to write new index file"));
>
> -       return 0;
> +       return ret;
>  }
> diff --git a/t/t3602-rm-sparse-checkout.sh b/t/t3602-rm-sparse-checkout.sh
> new file mode 100755
> index 0000000000..34f4debacf
> --- /dev/null
> +++ b/t/t3602-rm-sparse-checkout.sh
> @@ -0,0 +1,76 @@
> +#!/bin/sh
> +
> +test_description='git rm in sparse checked out working trees'
> +
> +. ./test-lib.sh
> +
> +test_expect_success 'setup' '
> +       mkdir -p sub/dir &&
> +       touch a b c sub/d sub/dir/e &&
> +       git add -A &&
> +       git commit -m files &&
> +
> +       cat >sparse_entry_b_error <<-EOF &&
> +       The following pathspecs only matched index entries outside the current
> +       sparse checkout:
> +       b
> +       EOF
> +
> +       cat >b_error_and_hint sparse_entry_b_error - <<-EOF
> +       hint: Disable or modify the sparsity rules if you intend to update such entries.
> +       hint: Disable this message with "git config advice.updateSparsePath false"
> +       EOF
> +'
> +
> +for opt in "" -f --dry-run
> +do
> +       test_expect_success "rm${opt:+ $opt} does not remove sparse entries" '
> +               git sparse-checkout set a &&
> +               test_must_fail git rm $opt b 2>stderr &&
> +               test_i18ncmp b_error_and_hint stderr &&
> +               git ls-files --error-unmatch b
> +       '
> +done
> +
> +test_expect_success 'recursive rm does not remove sparse entries' '
> +       git reset --hard &&
> +       git sparse-checkout set sub/dir &&
> +       git rm -r sub &&
> +       git status --porcelain -uno >actual &&
> +       echo "D  sub/dir/e" >expected &&
> +       test_cmp expected actual
> +'
> +
> +test_expect_success 'rm obeys advice.updateSparsePath' '
> +       git reset --hard &&
> +       git sparse-checkout set a &&
> +       test_must_fail git -c advice.updateSparsePath=false rm b 2>stderr &&
> +       test_i18ncmp sparse_entry_b_error stderr
> +'
> +
> +test_expect_success 'do not advice about sparse entries when they do not match the pathspec' '
> +       git reset --hard &&
> +       git sparse-checkout set a &&
> +       test_must_fail git rm nonexistent 2>stderr &&
> +       test_i18ngrep "fatal: pathspec .nonexistent. did not match any files" stderr &&
> +       test_i18ngrep ! "The following pathspecs only matched index entries" stderr
> +'
> +
> +test_expect_success 'do not warn about sparse entries when pathspec matches dense entries' '
> +       git reset --hard &&
> +       git sparse-checkout set a &&
> +       git rm "[ba]" 2>stderr &&
> +       test_must_be_empty stderr &&
> +       git ls-files --error-unmatch b &&
> +       test_must_fail git ls-files --error-unmatch a
> +'
> +
> +test_expect_success 'do not warn about sparse entries with --ignore-unmatch' '
> +       git reset --hard &&
> +       git sparse-checkout set a &&
> +       git rm --ignore-unmatch b 2>stderr &&
> +       test_must_be_empty stderr &&
> +       git ls-files --error-unmatch b
> +'
> +
> +test_done
> diff --git a/t/t7011-skip-worktree-reading.sh b/t/t7011-skip-worktree-reading.sh
> index 37525cae3a..f87749951f 100755
> --- a/t/t7011-skip-worktree-reading.sh
> +++ b/t/t7011-skip-worktree-reading.sh
> @@ -141,11 +141,6 @@ test_expect_success 'diff-files does not examine skip-worktree dirty entries' '
>         test -z "$(git diff-files -- one)"
>  '
>
> -test_expect_success 'git-rm succeeds on skip-worktree absent entries' '
> -       setup_absent &&
> -       git rm 1
> -'
> -
>  test_expect_success 'commit on skip-worktree absent entries' '
>         git reset &&
>         setup_absent &&
> --
> 2.30.1

I already reviewed the source code previously a few times now; it still looks good to me, and you didn't change it from the last version. You've added new tests and tweaked them a bit, but all those changes look good to me.

Elijah Newren· Feb 24, 2021, 07:05 UTC · re: Matheus Tavares · lore

Re: [PATCH v2 0/7] add/rm: honor sparse checkout and warn on sparse paths

Hi,

This time I'm responding a bit more promptly than the last few patch series you sent out. :-)

On Tue, Feb 23, 2021 at 8:05 PM Matheus Tavares <matheus.bernardino@usp.br> wrote:

Show 7 quoted lines
>
> Make `rm` honor sparse checkouts, and make both `rm` and `add` warn
> about pathspecs that only match sparse entries.
>
> This is now based on mt/add-chmod-fixes.
>
> Main changes since RFC:

I've read through the series. I'm obviously a fan of the overall direction, and after reading it over I think the implementation also looks pretty good. I had a few minor comments here and there, and one weird testcase that I came up with based on reading your code where I'm not sure what the right behavior is (because the testcase is convoluted); at worst, the current code just prints two warnings when we maybe want one, but two warnings might be considered the correct behavior. Maybe you or someone else can look at it with a fresh set of eyes and have a reason to prefer one behavior over the other for that testcase.

Show 599 quoted lines
>
> Ejected "add --chmod: don't update index when --dry-run is used", which
> is now part of mt/add-chmod-fixes.
>
> Patch 2:
> - Fixed style problems.
> - Fixed comment on setup_sparse_entry() about argument being passed to
>   printf.
> - Standardized on 'sparse entries' on test names.
> - Added test for 'git add .', and set up a .gitignore to run this and
>   the 'git add -A' test, to avoid smudging the index.
> - Added test for --dry-run.
> - Modified `git add --refresh` test to use `git ls-files --debug`
>   instead of diff-files.
>
> Added patch 5
>
> Patch 6:
> - Improved commit message.
> - Used the flag added in patch 5 instead of directly changing
>   refresh_index(), so that new users of seen[] can still have the
>   complete match array.
> - Added test for --ignore-missing.
> - Added test where the pathspec matches both sparse and dense entries.
> - Changed 'git add .' behavior where . only contains sparse entries. [1]
>
> Patch 7:
> - Moved rm-sparse-checkout tests to their own file. They need a repo
>   with a different setup than the one t3600 has, and the subshells (or
>   git -C ...) were making the tests too hard to read.
> - Added tests for -f, --dry-run and --ignore-unmatch.
> - Added test where the pathspec matches both sparse and dense entries.
> - Also test if the index entry is really not removed (besides testing
>   `git rm` exit code and stderr).
> - Passed "a" instead of "/a" to git sparse-checkout to avoid MSYS path
>   conversions on Windows that would make some of the new tests fail.
>
> [1]: Although the sparse entries warning is based on the ignored files
>      warning, they have some differences. One of them is that running
>      'git add dir' where 'dir/somefile' is ignored does not trigger a
>      warning, whereas if 'somefile' is a sparse entry, we warn. In
>      this sense, I think it's more coherent to also warn on 'git add .'
>      when . only has sparse entries. This is also consistent with what
>      'git rm -r .' does in the same scenario, after this series.
>
> Matheus Tavares (7):
>   add: include magic part of pathspec on --refresh error
>   t3705: add tests for `git add` in sparse checkouts
>   add: make --chmod and --renormalize honor sparse checkouts
>   pathspec: allow to ignore SKIP_WORKTREE entries on index matching
>   refresh_index(): add REFRESH_DONT_MARK_SPARSE_MATCHES flag
>   add: warn when pathspec only matches SKIP_WORKTREE entries
>   rm: honor sparse checkout patterns
>
>  Documentation/config/advice.txt  |   4 +
>  Documentation/git-rm.txt         |   4 +-
>  advice.c                         |  19 ++++
>  advice.h                         |   4 +
>  builtin/add.c                    |  75 ++++++++++++---
>  builtin/check-ignore.c           |   2 +-
>  builtin/rm.c                     |  35 ++++---
>  cache.h                          |  15 +--
>  pathspec.c                       |  25 ++++-
>  pathspec.h                       |  13 ++-
>  read-cache.c                     |   5 +-
>  t/t3602-rm-sparse-checkout.sh    |  76 +++++++++++++++
>  t/t3700-add.sh                   |   6 ++
>  t/t3705-add-sparse-checkout.sh   | 155 +++++++++++++++++++++++++++++++
>  t/t7011-skip-worktree-reading.sh |   5 -
>  t/t7012-skip-worktree-writing.sh |  19 ----
>  16 files changed, 398 insertions(+), 64 deletions(-)
>  create mode 100755 t/t3602-rm-sparse-checkout.sh
>  create mode 100755 t/t3705-add-sparse-checkout.sh
>
> Range-diff against v1:
> 1:  5612c57977 < -:  ---------- add --chmod: don't update index when --dry-run is used
> 2:  5a06223007 = 1:  2831fd5744 add: include magic part of pathspec on --refresh error
> 3:  4fc81a83b1 ! 2:  72b8787018 t3705: add tests for `git add` in sparse checkouts
>     @@ t/t3705-add-sparse-checkout.sh (new)
>      +
>      +SPARSE_ENTRY_BLOB=""
>      +
>     -+# Optionally take a string for the entry's contents
>     -+setup_sparse_entry()
>     -+{
>     -+  if test -f sparse_entry
>     -+  then
>     -+          rm sparse_entry
>     -+  fi &&
>     ++# Optionally take a printf format string to write to the sparse_entry file
>     ++setup_sparse_entry () {
>     ++  rm -f sparse_entry &&
>      +  git update-index --force-remove sparse_entry &&
>      +
>     -+  if test "$#" -eq 1
>     ++  if test $# -eq 1
>      +  then
>      +          printf "$1" >sparse_entry
>      +  else
>     -+          printf "" >sparse_entry
>     ++          >sparse_entry
>      +  fi &&
>      +  git add sparse_entry &&
>      +  git update-index --skip-worktree sparse_entry &&
>      +  SPARSE_ENTRY_BLOB=$(git rev-parse :sparse_entry)
>      +}
>      +
>     -+test_sparse_entry_unchanged() {
>     ++test_sparse_entry_unchanged () {
>      +  echo "100644 $SPARSE_ENTRY_BLOB 0       sparse_entry" >expected &&
>      +  git ls-files --stage sparse_entry >actual &&
>      +  test_cmp expected actual
>      +}
>      +
>     -+test_expect_success "git add does not remove SKIP_WORKTREE entries" '
>     ++setup_gitignore () {
>     ++  test_when_finished rm -f .gitignore &&
>     ++  cat >.gitignore <<-EOF
>     ++  *
>     ++  !/sparse_entry
>     ++  EOF
>     ++}
>     ++
>     ++test_expect_success 'git add does not remove sparse entries' '
>      +  setup_sparse_entry &&
>      +  rm sparse_entry &&
>      +  git add sparse_entry &&
>      +  test_sparse_entry_unchanged
>      +'
>      +
>     -+test_expect_success "git add -A does not remove SKIP_WORKTREE entries" '
>     ++test_expect_success 'git add -A does not remove sparse entries' '
>      +  setup_sparse_entry &&
>      +  rm sparse_entry &&
>     ++  setup_gitignore &&
>      +  git add -A &&
>      +  test_sparse_entry_unchanged
>      +'
>      +
>     -+for opt in "" -f -u --ignore-removal
>     -+do
>     -+  if test -n "$opt"
>     -+  then
>     -+          opt=" $opt"
>     -+  fi
>     ++test_expect_success 'git add . does not remove sparse entries' '
>     ++  setup_sparse_entry &&
>     ++  rm sparse_entry &&
>     ++  setup_gitignore &&
>     ++  git add . &&
>     ++  test_sparse_entry_unchanged
>     ++'
>      +
>     -+  test_expect_success "git add$opt does not update SKIP_WORKTREE entries" '
>     ++for opt in "" -f -u --ignore-removal --dry-run
>     ++do
>     ++  test_expect_success "git add${opt:+ $opt} does not update sparse entries" '
>      +          setup_sparse_entry &&
>      +          echo modified >sparse_entry &&
>      +          git add $opt sparse_entry &&
>     @@ t/t3705-add-sparse-checkout.sh (new)
>      +  '
>      +done
>      +
>     -+test_expect_success 'git add --refresh does not update SKIP_WORKTREE entries' '
>     ++test_expect_success 'git add --refresh does not update sparse entries' '
>      +  setup_sparse_entry &&
>     ++  git ls-files --debug sparse_entry | grep mtime >before &&
>      +  test-tool chmtime -60 sparse_entry &&
>      +  git add --refresh sparse_entry &&
>     -+
>     -+  # We must unset the SKIP_WORKTREE bit, otherwise
>     -+  # git diff-files would skip examining the file
>     -+  git update-index --no-skip-worktree sparse_entry &&
>     -+
>     -+  echo sparse_entry >expected &&
>     -+  git diff-files --name-only sparse_entry >actual &&
>     -+  test_cmp actual expected
>     ++  git ls-files --debug sparse_entry | grep mtime >after &&
>     ++  test_cmp before after
>      +'
>      +
>     -+test_expect_failure 'git add --chmod does not update SKIP_WORKTREE entries' '
>     ++test_expect_failure 'git add --chmod does not update sparse entries' '
>      +  setup_sparse_entry &&
>      +  git add --chmod=+x sparse_entry &&
>     -+  test_sparse_entry_unchanged
>     ++  test_sparse_entry_unchanged &&
>     ++  ! test -x sparse_entry
>      +'
>      +
>     -+test_expect_failure 'git add --renormalize does not update SKIP_WORKTREE entries' '
>     ++test_expect_failure 'git add --renormalize does not update sparse entries' '
>      +  test_config core.autocrlf false &&
>      +  setup_sparse_entry "LINEONE\r\nLINETWO\r\n" &&
>      +  echo "sparse_entry text=auto" >.gitattributes &&
> 4:  ccb05cc3d7 ! 3:  0f03adf241 add: make --chmod and --renormalize honor sparse checkouts
>     @@ Commit message
>          Signed-off-by: Matheus Tavares <matheus.bernardino@usp.br>
>
>       ## builtin/add.c ##
>     -@@ builtin/add.c: static void chmod_pathspec(struct pathspec *pathspec, char flip, int show_only)
>     -   for (i = 0; i < active_nr; i++) {
>     +@@ builtin/add.c: static int chmod_pathspec(struct pathspec *pathspec, char flip, int show_only)
>                 struct cache_entry *ce = active_cache[i];
>     +           int err;
>
>      +          if (ce_skip_worktree(ce))
>      +                  continue;
>     @@ builtin/add.c: static int renormalize_tracked_files(const struct pathspec *paths
>                 if (!S_ISREG(ce->ce_mode) && !S_ISLNK(ce->ce_mode))
>
>       ## t/t3705-add-sparse-checkout.sh ##
>     -@@ t/t3705-add-sparse-checkout.sh: test_expect_success 'git add --refresh does not update SKIP_WORKTREE entries' '
>     -   test_cmp actual expected
>     +@@ t/t3705-add-sparse-checkout.sh: test_expect_success 'git add --refresh does not update sparse entries' '
>     +   test_cmp before after
>       '
>
>     --test_expect_failure 'git add --chmod does not update SKIP_WORKTREE entries' '
>     -+test_expect_success 'git add --chmod does not update SKIP_WORKTREE entries' '
>     +-test_expect_failure 'git add --chmod does not update sparse entries' '
>     ++test_expect_success 'git add --chmod does not update sparse entries' '
>         setup_sparse_entry &&
>         git add --chmod=+x sparse_entry &&
>     -   test_sparse_entry_unchanged
>     +   test_sparse_entry_unchanged &&
>     +   ! test -x sparse_entry
>       '
>
>     --test_expect_failure 'git add --renormalize does not update SKIP_WORKTREE entries' '
>     -+test_expect_success 'git add --renormalize does not update SKIP_WORKTREE entries' '
>     +-test_expect_failure 'git add --renormalize does not update sparse entries' '
>     ++test_expect_success 'git add --renormalize does not update sparse entries' '
>         test_config core.autocrlf false &&
>         setup_sparse_entry "LINEONE\r\nLINETWO\r\n" &&
>         echo "sparse_entry text=auto" >.gitattributes &&
> 5:  00786cba82 = 4:  a8a8af22a0 pathspec: allow to ignore SKIP_WORKTREE entries on index matching
> -:  ---------- > 5:  d65b214dd1 refresh_index(): add REFRESH_DONT_MARK_SPARSE_MATCHES flag
> 6:  ce74a60e0d ! 6:  24e889ca9b add: warn when pathspec only matches SKIP_WORKTREE entries
>     @@ Commit message
>          add: warn when pathspec only matches SKIP_WORKTREE entries
>
>          `git add` already refrains from updating SKIP_WORKTREE entries, but it
>     -    silently succeeds when a pathspec only matches these entries. Instead,
>     -    let's warn the user and display a hint on how to update these entries.
>     +    silently exits with zero code when a pathspec only matches these
>     +    entries. Instead, let's warn the user and display a hint on how to
>     +    update these entries.
>
>          Note that the warning is only shown if the pathspec matches no untracked
>          paths in the working tree and only matches index entries with the
>     -    SKIP_WORKTREE bit set. Performance-wise, this patch doesn't change the
>     -    number of ce_path_match() calls in the worst case scenario (because we
>     -    still need to check the sparse entries for the warning). But in the
>     -    general case, it avoids unnecessarily calling this function for each
>     -    SKIP_WORKTREE entry.
>     -
>     -    A warning message was chosen over erroring out right away to reproduce
>     -    the same behavior `add` already exhibits with ignored files. This also
>     -    allow users to continue their workflow without having to invoke `add`
>     -    again with only the matching pathspecs, as the matched files will have
>     -    already been added.
>     -
>     -    Note: refresh_index() was changed to only mark matches with
>     -    no-SKIP-WORKTREE entries in the `seen` output parameter. This is exactly
>     -    the behavior we want for `add`, and only `add` calls this function with
>     -    a non-NULL `seen` pointer. So the change brings no side effect on
>     -    other callers.
>     +    SKIP_WORKTREE bit set. A warning message was chosen over erroring out
>     +    right away to reproduce the same behavior `add` already exhibits with
>     +    ignored files. This also allow users to continue their workflow without
>     +    having to invoke `add` again with only the matching pathspecs, as the
>     +    matched files will have already been added.
>
>          Signed-off-by: Matheus Tavares <matheus.bernardino@usp.br>
>
>     @@ builtin/add.c: static char *prune_directory(struct dir_struct *dir, struct paths
>      +  int i, ret = 0;
>      +  char *skip_worktree_seen = NULL;
>      +  struct string_list only_match_skip_worktree = STRING_LIST_INIT_NODUP;
>     ++  int flags = REFRESH_DONT_MARK_SPARSE_MATCHES |
>     ++              (verbose ? REFRESH_IN_PORCELAIN : REFRESH_QUIET);
>
>         seen = xcalloc(pathspec->nr, 1);
>     -   refresh_index(&the_index, verbose ? REFRESH_IN_PORCELAIN : REFRESH_QUIET,
>     -                 pathspec, seen, _("Unstaged changes after refreshing the index:"));
>     +-  refresh_index(&the_index, verbose ? REFRESH_IN_PORCELAIN : REFRESH_QUIET,
>     +-                pathspec, seen, _("Unstaged changes after refreshing the index:"));
>     ++  refresh_index(&the_index, flags, pathspec, seen,
>     ++                _("Unstaged changes after refreshing the index:"));
>         for (i = 0; i < pathspec->nr; i++) {
>      -          if (!seen[i])
>      -                  die(_("pathspec '%s' did not match any files"),
>     @@ builtin/add.c: int cmd_add(int argc, const char **argv, const char *prefix)
>      -                      ((pathspec.items[i].magic &
>      -                        (PATHSPEC_GLOB | PATHSPEC_ICASE)) ||
>      -                       !file_exists(path))) {
>     -+                  if (seen[i] || !path[0])
>     ++                  if (seen[i])
>      +                          continue;
>      +
>      +                  if (matches_skip_worktree(&pathspec, i, &skip_worktree_seen)) {
>     @@ builtin/add.c: int cmd_add(int argc, const char **argv, const char *prefix)
>      +                          continue;
>      +                  }
>      +
>     ++                  /* Don't complain at 'git add .' inside empty repo. */
>     ++                  if (!path[0])
>     ++                          continue;
>     ++
>      +                  if ((pathspec.items[i].magic & (PATHSPEC_GLOB | PATHSPEC_ICASE)) ||
>      +                      !file_exists(path)) {
>                                 if (ignore_missing) {
>     @@ pathspec.h: void add_pathspec_matches_against_index(const struct pathspec *paths
>                          const char *name, int namelen,
>                          const struct pathspec_item *item);
>
>     - ## read-cache.c ##
>     -@@ read-cache.c: int refresh_index(struct index_state *istate, unsigned int flags,
>     -           if (ignore_submodules && S_ISGITLINK(ce->ce_mode))
>     -                   continue;
>     -
>     --          if (pathspec && !ce_path_match(istate, ce, pathspec, seen))
>     -+          if (pathspec && !ce_path_match(istate, ce, pathspec,
>     -+                                         ce_skip_worktree(ce) ? NULL : seen))
>     -                   filtered = 1;
>     -
>     -           if (ce_stage(ce)) {
>     -
>       ## t/t3705-add-sparse-checkout.sh ##
>     -@@ t/t3705-add-sparse-checkout.sh: test_sparse_entry_unchanged() {
>     -   test_cmp expected actual
>     +@@ t/t3705-add-sparse-checkout.sh: setup_gitignore () {
>     +   EOF
>       }
>
>     -+cat >sparse_entry_error <<-EOF
>     -+The following pathspecs only matched index entries outside the current
>     -+sparse checkout:
>     -+sparse_entry
>     -+EOF
>     ++test_expect_success 'setup' '
>     ++  cat >sparse_error_header <<-EOF &&
>     ++  The following pathspecs only matched index entries outside the current
>     ++  sparse checkout:
>     ++  EOF
>     ++
>     ++  cat >sparse_hint <<-EOF &&
>     ++  hint: Disable or modify the sparsity rules if you intend to update such entries.
>     ++  hint: Disable this message with "git config advice.updateSparsePath false"
>     ++  EOF
>      +
>     -+cat >error_and_hint sparse_entry_error - <<-EOF
>     -+hint: Disable or modify the sparsity rules if you intend to update such entries.
>     -+hint: Disable this message with "git config advice.updateSparsePath false"
>     -+EOF
>     ++  echo sparse_entry | cat sparse_error_header - >sparse_entry_error &&
>     ++  cat sparse_entry_error sparse_hint >error_and_hint
>     ++'
>      +
>     - test_expect_success "git add does not remove SKIP_WORKTREE entries" '
>     + test_expect_success 'git add does not remove sparse entries' '
>         setup_sparse_entry &&
>         rm sparse_entry &&
>      -  git add sparse_entry &&
>     @@ t/t3705-add-sparse-checkout.sh: test_sparse_entry_unchanged() {
>         test_sparse_entry_unchanged
>       '
>
>     +@@ t/t3705-add-sparse-checkout.sh: test_expect_success 'git add -A does not remove sparse entries' '
>     +   setup_sparse_entry &&
>     +   rm sparse_entry &&
>     +   setup_gitignore &&
>     +-  git add -A &&
>     ++  git add -A 2>stderr &&
>     ++  test_must_be_empty stderr &&
>     +   test_sparse_entry_unchanged
>     + '
>     +
>     +@@ t/t3705-add-sparse-checkout.sh: test_expect_success 'git add . does not remove sparse entries' '
>     +   setup_sparse_entry &&
>     +   rm sparse_entry &&
>     +   setup_gitignore &&
>     +-  git add . &&
>     ++  test_must_fail git add . 2>stderr &&
>     ++
>     ++  cat sparse_error_header >expect &&
>     ++  echo . >>expect &&
>     ++  cat sparse_hint >>expect &&
>     ++
>     ++  test_i18ncmp expect stderr &&
>     +   test_sparse_entry_unchanged
>     + '
>     +
>      @@ t/t3705-add-sparse-checkout.sh: do
>     -   test_expect_success "git add$opt does not update SKIP_WORKTREE entries" '
>     +   test_expect_success "git add${opt:+ $opt} does not update sparse entries" '
>                 setup_sparse_entry &&
>                 echo modified >sparse_entry &&
>      -          git add $opt sparse_entry &&
>     @@ t/t3705-add-sparse-checkout.sh: do
>                 test_sparse_entry_unchanged
>         '
>       done
>     -@@ t/t3705-add-sparse-checkout.sh: done
>     - test_expect_success 'git add --refresh does not update SKIP_WORKTREE entries' '
>     +@@ t/t3705-add-sparse-checkout.sh: test_expect_success 'git add --refresh does not update sparse entries' '
>         setup_sparse_entry &&
>     +   git ls-files --debug sparse_entry | grep mtime >before &&
>         test-tool chmtime -60 sparse_entry &&
>      -  git add --refresh sparse_entry &&
>      +  test_must_fail git add --refresh sparse_entry 2>stderr &&
>      +  test_i18ncmp error_and_hint stderr &&
>     +   git ls-files --debug sparse_entry | grep mtime >after &&
>     +   test_cmp before after
>     + '
>
>     -   # We must unset the SKIP_WORKTREE bit, otherwise
>     -   # git diff-files would skip examining the file
>     -@@ t/t3705-add-sparse-checkout.sh: test_expect_success 'git add --refresh does not update SKIP_WORKTREE entries' '
>     -
>     - test_expect_success 'git add --chmod does not update SKIP_WORKTREE entries' '
>     + test_expect_success 'git add --chmod does not update sparse entries' '
>         setup_sparse_entry &&
>      -  git add --chmod=+x sparse_entry &&
>      +  test_must_fail git add --chmod=+x sparse_entry 2>stderr &&
>      +  test_i18ncmp error_and_hint stderr &&
>     -   test_sparse_entry_unchanged
>     +   test_sparse_entry_unchanged &&
>     +   ! test -x sparse_entry
>       '
>     -
>     -@@ t/t3705-add-sparse-checkout.sh: test_expect_success 'git add --renormalize does not update SKIP_WORKTREE entries
>     +@@ t/t3705-add-sparse-checkout.sh: test_expect_success 'git add --renormalize does not update sparse entries' '
>         test_config core.autocrlf false &&
>         setup_sparse_entry "LINEONE\r\nLINETWO\r\n" &&
>         echo "sparse_entry text=auto" >.gitattributes &&
>     @@ t/t3705-add-sparse-checkout.sh: test_expect_success 'git add --renormalize does
>         test_sparse_entry_unchanged
>       '
>
>     ++test_expect_success 'git add --dry-run --ignore-missing warn on sparse path' '
>     ++  setup_sparse_entry &&
>     ++  rm sparse_entry &&
>     ++  test_must_fail git add --dry-run --ignore-missing sparse_entry 2>stderr &&
>     ++  test_i18ncmp error_and_hint stderr &&
>     ++  test_sparse_entry_unchanged
>     ++'
>     ++
>      +test_expect_success 'do not advice about sparse entries when they do not match the pathspec' '
>      +  setup_sparse_entry &&
>     -+  test_must_fail git add nonexistent sp 2>stderr &&
>     ++  test_must_fail git add nonexistent 2>stderr &&
>      +  test_i18ngrep "fatal: pathspec .nonexistent. did not match any files" stderr &&
>      +  test_i18ngrep ! "The following pathspecs only matched index entries" stderr
>      +'
>      +
>     ++test_expect_success 'do not warn when pathspec matches dense entries' '
>     ++  setup_sparse_entry &&
>     ++  echo modified >sparse_entry &&
>     ++  >dense_entry &&
>     ++  git add "*_entry" 2>stderr &&
>     ++  test_must_be_empty stderr &&
>     ++  test_sparse_entry_unchanged &&
>     ++  git ls-files --error-unmatch dense_entry
>     ++'
>     ++
>      +test_expect_success 'add obeys advice.updateSparsePath' '
>      +  setup_sparse_entry &&
>      +  test_must_fail git -c advice.updateSparsePath=false add sparse_entry 2>stderr &&
> 7:  e76c7c6999 ! 7:  08f0c32bfc rm: honor sparse checkout patterns
>     @@ Commit message
>          rules, but `git rm` doesn't follow the same restrictions. This is
>          somewhat counter-intuitive and inconsistent. So make `rm` honor the
>          sparse checkout and advise on how to remove SKIP_WORKTREE entries, just
>     -    like `add` does. Also add a few tests for the new behavior.
>     +    like `add` does. Also add some tests for the new behavior.
>
>          Suggested-by: Elijah Newren <newren@gmail.com>
>          Signed-off-by: Matheus Tavares <matheus.bernardino@usp.br>
>     @@ builtin/rm.c: int cmd_rm(int argc, const char **argv, const char *prefix)
>      +  return ret;
>       }
>
>     - ## t/t3600-rm.sh ##
>     -@@ t/t3600-rm.sh: test_expect_success 'rm empty string should fail' '
>     -   test_must_fail git rm -rf ""
>     - '
>     -
>     -+test_expect_success 'setup repo for tests with sparse-checkout' '
>     -+  git init sparse &&
>     -+  (
>     -+          cd sparse &&
>     -+          mkdir -p sub/dir &&
>     -+          touch a b c sub/d sub/dir/e &&
>     -+          git add -A &&
>     -+          git commit -m files
>     -+  ) &&
>     + ## t/t3602-rm-sparse-checkout.sh (new) ##
>     +@@
>     ++#!/bin/sh
>     ++
>     ++test_description='git rm in sparse checked out working trees'
>     ++
>     ++. ./test-lib.sh
>     ++
>     ++test_expect_success 'setup' '
>     ++  mkdir -p sub/dir &&
>     ++  touch a b c sub/d sub/dir/e &&
>     ++  git add -A &&
>     ++  git commit -m files &&
>      +
>      +  cat >sparse_entry_b_error <<-EOF &&
>      +  The following pathspecs only matched index entries outside the current
>     @@ t/t3600-rm.sh: test_expect_success 'rm empty string should fail' '
>      +  EOF
>      +'
>      +
>     -+test_expect_success 'rm should respect sparse-checkout' '
>     -+  git -C sparse sparse-checkout set "/a" &&
>     -+  test_must_fail git -C sparse rm b 2>stderr &&
>     -+  test_i18ncmp b_error_and_hint stderr
>     ++for opt in "" -f --dry-run
>     ++do
>     ++  test_expect_success "rm${opt:+ $opt} does not remove sparse entries" '
>     ++          git sparse-checkout set a &&
>     ++          test_must_fail git rm $opt b 2>stderr &&
>     ++          test_i18ncmp b_error_and_hint stderr &&
>     ++          git ls-files --error-unmatch b
>     ++  '
>     ++done
>     ++
>     ++test_expect_success 'recursive rm does not remove sparse entries' '
>     ++  git reset --hard &&
>     ++  git sparse-checkout set sub/dir &&
>     ++  git rm -r sub &&
>     ++  git status --porcelain -uno >actual &&
>     ++  echo "D  sub/dir/e" >expected &&
>     ++  test_cmp expected actual
>      +'
>      +
>      +test_expect_success 'rm obeys advice.updateSparsePath' '
>     -+  git -C sparse reset --hard &&
>     -+  git -C sparse sparse-checkout set "/a" &&
>     -+  test_must_fail git -C sparse -c advice.updateSparsePath=false rm b 2>stderr &&
>     ++  git reset --hard &&
>     ++  git sparse-checkout set a &&
>     ++  test_must_fail git -c advice.updateSparsePath=false rm b 2>stderr &&
>      +  test_i18ncmp sparse_entry_b_error stderr
>     -+
>     -+'
>     -+
>     -+test_expect_success 'recursive rm should respect sparse-checkout' '
>     -+  (
>     -+          cd sparse &&
>     -+          git reset --hard &&
>     -+          git sparse-checkout set "sub/dir" &&
>     -+          git rm -r sub &&
>     -+          git status --porcelain -uno >../actual
>     -+  ) &&
>     -+  echo "D  sub/dir/e" >expected &&
>     -+  test_cmp expected actual
>      +'
>      +
>      +test_expect_success 'do not advice about sparse entries when they do not match the pathspec' '
>     -+  test_must_fail git -C sparse rm nonexistent 2>stderr &&
>     ++  git reset --hard &&
>     ++  git sparse-checkout set a &&
>     ++  test_must_fail git rm nonexistent 2>stderr &&
>      +  test_i18ngrep "fatal: pathspec .nonexistent. did not match any files" stderr &&
>      +  test_i18ngrep ! "The following pathspecs only matched index entries" stderr
>      +'
>      +
>     - test_done
>     ++test_expect_success 'do not warn about sparse entries when pathspec matches dense entries' '
>     ++  git reset --hard &&
>     ++  git sparse-checkout set a &&
>     ++  git rm "[ba]" 2>stderr &&
>     ++  test_must_be_empty stderr &&
>     ++  git ls-files --error-unmatch b &&
>     ++  test_must_fail git ls-files --error-unmatch a
>     ++'
>     ++
>     ++test_expect_success 'do not warn about sparse entries with --ignore-unmatch' '
>     ++  git reset --hard &&
>     ++  git sparse-checkout set a &&
>     ++  git rm --ignore-unmatch b 2>stderr &&
>     ++  test_must_be_empty stderr &&
>     ++  git ls-files --error-unmatch b
>     ++'
>     ++
>     ++test_done
>
>       ## t/t7011-skip-worktree-reading.sh ##
>      @@ t/t7011-skip-worktree-reading.sh: test_expect_success 'diff-files does not examine skip-worktree dirty entries' '
> --
> 2.30.1
>

← back to recent threads