threads / patch / 50739

patch, 4 partst3000: move non-submodule repo test to separate file

Subject: [PATCH 2/4] t3000: move non-submodule repo test to separate file

## tl;dr

16 messages between Mar 14, 2019 and Apr 4, 2019. Diffs are folded; open one to read it.

replies: 15people: 2as markdown or json

Kyle Meyer· Mar 14, 2019, 15:02 UTC · lore

[PATCH 0/4] dir: Treat a repository without commits as a repository

When the working tree contains a repository with no commits, it's treated as an empty directory, not a repository:

    $ git init
    $ git init no-commit && touch no-commit/untracked
    $ git init with-commit && touch with-commit/untracked
    $ git -C with-commit commit --allow-empty -mmsg
    $ git ls-files -o
    no-commit/untracked
    with-commit/

That's admittedly a weird case that is unlikely to happen in the wild, and indeed I didn't observe it in the wild. In DataLad, we rely on `ls-files -o` to stop at the boundary of a submodule (like it does with "with-commit" above), and we noticed the "repository with no commit" exception on a minimal snippet that was meant for debugging another issue [*].

This series makes the "is repository?" check in treat_directory() use is_nonbare_repository_dir() instead of resolve_gitlink_ref() so that a repository without any commits is treated like a repository rather than an empty directory. This makes the reporting from commands like ls-files consistent for the case shown above, and it also avoids the case where 'git add no-commit' (no trailing slash) adds the untracked files of the no-commit/ repository to the index of the current repository.

[*]: https://github.com/datalad/datalad/issues/3139#issuecomment-460542647
Kyle Meyer (4):
  submodule: refuse to add repository with no commits
  t3000: move non-submodule repo test to separate file
  t3009: test that ls-files -o traverses bogus repo
  dir: do not traverse repositories with no commits
 dir.c                                   |  6 ++-
 git-submodule.sh                        | 12 +++++-
 t/t3000-ls-files-others.sh              |  7 ----
 t/t3009-ls-files-others-nonsubmodule.sh | 56 +++++++++++++++++++++++++
 t/t3700-add.sh                          |  1 +
 t/t7400-submodule-basic.sh              | 11 ++++-
 6 files changed, 81 insertions(+), 12 deletions(-)
 create mode 100755 t/t3009-ls-files-others-nonsubmodule.sh
-- 
2.21.0
Kyle Meyer· Mar 14, 2019, 15:02 UTC · re: Kyle Meyer · lore

a2d5156c2b (resolve_gitlink_ref: ignore non-repository paths, 2016-01-22) added a test to t3000-ls-files-others.sh to check that 'ls-files -o' does not die() when given a subdirectory that looks like a repository but is actually a subdirectory containing a bogus .git file.

Move this test to a separate file in preparation for testing scenarios with non-submodule repositories that are not bogus.

Signed-off-by: Kyle Meyer <kyle@kyleam.com>
---
 t/t3000-ls-files-others.sh              |  7 -------
 t/t3009-ls-files-others-nonsubmodule.sh | 21 +++++++++++++++++++++
 2 files changed, 21 insertions(+), 7 deletions(-)
 create mode 100755 t/t3009-ls-files-others-nonsubmodule.sh
Show changes to 2 files +21 −7

t/t3000-ls-files-others.sh, t/t3009-ls-files-others-nonsubmodule.sh

diff --git a/t/t3000-ls-files-others.sh b/t/t3000-ls-files-others.sh
index afd4756134..b4f9fc4580 100755
--- a/t/t3000-ls-files-others.sh
+++ b/t/t3000-ls-files-others.sh
@@ -65,13 +65,6 @@ test_expect_success '--no-empty-directory hides empty directory' '
 	test_cmp expected3 output
 '
 
-test_expect_success 'ls-files --others handles non-submodule .git' '
-	mkdir not-a-submodule &&
-	echo foo >not-a-submodule/.git &&
-	git ls-files -o >output &&
-	test_cmp expected1 output
-'
-
 test_expect_success SYMLINKS 'ls-files --others with symlinked submodule' '
 	git init super &&
 	git init sub &&
diff --git a/t/t3009-ls-files-others-nonsubmodule.sh b/t/t3009-ls-files-others-nonsubmodule.sh
new file mode 100755
index 0000000000..cc66a4a14d
--- /dev/null
+++ b/t/t3009-ls-files-others-nonsubmodule.sh
@@ -0,0 +1,21 @@
+#!/bin/sh
+
+test_description='test git ls-files --others with non-submodule repositories'
+
+. ./test-lib.sh
+
+test_expect_success 'setup: expected output' '
+	cat >expected <<-EOF
+	expected
+	output
+	EOF
+'
+
+test_expect_success 'ls-files --others handles non-submodule .git' '
+	mkdir not-a-submodule &&
+	echo foo >not-a-submodule/.git &&
+	git ls-files -o >output &&
+	test_cmp expected output
+'
+
+test_done
-- 
2.21.0
Kyle Meyer· Mar 14, 2019, 15:02 UTC · re: Kyle Meyer · lore

[PATCH 4/4] dir: do not traverse repositories with no commits

When treat_directory() encounters a directory that is not in the index and DIR_NO_GITLINKS is unset, it calls resolve_gitlink_ref() to decide if a directory looks like a repository, in which case the directory won't be traversed. As a result, 'status -uall' and 'ls-files -o' will show only the directory, even when there are untracked files within the directory.

For the unusual case where a repository doesn't have any commits, resolve_gitlink_ref() returns -1 because HEAD cannot be resolved, and the directory is treated as a normal directory (i.e. traversal does not stop at the repository boundary). The status and ls-files commands above list untracked files within the repository rather than showing only the top-level directory.

The above case is a corner case in an already unusual situation of the working tree containing a repository that is not a tracked submodule, but we might as well treat anything that looks like a repository consistently. Loosen the "looks like a repository" criteria in treat_directory() by replacing resolve_gitlink_ref() with is_nonbare_repository_dir(), one of the checks that is performed downstream when resolve_gitlink_ref() is called with an empty repository.

As the required update to t3700-add shows, being looser with the check means that we're stricter when adding empty repositories to the index:

  % git add repo
  warning: adding embedded git repository: repo
  hint: You've added another git repository inside your current repository.
  hint: [...]
  error: unable to index file 'repo/'
  fatal: adding files failed

That error message isn't particularly helpful in this situation, but it seems preferable to the old behavior of adding the repository's untracked files. And if the caller really wants the previous behavior, they can get it by adding a trailing slash.

Signed-off-by: Kyle Meyer <kyle@kyleam.com>
---
 dir.c                                   |  6 ++++--
 t/t3009-ls-files-others-nonsubmodule.sh | 22 +++++++++++++++++++++-
 t/t3700-add.sh                          |  1 +
 3 files changed, 26 insertions(+), 3 deletions(-)
Show changes to 3 files +26 −3

dir.c, t/t3009-ls-files-others-nonsubmodule.sh, t/t3700-add.sh

diff --git a/dir.c b/dir.c
index b2cabadf25..a4e59eb351 100644
--- a/dir.c
+++ b/dir.c
@@ -1467,9 +1467,11 @@ static enum path_treatment treat_directory(struct dir_struct *dir,
 			return path_none;
 		}
 		if (!(dir->flags & DIR_NO_GITLINKS)) {
-			struct object_id oid;
-			if (resolve_gitlink_ref(dirname, "HEAD", &oid) == 0)
+			struct strbuf sb = STRBUF_INIT;
+			strbuf_addstr(&sb, dirname);
+			if (is_nonbare_repository_dir(&sb))
 				return exclude ? path_excluded : path_untracked;
+			strbuf_release(&sb);
 		}
 		return path_recurse;
 	}
diff --git a/t/t3009-ls-files-others-nonsubmodule.sh b/t/t3009-ls-files-others-nonsubmodule.sh
index 9ed75928aa..be4e7e26bc 100755
--- a/t/t3009-ls-files-others-nonsubmodule.sh
+++ b/t/t3009-ls-files-others-nonsubmodule.sh
@@ -8,6 +8,14 @@ This test runs git ls-files --others with the following working tree:
       directory with no files aside from a bogus .git file
     repo-bogus-untracked-file/
       directory with a bogus .git file and another untracked file
+    repo-no-commit-no-files/
+      git repository without a commit or a file
+    repo-no-commit-untracked-file/
+      git repository without a commit but with an untracked file
+    repo-with-commit-no-files/
+      git repository with a commit and no untracked files
+    repo-with-commit-untracked-file/
+      git repository with a commit and an untracked file
 '
 
 . ./test-lib.sh
@@ -17,6 +25,10 @@ test_expect_success 'setup: expected output' '
 	expected
 	output
 	repo-bogus-untracked-file/untracked
+	repo-no-commit-no-files/
+	repo-no-commit-untracked-file/
+	repo-with-commit-no-files/
+	repo-with-commit-untracked-file/
 	EOF
 '
 
@@ -25,7 +37,15 @@ test_expect_success 'setup: directories' '
 	echo foo >repo-bogus-no-files/.git &&
 	mkdir repo-bogus-untracked-file &&
 	echo foo >repo-bogus-untracked-file/.git &&
-	: >repo-bogus-untracked-file/untracked
+	: >repo-bogus-untracked-file/untracked &&
+	git init repo-no-commit-no-files &&
+	git init repo-no-commit-untracked-file &&
+	: >repo-no-commit-untracked-file/untracked &&
+	git init repo-with-commit-no-files &&
+	git -C repo-with-commit-no-files commit --allow-empty -mmsg &&
+	git init repo-with-commit-untracked-file &&
+	test_commit -C repo-with-commit-untracked-file msg &&
+	: >repo-with-commit-untracked-file/untracked
 '
 
 test_expect_success 'ls-files --others handles non-submodule .git' '
diff --git a/t/t3700-add.sh b/t/t3700-add.sh
index be582a513b..5a8425962b 100755
--- a/t/t3700-add.sh
+++ b/t/t3700-add.sh
@@ -396,6 +396,7 @@ test_expect_success 'no file status change if no pathspec is given in subdir' '
 '
 
 test_expect_success 'all statuses changed in folder if . is given' '
+	rm -fr empty &&
 	git add --chmod=+x . &&
 	test $(git ls-files --stage | grep ^100644 | wc -l) -eq 0 &&
 	git add --chmod=-x . &&
-- 
2.21.0
Kyle Meyer· Mar 14, 2019, 15:02 UTC · re: Kyle Meyer · lore

[PATCH 3/4] t3009: test that ls-files -o traverses bogus repo

When a2d5156c2b (resolve_gitlink_ref: ignore non-repository paths, 2016-01-22) added this test, the purpose was to check the 'ls-files -o' didn't die() when processing the bogus repository. The expected output didn't even need to be adjusted for the addition because the bogus repository is treated as an empty directory.

Let's add another scenario to test that 'ls-files -o' lists an untracked file within this directory. Doing so isn't useful for the original purpose of the test, but it will be helpful for highlighting that this traversal does _not_ happen in the non-bogus scenarios that will be added next. Rename the original subdirectory to highlight the difference with the new one.

Signed-off-by: Kyle Meyer <kyle@kyleam.com>
---
 t/t3009-ls-files-others-nonsubmodule.sh | 21 ++++++++++++++++++---
 1 file changed, 18 insertions(+), 3 deletions(-)
Show changes to t/t3009-ls-files-others-nonsubmodule.sh +18 −3
diff --git a/t/t3009-ls-files-others-nonsubmodule.sh b/t/t3009-ls-files-others-nonsubmodule.sh
index cc66a4a14d..9ed75928aa 100755
--- a/t/t3009-ls-files-others-nonsubmodule.sh
+++ b/t/t3009-ls-files-others-nonsubmodule.sh
@@ -1,6 +1,14 @@
 #!/bin/sh
 
-test_description='test git ls-files --others with non-submodule repositories'
+test_description='test git ls-files --others with non-submodule repositories
+
+This test runs git ls-files --others with the following working tree:
+
+    repo-bogus-no-files/
+      directory with no files aside from a bogus .git file
+    repo-bogus-untracked-file/
+      directory with a bogus .git file and another untracked file
+'
 
 . ./test-lib.sh
 
@@ -8,12 +16,19 @@ test_expect_success 'setup: expected output' '
 	cat >expected <<-EOF
 	expected
 	output
+	repo-bogus-untracked-file/untracked
 	EOF
 '
 
+test_expect_success 'setup: directories' '
+	mkdir repo-bogus-no-files &&
+	echo foo >repo-bogus-no-files/.git &&
+	mkdir repo-bogus-untracked-file &&
+	echo foo >repo-bogus-untracked-file/.git &&
+	: >repo-bogus-untracked-file/untracked
+'
+
 test_expect_success 'ls-files --others handles non-submodule .git' '
-	mkdir not-a-submodule &&
-	echo foo >not-a-submodule/.git &&
 	git ls-files -o >output &&
 	test_cmp expected output
 '
-- 
2.21.0
Kyle Meyer· Mar 14, 2019, 15:02 UTC · re: Kyle Meyer · lore

[PATCH 1/4] submodule: refuse to add repository with no commits

When the path given to 'git submodule add' is an existing repository that is not in the index, the repository is passed to 'git add'. If this repository doesn't have any commits, we don't get a useful result: there is no subproject OID to track, and any untracked files in the sub-repository are added to the current repository.

Detect if the path is a repository with no commits and abort to avoid getting into this state unless --force is used. Reacting to --force isn't very useful, especially because an upcoming commit will make 'git add' fail in this situation, but it allows us to use the same 'git add --dry-run' condition as the ignored path case.

Signed-off-by: Kyle Meyer <kyle@kyleam.com>
---
 git-submodule.sh           | 12 ++++++++++--
 t/t7400-submodule-basic.sh | 11 ++++++++++-
 2 files changed, 20 insertions(+), 3 deletions(-)
Show changes to 2 files +20 −3

git-submodule.sh, t/t7400-submodule-basic.sh

diff --git a/git-submodule.sh b/git-submodule.sh
index 514ede2596..6c74656027 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -234,10 +234,18 @@ cmd_add()
 	if test -z "$force" &&
 		! git add --dry-run --ignore-missing --no-warn-embedded-repo "$sm_path" > /dev/null 2>&1
 	then
-		eval_gettextln "The following path is ignored by one of your .gitignore files:
+		if test -d "$sm_path" &&
+			test -z $(git -C "$sm_path" rev-parse --show-cdup 2>/dev/null) &&
+			! git -C "$sm_path" rev-parse --verify -q HEAD >/dev/null
+		then
+			die "$(eval_gettext "'\$sm_path' does not have any commits")"
+		else
+			eval_gettextln "\
+The following path is ignored by one of your .gitignore files:
 \$sm_path
 Use -f if you really want to add it." >&2
-		exit 1
+			exit 1
+		fi
 	fi
 
 	if test -n "$custom_name"
diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh
index aba2d4d6ee..6adf640143 100755
--- a/t/t7400-submodule-basic.sh
+++ b/t/t7400-submodule-basic.sh
@@ -46,6 +46,15 @@ test_expect_success 'submodule update aborts on missing gitmodules url' '
 	test_must_fail git submodule init
 '
 
+test_expect_success 'add aborts on repository with no commits' '
+	cat >expected <<-\EOF &&
+	'"'repo-no-commits'"' does not have any commits
+	EOF
+	git init repo-no-commits &&
+	test_must_fail git submodule add ../a ./repo-no-commits 2>actual &&
+	test_i18ncmp expected actual
+'
+
 test_expect_success 'setup - repository in init subdirectory' '
 	mkdir init &&
 	(
@@ -809,7 +818,7 @@ test_expect_success '../bar/a/b/c works with relative local path - ../foo/bar.gi
 		cp pristine-.git-config .git/config &&
 		cp pristine-.gitmodules .gitmodules &&
 		mkdir -p a/b/c &&
-		(cd a/b/c && git init) &&
+		(cd a/b/c && git init && test_commit msg) &&
 		git config remote.origin.url ../foo/bar.git &&
 		git submodule add ../bar/a/b/c ./a/b/c &&
 		git submodule init &&
-- 
2.21.0
Kyle Meyer· Mar 16, 2019, 15:40 UTC · re: Kyle Meyer · lore

Re: [PATCH 1/4] submodule: refuse to add repository with no commits

Kyle Meyer <kyle@kyleam.com> writes:
[...]
Show 22 quoted lines
> diff --git a/git-submodule.sh b/git-submodule.sh
> index 514ede2596..6c74656027 100755
> --- a/git-submodule.sh
> +++ b/git-submodule.sh
> @@ -234,10 +234,18 @@ cmd_add()
>  	if test -z "$force" &&
>  		! git add --dry-run --ignore-missing --no-warn-embedded-repo "$sm_path" > /dev/null 2>&1
>  	then
> -		eval_gettextln "The following path is ignored by one of your .gitignore files:
> +		if test -d "$sm_path" &&
> +			test -z $(git -C "$sm_path" rev-parse --show-cdup 2>/dev/null) &&
> +			! git -C "$sm_path" rev-parse --verify -q HEAD >/dev/null
> +		then
> +			die "$(eval_gettext "'\$sm_path' does not have any commits")"
> +		else
> +			eval_gettextln "\
> +The following path is ignored by one of your .gitignore files:
>  \$sm_path
>  Use -f if you really want to add it." >&2
> -		exit 1
> +			exit 1
> +		fi

I didn't think through this check, which would have been obvious had I ran the added test without the rest of the patches in this series. It assumes that the 'git add --dry-run' call fails, but that failure depends on the last patch of this series. So I'd need to move this patch to the end or find a new place for this "no commits" check.

Kyle Meyer· Apr 2, 2019, 18:35 UTC · re: Kyle Meyer · lore

[PATCH v2 0/4] dir: Treat a repository without commits as a repository

Show 32 quoted lines
> Kyle Meyer <kyle@kyleam.com> writes:
>
> [...]
>
>> diff --git a/git-submodule.sh b/git-submodule.sh
>> index 514ede2596..6c74656027 100755
>> --- a/git-submodule.sh
>> +++ b/git-submodule.sh
>> @@ -234,10 +234,18 @@ cmd_add()
>>  	if test -z "$force" &&
>>  		! git add --dry-run --ignore-missing --no-warn-embedded-repo "$sm_path" > /dev/null 2>&1
>>  	then
>> -		eval_gettextln "The following path is ignored by one of your .gitignore files:
>> +		if test -d "$sm_path" &&
>> +			test -z $(git -C "$sm_path" rev-parse --show-cdup 2>/dev/null) &&
>> +			! git -C "$sm_path" rev-parse --verify -q HEAD >/dev/null
>> +		then
>> +			die "$(eval_gettext "'\$sm_path' does not have any commits")"
>> +		else
>> +			eval_gettextln "\
>> +The following path is ignored by one of your .gitignore files:
>>  \$sm_path
>>  Use -f if you really want to add it." >&2
>> -		exit 1
>> +			exit 1
>> +		fi
>
> I didn't think through this check, which would have been obvious had I
> ran the added test without the rest of the patches in this series.  It
> assumes that the 'git add --dry-run' call fails, but that failure
> depends on the last patch of this series.  So I'd need to move this
> patch to the end or find a new place for this "no commits" check.

v2 moves the "no commits" check outside of the 'git add --dry-run' failure condition so that the first patch doesn't depend on the final patch in this series.

Kyle Meyer (4):
  submodule: refuse to add repository with no commits
  t3000: move non-submodule repo test to separate file
  t3009: test that ls-files -o traverses bogus repo
  dir: do not traverse repositories with no commits
 dir.c                                   |  6 ++-
 git-submodule.sh                        |  7 ++++
 t/t3000-ls-files-others.sh              |  7 ----
 t/t3009-ls-files-others-nonsubmodule.sh | 56 +++++++++++++++++++++++++
 t/t3700-add.sh                          |  1 +
 t/t7400-submodule-basic.sh              | 11 ++++-
 6 files changed, 78 insertions(+), 10 deletions(-)
 create mode 100755 t/t3009-ls-files-others-nonsubmodule.sh
Range-diff against v1:
1:  e0db7e3c3c ! 1:  b080e2c557 submodule: refuse to add repository with no commits
    @@ -9,10 +9,9 @@
         in the sub-repository are added to the current repository.
     
         Detect if the path is a repository with no commits and abort to avoid
    -    getting into this state unless --force is used.  Reacting to --force
    -    isn't very useful, especially because an upcoming commit will make
    -    'git add' fail in this situation, but it allows us to use the same
    -    'git add --dry-run' condition as the ignored path case.
    +    getting into this state.  Note that this check must come before the
    +    'git add --dry-run' check because an upcoming commit will make 'git
    +    add' fail in this situation.
     
         Signed-off-by: Kyle Meyer <kyle@kyleam.com>
     
    @@ -20,26 +19,19 @@
      --- a/git-submodule.sh
      +++ b/git-submodule.sh
     @@
    + 		die "$(eval_gettext "'\$sm_path' already exists in the index and is not a submodule")"
    + 	fi
    + 
    ++	if test -d "$sm_path" &&
    ++		test -z $(git -C "$sm_path" rev-parse --show-cdup 2>/dev/null)
    ++	then
    ++	    git -C "$sm_path" rev-parse --verify -q HEAD >/dev/null ||
    ++	    die "$(eval_gettext "'\$sm_path' does not have any commits")"
    ++	fi
    ++
      	if test -z "$force" &&
      		! git add --dry-run --ignore-missing --no-warn-embedded-repo "$sm_path" > /dev/null 2>&1
      	then
    --		eval_gettextln "The following path is ignored by one of your .gitignore files:
    -+		if test -d "$sm_path" &&
    -+			test -z $(git -C "$sm_path" rev-parse --show-cdup 2>/dev/null) &&
    -+			! git -C "$sm_path" rev-parse --verify -q HEAD >/dev/null
    -+		then
    -+			die "$(eval_gettext "'\$sm_path' does not have any commits")"
    -+		else
    -+			eval_gettextln "\
    -+The following path is ignored by one of your .gitignore files:
    - \$sm_path
    - Use -f if you really want to add it." >&2
    --		exit 1
    -+			exit 1
    -+		fi
    - 	fi
    - 
    - 	if test -n "$custom_name"
     
      diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh
      --- a/t/t7400-submodule-basic.sh
2:  6eed1f5daf = 2:  c027701842 t3000: move non-submodule repo test to separate file
3:  7ba3209762 = 3:  97f53e30c0 t3009: test that ls-files -o traverses bogus repo
4:  2901375dc1 = 4:  a926b87102 dir: do not traverse repositories with no commits
-- 
2.21.0
Kyle Meyer· Apr 2, 2019, 18:35 UTC · re: Kyle Meyer · lore

[PATCH v2 1/4] submodule: refuse to add repository with no commits

When the path given to 'git submodule add' is an existing repository that is not in the index, the repository is passed to 'git add'. If this repository doesn't have any commits, we don't get a useful result: there is no subproject OID to track, and any untracked files in the sub-repository are added to the current repository.

Detect if the path is a repository with no commits and abort to avoid getting into this state. Note that this check must come before the 'git add --dry-run' check because an upcoming commit will make 'git add' fail in this situation.

Signed-off-by: Kyle Meyer <kyle@kyleam.com>
---
 git-submodule.sh           |  7 +++++++
 t/t7400-submodule-basic.sh | 11 ++++++++++-
 2 files changed, 17 insertions(+), 1 deletion(-)
Show changes to 2 files +17 −1

git-submodule.sh, t/t7400-submodule-basic.sh

diff --git a/git-submodule.sh b/git-submodule.sh
index 514ede2596..5a9dc9bf7e 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -231,6 +231,13 @@ cmd_add()
 		die "$(eval_gettext "'\$sm_path' already exists in the index and is not a submodule")"
 	fi
 
+	if test -d "$sm_path" &&
+		test -z $(git -C "$sm_path" rev-parse --show-cdup 2>/dev/null)
+	then
+	    git -C "$sm_path" rev-parse --verify -q HEAD >/dev/null ||
+	    die "$(eval_gettext "'\$sm_path' does not have any commits")"
+	fi
+
 	if test -z "$force" &&
 		! git add --dry-run --ignore-missing --no-warn-embedded-repo "$sm_path" > /dev/null 2>&1
 	then
diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh
index aba2d4d6ee..6adf640143 100755
--- a/t/t7400-submodule-basic.sh
+++ b/t/t7400-submodule-basic.sh
@@ -46,6 +46,15 @@ test_expect_success 'submodule update aborts on missing gitmodules url' '
 	test_must_fail git submodule init
 '
 
+test_expect_success 'add aborts on repository with no commits' '
+	cat >expected <<-\EOF &&
+	'"'repo-no-commits'"' does not have any commits
+	EOF
+	git init repo-no-commits &&
+	test_must_fail git submodule add ../a ./repo-no-commits 2>actual &&
+	test_i18ncmp expected actual
+'
+
 test_expect_success 'setup - repository in init subdirectory' '
 	mkdir init &&
 	(
@@ -809,7 +818,7 @@ test_expect_success '../bar/a/b/c works with relative local path - ../foo/bar.gi
 		cp pristine-.git-config .git/config &&
 		cp pristine-.gitmodules .gitmodules &&
 		mkdir -p a/b/c &&
-		(cd a/b/c && git init) &&
+		(cd a/b/c && git init && test_commit msg) &&
 		git config remote.origin.url ../foo/bar.git &&
 		git submodule add ../bar/a/b/c ./a/b/c &&
 		git submodule init &&
-- 
2.21.0
Junio C Hamano· Apr 4, 2019, 07:24 UTC · re: Kyle Meyer · lore

Re: [PATCH v2 1/4] submodule: refuse to add repository with no commits

Kyle Meyer <kyle@kyleam.com> writes:
Show 10 quoted lines
> When the path given to 'git submodule add' is an existing repository
> that is not in the index, the repository is passed to 'git add'.  If
> this repository doesn't have any commits, we don't get a useful
> result: there is no subproject OID to track, and any untracked files
> in the sub-repository are added to the current repository.
>
> Detect if the path is a repository with no commits and abort to avoid
> getting into this state.  Note that this check must come before the
> 'git add --dry-run' check because an upcoming commit will make 'git
> add' fail in this situation.
OK, in short, in an already-populated repository,
	$ git init junk ;# there was no 'junk' subdirectory
	$ git add junk

would have been a successful no-op, and after the above two commands,

	$ >junk/one
	$ git add junk

would have added junk/one as a normal blob to the top-level project, ignoring the fact that the user wanted to eventually populate junk as a repository and add it as a submodule. You'll be teaching "git add" not to do so, and pay attention to the fact that junk/.git exists, even though junk/.git/HEAD is on an unborn branch.

Which makes sense to me.
Show 53 quoted lines
> Signed-off-by: Kyle Meyer <kyle@kyleam.com>
> ---
>  git-submodule.sh           |  7 +++++++
>  t/t7400-submodule-basic.sh | 11 ++++++++++-
>  2 files changed, 17 insertions(+), 1 deletion(-)
>
> diff --git a/git-submodule.sh b/git-submodule.sh
> index 514ede2596..5a9dc9bf7e 100755
> --- a/git-submodule.sh
> +++ b/git-submodule.sh
> @@ -231,6 +231,13 @@ cmd_add()
>  		die "$(eval_gettext "'\$sm_path' already exists in the index and is not a submodule")"
>  	fi
>  
> +	if test -d "$sm_path" &&
> +		test -z $(git -C "$sm_path" rev-parse --show-cdup 2>/dev/null)
> +	then
> +	    git -C "$sm_path" rev-parse --verify -q HEAD >/dev/null ||
> +	    die "$(eval_gettext "'\$sm_path' does not have any commits")"
> +	fi
> +
>  	if test -z "$force" &&
>  		! git add --dry-run --ignore-missing --no-warn-embedded-repo "$sm_path" > /dev/null 2>&1
>  	then
> diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh
> index aba2d4d6ee..6adf640143 100755
> --- a/t/t7400-submodule-basic.sh
> +++ b/t/t7400-submodule-basic.sh
> @@ -46,6 +46,15 @@ test_expect_success 'submodule update aborts on missing gitmodules url' '
>  	test_must_fail git submodule init
>  '
>  
> +test_expect_success 'add aborts on repository with no commits' '
> +	cat >expected <<-\EOF &&
> +	'"'repo-no-commits'"' does not have any commits
> +	EOF
> +	git init repo-no-commits &&
> +	test_must_fail git submodule add ../a ./repo-no-commits 2>actual &&
> +	test_i18ncmp expected actual
> +'
> +
>  test_expect_success 'setup - repository in init subdirectory' '
>  	mkdir init &&
>  	(
> @@ -809,7 +818,7 @@ test_expect_success '../bar/a/b/c works with relative local path - ../foo/bar.gi
>  		cp pristine-.git-config .git/config &&
>  		cp pristine-.gitmodules .gitmodules &&
>  		mkdir -p a/b/c &&
> -		(cd a/b/c && git init) &&
> +		(cd a/b/c && git init && test_commit msg) &&
>  		git config remote.origin.url ../foo/bar.git &&
>  		git submodule add ../bar/a/b/c ./a/b/c &&
>  		git submodule init &&
Kyle Meyer· Apr 2, 2019, 18:35 UTC · re: Kyle Meyer · lore

[PATCH v2 2/4] t3000: move non-submodule repo test to separate file

a2d5156c2b (resolve_gitlink_ref: ignore non-repository paths, 2016-01-22) added a test to t3000-ls-files-others.sh to check that 'ls-files -o' does not die() when given a subdirectory that looks like a repository but is actually a subdirectory containing a bogus .git file.

Move this test to a separate file in preparation for testing scenarios with non-submodule repositories that are not bogus.

Signed-off-by: Kyle Meyer <kyle@kyleam.com>
---
 t/t3000-ls-files-others.sh              |  7 -------
 t/t3009-ls-files-others-nonsubmodule.sh | 21 +++++++++++++++++++++
 2 files changed, 21 insertions(+), 7 deletions(-)
 create mode 100755 t/t3009-ls-files-others-nonsubmodule.sh
Show changes to 2 files +21 −7

t/t3000-ls-files-others.sh, t/t3009-ls-files-others-nonsubmodule.sh

diff --git a/t/t3000-ls-files-others.sh b/t/t3000-ls-files-others.sh
index afd4756134..b4f9fc4580 100755
--- a/t/t3000-ls-files-others.sh
+++ b/t/t3000-ls-files-others.sh
@@ -65,13 +65,6 @@ test_expect_success '--no-empty-directory hides empty directory' '
 	test_cmp expected3 output
 '
 
-test_expect_success 'ls-files --others handles non-submodule .git' '
-	mkdir not-a-submodule &&
-	echo foo >not-a-submodule/.git &&
-	git ls-files -o >output &&
-	test_cmp expected1 output
-'
-
 test_expect_success SYMLINKS 'ls-files --others with symlinked submodule' '
 	git init super &&
 	git init sub &&
diff --git a/t/t3009-ls-files-others-nonsubmodule.sh b/t/t3009-ls-files-others-nonsubmodule.sh
new file mode 100755
index 0000000000..cc66a4a14d
--- /dev/null
+++ b/t/t3009-ls-files-others-nonsubmodule.sh
@@ -0,0 +1,21 @@
+#!/bin/sh
+
+test_description='test git ls-files --others with non-submodule repositories'
+
+. ./test-lib.sh
+
+test_expect_success 'setup: expected output' '
+	cat >expected <<-EOF
+	expected
+	output
+	EOF
+'
+
+test_expect_success 'ls-files --others handles non-submodule .git' '
+	mkdir not-a-submodule &&
+	echo foo >not-a-submodule/.git &&
+	git ls-files -o >output &&
+	test_cmp expected output
+'
+
+test_done
-- 
2.21.0
Junio C Hamano· Apr 3, 2019, 07:59 UTC · re: Kyle Meyer · lore

Re: [PATCH v2 2/4] t3000: move non-submodule repo test to separate file

Kyle Meyer <kyle@kyleam.com> writes:
Show 8 quoted lines
> a2d5156c2b (resolve_gitlink_ref: ignore non-repository paths,
> 2016-01-22) added a test to t3000-ls-files-others.sh to check that
> 'ls-files -o' does not die() when given a subdirectory that looks like
> a repository but is actually a subdirectory containing a bogus .git
> file.
>
> Move this test to a separate file in preparation for testing scenarios
> with non-submodule repositories that are not bogus.
It is unclear to me why this is needed.
Show 14 quoted lines
> +++ b/t/t3009-ls-files-others-nonsubmodule.sh
> @@ -0,0 +1,21 @@
> +#!/bin/sh
> +
> +test_description='test git ls-files --others with non-submodule repositories'
> +
> +. ./test-lib.sh
> +
> +test_expect_success 'setup: expected output' '
> +	cat >expected <<-EOF
> +	expected
> +	output
> +	EOF
> +'

I think this is overkill. Usually we have one expectation for a single test, so having the above inside the actual test below makes more sense.

Or are you planning to add more tests before the test_done we see below, all of which expect the above output? It would make perfect sense if it were the case, but I do not think that is what is happenning here...

Show 8 quoted lines
> +test_expect_success 'ls-files --others handles non-submodule .git' '
> +	mkdir not-a-submodule &&
> +	echo foo >not-a-submodule/.git &&
> +	git ls-files -o >output &&
> +	test_cmp expected output
> +'
> +
> +test_done
Kyle Meyer· Apr 3, 2019, 22:21 UTC · re: Junio C Hamano · lore

Re: [PATCH v2 2/4] t3000: move non-submodule repo test to separate file

Junio C Hamano <gitster@pobox.com> writes:
Show 12 quoted lines
> Kyle Meyer <kyle@kyleam.com> writes:
>
>> a2d5156c2b (resolve_gitlink_ref: ignore non-repository paths,
>> 2016-01-22) added a test to t3000-ls-files-others.sh to check that
>> 'ls-files -o' does not die() when given a subdirectory that looks like
>> a repository but is actually a subdirectory containing a bogus .git
>> file.
>>
>> Move this test to a separate file in preparation for testing scenarios
>> with non-submodule repositories that are not bogus.
>
> It is unclear to me why this is needed.

It's not needed. My thinking, which I didn't do a good job of spelling out above, is

    We're going to be adding a test that checks how 'ls-files -o'
    handles a few different scenarios involving untracked repositories.
    This new test should go into a separate file rather than
    t3000-ls-files-others.sh because it substantially changes the shared
    directory layout that the t3000 tests work on.  Like the upcoming
    test, the "non-submodule .git" test from t3000 deals with a (bogus)
    untracked repository, so let's split it off into a separate test
    file that will be extended with the other scenarios.

Perhaps that's not a good reason to touch t3000, though. I could drop this patch, as well as the next one, and just add the new test file in the final patch.

Show 18 quoted lines
>> +++ b/t/t3009-ls-files-others-nonsubmodule.sh
>> @@ -0,0 +1,21 @@
>> +#!/bin/sh
>> +
>> +test_description='test git ls-files --others with non-submodule repositories'
>> +
>> +. ./test-lib.sh
>> +
>> +test_expect_success 'setup: expected output' '
>> +	cat >expected <<-EOF
>> +	expected
>> +	output
>> +	EOF
>> +'
>
> I think this is overkill.  Usually we have one expectation for a
> single test, so having the above inside the actual test below makes
> more sense.
OK, I'll move this into the test.
Kyle Meyer· Apr 2, 2019, 18:35 UTC · re: Kyle Meyer · lore

[PATCH v2 3/4] t3009: test that ls-files -o traverses bogus repo

When a2d5156c2b (resolve_gitlink_ref: ignore non-repository paths, 2016-01-22) added this test, the purpose was to check the 'ls-files -o' didn't die() when processing the bogus repository. The expected output didn't even need to be adjusted for the addition because the bogus repository is treated as an empty directory.

Let's add another scenario to test that 'ls-files -o' lists an untracked file within this directory. Doing so isn't useful for the original purpose of the test, but it will be helpful for highlighting that this traversal does _not_ happen in the non-bogus scenarios that will be added next. Rename the original subdirectory to highlight the difference with the new one.

Signed-off-by: Kyle Meyer <kyle@kyleam.com>
---
 t/t3009-ls-files-others-nonsubmodule.sh | 21 ++++++++++++++++++---
 1 file changed, 18 insertions(+), 3 deletions(-)
Show changes to t/t3009-ls-files-others-nonsubmodule.sh +18 −3
diff --git a/t/t3009-ls-files-others-nonsubmodule.sh b/t/t3009-ls-files-others-nonsubmodule.sh
index cc66a4a14d..9ed75928aa 100755
--- a/t/t3009-ls-files-others-nonsubmodule.sh
+++ b/t/t3009-ls-files-others-nonsubmodule.sh
@@ -1,6 +1,14 @@
 #!/bin/sh
 
-test_description='test git ls-files --others with non-submodule repositories'
+test_description='test git ls-files --others with non-submodule repositories
+
+This test runs git ls-files --others with the following working tree:
+
+    repo-bogus-no-files/
+      directory with no files aside from a bogus .git file
+    repo-bogus-untracked-file/
+      directory with a bogus .git file and another untracked file
+'
 
 . ./test-lib.sh
 
@@ -8,12 +16,19 @@ test_expect_success 'setup: expected output' '
 	cat >expected <<-EOF
 	expected
 	output
+	repo-bogus-untracked-file/untracked
 	EOF
 '
 
+test_expect_success 'setup: directories' '
+	mkdir repo-bogus-no-files &&
+	echo foo >repo-bogus-no-files/.git &&
+	mkdir repo-bogus-untracked-file &&
+	echo foo >repo-bogus-untracked-file/.git &&
+	: >repo-bogus-untracked-file/untracked
+'
+
 test_expect_success 'ls-files --others handles non-submodule .git' '
-	mkdir not-a-submodule &&
-	echo foo >not-a-submodule/.git &&
 	git ls-files -o >output &&
 	test_cmp expected output
 '
-- 
2.21.0
Kyle Meyer· Apr 2, 2019, 18:35 UTC · re: Kyle Meyer · lore

[PATCH v2 4/4] dir: do not traverse repositories with no commits

When treat_directory() encounters a directory that is not in the index and DIR_NO_GITLINKS is unset, it calls resolve_gitlink_ref() to decide if a directory looks like a repository, in which case the directory won't be traversed. As a result, 'status -uall' and 'ls-files -o' will show only the directory, even when there are untracked files within the directory.

For the unusual case where a repository doesn't have any commits, resolve_gitlink_ref() returns -1 because HEAD cannot be resolved, and the directory is treated as a normal directory (i.e. traversal does not stop at the repository boundary). The status and ls-files commands above list untracked files within the repository rather than showing only the top-level directory.

The above case is a corner case in an already unusual situation of the working tree containing a repository that is not a tracked submodule, but we might as well treat anything that looks like a repository consistently. Loosen the "looks like a repository" criteria in treat_directory() by replacing resolve_gitlink_ref() with is_nonbare_repository_dir(), one of the checks that is performed downstream when resolve_gitlink_ref() is called with an empty repository.

As the required update to t3700-add shows, being looser with the check means that we're stricter when adding empty repositories to the index:

  % git add repo
  warning: adding embedded git repository: repo
  hint: You've added another git repository inside your current repository.
  hint: [...]
  error: unable to index file 'repo/'
  fatal: adding files failed

That error message isn't particularly helpful in this situation, but it seems preferable to the old behavior of adding the repository's untracked files. And if the caller really wants the previous behavior, they can get it by adding a trailing slash.

Signed-off-by: Kyle Meyer <kyle@kyleam.com>
---
 dir.c                                   |  6 ++++--
 t/t3009-ls-files-others-nonsubmodule.sh | 22 +++++++++++++++++++++-
 t/t3700-add.sh                          |  1 +
 3 files changed, 26 insertions(+), 3 deletions(-)
Show changes to 3 files +26 −3

dir.c, t/t3009-ls-files-others-nonsubmodule.sh, t/t3700-add.sh

diff --git a/dir.c b/dir.c
index b2cabadf25..a4e59eb351 100644
--- a/dir.c
+++ b/dir.c
@@ -1467,9 +1467,11 @@ static enum path_treatment treat_directory(struct dir_struct *dir,
 			return path_none;
 		}
 		if (!(dir->flags & DIR_NO_GITLINKS)) {
-			struct object_id oid;
-			if (resolve_gitlink_ref(dirname, "HEAD", &oid) == 0)
+			struct strbuf sb = STRBUF_INIT;
+			strbuf_addstr(&sb, dirname);
+			if (is_nonbare_repository_dir(&sb))
 				return exclude ? path_excluded : path_untracked;
+			strbuf_release(&sb);
 		}
 		return path_recurse;
 	}
diff --git a/t/t3009-ls-files-others-nonsubmodule.sh b/t/t3009-ls-files-others-nonsubmodule.sh
index 9ed75928aa..be4e7e26bc 100755
--- a/t/t3009-ls-files-others-nonsubmodule.sh
+++ b/t/t3009-ls-files-others-nonsubmodule.sh
@@ -8,6 +8,14 @@ This test runs git ls-files --others with the following working tree:
       directory with no files aside from a bogus .git file
     repo-bogus-untracked-file/
       directory with a bogus .git file and another untracked file
+    repo-no-commit-no-files/
+      git repository without a commit or a file
+    repo-no-commit-untracked-file/
+      git repository without a commit but with an untracked file
+    repo-with-commit-no-files/
+      git repository with a commit and no untracked files
+    repo-with-commit-untracked-file/
+      git repository with a commit and an untracked file
 '
 
 . ./test-lib.sh
@@ -17,6 +25,10 @@ test_expect_success 'setup: expected output' '
 	expected
 	output
 	repo-bogus-untracked-file/untracked
+	repo-no-commit-no-files/
+	repo-no-commit-untracked-file/
+	repo-with-commit-no-files/
+	repo-with-commit-untracked-file/
 	EOF
 '
 
@@ -25,7 +37,15 @@ test_expect_success 'setup: directories' '
 	echo foo >repo-bogus-no-files/.git &&
 	mkdir repo-bogus-untracked-file &&
 	echo foo >repo-bogus-untracked-file/.git &&
-	: >repo-bogus-untracked-file/untracked
+	: >repo-bogus-untracked-file/untracked &&
+	git init repo-no-commit-no-files &&
+	git init repo-no-commit-untracked-file &&
+	: >repo-no-commit-untracked-file/untracked &&
+	git init repo-with-commit-no-files &&
+	git -C repo-with-commit-no-files commit --allow-empty -mmsg &&
+	git init repo-with-commit-untracked-file &&
+	test_commit -C repo-with-commit-untracked-file msg &&
+	: >repo-with-commit-untracked-file/untracked
 '
 
 test_expect_success 'ls-files --others handles non-submodule .git' '
diff --git a/t/t3700-add.sh b/t/t3700-add.sh
index be582a513b..5a8425962b 100755
--- a/t/t3700-add.sh
+++ b/t/t3700-add.sh
@@ -396,6 +396,7 @@ test_expect_success 'no file status change if no pathspec is given in subdir' '
 '
 
 test_expect_success 'all statuses changed in folder if . is given' '
+	rm -fr empty &&
 	git add --chmod=+x . &&
 	test $(git ls-files --stage | grep ^100644 | wc -l) -eq 0 &&
 	git add --chmod=-x . &&
-- 
2.21.0
Junio C Hamano· Apr 3, 2019, 08:05 UTC · re: Kyle Meyer · lore

Re: [PATCH v2 4/4] dir: do not traverse repositories with no commits

Kyle Meyer <kyle@kyleam.com> writes:
> The above case is a corner case in an already unusual situation of the
> working tree containing a repository that is not a tracked submodule,
> but we might as well treat anything that looks like a repository
> consistently.
Makes sense.
Show 9 quoted lines
> As the required update to t3700-add shows, being looser with the check
> means that we're stricter when adding empty repositories to the index:
>
>   % git add repo
>   warning: adding embedded git repository: repo
>   hint: You've added another git repository inside your current repository.
>   hint: [...]
>   error: unable to index file 'repo/'
>   fatal: adding files failed

Hmph, why should we have the initial "warning" with hint? Shouldn't the above result in an outright error? Something like:

    $ git add repo ;# or git add repo/
    error: repo does not have a commit checked out
    fatal: adding files failed

because the entire purpose of "git add repo" (or "git add repo/") when "repo" is a repository is to register the commit that is checked out in that sub-repository to the index of the top-level repository, as part of updating (or starting) a submodule.

I very much like the direction these four patches want to take us. I just have small troubles here and there in their execution, though ;-)

Thanks.
Kyle Meyer· Apr 3, 2019, 22:25 UTC · re: Junio C Hamano · lore

Re: [PATCH v2 4/4] dir: do not traverse repositories with no commits

Junio C Hamano <gitster@pobox.com> writes:
> Kyle Meyer <kyle@kyleam.com> writes:
Show 21 quoted lines
>> As the required update to t3700-add shows, being looser with the check
>> means that we're stricter when adding empty repositories to the index:
>>
>>   % git add repo
>>   warning: adding embedded git repository: repo
>>   hint: You've added another git repository inside your current repository.
>>   hint: [...]
>>   error: unable to index file 'repo/'
>>   fatal: adding files failed
>
> Hmph, why should we have the initial "warning" with hint?  Shouldn't
> the above result in an outright error?  Something like:
>
>     $ git add repo ;# or git add repo/
>     error: repo does not have a commit checked out
>     fatal: adding files failed
>
> because the entire purpose of "git add repo" (or "git add repo/") when
> "repo" is a repository is to register the commit that is checked out
> in that sub-repository to the index of the top-level repository, as
> part of updating (or starting) a submodule.

Right, the hint makes no sense here. I'll look into adjusting the output to be appropriate.

As a minor note: I believe your "git add repo/" comments above describe what should be and not what currently is. The trailing slash results in adding the content from the specified repository to the current repository rather than registering the repository as a submodule:

  $ git --version
  git version 2.21.0
  $ git init
  $ git init a
  $ git -C a commit --allow-empty -mmsg
  $ touch a/f
  $ git add a/
  $ git status -s
  A  a/f

This behavior (and fixing it) was discussed at https://public-inbox.org/git/20180618111919.GA10085@book.hvoigt.net

← back to recent threads