threads / rfc / 36290

RFC patch, 4 partstest-lib: add test_dir_is_empty()

Subject: [RFC/PATCH 1/4] test-lib: add test_dir_is_empty()

## tl;dr

17 messages between Mar 25, 2014 and Apr 18, 2014. Diffs are folded; open one to read it.

replies: 16people: 6as markdown or json

Jens Lehmann· Mar 25, 2014, 17:03 UTC · lore

[RFC/PATCH 0/4] Add submodule test harness

So here is my proposal for a single test harness for submodule updates. It is intended to be easily applicable to all work tree manipulating commands. The current version tests the status quo (and it documents two bugs in current Git). This framework will be extended to cover recursive submodule update too in a later series.

The first patch adds a simple helper function to the test lib which makes it easier to test for an empty submodule directory.

The second patch contains the heavy lifting, it adds the test framework for switching submodules. Currently only transitions without merge conflicts are tested for, I intend to add others producing merge conflicts in a follow-up series.

The third and forth patch show how to apply this framework to a simple command (checkout) and a more complicated case where a helper function is used to execute a preparing command (diff) which produces the input for the then to-be-tested command (apply).

I'm currently working on adding this harness to more commands. read-tree and reset are finished, merge and pull still show some oddities when used with the --no-ff option, also cherry-pick and checkout-index are not working as expected yet. And then there are am, apply, bisect, rebase, revert & stash apply which still need to be covered.

Jens Lehmann (4):
  test-lib: add test_dir_is_empty()
  Submodules: Add the lib-submodule-update.sh test library
  checkout: call the new submodule update test framework
  apply: add t4137 for submodule updates
 t/lib-submodule-update.sh     | 627 ++++++++++++++++++++++++++++++++++++++++++
 t/t2013-checkout-submodule.sh |   5 +
 t/t4137-apply-submodule.sh    |  20 ++
 t/test-lib-functions.sh       |  11 +
 4 files changed, 663 insertions(+)
 create mode 100755 t/lib-submodule-update.sh
 create mode 100755 t/t4137-apply-submodule.sh
-- 
1.9.1.327.g3d8d896
Jens Lehmann· Mar 25, 2014, 17:04 UTC · re: Jens Lehmann · lore

For the upcoming submodule test framework we often need to assert that an empty directory exists in the work tree. Add the test_dir_is_empty() function which asserts that the given argument is an empty directory.

Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>
---

I believe this one is pretty straightforward (unless I missed that this functionality already exists someplace I forgot to look ;-).

 t/test-lib-functions.sh | 11 +++++++++++
 1 file changed, 11 insertions(+)
Show changes to t/test-lib-functions.sh +11 −0
diff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh
index 158e10a..93d10cd 100644
--- a/t/test-lib-functions.sh
+++ b/t/test-lib-functions.sh
@@ -489,6 +489,17 @@ test_path_is_dir () {
 	fi
 }

+# Check if the directory exists and is empty as expected, barf otherwise.
+test_dir_is_empty () {
+	test_path_is_dir "$1" &&
+	if test $(ls -a1 "$1" | wc -l) != 2
+	then
+		echo "Directory '$1' is not empty, it contains:"
+		ls -la "$1"
+		return 1
+	fi
+}
+
 test_path_is_missing () {
 	if [ -e "$1" ]
 	then

-- 
1.9.1.327.g3d8d896
Junio C Hamano· Mar 25, 2014, 20:49 UTC · re: Jens Lehmann · lore

Re: [RFC/PATCH 1/4] test-lib: add test_dir_is_empty()

Jens Lehmann <Jens.Lehmann@web.de> writes:
Show 9 quoted lines
> For the upcoming submodule test framework we often need to assert that an
> empty directory exists in the work tree. Add the test_dir_is_empty()
> function which asserts that the given argument is an empty directory.
>
> Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>
> ---
>
> I believe this one is pretty straightforward (unless I missed that this
> functionality already exists someplace I forgot to look ;-).

I am not very thrilled to see that it depends on "." and ".." to always exist, which may be true for all POSIX filesystems, but still...

Do expected callsites of this helper care if "$1" is a symbolic link that points at an empty directory?

What do expected callsites really want to ensure? In other words, why do they care if the directory is empty? Is it to make sure, after some operation, they can "rmdir" the directory?

Show 25 quoted lines
>  t/test-lib-functions.sh | 11 +++++++++++
>  1 file changed, 11 insertions(+)
>
> diff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh
> index 158e10a..93d10cd 100644
> --- a/t/test-lib-functions.sh
> +++ b/t/test-lib-functions.sh
> @@ -489,6 +489,17 @@ test_path_is_dir () {
>  	fi
>  }
>
> +# Check if the directory exists and is empty as expected, barf otherwise.
> +test_dir_is_empty () {
> +	test_path_is_dir "$1" &&
> +	if test $(ls -a1 "$1" | wc -l) != 2
> +	then
> +		echo "Directory '$1' is not empty, it contains:"
> +		ls -la "$1"
> +		return 1
> +	fi
> +}
> +
>  test_path_is_missing () {
>  	if [ -e "$1" ]
>  	then
David Kastrup· Mar 25, 2014, 21:06 UTC · re: Junio C Hamano · lore

Re: [RFC/PATCH 1/4] test-lib: add test_dir_is_empty()

Junio C Hamano <gitster@pobox.com> writes:
Show 15 quoted lines
> Jens Lehmann <Jens.Lehmann@web.de> writes:
>
>> For the upcoming submodule test framework we often need to assert that an
>> empty directory exists in the work tree. Add the test_dir_is_empty()
>> function which asserts that the given argument is an empty directory.
>>
>> Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>
>> ---
>>
>> I believe this one is pretty straightforward (unless I missed that this
>> functionality already exists someplace I forgot to look ;-).
>
> I am not very thrilled to see that it depends on "." and ".." to
> always exist, which may be true for all POSIX filesystems, but
> still...

Not even there, though few people will likely use / as their work tree...

-- 
David Kastrup
Jens Lehmann· Mar 26, 2014, 08:29 UTC · re: Junio C Hamano · lore

Re: [RFC/PATCH 1/4] test-lib: add test_dir_is_empty()

Am 25.03.2014 21:49, schrieb Junio C Hamano:
Show 15 quoted lines
> Jens Lehmann <Jens.Lehmann@web.de> writes:
> 
>> For the upcoming submodule test framework we often need to assert that an
>> empty directory exists in the work tree. Add the test_dir_is_empty()
>> function which asserts that the given argument is an empty directory.
>>
>> Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>
>> ---
>>
>> I believe this one is pretty straightforward (unless I missed that this
>> functionality already exists someplace I forgot to look ;-).
> 
> I am not very thrilled to see that it depends on "." and ".." to
> always exist, which may be true for all POSIX filesystems, but
> still...

Agreed. I didn't find any one-liners to do that ("ls -A" isn't POSIX), so I decided to wrap that in a function. Testing that "rmdir" on the directory succeeds (because the directory is empty) would kinda work, but then we'd have to re-create the directory afterwards, which really doesn't sound like a good strategy either as the test would manipulate the to-be-tested object. I'm not terribly happy with depending on "." and ".." either, but couldn't come up with something better. At least the test should fail for any filesystem not having the dot files ...

> Do expected callsites of this helper care if "$1" is a symbolic link
> that points at an empty directory?

Yep, a symbolic link pointing to an empty directory should make the test fail.

> What do expected callsites really want to ensure?  In other words,
> why do they care if the directory is empty?  Is it to make sure,
> after some operation, they can "rmdir" the directory?

To assert that a submodule is created but *not* populated. This is intended to catch any possible fallout from the recursive checkout later, in case that would kick in when it shouldn't.

Show 30 quoted lines
>>  t/test-lib-functions.sh | 11 +++++++++++
>>  1 file changed, 11 insertions(+)
>>
>> diff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh
>> index 158e10a..93d10cd 100644
>> --- a/t/test-lib-functions.sh
>> +++ b/t/test-lib-functions.sh
>> @@ -489,6 +489,17 @@ test_path_is_dir () {
>>  	fi
>>  }
>>
>> +# Check if the directory exists and is empty as expected, barf otherwise.
>> +test_dir_is_empty () {
>> +	test_path_is_dir "$1" &&
>> +	if test $(ls -a1 "$1" | wc -l) != 2
>> +	then
>> +		echo "Directory '$1' is not empty, it contains:"
>> +		ls -la "$1"
>> +		return 1
>> +	fi
>> +}
>> +
>>  test_path_is_missing () {
>>  	if [ -e "$1" ]
>>  	then
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
Michael Haggerty· Mar 26, 2014, 10:43 UTC · re: Jens Lehmann · lore

Re: [RFC/PATCH 1/4] test-lib: add test_dir_is_empty()

On 03/26/2014 09:29 AM, Jens Lehmann wrote:
Show 67 quoted lines
> Am 25.03.2014 21:49, schrieb Junio C Hamano:
>> Jens Lehmann <Jens.Lehmann@web.de> writes:
>>
>>> For the upcoming submodule test framework we often need to assert that an
>>> empty directory exists in the work tree. Add the test_dir_is_empty()
>>> function which asserts that the given argument is an empty directory.
>>>
>>> Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>
>>> ---
>>>
>>> I believe this one is pretty straightforward (unless I missed that this
>>> functionality already exists someplace I forgot to look ;-).
>>
>> I am not very thrilled to see that it depends on "." and ".." to
>> always exist, which may be true for all POSIX filesystems, but
>> still...
> 
> Agreed. I didn't find any one-liners to do that ("ls -A" isn't
> POSIX), so I decided to wrap that in a function. Testing that
> "rmdir" on the directory succeeds (because the directory is
> empty) would kinda work, but then we'd have to re-create the
> directory afterwards, which really doesn't sound like a good
> strategy either as the test would manipulate the to-be-tested
> object. I'm not terribly happy with depending on "." and ".."
> either, but couldn't come up with something better. At least
> the test should fail for any filesystem not having the dot
> files ...
> 
>> Do expected callsites of this helper care if "$1" is a symbolic link
>> that points at an empty directory?
> 
> Yep, a symbolic link pointing to an empty directory should make
> the test fail.
> 
>> What do expected callsites really want to ensure?  In other words,
>> why do they care if the directory is empty?  Is it to make sure,
>> after some operation, they can "rmdir" the directory?
> 
> To assert that a submodule is created but *not* populated. This
> is intended to catch any possible fallout from the recursive
> checkout later, in case that would kick in when it shouldn't.
> 
>>>  t/test-lib-functions.sh | 11 +++++++++++
>>>  1 file changed, 11 insertions(+)
>>>
>>> diff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh
>>> index 158e10a..93d10cd 100644
>>> --- a/t/test-lib-functions.sh
>>> +++ b/t/test-lib-functions.sh
>>> @@ -489,6 +489,17 @@ test_path_is_dir () {
>>>  	fi
>>>  }
>>>
>>> +# Check if the directory exists and is empty as expected, barf otherwise.
>>> +test_dir_is_empty () {
>>> +	test_path_is_dir "$1" &&
>>> +	if test $(ls -a1 "$1" | wc -l) != 2
>>> +	then
>>> +		echo "Directory '$1' is not empty, it contains:"
>>> +		ls -la "$1"
>>> +		return 1
>>> +	fi
>>> +}
>>> +
>>>  test_path_is_missing () {
>>>  	if [ -e "$1" ]
>>>  	then
Why not do something like
    test -z "$(ls -a1 "$1" | egrep -v '^\.\.?$')"

I.e., make the test ignore "." and ".." without depending on their existence?

Michael
-- 
Michael Haggerty
mhagger@alum.mit.edu
http://softwareswirl.blogspot.com/
Jens Lehmann· Mar 26, 2014, 19:22 UTC · re: Michael Haggerty · lore

Re: [RFC/PATCH 1/4] test-lib: add test_dir_is_empty()

Am 26.03.2014 11:43, schrieb Michael Haggerty:
Show 35 quoted lines
> On 03/26/2014 09:29 AM, Jens Lehmann wrote:
>> Am 25.03.2014 21:49, schrieb Junio C Hamano:
>>> Jens Lehmann <Jens.Lehmann@web.de> writes:
>>>>  t/test-lib-functions.sh | 11 +++++++++++
>>>>  1 file changed, 11 insertions(+)
>>>>
>>>> diff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh
>>>> index 158e10a..93d10cd 100644
>>>> --- a/t/test-lib-functions.sh
>>>> +++ b/t/test-lib-functions.sh
>>>> @@ -489,6 +489,17 @@ test_path_is_dir () {
>>>>  	fi
>>>>  }
>>>>
>>>> +# Check if the directory exists and is empty as expected, barf otherwise.
>>>> +test_dir_is_empty () {
>>>> +	test_path_is_dir "$1" &&
>>>> +	if test $(ls -a1 "$1" | wc -l) != 2
>>>> +	then
>>>> +		echo "Directory '$1' is not empty, it contains:"
>>>> +		ls -la "$1"
>>>> +		return 1
>>>> +	fi
>>>> +}
>>>> +
>>>>  test_path_is_missing () {
>>>>  	if [ -e "$1" ]
>>>>  	then
> 
> Why not do something like
> 
>     test -z "$(ls -a1 "$1" | egrep -v '^\.\.?$')"
> 
> I.e., make the test ignore "." and ".." without depending on their
> existence?
Thanks, will do so in the next round.
Jens Lehmann· Mar 25, 2014, 17:05 UTC · re: Jens Lehmann · lore

[RFC/PATCH 2/4] Submodules: Add the lib-submodule-update.sh test library

Add this test library to simplify covering all combinations of submodule update scenarios without having to add those to a test of each work tree manipulating command over and over again.

The functions test_submodule_switch() and test_submodule_forced_switch() are intended to be called from a test script with a single argument. This argument is either a work tree manipulating command (including any command line options) or a function (when more than a single git command is needed to switch work trees from the current HEAD to another commit). This command (or function) is passed a target branch as argument. The two new functions check that each submodule transition is handled as expected, which currently means that submodule work trees are not affected until "git submodule update" is called. The "forced" variant is for commands using their '-f' or '--hard' option and expects them to overwrite local modifications as a result. Each of these two functions contains 14 tests_expect_* calls.

Calling one of these test functions the first time creates a repository named "submodule_update_repo". At first it contains two files, then a single submodule is added in another commit followed by commits covering all relevant submodule modifications. This repository is newly cloned into the "submodule_update" for each test_expect_* to avoid interference between different parts of the test functions (some to-be-tested commands also manipulate refs along with the work tree, e.g. "git reset").

Follow-up commits will then call these two test functions for all work tree manipulating commands (with a combination of all their options relevant to what they do with the work tree) making sure they work as expected. Later this test library will be extended to cover merges resulting in conflicts too. Also it is intended to be easily extendable for the recursive update functionality, where even more combinations of submodule modifications have to be tested for.

This version documents two bugs in current Git with expected failures:
*) When a submodule is replaced with a tracked file of the same name the
   submodule work tree including any local modifications (and even the
   whole history if it uses a .git directory instead of a gitfile!) is
   simply removed.
*) Forced work tree updates happily manipulate files in the directory of a
   submodule that has just been removed in the superproject (but is of
   course still present in the work tree due to the way submodules are
   currently handled). This becomes dangerous when files in the submodule
   directory are overwritten by files from the new superproject commit, as
   any modifications to the submodule files will be lost) and is expected
   to also destroy history in the - admittedly unlikely case - the new
   commit adds a file named ".git" to the submodule directory.
Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>
---

I think the first bug really needs to be fixed, as that behavior is extremely nasty. We should always protect work tree modifications (unless forced) and *never* remove a .git directory (even when forced).

I'm not so sure about the second one. Even though I believe the current behavior is not correct (switching commits should never mess around in a submodule directory) it may be that people who committed a directory => submodule transition (or vice versa) are depending on the current behavior. Fixing the bug would forbid to repeat the submodule => directory transition and only allow that after the user removed the submodule directory manually. And as the potential fallout of this bug is not as disastrous as for the first bug, I might be convinced we should not fix it.

 t/lib-submodule-update.sh | 627 ++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 627 insertions(+)
 create mode 100755 t/lib-submodule-update.sh
Show changes to t/lib-submodule-update.sh +627 −0
diff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh
new file mode 100755
index 0000000..de5d54e
--- /dev/null
+++ b/t/lib-submodule-update.sh
@@ -0,0 +1,627 @@
+# Create a submodule layout used for all tests below.
+#
+# The following use cases are covered:
+# - New submodule (no_submodule => add_sub1)
+# - Removed submodule (add_sub1 => remove_sub1)
+# - Updated submodule (add_sub1 => modify_sub1)
+# - Submodule updated to invalid commit (add_sub1 => invalid_sub1)
+# - Submodule updated from invalid commit (invalid_sub1 => valid_sub1)
+# - Submodule replaced by tracked files in directory (add_sub1 =>
+#   replace_sub1_with_directory)
+# - Directory containing tracked files replaced by submodule
+#   (replace_sub1_with_directory => replace_directory_with_sub1)
+# - Submodule replaced by tracked file with the same name (add_sub1 =>
+#   replace_sub1_with_file)
+# - Tracked file replaced by submodule (replace_sub1_with_file =>
+#   replace_file_with_sub1)
+#
+#                   --O-----O
+#                  /  ^     replace_directory_with_sub1
+#                 /   replace_sub1_with_directory
+#                /----O
+#               /     ^
+#              /      modify_sub1
+#      O------O-------O
+#      ^      ^\      ^
+#      |      | \     remove_sub1
+#      |      |  -----O-----O
+#      |      |   \   ^     replace_file_with_sub1
+#      |      |    \  replace_sub1_with_file
+#      |   add_sub1 --O-----O
+# no_submodule        ^     valid_sub1
+#                     invalid_sub1
+#
+create_lib_submodule_repo () {
+	git init submodule_update_repo &&
+	(
+		cd submodule_update_repo &&
+		echo "expect" >>.gitignore &&
+		echo "actual" >>.gitignore &&
+		echo "x" >file1 &&
+		echo "y" >file2 &&
+		git add .gitignore file1 file2 &&
+		git commit -m "Base" &&
+		git branch "no_submodule" &&
+
+		git checkout -b "add_sub1" &&
+		git submodule add ./. sub1 &&
+		git commit -m "Add sub1" &&
+		git checkout -b remove_sub1 &&
+		git revert HEAD &&
+
+		git checkout -b "modify_sub1" "add_sub1" &&
+		git submodule update &&
+		(
+			cd sub1 &&
+			git fetch &&
+			git checkout -b "modifications" &&
+			echo "z" >file2 &&
+			echo "x" >file3 &&
+			git add file2 file3 &&
+			git commit -m "modified file2 and added file3" &&
+			git push origin modifications
+		) &&
+		git add sub1 &&
+		git commit -m "Modify sub1" &&
+
+		git checkout -b "replace_sub1_with_directory" "add_sub1" &&
+		git submodule update &&
+		(
+			cd sub1 &&
+			git checkout modifications
+		) &&
+		git rm --cached sub1 &&
+		rm sub1/.git* &&
+		git config -f .gitmodules --remove-section "submodule.sub1" &&
+		git add .gitmodules sub1/* &&
+		git commit -m "Replace sub1 with directory" &&
+		git checkout -b replace_directory_with_sub1 &&
+		git revert HEAD &&
+
+		git checkout -b "replace_sub1_with_file" "add_sub1" &&
+		git rm sub1 &&
+		echo "content" >sub1 &&
+		git add sub1 &&
+		git commit -m "Replace sub1 with file" &&
+		git checkout -b replace_file_with_sub1 &&
+		git revert HEAD &&
+
+		git checkout -b "invalid_sub1" "add_sub1" &&
+		git update-index --cacheinfo 160000 0123456789012345678901234567890123456789 sub1 &&
+		git commit -m "Invalid sub1 commit" &&
+		git checkout -b valid_sub1 &&
+		git revert HEAD &&
+		git checkout master
+	)
+}
+
+# Helper function to replace gitfile with .git directory
+replace_gitfile_with_git_dir () {
+	(
+		cd "$1" &&
+		git_dir="$(git rev-parse --git-dir)" &&
+		rm -f .git &&
+		cp -a "$git_dir" .git &&
+		GIT_WORK_TREE=. git config --unset core.worktree
+	)
+}
+
+# Test that the .git directory in the submodule is unchanged (except for the
+# core.worktree setting)
+test_git_directory_is_unchanged () {
+	(
+		cd "$1" &&
+		git config core.worktree "../../../$1"
+	) &&
+	git diff -r ".git/modules/$1" "$1/.git" &&
+	(
+		cd "$1" &&
+		GIT_WORK_TREE=. git config --unset core.worktree
+	)
+}
+
+# Helper function to be executed at the start of every test below, it sets up
+# the submodule repo if it doesn't exist and configures the most problematic
+# settings for diff.ignoreSubmodules.
+prolog () {
+	(test -d submodule_update_repo || create_lib_submodule_repo) &&
+	test_config_global diff.ignoreSubmodules all &&
+	test_config diff.ignoreSubmodules all
+}
+
+# Helper function to bring work tree back into the state given by the
+# commit. This includes trying to populate sub1 accordingly if it exists and
+# should be updated to an existing commit.
+reset_work_tree_to () {
+	rm -rf submodule_update &&
+	git clone submodule_update_repo submodule_update &&
+	(
+		cd submodule_update &&
+		rm -rf sub1 &&
+		git checkout -f "$1" &&
+		git status -u -s >actual &&
+		test_must_be_empty actual &&
+		sha1=$(git ls-tree HEAD "sub1" 2>/dev/null | grep 160000 | tr '\t' ' ' | cut -d ' ' -f3) &&
+		if test -n "$sha1" &&
+		   test $(cd "sub1" && git rev-parse --verify "$sha1^{commit}")
+		then
+			git submodule update --init --recursive "sub1"
+		fi
+	)
+}
+
+# Test that the superproject contains the content according to commit "$1"
+# (the work tree must match the index for everything but submodules but the
+# index must exactly match the given commit including any submodule SHA-1s).
+test_superproject_content () {
+	git diff-index --cached "$1" >actual &&
+	test_must_be_empty actual &&
+	git diff-files --ignore-submodules >actual &&
+	test_must_be_empty actual
+}
+
+# Test that the given submodule at path "$1" contains the content according
+# to the submodule commit recorded in the superproject's commit "$2"
+test_submodule_content () {
+	if test $# != 2
+	then
+		echo "test_submodule_content needs two arguments"
+		return 1
+	fi &&
+	submodule="$1" &&
+	commit="$2" &&
+	test -d "$submodule"/ &&
+	if ! test -f "$submodule"/.git && ! test -d "$submodule"/.git
+	then
+		echo "Submodule $submodule is not populated"
+		return 1
+	fi &&
+	sha1=$(git ls-tree "$commit" "$submodule" 2>/dev/null | tr '\t' ' ' | cut -d ' ' -f3) &&
+	if test -z "$sha1"
+	then
+		echo "Couldn't retrieve SHA-1 of $submodule for $commit"
+		return 1
+	fi &&
+	(
+		cd "$submodule" &&
+		git status -u -s >actual &&
+		test_must_be_empty actual &&
+		git diff "$sha1" >actual &&
+		test_must_be_empty actual
+	)
+}
+
+# Test that the following transitions are correctly handled:
+# - Updated submodule
+# - New submodule
+# - Removed submodule
+# - Directory containing tracked files replaced by submodule
+# - Submodule replaced by tracked files in directory
+# - Submodule replaced by tracked file with the same name
+# - tracked file replaced by submodule
+#
+# The default is that submodule contents aren't changed until "git submodule
+# update" is run. And even then that command doesn't delete the work tree of
+# a removed submodule.
+#
+# Removing a submodule containing a .git directory must fail even when forced
+# to protect the history!
+#
+
+# Test that submodule contents are currently not updated when switching
+# between commits that change a submodule.
+test_submodule_switch () {
+	command="$1"
+	######################### Appearing submodule #########################
+	# Switching to a commit letting a submodule appear creates empty dir ...
+	test_expect_success "$command: added submodule creates empty directory" '
+		prolog &&
+		reset_work_tree_to no_submodule &&
+		(
+			cd submodule_update &&
+			git branch -t add_sub1 origin/add_sub1 &&
+			$command add_sub1 &&
+			test_superproject_content origin/add_sub1 &&
+			test_dir_is_empty sub1 &&
+			git submodule update --init --recursive &&
+			test_submodule_content sub1 origin/add_sub1
+		)
+	'
+	# ... and doesn't care if it already exists ...
+	test_expect_success "$command: added submodule leaves existing empty directory alone" '
+		prolog &&
+		reset_work_tree_to no_submodule &&
+		(
+			cd submodule_update &&
+			mkdir sub1 &&
+			git branch -t add_sub1 origin/add_sub1 &&
+			$command add_sub1 &&
+			test_superproject_content origin/add_sub1 &&
+			test_dir_is_empty sub1 &&
+			git submodule update --init --recursive &&
+			test_submodule_content sub1 origin/add_sub1
+		)
+	'
+	# ... unless there is an untracked file in its place.
+	test_expect_success "$command: added submodule doesn't remove untracked unignored file with same name" '
+		prolog &&
+		reset_work_tree_to no_submodule &&
+		(
+			cd submodule_update &&
+			git branch -t add_sub1 origin/add_sub1 &&
+			echo -n >sub1 &&
+			test_must_fail $command add_sub1 &&
+			test_superproject_content origin/no_submodule &&
+			test_must_be_empty sub1
+		)
+	'
+	# Replacing a tracked file with a submodule produces an empty
+	# directory ...
+	test_expect_success "$command: replace tracked file with submodule creates empty directory" '
+		prolog &&
+		reset_work_tree_to replace_sub1_with_file &&
+		(
+			cd submodule_update &&
+			git branch -t replace_file_with_sub1 origin/replace_file_with_sub1 &&
+			$command replace_file_with_sub1 &&
+			test_superproject_content origin/replace_file_with_sub1 &&
+			test_dir_is_empty sub1 &&
+			git submodule update --init --recursive &&
+			test_submodule_content sub1 origin/replace_file_with_sub1
+		)
+	'
+	# ... as does removing a directory with tracked files with a
+	# submodule.
+	test_expect_success "$command: replace directory with submodule" '
+		prolog &&
+		reset_work_tree_to replace_sub1_with_directory &&
+		(
+			cd submodule_update &&
+			git branch -t replace_directory_with_sub1 origin/replace_directory_with_sub1 &&
+			$command replace_directory_with_sub1 &&
+			test_superproject_content origin/replace_directory_with_sub1 &&
+			test_dir_is_empty sub1 &&
+			git submodule update --init --recursive &&
+			test_submodule_content sub1 origin/replace_directory_with_sub1
+		)
+	'
+
+	######################## Disappearing submodule #######################
+	# Removing a submodule doesn't remove its work tree ...
+	test_expect_success "$command: removed submodule leaves submodule directory and its contents in place" '
+		prolog &&
+		reset_work_tree_to add_sub1 &&
+		(
+			cd submodule_update &&
+			git branch -t remove_sub1 origin/remove_sub1 &&
+			$command remove_sub1 &&
+			test_superproject_content origin/remove_sub1 &&
+			test_submodule_content sub1 origin/add_sub1
+		)
+	'
+	# ... especially when it contains a .git directory.
+	test_expect_success "$command: removed submodule leaves submodule containing a .git directory alone" '
+		prolog &&
+		reset_work_tree_to add_sub1 &&
+		(
+			cd submodule_update &&
+			git branch -t remove_sub1 origin/remove_sub1 &&
+			replace_gitfile_with_git_dir sub1 &&
+			$command remove_sub1 &&
+			test_superproject_content origin/remove_sub1 &&
+			test_submodule_content sub1 origin/add_sub1 &&
+			test_git_directory_is_unchanged sub1
+		)
+	'
+	# Replacing a submodule with files in a directory must fail as the
+	# submodule work tree isn't removed ...
+	test_expect_success "$command: replace submodule with a directory fails" '
+		prolog &&
+		reset_work_tree_to add_sub1 &&
+		(
+			cd submodule_update &&
+			git branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&
+			test_must_fail $command replace_sub1_with_directory &&
+			test_superproject_content origin/add_sub1 &&
+			test_submodule_content sub1 origin/add_sub1
+		)
+	'
+	# ... especially when it contains a .git directory.
+	test_expect_success "$command: replace submodule containing a .git directory with a directory fails" '
+		prolog &&
+		reset_work_tree_to add_sub1 &&
+		(
+			cd submodule_update &&
+			git branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&
+			replace_gitfile_with_git_dir sub1 &&
+			test_must_fail $command replace_sub1_with_directory &&
+			test_superproject_content origin/add_sub1 &&
+			test_submodule_content sub1 origin/add_sub1 &&
+			test_git_directory_is_unchanged sub1
+		)
+	'
+	# Replacing it with a file must fail as it could throw away any local
+	# work tree changes ...
+	test_expect_failure "$command: replace submodule directory with a file must fail" '
+		prolog &&
+		reset_work_tree_to add_sub1 &&
+		(
+			cd submodule_update &&
+			git branch -t replace_sub1_with_file origin/replace_sub1_with_file &&
+			test_must_fail $command replace_sub1_with_file &&
+			test_superproject_content origin/add_sub1 &&
+			test_submodule_content sub1 origin/add_sub1
+		)
+	'
+	# ... or even destroy unpushed parts of submodule history if that
+	# still uses a .git directory.
+	test_expect_failure "$command: replace submodule directory with a file must fail" '
+		prolog &&
+		reset_work_tree_to add_sub1 &&
+		(
+			cd submodule_update &&
+			git branch -t replace_sub1_with_file origin/replace_sub1_with_file &&
+			replace_gitfile_with_git_dir sub1 &&
+			test_must_fail $command replace_sub1_with_file &&
+			test_superproject_content origin/add_sub1 &&
+			test_submodule_content sub1 origin/add_sub1 &&
+			test_git_directory_is_unchanged sub1
+		)
+	'
+
+	########################## Modified submodule #########################
+	# Updating a submodule sha1 doesn't update the submodule's work tree
+	test_expect_success "$command: modified submodule does not update submodule work tree" '
+		prolog &&
+		reset_work_tree_to add_sub1 &&
+		(
+			cd submodule_update &&
+			git branch -t modify_sub1 origin/modify_sub1 &&
+			$command modify_sub1 &&
+			test_superproject_content origin/modify_sub1 &&
+			test_submodule_content sub1 origin/add_sub1 &&
+			git submodule update &&
+			test_submodule_content sub1 origin/modify_sub1
+		)
+	'
+
+	# Updating a submodule to an invalid sha1 doesn't update the
+	# submodule's work tree, subsequent update will fail
+	test_expect_success "$command: modified submodule does not update submodule work tree to invalid commit" '
+		prolog &&
+		reset_work_tree_to add_sub1 &&
+		(
+			cd submodule_update &&
+			git branch -t invalid_sub1 origin/invalid_sub1 &&
+			$command invalid_sub1 &&
+			test_superproject_content origin/invalid_sub1 &&
+			test_submodule_content sub1 origin/add_sub1 &&
+			test_must_fail git submodule update &&
+			test_submodule_content sub1 origin/add_sub1
+		)
+	'
+	# Updating a submodule from an invalid sha1 doesn't update the
+	# submodule's work tree, subsequent update will succeed
+	test_expect_success "$command: modified submodule does not update submodule work tree from invalid commit" '
+		prolog &&
+		reset_work_tree_to invalid_sub1 &&
+		(
+			cd submodule_update &&
+			git branch -t valid_sub1 origin/valid_sub1 &&
+			$command valid_sub1 &&
+			test_superproject_content origin/valid_sub1 &&
+			test_dir_is_empty sub1 &&
+			git submodule update --init --recursive &&
+			test_submodule_content sub1 origin/valid_sub1
+		)
+	'
+}
+
+# Test that submodule contents are currently not updated when switching
+# between commits that change a submodule, but throwing away local changes in
+# the superproject is allowed.
+test_submodule_forced_switch () {
+	command="$1"
+	######################### Appearing submodule #########################
+	# Switching to a commit letting a submodule appear creates empty dir ...
+	test_expect_success "$command: added submodule creates empty directory" '
+		prolog &&
+		reset_work_tree_to no_submodule &&
+		(
+			cd submodule_update &&
+			git branch -t add_sub1 origin/add_sub1 &&
+			$command add_sub1 &&
+			test_superproject_content origin/add_sub1 &&
+			test_dir_is_empty sub1 &&
+			git submodule update --init --recursive &&
+			test_submodule_content sub1 origin/add_sub1
+		)
+	'
+	# ... and doesn't care if it already exists ...
+	test_expect_success "$command: added submodule leaves existing empty directory alone" '
+		prolog &&
+		reset_work_tree_to no_submodule &&
+		(
+			cd submodule_update &&
+			git branch -t add_sub1 origin/add_sub1 &&
+			mkdir sub1 &&
+			$command add_sub1 &&
+			test_superproject_content origin/add_sub1 &&
+			test_dir_is_empty sub1 &&
+			git submodule update --init --recursive &&
+			test_submodule_content sub1 origin/add_sub1
+		)
+	'
+	# ... unless there is an untracked file in its place.
+	test_expect_success "$command: added submodule does remove untracked unignored file with same name when forced" '
+		prolog &&
+		reset_work_tree_to no_submodule &&
+		(
+			cd submodule_update &&
+			git branch -t add_sub1 origin/add_sub1 &&
+			echo -n >sub1 &&
+			$command add_sub1 &&
+			test_superproject_content origin/add_sub1 &&
+			test_dir_is_empty sub1
+		)
+	'
+	# Replacing a tracked file with a submodule produces an empty
+	# directory ...
+	test_expect_success "$command: replace tracked file with submodule creates empty directory" '
+		prolog &&
+		reset_work_tree_to replace_sub1_with_file &&
+		(
+			cd submodule_update &&
+			git branch -t replace_file_with_sub1 origin/replace_file_with_sub1 &&
+			$command replace_file_with_sub1 &&
+			test_superproject_content origin/replace_file_with_sub1 &&
+			test_dir_is_empty sub1 &&
+			git submodule update --init --recursive &&
+			test_submodule_content sub1 origin/replace_file_with_sub1
+		)
+	'
+	# ... as does removing a directory with tracked files with a
+	# submodule.
+	test_expect_success "$command: replace directory with submodule" '
+		prolog &&
+		reset_work_tree_to replace_sub1_with_directory &&
+		(
+			cd submodule_update &&
+			git branch -t replace_directory_with_sub1 origin/replace_directory_with_sub1 &&
+			$command replace_directory_with_sub1 &&
+			test_superproject_content origin/replace_directory_with_sub1 &&
+			test_dir_is_empty sub1 &&
+			git submodule update --init --recursive &&
+			test_submodule_content sub1 origin/replace_directory_with_sub1
+		)
+	'
+
+	######################## Disappearing submodule #######################
+	# Removing a submodule doesn't remove its work tree ...
+	test_expect_success "$command: removed submodule leaves submodule directory and its contents in place" '
+		prolog &&
+		reset_work_tree_to add_sub1 &&
+		(
+			cd submodule_update &&
+			git branch -t remove_sub1 origin/remove_sub1 &&
+			$command remove_sub1 &&
+			test_superproject_content origin/remove_sub1 &&
+			test_submodule_content sub1 origin/add_sub1
+		)
+	'
+	# ... especially when it contains a .git directory.
+	test_expect_success "$command: removed submodule leaves submodule containing a .git directory alone" '
+		prolog &&
+		reset_work_tree_to add_sub1 &&
+		(
+			cd submodule_update &&
+			git branch -t remove_sub1 origin/remove_sub1 &&
+			replace_gitfile_with_git_dir sub1 &&
+			$command remove_sub1 &&
+			test_superproject_content origin/remove_sub1 &&
+			test_submodule_content sub1 origin/add_sub1 &&
+			test_git_directory_is_unchanged sub1
+		)
+	'
+	# Replacing a submodule with files in a directory must fail as the
+	# submodule work tree isn't removed ...
+	test_expect_failure "$command: replace submodule with a directory fails" '
+		prolog &&
+		reset_work_tree_to add_sub1 &&
+		(
+			cd submodule_update &&
+			git branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&
+			test_must_fail $command replace_sub1_with_directory &&
+			test_superproject_content origin/add_sub1 &&
+			test_submodule_content sub1 origin/add_sub1
+		)
+	'
+	# ... especially when it contains a .git directory.
+	test_expect_failure "$command: replace submodule containing a .git directory with a directory fails" '
+		prolog &&
+		reset_work_tree_to add_sub1 &&
+		(
+			cd submodule_update &&
+			git branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&
+			replace_gitfile_with_git_dir sub1 &&
+			test_must_fail $command replace_sub1_with_directory &&
+			test_superproject_content origin/add_sub1 &&
+			test_submodule_content sub1 origin/add_sub1 &&
+			test_git_directory_is_unchanged sub1
+		)
+	'
+	# Replacing it with a file must fail as it could throw away any local
+	# work tree changes ...
+	test_expect_failure "$command: replace submodule directory with a file must fail" '
+		prolog &&
+		reset_work_tree_to add_sub1 &&
+		(
+			cd submodule_update &&
+			git branch -t replace_sub1_with_file origin/replace_sub1_with_file &&
+			test_must_fail $command replace_sub1_with_file &&
+			test_superproject_content origin/add_sub1 &&
+			test_submodule_content sub1 origin/add_sub1
+		)
+	'
+	# ... or even destroy unpushed parts of submodule history if that
+	# still uses a .git directory.
+	test_expect_failure "$command: replace submodule directory with a file must fail" '
+		prolog &&
+		reset_work_tree_to add_sub1 &&
+		(
+			cd submodule_update &&
+			git branch -t replace_sub1_with_file origin/replace_sub1_with_file &&
+			replace_gitfile_with_git_dir sub1 &&
+			test_must_fail $command replace_sub1_with_file &&
+			test_superproject_content origin/add_sub1 &&
+			test_submodule_content sub1 origin/add_sub1 &&
+			test_git_directory_is_unchanged sub1
+		)
+	'
+
+	########################## Modified submodule #########################
+	# Updating a submodule sha1 doesn't update the submodule's work tree
+	test_expect_success "$command: modified submodule does not update submodule work tree" '
+		prolog &&
+		reset_work_tree_to add_sub1 &&
+		(
+			cd submodule_update &&
+			git branch -t modify_sub1 origin/modify_sub1 &&
+			$command modify_sub1 &&
+			test_superproject_content origin/modify_sub1 &&
+			test_submodule_content sub1 origin/add_sub1 &&
+			git submodule update &&
+			test_submodule_content sub1 origin/modify_sub1
+		)
+	'
+	# Updating a submodule to an invalid sha1 doesn't update the
+	# submodule's work tree, subsequent update will fail
+	test_expect_success "$command: modified submodule does not update submodule work tree to invalid commit" '
+		prolog &&
+		reset_work_tree_to add_sub1 &&
+		(
+			cd submodule_update &&
+			git branch -t invalid_sub1 origin/invalid_sub1 &&
+			$command invalid_sub1 &&
+			test_superproject_content origin/invalid_sub1 &&
+			test_submodule_content sub1 origin/add_sub1 &&
+			test_must_fail git submodule update &&
+			test_submodule_content sub1 origin/add_sub1
+		)
+	'
+	# Updating a submodule from an invalid sha1 doesn't update the
+	# submodule's work tree, subsequent update will succeed
+	test_expect_success "$command: modified submodule does not update submodule work tree from invalid commit" '
+		prolog &&
+		reset_work_tree_to invalid_sub1 &&
+		(
+			cd submodule_update &&
+			git branch -t valid_sub1 origin/valid_sub1 &&
+			$command valid_sub1 &&
+			test_superproject_content origin/valid_sub1 &&
+			test_dir_is_empty sub1 &&
+			git submodule update --init --recursive &&
+			test_submodule_content sub1 origin/valid_sub1
+		)
+	'
+}

-- 
1.9.1.327.g3d8d896
W. Trevor King· Apr 17, 2014, 16:41 UTC · re: Jens Lehmann · lore

Re: [RFC/PATCH 2/4] Submodules: Add the lib-submodule-update.sh test library

On Tue, Mar 25, 2014 at 06:05:05PM +0100, Jens Lehmann wrote:
Show 9 quoted lines
> *) When a submodule is replaced with a tracked file of the same name
>    the submodule work tree including any local modifications (and
>    even the whole history if it uses a .git directory instead of a
>    gitfile!) is simply removed.
> …
> I think the first bug really needs to be fixed, as that behavior is
> extremely nasty. We should always protect work tree modifications
> (unless forced) and *never* remove a .git directory (even when
> forced).

I think this should be covered by the usual “don't allow checkouts from dirty workdirs unless the dirty-ing changes are easily applied to the target tree”.

Are we waiting to land this series (or a successor) before starting on a fix for this issue? There have been a number of submodule series in flight recently, and I'm having trouble keeping track of them all ;).

Show 13 quoted lines
> *) Forced work tree updates happily manipulate files in the
>    directory of a submodule that has just been removed in the
>    superproject (but is of course still present in the work tree due
>    to the way submodules are currently handled). This becomes
>    dangerous when files in the submodule directory are overwritten
>    by files from the new superproject commit, as any modifications
>    to the submodule files will be lost) and is expected to also
>    destroy history in the - admittedly unlikely case - the new
>    commit adds a file named ".git" to the submodule directory.
> …
> I'm not so sure about the second one. Even though I believe the
> current behavior is not correct (switching commits should never mess
> around in a submodule directory)

This should also be covered by the usual “don't allow checkouts from dirty workdirs unless the dirty-ing changes are easily applied to the target tree”. We don't implement this yet, but I'd like to force users to move any about-to-be-clobbered state from their submodule into .git/modules/<name>/ (via commits or stashes) before allowing them to begin the checkout. Once we've ensured that the state is preserved out-of-tree, then clobber away ;).

Cheers, Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy
Junio C Hamano· Apr 17, 2014, 19:23 UTC · re: W. Trevor King · lore

Re: [RFC/PATCH 2/4] Submodules: Add the lib-submodule-update.sh test library

"W. Trevor King" <wking@tremily.us> writes:
> There have been a number of submodule series in
> flight recently, and I'm having trouble keeping track of them all ;).
Unfortunately I share that same feeling X-<.

Could you guys collectively summarize what issues each of these in-flight topics try to address and how, how close it is to achieve concensus, and how it interact with other proposed topics?

Thanks.
Jens Lehmann· Apr 17, 2014, 21:30 UTC · re: Junio C Hamano · lore

Re: [RFC/PATCH 2/4] Submodules: Add the lib-submodule-update.sh test library

Am 17.04.2014 21:23, schrieb Junio C Hamano:
Show 10 quoted lines
> "W. Trevor King" <wking@tremily.us> writes:
> 
>> There have been a number of submodule series in
>> flight recently, and I'm having trouble keeping track of them all ;).
> 
> Unfortunately I share that same feeling X-<.
> 
> Could you guys collectively summarize what issues each of these
> in-flight topics try to address and how, how close it is to achieve
> concensus, and how it interact with other proposed topics?
I'm aware of these topics:
- My 4 "Never ignore staged but ignored submodules" patches
  I recall that everyone agreed that this is a good change.
- Johan's "Test various 'git submodule update' scenarios"
  Intended to document current behavior of the submodule.<name>.branch
  setting (and others) as a starting point for a discussion of how
  that could (and should) evolve. Needs some cooking.
- My "submodule test harness" RFC series (currently 14 patches)
  Similar to Johan's patch I try to document the current behavior
  of Git, but with the focus on all work tree manipulating commands
  (not only 'submodule update' handling all submodule changes. Will
  send to the list again when I resolved the last outstanding issues,
  current state can be seen in the submodule-test-harness branch of
  my GitHub repo. My next to-be-finished topic.
- Ronald's two "Teach add and commit the --ignore-submodules option"
  options
  Will review v4 soonish, looking good from the first cursory look.
- My "recursive submodule checkout" series
  Needs to be rerolled, I intend to extend my "submodule test harness"
  to cover all relevant scenarios for this series.
- Heiko's "config cache for submodules" patch
  Needed for my recursive checkout series to populate new submodules.

And then a not yet surfaced "do not replace submodules with a file" fix I intend to send between the "submodule test harness" and the "recursive submodule checkout" series.

Hope that makes it clearer ;-)
Heiko Voigt· Apr 18, 2014, 12:39 UTC · re: Jens Lehmann · lore

Re: Re: [RFC/PATCH 2/4] Submodules: Add the lib-submodule-update.sh test library

On Thu, Apr 17, 2014 at 11:30:04PM +0200, Jens Lehmann wrote:
> - Heiko's "config cache for submodules" patch
> 
>   Needed for my recursive checkout series to populate new submodules.

Which will be followed by my recursive fetch series, which is also the last part of what started with this:

http://thread.gmane.org/gmane.comp.version-control.git/217018
and is described here:
https://github.com/hvoigt/git/wiki/submodule-fetch-config
Cheers Heiko
Jens Lehmann· Apr 17, 2014, 21:08 UTC · re: W. Trevor King · lore

Re: [RFC/PATCH 2/4] Submodules: Add the lib-submodule-update.sh test library

Am 17.04.2014 18:41, schrieb W. Trevor King:
Show 14 quoted lines
> On Tue, Mar 25, 2014 at 06:05:05PM +0100, Jens Lehmann wrote:
>> *) When a submodule is replaced with a tracked file of the same name
>>    the submodule work tree including any local modifications (and
>>    even the whole history if it uses a .git directory instead of a
>>    gitfile!) is simply removed.
>> …
>> I think the first bug really needs to be fixed, as that behavior is
>> extremely nasty. We should always protect work tree modifications
>> (unless forced) and *never* remove a .git directory (even when
>> forced).
> 
> I think this should be covered by the usual “don't allow checkouts
> from dirty workdirs unless the dirty-ing changes are easily applied to
> the target tree”.

Nope, the target tree will be removed completely and everything in it is silently nuked. It should be allowed with '-f', but only if the submodule contains a gitfile, and never if it contains a .git directory (which is just what we do for rm too).

> Are we waiting to land this series (or a successor) before starting on
> a fix for this issue?

I think so, as this bug is there for a long time (so I see no urge to fix it very soon) and my test harness is intended to document this current bug (and then soon its fix).

Show 21 quoted lines
>> *) Forced work tree updates happily manipulate files in the
>>    directory of a submodule that has just been removed in the
>>    superproject (but is of course still present in the work tree due
>>    to the way submodules are currently handled). This becomes
>>    dangerous when files in the submodule directory are overwritten
>>    by files from the new superproject commit, as any modifications
>>    to the submodule files will be lost) and is expected to also
>>    destroy history in the - admittedly unlikely case - the new
>>    commit adds a file named ".git" to the submodule directory.
>> …
>> I'm not so sure about the second one. Even though I believe the
>> current behavior is not correct (switching commits should never mess
>> around in a submodule directory)
> 
> This should also be covered by the usual “don't allow checkouts from
> dirty workdirs unless the dirty-ing changes are easily applied to the
> target tree”.  We don't implement this yet, but I'd like to force
> users to move any about-to-be-clobbered state from their submodule
> into .git/modules/<name>/ (via commits or stashes) before allowing
> them to begin the checkout.  Once we've ensured that the state is
> preserved out-of-tree, then clobber away ;).

I'm intending to fix this in the recursive checkout series, as I'm a) not sure if any users currently depend on that for a submodule to directory transition and b) recursive checkout is the place to consistently care about submodule modifications (the submodule script doesn't do that and it is impossible to change that without causing trouble to a lot of users.

W. Trevor King· Apr 17, 2014, 21:55 UTC · re: Jens Lehmann · lore

Re: [RFC/PATCH 2/4] Submodules: Add the lib-submodule-update.sh test library

On Thu, Apr 17, 2014 at 11:08:06PM +0200, Jens Lehmann wrote:
Show 20 quoted lines
> Am 17.04.2014 18:41, schrieb W. Trevor King:
> > On Tue, Mar 25, 2014 at 06:05:05PM +0100, Jens Lehmann wrote:
> >> *) When a submodule is replaced with a tracked file of the same name
> >>    the submodule work tree including any local modifications (and
> >>    even the whole history if it uses a .git directory instead of a
> >>    gitfile!) is simply removed.
> >> …
> >> I think the first bug really needs to be fixed, as that behavior is
> >> extremely nasty. We should always protect work tree modifications
> >> (unless forced) and *never* remove a .git directory (even when
> >> forced).
> > 
> > I think this should be covered by the usual “don't allow checkouts
> > from dirty workdirs unless the dirty-ing changes are easily applied to
> > the target tree”.
> 
> Nope, the target tree will be removed completely and everything in
> it is silently nuked. It should be allowed with '-f', but only if
> the submodule contains a gitfile, and never if it contains a .git
> directory (which is just what we do for rm too).

I think it's not covered *now* because of a flaw in our “are dirty-ing changes easily applied to the target tree” detection logic, and the solution should involve updating that logic to hit on this case.

> b) recursive checkout is the place to consistently care about
> submodule modifications (the submodule script doesn't do that and it
> is impossible to change that without causing trouble to a lot of
> users.

I agree that the submodule script is not the place for this, and the core checkout code is. I'd like checkouts to always be recursive, and see --[no-]recurse-submodules as a finger-breaking stop-gap until we can complete that transition for checkout, bisect, merge, reset, and other work-tree altering commands. I think this is one reason why some folks prefer the stiffer joints you get from a subtree approach.

Cheers, Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy
Jens Lehmann· Apr 18, 2014, 12:31 UTC · re: W. Trevor King · lore

Re: [RFC/PATCH 2/4] Submodules: Add the lib-submodule-update.sh test library

Am 17.04.2014 23:55, schrieb W. Trevor King:
Show 25 quoted lines
> On Thu, Apr 17, 2014 at 11:08:06PM +0200, Jens Lehmann wrote:
>> Am 17.04.2014 18:41, schrieb W. Trevor King:
>>> On Tue, Mar 25, 2014 at 06:05:05PM +0100, Jens Lehmann wrote:
>>>> *) When a submodule is replaced with a tracked file of the same name
>>>>    the submodule work tree including any local modifications (and
>>>>    even the whole history if it uses a .git directory instead of a
>>>>    gitfile!) is simply removed.
>>>> …
>>>> I think the first bug really needs to be fixed, as that behavior is
>>>> extremely nasty. We should always protect work tree modifications
>>>> (unless forced) and *never* remove a .git directory (even when
>>>> forced).
>>>
>>> I think this should be covered by the usual “don't allow checkouts
>>> from dirty workdirs unless the dirty-ing changes are easily applied to
>>> the target tree”.
>>
>> Nope, the target tree will be removed completely and everything in
>> it is silently nuked. It should be allowed with '-f', but only if
>> the submodule contains a gitfile, and never if it contains a .git
>> directory (which is just what we do for rm too).
> 
> I think it's not covered *now* because of a flaw in our “are dirty-ing
> changes easily applied to the target tree” detection logic, and the
> solution should involve updating that logic to hit on this case.
Yup.
Show 11 quoted lines
>> b) recursive checkout is the place to consistently care about
>> submodule modifications (the submodule script doesn't do that and it
>> is impossible to change that without causing trouble to a lot of
>> users.
> 
> I agree that the submodule script is not the place for this, and the
> core checkout code is.  I'd like checkouts to always be recursive, and
> see --[no-]recurse-submodules as a finger-breaking stop-gap until we
> can complete that transition for checkout, bisect, merge, reset, and
> other work-tree altering commands.  I think this is one reason why
> some folks prefer the stiffer joints you get from a subtree approach.
We are definitely in the same boat here :-)
Jens Lehmann· Mar 25, 2014, 17:05 UTC · re: Jens Lehmann · lore

[RFC/PATCH 3/4] checkout: call the new submodule update test framework

Test that the checkout command updates the work tree as expected with or without the '-f' flag.

Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>
---

I think this should explain how to use the framework with a single command and some options.

 t/t2013-checkout-submodule.sh | 5 +++++
 1 file changed, 5 insertions(+)
Show changes to t/t2013-checkout-submodule.sh +5 −0
diff --git a/t/t2013-checkout-submodule.sh b/t/t2013-checkout-submodule.sh
index 06b18f8..6847f75 100755
--- a/t/t2013-checkout-submodule.sh
+++ b/t/t2013-checkout-submodule.sh
@@ -3,6 +3,7 @@
 test_description='checkout can handle submodules'

 . ./test-lib.sh
+. "$TEST_DIRECTORY"/lib-submodule-update.sh

 test_expect_success 'setup' '
 	mkdir submodule &&
@@ -62,4 +63,8 @@ test_expect_success '"checkout <submodule>" honors submodule.*.ignore from .git/
 	! test -s actual
 '

+test_submodule_switch "git checkout"
+
+test_submodule_forced_switch "git checkout -f"
+
 test_done

-- 
1.9.1.327.g3d8d896
Jens Lehmann· Mar 25, 2014, 17:06 UTC · re: Jens Lehmann · lore

[RFC/PATCH 4/4] apply: add t4137 for submodule updates

Test that the apply command updates the work tree as expected for the '--index' and the '--3way' options (for submodule changes which don't result in conflicts).

Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>
---

And this shows how to use the new framework when more than a single command is needed to switch to a new work tree.

 t/t4137-apply-submodule.sh | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)
 create mode 100755 t/t4137-apply-submodule.sh
Show changes to t/t4137-apply-submodule.sh +20 −0
diff --git a/t/t4137-apply-submodule.sh b/t/t4137-apply-submodule.sh
new file mode 100755
index 0000000..f0a0500
--- /dev/null
+++ b/t/t4137-apply-submodule.sh
@@ -0,0 +1,20 @@
+#!/bin/sh
+
+test_description='git am handling submodules'
+
+. ./test-lib.sh
+. "$TEST_DIRECTORY"/lib-submodule-update.sh
+
+apply_index () {
+    git diff --ignore-submodules=dirty "..$1" | git apply --index -
+}
+
+test_submodule_switch "apply_index"
+
+apply_3way () {
+    git diff --ignore-submodules=dirty "..$1" | git apply --3way -
+}
+
+test_submodule_switch "apply_3way"
+
+test_done

-- 
1.9.1.327.g3d8d896

← back to recent threads