threads / patch / 42658

patchpathspec: prevent empty strings as pathspecs

Subject: [PATCH] pathspec: prevent empty strings as pathspecs

## tl;dr

7 messages between Jun 19, 2016 and Jun 19, 2016. Diffs are folded; open one to read it.

replies: 6people: 4as markdown or json

Emily Xie· Jun 19, 2016, 00:57 UTC · lore

For any command that takes a pathspec, passing an empty string will execute the command on all files in the current directory. This results in unexpected behavior. For example, git add "" adds all files to staging, while git rm -rf "" recursively removes all files from the working tree and index. This patch prevents such cases by throwing an error message whenever an empty string is detected as a pathspec.

Signed-off-by: Emily Xie <emilyxxie@gmail.com>
Reported-by: David Turner <novalis@novalis.org>
Mentored-by: Michail Denchev <mdenchev@gmail.com>
Thanks-to: Sarah Sharp <sarah@thesharps.us> and James Sharp <jamey@minilop.net>
---
 pathspec.c     | 6 +++++-
 t/t3600-rm.sh  | 4 ++++
 t/t3700-add.sh | 4 ++++
 3 files changed, 13 insertions(+), 1 deletion(-)
Show changes to 3 files +13 −1

pathspec.c, t/t3600-rm.sh, t/t3700-add.sh

diff --git a/pathspec.c b/pathspec.c
index c9e9b6c..11901a2 100644
--- a/pathspec.c
+++ b/pathspec.c
@@ -402,8 +402,12 @@ void parse_pathspec(struct pathspec *pathspec,
 	}
 
 	n = 0;
-	while (argv[n])
+	while (argv[n]) {
+		if (*argv[n] == '\0') {
+			die("Empty string is not a valid pathspec.");
+		}
 		n++;
+	}
 
 	pathspec->nr = n;
 	ALLOC_ARRAY(pathspec->items, n);
diff --git a/t/t3600-rm.sh b/t/t3600-rm.sh
index d046d98..1d7dc79 100755
--- a/t/t3600-rm.sh
+++ b/t/t3600-rm.sh
@@ -791,6 +791,10 @@ test_expect_success 'setup for testing rm messages' '
 	git add bar.txt foo.txt
 '
 
+test_expect_success 'rm files with empty string pathspec' '
+  test_must_fail git rm ""
+'
+
 test_expect_success 'rm files with different staged content' '
 	cat >expect <<-\EOF &&
 	error: the following files have staged content different from both the
diff --git a/t/t3700-add.sh b/t/t3700-add.sh
index f14a665..49cb080 100755
--- a/t/t3700-add.sh
+++ b/t/t3700-add.sh
@@ -207,6 +207,10 @@ test_expect_success POSIXPERM,SANITY 'git add should fail atomically upon an unr
 	! ( git ls-files foo1 | grep foo1 )
 '
 
+test_expect_success 'git add empty string as pathspec must fail' '
+  test_must_fail git add ""
+'
+
 rm -f foo2
 
 test_expect_success POSIXPERM,SANITY 'git add --ignore-errors' '

-- 
2.8.4
David Turner· Jun 19, 2016, 01:49 UTC · re: Emily Xie · lore

Re: [PATCH] pathspec: prevent empty strings as pathspecs

On Sat, 2016-06-18 at 20:57 -0400, Emily Xie wrote:
Show 13 quoted lines
> For any command that takes a pathspec, passing an empty string will
> execute the command on all files in the current directory. This
> results in unexpected behavior. For example, git add "" adds all
> files to staging, while git rm -rf "" recursively removes all files
> from the working tree and index. This patch prevents such cases by
> throwing an error message whenever an empty string is detected as a
> pathspec.
> 
> Signed-off-by: Emily Xie <emilyxxie@gmail.com>
> Reported-by: David Turner <novalis@novalis.org>
> Mentored-by: Michail Denchev <mdenchev@gmail.com>
> Thanks-to: Sarah Sharp <sarah@thesharps.us> and James Sharp <jamey@minilop.net>
> ---
Overall, lgtm.

The reason for this behavior is that, generally, traversals start at some tree, and if there are no elements in the pathspec, the recursion ends with that tree. Because this is a UX issue, fixing it at the pathspec level seems correct to me.

Show 18 quoted lines
>  pathspec.c     | 6 +++++-
>  t/t3600-rm.sh  | 4 ++++
>  t/t3700-add.sh | 4 ++++
>  3 files changed, 13 insertions(+), 1 deletion(-)
> 
> diff --git a/pathspec.c b/pathspec.c
> index c9e9b6c..11901a2 100644
> --- a/pathspec.c
> +++ b/pathspec.c
> @@ -402,8 +402,12 @@ void parse_pathspec(struct pathspec *pathspec,
>  	}
>  
>  	n = 0;
> -	while (argv[n])
> +	while (argv[n]) {
> +		if (*argv[n] == '\0') {
> +			die("Empty string is not a valid pathspec.");
> +		}
nit: git style doesn't use {} on one-statement ifs.
Show 17 quoted lines
>  		n++;
> +	}
>  
>  	pathspec->nr = n;
>  	ALLOC_ARRAY(pathspec->items, n);
> diff --git a/t/t3600-rm.sh b/t/t3600-rm.sh
> index d046d98..1d7dc79 100755
> --- a/t/t3600-rm.sh
> +++ b/t/t3600-rm.sh
> @@ -791,6 +791,10 @@ test_expect_success 'setup for testing rm messages' '
>  	git add bar.txt foo.txt
>  '
>  
> +test_expect_success 'rm files with empty string pathspec' '
> +  test_must_fail git rm ""
> +'
> +
Maybe check the error message here?
Eric Sunshine· Jun 19, 2016, 02:23 UTC · re: David Turner · lore

Re: [PATCH] pathspec: prevent empty strings as pathspecs

On Sat, Jun 18, 2016 at 9:49 PM, David Turner <novalis@novalis.org> wrote:
Show 7 quoted lines
> On Sat, 2016-06-18 at 20:57 -0400, Emily Xie wrote:
>> +     while (argv[n]) {
>> +             if (*argv[n] == '\0') {
>> +                     die("Empty string is not a valid pathspec.");
>> +             }
>
> nit: git style doesn't use {} on one-statement ifs.

Also, drop the full-stop (period) from the error message and, to be consistent with modern practice, don't capitalize the first word. Finally, the string should be localizable, so wrap it in _(...):

    die(_("empty string not valid pathspec"));
Junio C Hamano· Jun 19, 2016, 03:10 UTC · re: Emily Xie · lore

Re: [PATCH] pathspec: prevent empty strings as pathspecs

Emily Xie <emilyxxie@gmail.com> writes:
Show 7 quoted lines
> For any command that takes a pathspec, passing an empty string will
> execute the command on all files in the current directory. This
> results in unexpected behavior. For example, git add "" adds all
> files to staging, while git rm -rf "" recursively removes all files
> from the working tree and index. This patch prevents such cases by
> throwing an error message whenever an empty string is detected as a
> pathspec.
This change is likely to break people's existing scripts.

For example, a script may want to inspect some state of the whole working tree and then after the inspection was satisfactory do something that affects paths that are inside the directory you were originally in. You would write something like this:

	#!/bin/sh
	original=$(git rev-parse --show-prefix)
        cd "$(git rev-parse --show-toplevel)"
        ... do the whole-tree pre-inspection
        ... using various git commands here
	# Then finally do the "action"
        git add "$original"

This works even if you happen to be at the top-level of the working tree, but with your patch it suddenly breaks. It may be trivial to update the script to use "." when $original is empty, but that is not an excuse. The important thing is that they need to be told about having to change their script before this change actually happens. Suddenly breaking other people without telling is simply rude and unacceptable.

At least you would need two-step process to introduce a change like this to warn the people whose tools and workflows you are breaking. That is, (1) in one release, you add code to only detect the case you will be changing the behaviour in a later version and give warning messages, and (2) in another release that is several release cycles later, stop warning and actually change the behaviour.

That is, if we assume that passing an empty string by mistake is a problem in the real world that we want to save users from. After all, even though there is _no_ good reason why we _need_ to take 'git add ""' as the same as 'git add .', there is no need not to, either. When a user says 'git add sub/sub/' we would add everything in that directory, 'git add sub/' would add even more, and it is logically consistent for 'git add ""' to add everything. This is a border-line "doctor, if I pass an empty string, it is taken the same way as a dot. -- don't do that then."

A kind of change that satisifies the following criteria:
 (1) I can tell that the author of the change really means well.
 (2) The problem the change is fixing does not seem to be a huge
     issue in real life.
 (3) The code with the patch would behave is not wrong per-se; it
     probably is how we would have designed the feature if we knew
     better.
 (4) However, we are not designing things from scratch without
     existing users, and the change potentially breaks uncountable
     number of existing users.
is something I really hate having to evaluate.  This is one of them.
So, I am of two minds.

I do not mind a two-step breaking of compatibility to address this issue; I would also understand if the author thinks it is not worth the hassle to do so. The sudden behaviour change with this patch alone is however not acceptable, I would think.

David Turner· Jun 19, 2016, 04:47 UTC · re: Junio C Hamano · lore

Re: [PATCH] pathspec: prevent empty strings as pathspecs

On 06/18/2016 11:10 PM, Junio C Hamano wrote:
Show 34 quoted lines
> Emily Xie <emilyxxie@gmail.com> writes:
>
>> For any command that takes a pathspec, passing an empty string will
>> execute the command on all files in the current directory. This
>> results in unexpected behavior. For example, git add "" adds all
>> files to staging, while git rm -rf "" recursively removes all files
>> from the working tree and index. This patch prevents such cases by
>> throwing an error message whenever an empty string is detected as a
>> pathspec.
>
> This change is likely to break people's existing scripts.
>
> For example, a script may want to inspect some state of the whole
> working tree and then after the inspection was satisfactory do
> something that affects paths that are inside the directory you were
> originally in.  You would write something like this:
>
> 	#!/bin/sh
> 	original=$(git rev-parse --show-prefix)
>
>          cd "$(git rev-parse --show-toplevel)"
>          ... do the whole-tree pre-inspection
>          ... using various git commands here
>
> 	# Then finally do the "action"
>          git add "$original"
>
> This works even if you happen to be at the top-level of the working
> tree, but with your patch it suddenly breaks.  It may be trivial to
> update the script to use "." when $original is empty, but that is
> not an excuse.  The important thing is that they need to be told
> about having to change their script before this change actually
> happens.  Suddenly breaking other people without telling is simply
> rude and unacceptable.

The help for rev-parse --show-prefix says: "When the command is invoked from a subdirectory, show the path of the current directory relative to the top-level directory."

This doesn't say what it does when invoked at the top-level. Maybe we should just change it to output "./", which would make such scripts work? It would not necessarily fix all possible scripts, but it would fix these. It seem odd to insist on preserving this undocumented behavior of git add/rm.

Show 9 quoted lines
> At least you would need two-step process to introduce a change like
> this to warn the people whose tools and workflows you are breaking.
> That is, (1) in one release, you add code to only detect the case
> you will be changing the behaviour in a later version and give
> warning messages, and (2) in another release that is several release
> cycles later, stop warning and actually change the behaviour.
>
> That is, if we assume that passing an empty string by mistake is a
> problem in the real world that we want to save users from.

I discovered this when one of our shell-scripts had a variable was accidentally empty. The user I was supporting was very confused. So I think it is a problem in the real world.

Junio C Hamano· Jun 19, 2016, 06:18 UTC · re: David Turner · lore

Re: [PATCH] pathspec: prevent empty strings as pathspecs

David Turner <novalis@novalis.org> writes:
Show 9 quoted lines
> The help for rev-parse --show-prefix says:
> "When the command is invoked from a subdirectory, show the path of
> the current directory relative to the top-level directory."
>
> This doesn't say what it does when invoked at the top-level.  Maybe we
> should just change it to output "./", which would make such scripts
> work?  It would not necessarily fix all possible scripts, but it would
> fix these.  It seem odd to insist on preserving this undocumented
> behavior of git add/rm.
For two obvious reasons, you do not want to go there.

As you seem to understand in the above, "--show-prefix" was given merely as an example, so updating its behaviour would not change the fundamental problem with the patch under discussion at all. It will break scripts that knew, documented or not, that pathspec match is a prefix substring match that honors directory boundary and an empty string is defined long time ago to match any path and it does not matter if that definition was made by mistake.

Also, if you change "--show-prefix" output, documented or not, that will break other scripts that knew that checking it against an empty string is a valid way to check if you are already at the top level of the working tree. And I highly suspect that such a change is much harder to advertise and warn existing scripts, compared to the change proposed by the patch under discussion. The output routine in show-prefix knows what it is giving to the caller, but does not know how the caller uses it and what for. Compared to that, ...

Show 6 quoted lines
>> At least you would need two-step process to introduce a change like
>> this to warn the people whose tools and workflows you are breaking.
>> That is, (1) in one release, you add code to only detect the case
>> you will be changing the behaviour in a later version and give
>> warning messages, and (2) in another release that is several release
>> cycles later, stop warning and actually change the behaviour.

... is a lot easier transition (again, assuming that this change is worth doing in the first place).

Emily Xie· Jun 19, 2016, 20:32 UTC · re: Junio C Hamano · lore

Re: [PATCH] pathspec: prevent empty strings as pathspecs

Thanks for the quick responses on this patch!
In response to Junio:
Show 6 quoted lines
> "At least you would need two-step process to introduce a change like
> this to warn the people whose tools and workflows you are breaking.
> That is, (1) in one release, you add code to only detect the case
> you will be changing the behaviour in a later version and give
> warning messages, and (2) in another release that is several release
> cycles later, stop warning and actually change the behaviour....
> I do not mind a two-step breaking of compatibility to address this
> issue; I would also understand if the author thinks it is not worth
> the hassle to do so.  The sudden behaviour change with this patch
> alone is however not acceptable, I would think."

I understand your hesitance with the original method. If the agreed solution is a two-step implementation, I think it would definitely be worth my time and hassle----in part because I'm particularly excited about the prospect of contributing to Git, but mostly because I do believe that it would be a good improvement.

Given this, I'll edit the patch and re-submit to only emit warning messages for now.

- Emily

← back to recent threads