threads / patch / 34747

patch, 2 partsfix t3010 failure when core.ignorecase=true

Subject: [PATCH 0/2] fix t3010 failure when core.ignorecase=true

## tl;dr

7 messages between Aug 23, 2013 and Aug 26, 2013. Diffs are folded; open one to read it.

replies: 6people: 3as markdown or json

Eric Sunshine· Aug 23, 2013, 04:29 UTC · lore

This series fixes a bug in dir.c which causes t3010 to fail [1] when core.ignorecase is true. The problem is that directory_exists_in_index(dirname,len) and directory_exists_in_index_icase() behave differently if dirname[len] is not a '/', even though this is beyond end-of-string. 2eac2a4cc4bdc8d7 (ls-files -k: a directory only can be killed if the index has a non-directory; 2013-08-15) adds a caller which neglects to ensure that the the required '/' is present, hence the failure.

I am not happy with the fix, which is too add a '/' after the last character in dirname just at the call site introduced by 2eac2a4cc4bdc8d7.

The reason for my unhappiness is that directory_exists_in_index_icase() makes the assumption, not only that it can access the character beyond the end-of-string, but also that that character will unconditionally be '/'. I presume that this was done for the sake of speed (existing callers always had a '/' beyond end-of-string), but it feels like an ugly wart. Since the required trailing '/' is purely an implementation detail of directory_exists_in_index_icase(), and not of directory_exists_in_index(), a cleaner fix would be for directory_exists_in_index_icase() to add the '/' it needs, and not expect the passed in dirname to have a '/' after its last character. Unfortunately, such a fix would probably negate any optimization benefit gained by the present implementation.

[1]: http://thread.gmane.org/gmane.comp.version-control.git/232727
Eric Sunshine (2):
  t3103: demonstrate dir.c:treat_one_path() core.ignorecase failure
  dir: test_one_path: fix inconsistent behavior due to missing '/'
 dir.c                   | 12 +++++++++---
 t/t3103-ls-tree-misc.sh | 15 +++++++++++++++
 2 files changed, 24 insertions(+), 3 deletions(-)
-- 
1.8.4.rc4.529.g78818d7
Eric Sunshine· Aug 23, 2013, 04:29 UTC · re: Eric Sunshine · lore

[PATCH 1/2] t3103: demonstrate dir.c:treat_one_path() core.ignorecase failure

2eac2a4cc4bdc8d7 (ls-files -k: a directory only can be killed if the index has a non-directory; 2013-08-15) adds a caller of directory_exists_in_index(dirname,len) which forgets to satisfy the undocumented requirement that a '/' must be present at dirname[len] (despite being past the end-of-string). This oversight leads to incorrect behavior when core.ignorecase is true. Demonstrate this.

Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
---
 t/t3103-ls-tree-misc.sh | 15 +++++++++++++++
 1 file changed, 15 insertions(+)
Show changes to t/t3103-ls-tree-misc.sh +15 −0
diff --git a/t/t3103-ls-tree-misc.sh b/t/t3103-ls-tree-misc.sh
index 09dcf04..fd95991 100755
--- a/t/t3103-ls-tree-misc.sh
+++ b/t/t3103-ls-tree-misc.sh
@@ -4,6 +4,7 @@ test_description='
 Miscellaneous tests for git ls-tree.
 
 	      1. git ls-tree fails in presence of tree damage.
+	      2. git ls-tree acts sanely with core.ignorecase.
 
 '
 
@@ -21,4 +22,18 @@ test_expect_success 'ls-tree fails with non-zero exit code on broken tree' '
 	test_must_fail git ls-tree -r HEAD
 '
 
+test_expect_failure 'ls-tree directory core.ignorecase' '
+	cat >expect <<-\EOF &&
+	d/e/f
+	EOF
+	mkdir d &&
+	>d/e &&
+	git update-index --add -- d/e &&
+	rm d/e &&
+	mkdir d/e &&
+	>d/e/f &&
+	git -c core.ignorecase=true ls-files -k >actual &&
+	test_cmp expect actual
+'
+
 test_done

-- 
1.8.4.rc4.529.g78818d7
Junio C Hamano· Aug 23, 2013, 17:21 UTC · re: Eric Sunshine · lore

Re: [PATCH 1/2] t3103: demonstrate dir.c:treat_one_path() core.ignorecase failure

Eric Sunshine <sunshine@sunshineco.com> writes:
Show 17 quoted lines
> @@ -21,4 +22,18 @@ test_expect_success 'ls-tree fails with non-zero exit code on broken tree' '
>  	test_must_fail git ls-tree -r HEAD
>  '
>  
> +test_expect_failure 'ls-tree directory core.ignorecase' '
> +	cat >expect <<-\EOF &&
> +	d/e/f
> +	EOF
> +	mkdir d &&
> +	>d/e &&
> +	git update-index --add -- d/e &&
> +	rm d/e &&
> +	mkdir d/e &&
> +	>d/e/f &&
> +	git -c core.ignorecase=true ls-files -k >actual &&
> +	test_cmp expect actual
> +'

Hmm. Wouldn't it be a clearer demonstration to add to t3010 another invocation of "ls-files -k" that was added by 3c568751 (t3010: update to demonstrate "ls-files -k" optimization pitfalls, 2013-08-15) but with core.ignorecase=true? Something like...

 t/t3010-ls-files-killed-modified.sh | 11 +++++++++++
 1 file changed, 11 insertions(+)
Show changes to t/t3010-ls-files-killed-modified.sh +11 −0
diff --git a/t/t3010-ls-files-killed-modified.sh b/t/t3010-ls-files-killed-modified.sh
index 6ea7ca8..f4783ed 100755
--- a/t/t3010-ls-files-killed-modified.sh
+++ b/t/t3010-ls-files-killed-modified.sh
@@ -90,6 +90,17 @@ pathx/ju/nk
 EOF
 
 test_expect_success \
+    'git ls-files -k to show killed files (with icase)' \
+    'git -c core.ignorecase=true ls-files -k >.output'
+cat >.expected <<EOF
+path0/file0
+path1/file1
+path2
+path3
+pathx/ju/nk
+EOF
+
+test_expect_success \
     'validate git ls-files -k output.' \
     'test_cmp .expected .output'
Eric Sunshine· Aug 23, 2013, 04:29 UTC · re: Eric Sunshine · lore

[PATCH 2/2] dir: test_one_path: fix inconsistent behavior due to missing '/'

Although undocumented, directory_exists_in_index_icase(dirname,len) unconditionally assumes the presence of a '/' at dirname[len] (despite being past the end-of-string). Callers are expected to respect this assumption by ensuring that a '/' is present beyond the last character of the passed path. directory_exists_in_index(), on the other hand, does not assume nor care about a trailing '/' beyond end-of-string.

2eac2a4cc4bdc8d7 (ls-files -k: a directory only can be killed if the index has a non-directory; 2013-08-15) adds a caller which forgets to ensure the trailing '/', thus leading to inconsistent behavior between directory_exists_in_index() and directory_exists_in_index_icase() depending upon the setting of core.ignorecase. Fix this problem.

This also fixes an initially-unnoticed failure in a t3010 test added by 3c56875176390eee (t3010: update to demonstrate "ls-files -k" optimization pitfalls; 2013-08-15) when core.ignorecase is true.

Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
---
 dir.c                   | 12 +++++++++---
 t/t3103-ls-tree-misc.sh |  2 +-
 2 files changed, 10 insertions(+), 4 deletions(-)
Show changes to 2 files +10 −4

dir.c, t/t3103-ls-tree-misc.sh

diff --git a/dir.c b/dir.c
index edd666a..a52c6f9 100644
--- a/dir.c
+++ b/dir.c
@@ -1160,9 +1160,15 @@ static enum path_treatment treat_one_path(struct dir_struct *dir,
 	 */
 	if ((dir->flags & DIR_COLLECT_KILLED_ONLY) &&
 	    (dtype == DT_DIR) &&
-	    !has_path_in_index &&
-	    (directory_exists_in_index(path->buf, path->len) == index_nonexistent))
-		return path_none;
+	    !has_path_in_index) {
+		strbuf_addch(path, '/');
+		if (directory_exists_in_index(path->buf, path->len - 1) ==
+		    index_nonexistent) {
+			strbuf_setlen(path, path->len - 1);
+			return path_none;
+		}
+		strbuf_setlen(path, path->len - 1);
+	}
 
 	exclude = is_excluded(dir, path->buf, &dtype);
 
diff --git a/t/t3103-ls-tree-misc.sh b/t/t3103-ls-tree-misc.sh
index fd95991..9fb1706 100755
--- a/t/t3103-ls-tree-misc.sh
+++ b/t/t3103-ls-tree-misc.sh
@@ -22,7 +22,7 @@ test_expect_success 'ls-tree fails with non-zero exit code on broken tree' '
 	test_must_fail git ls-tree -r HEAD
 '
 
-test_expect_failure 'ls-tree directory core.ignorecase' '
+test_expect_success 'ls-tree directory core.ignorecase' '
 	cat >expect <<-\EOF &&
 	d/e/f
 	EOF

-- 
1.8.4.rc4.529.g78818d7
Jonathan Nieder· Aug 25, 2013, 06:00 UTC · re: Eric Sunshine · lore

Re: [PATCH 2/2] dir: test_one_path: fix inconsistent behavior due to missing '/'

Eric Sunshine wrote:
> Although undocumented, directory_exists_in_index_icase(dirname,len)
> unconditionally assumes the presence of a '/' at dirname[len] (despite
> being past the end-of-string). Callers are expected to respect
[...]
>                                                Fix this problem.

So, does this fix the problem by changing directory_exists_in_index_icase() to be more liberal in what it accepts, or callers to be more conservative in what they pass in?

Please forgive my laziness. I ask in order to save future readers the time of digging into the code.

Thanks, Jonathan

Eric Sunshine· Aug 25, 2013, 08:05 UTC · re: Jonathan Nieder · lore

Re: [PATCH 2/2] dir: test_one_path: fix inconsistent behavior due to missing '/'

On Sun, Aug 25, 2013 at 2:00 AM, Jonathan Nieder <jrnieder@gmail.com> wrote:
Show 11 quoted lines
> Eric Sunshine wrote:
>
>> Although undocumented, directory_exists_in_index_icase(dirname,len)
>> unconditionally assumes the presence of a '/' at dirname[len] (despite
>> being past the end-of-string). Callers are expected to respect
> [...]
>>                                                Fix this problem.
>
> So, does this fix the problem by changing
> directory_exists_in_index_icase() to be more liberal in what it
> accepts, or callers to be more conservative in what they pass in?

It places the onus upon the caller. As mentioned in the cover letter [1], I was not happy with this solution. Junio felt likewise. A follow-up series [2] fixes both directory_exists_in_index() and directory_exists_in_index_icase() to be more liberal in what they accept, relieving the caller of the burden. By the time that series was posted, however, Junio and Peff had decided that a fix at a more fundamental level would be better (a conclusion with which I agree, but for which I do not yet have sufficient knowledge about git internals to implement). In the meantime, as an interim bug fix, Junio decided [3] to apply the patch to which you responded (but with an updated commit message).

[1]: http://thread.gmane.org/gmane.comp.version-control.git/232796 [2]: http://thread.gmane.org/gmane.comp.version-control.git/232833 [3]: http://thread.gmane.org/gmane.comp.version-control.git/232833/focus=232837

> Please forgive my laziness.  I ask in order to save future readers the
> time of digging into the code.
Junio C Hamano· Aug 26, 2013, 05:38 UTC · re: Eric Sunshine · lore

Re: [PATCH 2/2] dir: test_one_path: fix inconsistent behavior due to missing '/'

Eric Sunshine <sunshine@sunshineco.com> writes:
Show 6 quoted lines
> ... A
> follow-up series [2] fixes both directory_exists_in_index() and
> directory_exists_in_index_icase() to be more liberal in what they
> accept, relieving the caller of the burden. By the time that series
> was posted, however, Junio and Peff had decided that a fix at a more
> fundamental level would be better (a conclusion with which I agree,

Thanks for a summary; the last "would be better" is a qualified one, though.

My understanding is that we agreed that it would be better *if* we can fix it at a more fundamental level. I looked at the codepaths involved just enough to make that off the cuff suggestion and no deeper than that, and I suspect neither did Peff.

If any of us looked at the problem deep enough, we may realize that it would affect too many things, and $gmane/232833/focus=232835 (your second round) may turn out to be a better solution. I think none of us know yet.

← back to recent threads