threads / patch / 55212

patchconvert: fail gracefully upon missing clean cmd on required filter

Subject: [PATCH] convert: fail gracefully upon missing clean cmd on required filter

## tl;dr

5 messages between Feb 25, 2021 and Feb 28, 2021. Diffs are folded; open one to read it.

replies: 4people: 3as markdown or json

Matheus Tavares· Feb 25, 2021, 18:16 UTC · lore

The gitattributes documentation mentions that either the clean cmd or the smudge cmd can be left unspecified in a filter definition. However, when the filter is marked as 'required', the absence of any one of these two should be treated as an error. Git already fails under these circumstances, but not always in a pleasant way: omitting a clean cmd in a required filter triggers an assertion error which leaves the user with a quite verbose message:

git: convert.c:1462: convert_to_git_filter_fd: Assertion "ca.drv->clean || ca.drv->process" failed.

This assertion and the one above it are not really necessary, as the apply_filter() call bellow them already performs the same checks. And when these conditions are not met, the function returns 0, making the caller die() with a much nicer message. (Also note that die()-ing here is the right behavior as `would_convert_to_git_filter_fd() == true` is a precondition to use convert_to_git_filter_fd(), and the former is only true when the filter is required.) So remove both assertions and add two regression tests to make sure that git fails nicely when either the smudge or clean command is missing on a required filter.

Signed-off-by: Matheus Tavares <matheus.bernardino@usp.br>
---
 convert.c             |  3 ---
 t/t0021-conversion.sh | 24 ++++++++++++++++++++++++
 2 files changed, 24 insertions(+), 3 deletions(-)
Show changes to 2 files +24 −3

convert.c, t/t0021-conversion.sh

diff --git a/convert.c b/convert.c
index ee360c2f07..48fbdac676 100644
--- a/convert.c
+++ b/convert.c
@@ -1455,9 +1455,6 @@ void convert_to_git_filter_fd(const struct index_state *istate,
 	struct conv_attrs ca;
 	convert_attrs(istate, &ca, path);
 
-	assert(ca.drv);
-	assert(ca.drv->clean || ca.drv->process);
-
 	if (!apply_filter(path, NULL, 0, fd, dst, ca.drv, CAP_CLEAN, NULL, NULL))
 		die(_("%s: clean filter '%s' failed"), path, ca.drv->name);
 
diff --git a/t/t0021-conversion.sh b/t/t0021-conversion.sh
index e828ee964c..4f8415d419 100755
--- a/t/t0021-conversion.sh
+++ b/t/t0021-conversion.sh
@@ -257,6 +257,30 @@ test_expect_success 'required filter clean failure' '
 	test_must_fail git add test.fc
 '
 
+test_expect_success 'required filter with absent clean field' '
+	test_config filter.absentclean.smudge cat &&
+	test_config filter.absentclean.required true &&
+
+	echo "*.ac filter=absentclean" >.gitattributes &&
+
+	echo test >test.ac &&
+	test_must_fail git add test.ac 2>stderr &&
+	test_i18ngrep "fatal: test.ac: clean filter .absentclean. failed" stderr
+'
+
+test_expect_success 'required filter with absent smudge field' '
+	test_config filter.absentsmudge.clean cat &&
+	test_config filter.absentsmudge.required true &&
+
+	echo "*.as filter=absentsmudge" >.gitattributes &&
+
+	echo test >test.as &&
+	git add test.as &&
+	rm -f test.as &&
+	test_must_fail git checkout -- test.as 2>stderr &&
+	test_i18ngrep "fatal: test.as: smudge filter absentsmudge failed" stderr
+'
+
 test_expect_success 'filtering large input to small output should use little memory' '
 	test_config filter.devnull.clean "cat >/dev/null" &&
 	test_config filter.devnull.required true &&

-- 
2.30.1
Junio C Hamano· Feb 25, 2021, 23:05 UTC · re: Matheus Tavares · lore

Re: [PATCH] convert: fail gracefully upon missing clean cmd on required filter

Matheus Tavares <matheus.bernardino@usp.br> writes:
Show 19 quoted lines
> The gitattributes documentation mentions that either the clean cmd or
> the smudge cmd can be left unspecified in a filter definition. However,
> when the filter is marked as 'required', the absence of any one of these
> two should be treated as an error. Git already fails under these
> circumstances, but not always in a pleasant way: omitting a clean cmd in
> a required filter triggers an assertion error which leaves the user with
> a quite verbose message:
>
> git: convert.c:1462: convert_to_git_filter_fd: Assertion "ca.drv->clean || ca.drv->process" failed.
>
> This assertion and the one above it are not really necessary, as the
> apply_filter() call bellow them already performs the same checks. And
> when these conditions are not met, the function returns 0, making the
> caller die() with a much nicer message. (Also note that die()-ing here
> is the right behavior as `would_convert_to_git_filter_fd() == true` is a
> precondition to use convert_to_git_filter_fd(), and the former is only
> true when the filter is required.) So remove both assertions and add two
> regression tests to make sure that git fails nicely when either the
> smudge or clean command is missing on a required filter.
Makes sense.
Will queue, thanks.
Martin Ågren· Feb 26, 2021, 06:27 UTC · re: Matheus Tavares · lore

Re: [PATCH] convert: fail gracefully upon missing clean cmd on required filter

On Thu, 25 Feb 2021 at 19:18, Matheus Tavares <matheus.bernardino@usp.br> wrote:
> This assertion and the one above it are not really necessary, as the
> apply_filter() call bellow them already performs the same checks. And
s/bellow/below/
> when these conditions are not met, the function returns 0, making the
> caller die() with a much nicer message. (Also note that die()-ing here
Makes sense. I noticed one thing:
> -       assert(ca.drv);
If ca.drv is NULL ....
> -       assert(ca.drv->clean || ca.drv->process);
> -
>         if (!apply_filter(path, NULL, 0, fd, dst, ca.drv, CAP_CLEAN, NULL, NULL))
... the return value will be 0, so this will trigger ...
>                 die(_("%s: clean filter '%s' failed"), path, ca.drv->name);
... and we'll dereference NULL to grab the name.

It seems like you could leave that first assertion and your new tests would still pass. Hitting an assertion is arguably better than wandering off into undefined behavior. (What will probably happen is ca.drv->name will effectively also be NULL, because it's at the top of the struct. Some implementations will format this as "(null)", others will crash.)

As you note, and my reading agrees, you can't really have ca.drv be NULL here. So this is like, if and when we grow a bug somewhere and actually do have NULL, maybe we would rather hit that "assert" than go dereferencing NULL.

Martin
Matheus Tavares· Feb 26, 2021, 16:23 UTC · re: Matheus Tavares · lore

[PATCH v2] convert: fail gracefully upon missing clean cmd on required filter

The gitattributes documentation mentions that either the clean cmd or the smudge cmd can be left unspecified in a filter definition. However, when the filter is marked as 'required', the absence of any one of these two should be treated as an error. Git already fails under these circumstances, but not always in a pleasant way: omitting a clean cmd in a required filter triggers an assertion error which leaves the user with a quite verbose message:

git: convert.c:1459: convert_to_git_filter_fd: Assertion "ca.drv->clean || ca.drv->process" failed.

This assertion is not really necessary, as the apply_filter() call below it already performs the same check. And when this condition is not met, the function returns 0, making the caller die() with a much nicer message. (Also note that die()-ing here is the right behavior as `would_convert_to_git_filter_fd() == true` is a precondition to use convert_to_git_filter_fd(), and the former is only true when the filter is required.) So remove the assertion and add two regression tests to make sure that git fails nicely when either the smudge or clean command is missing on a required filter.

Signed-off-by: Matheus Tavares <matheus.bernardino@usp.br>
---

Changes since v1: didn't remove the first assertion and adjusted the commit message. Thanks for catching that, Martin.

 convert.c             |  1 -
 t/t0021-conversion.sh | 24 ++++++++++++++++++++++++
 2 files changed, 24 insertions(+), 1 deletion(-)
Show changes to 2 files +24 −1

convert.c, t/t0021-conversion.sh

diff --git a/convert.c b/convert.c
index ee360c2f07..2c2c2adf61 100644
--- a/convert.c
+++ b/convert.c
@@ -1456,7 +1456,6 @@ void convert_to_git_filter_fd(const struct index_state *istate,
 	convert_attrs(istate, &ca, path);
 
 	assert(ca.drv);
-	assert(ca.drv->clean || ca.drv->process);
 
 	if (!apply_filter(path, NULL, 0, fd, dst, ca.drv, CAP_CLEAN, NULL, NULL))
 		die(_("%s: clean filter '%s' failed"), path, ca.drv->name);
diff --git a/t/t0021-conversion.sh b/t/t0021-conversion.sh
index e828ee964c..4f8415d419 100755
--- a/t/t0021-conversion.sh
+++ b/t/t0021-conversion.sh
@@ -257,6 +257,30 @@ test_expect_success 'required filter clean failure' '
 	test_must_fail git add test.fc
 '
 
+test_expect_success 'required filter with absent clean field' '
+	test_config filter.absentclean.smudge cat &&
+	test_config filter.absentclean.required true &&
+
+	echo "*.ac filter=absentclean" >.gitattributes &&
+
+	echo test >test.ac &&
+	test_must_fail git add test.ac 2>stderr &&
+	test_i18ngrep "fatal: test.ac: clean filter .absentclean. failed" stderr
+'
+
+test_expect_success 'required filter with absent smudge field' '
+	test_config filter.absentsmudge.clean cat &&
+	test_config filter.absentsmudge.required true &&
+
+	echo "*.as filter=absentsmudge" >.gitattributes &&
+
+	echo test >test.as &&
+	git add test.as &&
+	rm -f test.as &&
+	test_must_fail git checkout -- test.as 2>stderr &&
+	test_i18ngrep "fatal: test.as: smudge filter absentsmudge failed" stderr
+'
+
 test_expect_success 'filtering large input to small output should use little memory' '
 	test_config filter.devnull.clean "cat >/dev/null" &&
 	test_config filter.devnull.required true &&

-- 
2.30.1
Martin Ågren· Feb 28, 2021, 20:01 UTC · re: Matheus Tavares · lore

Re: [PATCH v2] convert: fail gracefully upon missing clean cmd on required filter

On Fri, 26 Feb 2021 at 17:24, Matheus Tavares <matheus.bernardino@usp.br> wrote:
> This assertion is not really necessary, as the apply_filter() call below
> it already performs the same check. And when this condition is not met,
> -       assert(ca.drv->clean || ca.drv->process);
FWIW, this LGTM.
> +       test_i18ngrep "fatal: test.ac: clean filter .absentclean. failed" stderr
> +       test_i18ngrep "fatal: test.as: smudge filter absentsmudge failed" stderr

(Funny how these two error messages differ in quoting. That's obviously not anything your patch needs to fix.)

Martin

← back to recent threads