{"thread":{"id":"55212","subject":"[PATCH] convert: fail gracefully upon missing clean cmd on required filter","startedAt":"2021-02-25T18:17:54Z","lastAt":"2021-02-28T20:02:12Z","messageCount":5,"participants":["Matheus Tavares","Junio C Hamano","Martin Ågren"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"417826","messageId":"b33d0dc82f5ff1fac6772e533bbf21eecfae44ed.1614277014.git.matheus.bernardino@usp.br","threadId":"55212","inReplyTo":null,"subject":"[PATCH] convert: fail gracefully upon missing clean cmd on required filter","fromName":"Matheus Tavares","fromEmail":"matheus.bernardino@usp.br","sentAt":"2021-02-25T18:16:56Z","receivedAt":"2021-02-25T18:17:54Z","isPatch":true,"sender":{"key":"matheus.tavb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/12701583?v=4"},"body":"The gitattributes documentation mentions that either the clean cmd or\nthe smudge cmd can be left unspecified in a filter definition. However,\nwhen the filter is marked as 'required', the absence of any one of these\ntwo should be treated as an error. Git already fails under these\ncircumstances, but not always in a pleasant way: omitting a clean cmd in\na required filter triggers an assertion error which leaves the user with\na quite verbose message:\n\ngit: convert.c:1462: convert_to_git_filter_fd: Assertion \"ca.drv->clean || ca.drv->process\" failed.\n\nThis assertion and the one above it are not really necessary, as the\napply_filter() call bellow them already performs the same checks. And\nwhen these conditions are not met, the function returns 0, making the\ncaller die() with a much nicer message. (Also note that die()-ing here\nis the right behavior as `would_convert_to_git_filter_fd() == true` is a\nprecondition to use convert_to_git_filter_fd(), and the former is only\ntrue when the filter is required.) So remove both assertions and add two\nregression tests to make sure that git fails nicely when either the\nsmudge or clean command is missing on a required filter.\n\nSigned-off-by: Matheus Tavares <matheus.bernardino@usp.br>\n---\n convert.c             |  3 ---\n t/t0021-conversion.sh | 24 ++++++++++++++++++++++++\n 2 files changed, 24 insertions(+), 3 deletions(-)\n\ndiff --git a/convert.c b/convert.c\nindex ee360c2f07..48fbdac676 100644\n--- a/convert.c\n+++ b/convert.c\n@@ -1455,9 +1455,6 @@ void convert_to_git_filter_fd(const struct index_state *istate,\n \tstruct conv_attrs ca;\n \tconvert_attrs(istate, &ca, path);\n \n-\tassert(ca.drv);\n-\tassert(ca.drv->clean || ca.drv->process);\n-\n \tif (!apply_filter(path, NULL, 0, fd, dst, ca.drv, CAP_CLEAN, NULL, NULL))\n \t\tdie(_(\"%s: clean filter '%s' failed\"), path, ca.drv->name);\n \ndiff --git a/t/t0021-conversion.sh b/t/t0021-conversion.sh\nindex e828ee964c..4f8415d419 100755\n--- a/t/t0021-conversion.sh\n+++ b/t/t0021-conversion.sh\n@@ -257,6 +257,30 @@ test_expect_success 'required filter clean failure' '\n \ttest_must_fail git add test.fc\n '\n \n+test_expect_success 'required filter with absent clean field' '\n+\ttest_config filter.absentclean.smudge cat &&\n+\ttest_config filter.absentclean.required true &&\n+\n+\techo \"*.ac filter=absentclean\" >.gitattributes &&\n+\n+\techo test >test.ac &&\n+\ttest_must_fail git add test.ac 2>stderr &&\n+\ttest_i18ngrep \"fatal: test.ac: clean filter .absentclean. failed\" stderr\n+'\n+\n+test_expect_success 'required filter with absent smudge field' '\n+\ttest_config filter.absentsmudge.clean cat &&\n+\ttest_config filter.absentsmudge.required true &&\n+\n+\techo \"*.as filter=absentsmudge\" >.gitattributes &&\n+\n+\techo test >test.as &&\n+\tgit add test.as &&\n+\trm -f test.as &&\n+\ttest_must_fail git checkout -- test.as 2>stderr &&\n+\ttest_i18ngrep \"fatal: test.as: smudge filter absentsmudge failed\" stderr\n+'\n+\n test_expect_success 'filtering large input to small output should use little memory' '\n \ttest_config filter.devnull.clean \"cat >/dev/null\" &&\n \ttest_config filter.devnull.required true &&\n-- \n2.30.1\n\n"},{"id":"417853","messageId":"xmqqsg5jsq7l.fsf@gitster.g","threadId":"55212","inReplyTo":"b33d0dc82f5ff1fac6772e533bbf21eecfae44ed.1614277014.git.matheus.bernardino@usp.br","subject":"Re: [PATCH] convert: fail gracefully upon missing clean cmd on required filter","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-02-25T23:05:50Z","receivedAt":"2021-02-25T23:06:53Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Matheus Tavares <matheus.bernardino@usp.br> writes:\n\n> The gitattributes documentation mentions that either the clean cmd or\n> the smudge cmd can be left unspecified in a filter definition. However,\n> when the filter is marked as 'required', the absence of any one of these\n> two should be treated as an error. Git already fails under these\n> circumstances, but not always in a pleasant way: omitting a clean cmd in\n> a required filter triggers an assertion error which leaves the user with\n> a quite verbose message:\n>\n> git: convert.c:1462: convert_to_git_filter_fd: Assertion \"ca.drv->clean || ca.drv->process\" failed.\n>\n> This assertion and the one above it are not really necessary, as the\n> apply_filter() call bellow them already performs the same checks. And\n> when these conditions are not met, the function returns 0, making the\n> caller die() with a much nicer message. (Also note that die()-ing here\n> is the right behavior as `would_convert_to_git_filter_fd() == true` is a\n> precondition to use convert_to_git_filter_fd(), and the former is only\n> true when the filter is required.) So remove both assertions and add two\n> regression tests to make sure that git fails nicely when either the\n> smudge or clean command is missing on a required filter.\n\nMakes sense.\n\nWill queue, thanks.\n"},{"id":"417873","messageId":"CAN0heSptkHY3A0MHEDwfaMQ0cqhhGwg7hTy70NOsMcp-otROSg@mail.gmail.com","threadId":"55212","inReplyTo":"b33d0dc82f5ff1fac6772e533bbf21eecfae44ed.1614277014.git.matheus.bernardino@usp.br","subject":"Re: [PATCH] convert: fail gracefully upon missing clean cmd on required filter","fromName":"Martin Ågren","fromEmail":"martin.agren@gmail.com","sentAt":"2021-02-26T06:27:19Z","receivedAt":"2021-02-26T06:28:18Z","isPatch":true,"sender":{"key":"martin.agren@gmail.com","avatar":null},"body":"On Thu, 25 Feb 2021 at 19:18, Matheus Tavares <matheus.bernardino@usp.br> wrote:\n> This assertion and the one above it are not really necessary, as the\n> apply_filter() call bellow them already performs the same checks. And\n\ns/bellow/below/\n\n> when these conditions are not met, the function returns 0, making the\n> caller die() with a much nicer message. (Also note that die()-ing here\n\nMakes sense. I noticed one thing:\n\n> -       assert(ca.drv);\n\nIf ca.drv is NULL ....\n\n> -       assert(ca.drv->clean || ca.drv->process);\n> -\n>         if (!apply_filter(path, NULL, 0, fd, dst, ca.drv, CAP_CLEAN, NULL, NULL))\n\n... the return value will be 0, so this will trigger ...\n\n>                 die(_(\"%s: clean filter '%s' failed\"), path, ca.drv->name);\n\n... and we'll dereference NULL to grab the name.\n\nIt seems like you could leave that first assertion and your new tests\nwould still pass. Hitting an assertion is arguably better than wandering\noff into undefined behavior. (What will probably happen is ca.drv->name\nwill effectively also be NULL, because it's at the top of the struct.\nSome implementations will format this as \"(null)\", others will crash.)\n\nAs you note, and my reading agrees, you can't really have ca.drv be NULL\nhere. So this is like, if and when we grow a bug somewhere and actually\ndo have NULL, maybe we would rather hit that \"assert\" than go\ndereferencing NULL.\n\nMartin\n"},{"id":"417900","messageId":"ead5b3145c0cbf726c2fe06f5c95536373c85265.1614354223.git.matheus.bernardino@usp.br","threadId":"55212","inReplyTo":"b33d0dc82f5ff1fac6772e533bbf21eecfae44ed.1614277014.git.matheus.bernardino@usp.br","subject":"[PATCH v2] convert: fail gracefully upon missing clean cmd on required filter","fromName":"Matheus Tavares","fromEmail":"matheus.bernardino@usp.br","sentAt":"2021-02-26T16:23:54Z","receivedAt":"2021-02-26T16:25:04Z","isPatch":true,"sender":{"key":"matheus.tavb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/12701583?v=4"},"body":"The gitattributes documentation mentions that either the clean cmd or\nthe smudge cmd can be left unspecified in a filter definition. However,\nwhen the filter is marked as 'required', the absence of any one of these\ntwo should be treated as an error. Git already fails under these\ncircumstances, but not always in a pleasant way: omitting a clean cmd in\na required filter triggers an assertion error which leaves the user with\na quite verbose message:\n\ngit: convert.c:1459: convert_to_git_filter_fd: Assertion \"ca.drv->clean || ca.drv->process\" failed.\n\nThis assertion is not really necessary, as the apply_filter() call below\nit already performs the same check. And when this condition is not met,\nthe function returns 0, making the caller die() with a much nicer\nmessage. (Also note that die()-ing here is the right behavior as\n`would_convert_to_git_filter_fd() == true` is a precondition to use\nconvert_to_git_filter_fd(), and the former is only true when the filter\nis required.) So remove the assertion and add two regression tests to\nmake sure that git fails nicely when either the smudge or clean command\nis missing on a required filter.\n\nSigned-off-by: Matheus Tavares <matheus.bernardino@usp.br>\n---\n\nChanges since v1: didn't remove the first assertion and adjusted the\ncommit message. Thanks for catching that, Martin.\n\n convert.c             |  1 -\n t/t0021-conversion.sh | 24 ++++++++++++++++++++++++\n 2 files changed, 24 insertions(+), 1 deletion(-)\n\ndiff --git a/convert.c b/convert.c\nindex ee360c2f07..2c2c2adf61 100644\n--- a/convert.c\n+++ b/convert.c\n@@ -1456,7 +1456,6 @@ void convert_to_git_filter_fd(const struct index_state *istate,\n \tconvert_attrs(istate, &ca, path);\n \n \tassert(ca.drv);\n-\tassert(ca.drv->clean || ca.drv->process);\n \n \tif (!apply_filter(path, NULL, 0, fd, dst, ca.drv, CAP_CLEAN, NULL, NULL))\n \t\tdie(_(\"%s: clean filter '%s' failed\"), path, ca.drv->name);\ndiff --git a/t/t0021-conversion.sh b/t/t0021-conversion.sh\nindex e828ee964c..4f8415d419 100755\n--- a/t/t0021-conversion.sh\n+++ b/t/t0021-conversion.sh\n@@ -257,6 +257,30 @@ test_expect_success 'required filter clean failure' '\n \ttest_must_fail git add test.fc\n '\n \n+test_expect_success 'required filter with absent clean field' '\n+\ttest_config filter.absentclean.smudge cat &&\n+\ttest_config filter.absentclean.required true &&\n+\n+\techo \"*.ac filter=absentclean\" >.gitattributes &&\n+\n+\techo test >test.ac &&\n+\ttest_must_fail git add test.ac 2>stderr &&\n+\ttest_i18ngrep \"fatal: test.ac: clean filter .absentclean. failed\" stderr\n+'\n+\n+test_expect_success 'required filter with absent smudge field' '\n+\ttest_config filter.absentsmudge.clean cat &&\n+\ttest_config filter.absentsmudge.required true &&\n+\n+\techo \"*.as filter=absentsmudge\" >.gitattributes &&\n+\n+\techo test >test.as &&\n+\tgit add test.as &&\n+\trm -f test.as &&\n+\ttest_must_fail git checkout -- test.as 2>stderr &&\n+\ttest_i18ngrep \"fatal: test.as: smudge filter absentsmudge failed\" stderr\n+'\n+\n test_expect_success 'filtering large input to small output should use little memory' '\n \ttest_config filter.devnull.clean \"cat >/dev/null\" &&\n \ttest_config filter.devnull.required true &&\n-- \n2.30.1\n\n"},{"id":"417996","messageId":"CAN0heSpZ4938f+Wd_LOnykCpLbWernXzLJU02LA79axcGCA_-Q@mail.gmail.com","threadId":"55212","inReplyTo":"ead5b3145c0cbf726c2fe06f5c95536373c85265.1614354223.git.matheus.bernardino@usp.br","subject":"Re: [PATCH v2] convert: fail gracefully upon missing clean cmd on required filter","fromName":"Martin Ågren","fromEmail":"martin.agren@gmail.com","sentAt":"2021-02-28T20:01:03Z","receivedAt":"2021-02-28T20:02:12Z","isPatch":true,"sender":{"key":"martin.agren@gmail.com","avatar":null},"body":"On Fri, 26 Feb 2021 at 17:24, Matheus Tavares <matheus.bernardino@usp.br> wrote:\n> This assertion is not really necessary, as the apply_filter() call below\n> it already performs the same check. And when this condition is not met,\n\n> -       assert(ca.drv->clean || ca.drv->process);\n\nFWIW, this LGTM.\n\n> +       test_i18ngrep \"fatal: test.ac: clean filter .absentclean. failed\" stderr\n\n> +       test_i18ngrep \"fatal: test.as: smudge filter absentsmudge failed\" stderr\n\n(Funny how these two error messages differ in quoting. That's obviously\nnot anything your patch needs to fix.)\n\nMartin\n"}]}