threads / patch / 61037

patchtest-lib-functions: simplify `test_file_not_empty` failure message

Subject: [PATCH] test-lib-functions: simplify `test_file_not_empty` failure message

## tl;dr

8 messages between Mar 1, 2024 and Mar 3, 2024. Diffs are folded; open one to read it.

replies: 7people: 5as markdown or json

Eric Sunshine· Mar 1, 2024, 20:49 UTC · lore
From: Eric Sunshine <sunshine@sunshineco.com>

The function `test_file_not_empty` asserts that a file exists and is not empty. When the assertion fails, it complains:

    'foo' is not a non-empty file.

which is difficult to interpret due to the double-negative. To make it easier to understand the problem, simplify the message by dropping the double-negative and stating the problem more directly:

    'foo' is empty but should not be

(The full-stop is also dropped from the message to reflect the style of messages issued by other `test_path_*` functions.)

Note: Technically, the revised message is slightly less accurate since
the function asserts both that the file exists and that it is non-empty,
but the new message talks only about the emptiness of the file, not
whether it exists. A more accurate message might be "'foo' is empty but
should not be (or doesn't exist)", but that's unnecessarily long-winded
and adds little information that the test author couldn't discover by
noticing the file's absence.
Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
---
This is a tangential follow-up to the discussion at [1].
[1]: https://lore.kernel.org/git/CAPig+cQ+JNBwydUq0CsTZGs8mHs3L3fJDuSosd+-WdKwWWw=gg@mail.gmail.com/
 t/test-lib-functions.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Show changes to t/test-lib-functions.sh +1 −1
diff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh
index b5eaf7fdc1..9e97b324c5 100644
--- a/t/test-lib-functions.sh
+++ b/t/test-lib-functions.sh
@@ -991,7 +991,7 @@ test_file_not_empty () {
 	test "$#" = 2 && BUG "2 param"
 	if ! test -s "$1"
 	then
-		echo "'$1' is not a non-empty file."
+		echo "'$1' is empty but should not be"
 		false
 	fi
 }

-- 
2.44.0
Junio C Hamano· Mar 1, 2024, 22:11 UTC · re: Eric Sunshine · lore

Re: [PATCH] test-lib-functions: simplify `test_file_not_empty` failure message

Eric Sunshine <ericsunshine@charter.net> writes:
Show 9 quoted lines
> Note: Technically, the revised message is slightly less accurate since
> the function asserts both that the file exists and that it is non-empty,
> but the new message talks only about the emptiness of the file, not
> whether it exists.
>
> A more accurate message might be "'foo' is empty but
> should not be (or doesn't exist)", but that's unnecessarily long-winded
> and adds little information that the test author couldn't discover by
> noticing the file's absence.

Besides, that is way too confusing. "<foo> is empty or it does not exist" I may understand, but with your construct, I wouldn't be able to tell how I am supposed to interpret the "(or doesn't exist)" part.

Show 10 quoted lines
> diff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh
> index b5eaf7fdc1..9e97b324c5 100644
> --- a/t/test-lib-functions.sh
> +++ b/t/test-lib-functions.sh
> @@ -991,7 +991,7 @@ test_file_not_empty () {
>  	test "$#" = 2 && BUG "2 param"
>  	if ! test -s "$1"
>  	then
> -		echo "'$1' is not a non-empty file."
> +		echo "'$1' is empty but should not be"
The "adds little information" version may be
		echo "'$1' is either missing or empty, but should not be"
And avoiding "X is Y, but should  be ~Y" construct, perhaps
		echo "'$1' should be a file with non-empty contents"
would work better?  I dunno.
Eric Sunshine· Mar 1, 2024, 22:59 UTC · re: Junio C Hamano · lore

Re: [PATCH] test-lib-functions: simplify `test_file_not_empty` failure message

On Fri, Mar 1, 2024 at 5:11 PM Junio C Hamano <gitster@pobox.com> wrote:
Show 15 quoted lines
> Eric Sunshine <ericsunshine@charter.net> writes:
> > A more accurate message might be "'foo' is empty but
> > should not be (or doesn't exist)", but that's unnecessarily long-winded
> > and adds little information that the test author couldn't discover by
> > noticing the file's absence.
>
> The "adds little information" version may be
>
>                 echo "'$1' is either missing or empty, but should not be"
>
> And avoiding "X is Y, but should  be ~Y" construct, perhaps
>
>                 echo "'$1' should be a file with non-empty contents"
>
> would work better?  I dunno.

I find "'$1' is either missing or empty, but should not be" suggestion clear and easily understood. I'll reroll with that.

Dirk Gouders· Mar 2, 2024, 07:07 UTC · re: Eric Sunshine · lore

Re: [PATCH] test-lib-functions: simplify `test_file_not_empty` failure message

Eric Sunshine <sunshine@sunshineco.com> writes:
Show 19 quoted lines
> On Fri, Mar 1, 2024 at 5:11 PM Junio C Hamano <gitster@pobox.com> wrote:
>> Eric Sunshine <ericsunshine@charter.net> writes:
>> > A more accurate message might be "'foo' is empty but
>> > should not be (or doesn't exist)", but that's unnecessarily long-winded
>> > and adds little information that the test author couldn't discover by
>> > noticing the file's absence.
>>
>> The "adds little information" version may be
>>
>>                 echo "'$1' is either missing or empty, but should not be"
>>
>> And avoiding "X is Y, but should  be ~Y" construct, perhaps
>>
>>                 echo "'$1' should be a file with non-empty contents"
>>
>> would work better?  I dunno.
>
> I find "'$1' is either missing or empty, but should not be" suggestion
> clear and easily understood. I'll reroll with that.
This is a view from a position with more distance:

I find that not so easily understood -- the "but should not be" part is rather unexpected and I feel, it doesn't provide necessary information, e.g.:

test_path_is_executable () {
...
		echo "$1 is not executable"
...

also doesn't state what is wanted and I doubt that message doesn't clearly describe the problem.

While I looked at it: there is another `test -s` in test_grep () that perhaps could be fixed the same way:

	if test -s "$last_arg"
	then
		cat >&4 "$last_arg"
	else
		echo >&4 "<File '$last_arg' is empty>"
	fi
Dirk
Junio C Hamano· Mar 2, 2024, 17:44 UTC · re: Dirk Gouders · lore

Re: [PATCH] test-lib-functions: simplify `test_file_not_empty` failure message

Dirk Gouders <dirk@gouders.net> writes:
Show 29 quoted lines
> Eric Sunshine <sunshine@sunshineco.com> writes:
>
>> On Fri, Mar 1, 2024 at 5:11 PM Junio C Hamano <gitster@pobox.com> wrote:
>>> Eric Sunshine <ericsunshine@charter.net> writes:
>>> > A more accurate message might be "'foo' is empty but
>>> > should not be (or doesn't exist)", but that's unnecessarily long-winded
>>> > and adds little information that the test author couldn't discover by
>>> > noticing the file's absence.
>>>
>>> The "adds little information" version may be
>>>
>>>                 echo "'$1' is either missing or empty, but should not be"
>>> ...
>> I find "'$1' is either missing or empty, but should not be" suggestion
>> clear and easily understood. I'll reroll with that.
>
> This is a view from a position with more distance:
>
> I find that not so easily understood -- the "but should not
> be" part is rather unexpected and I feel, it doesn't provide necessary
> information, e.g.:
>
> test_path_is_executable () {
> ...
> 		echo "$1 is not executable"
> ...
>
> also doesn't state what is wanted and I doubt that message doesn't
> clearly describe the problem.

I cannot tell if you really meant the double negative involving "doubt", but assuming you did, you are saying that

    With "X is not Y", it is clear enough that we expect X to be Y
    (if it were not clear to somebody who read "X is not Y" that we
    want X to be Y, then "X is not Y, but it should be" may needed,
    but "X is not Y" is clear enough).

So you think "$1 is either missing or empty" is better without "but should not be" added to the end? Am I reading you correctly?

I think this takes us back pretty much to square one ;-) but that is also fine.

But the above argument depends on an untold assumption. The message "X is not Y" must be clearly understood as a complaint, not a mere statement of a fact. I am not sure if that is the case.

Instead of "X is not Y, but it should be", the way to clarify these messages may be to say "error: X is not Y", perhaps?

Show 9 quoted lines
> While I looked at it: there is another `test -s` in test_grep () that
> perhaps could be fixed the same way:
>
> 	if test -s "$last_arg"
> 	then
> 		cat >&4 "$last_arg"
> 	else
> 		echo >&4 "<File '$last_arg' is empty>"
> 	fi

If you are worried about "test -s" failing because "$last_arg" does not exist, then you are worried too much. We upfront guard the test_grep() helper with "test -f" of the same file and diagnoses the lack of the file as a bug in the test. And we do not assume gremlins removing random files while we are running tests.

Dirk Gouders· Mar 3, 2024, 06:42 UTC · re: Junio C Hamano · lore

Re: [PATCH] test-lib-functions: simplify `test_file_not_empty` failure message

Junio C Hamano <gitster@pobox.com> writes:
Show 34 quoted lines
> Dirk Gouders <dirk@gouders.net> writes:
>
>> Eric Sunshine <sunshine@sunshineco.com> writes:
>>
>>> On Fri, Mar 1, 2024 at 5:11 PM Junio C Hamano <gitster@pobox.com> wrote:
>>>> Eric Sunshine <ericsunshine@charter.net> writes:
>>>> > A more accurate message might be "'foo' is empty but
>>>> > should not be (or doesn't exist)", but that's unnecessarily long-winded
>>>> > and adds little information that the test author couldn't discover by
>>>> > noticing the file's absence.
>>>>
>>>> The "adds little information" version may be
>>>>
>>>>                 echo "'$1' is either missing or empty, but should not be"
>>>> ...
>>> I find "'$1' is either missing or empty, but should not be" suggestion
>>> clear and easily understood. I'll reroll with that.
>>
>> This is a view from a position with more distance:
>>
>> I find that not so easily understood -- the "but should not
>> be" part is rather unexpected and I feel, it doesn't provide necessary
>> information, e.g.:
>>
>> test_path_is_executable () {
>> ...
>> 		echo "$1 is not executable"
>> ...
>>
>> also doesn't state what is wanted and I doubt that message doesn't
>> clearly describe the problem.
>
> I cannot tell if you really meant the double negative involving
> "doubt", but assuming you did, you are saying that

I'm sorry about that double negative which was probably wrong wording of a non-native speaker.

Show 17 quoted lines
>     With "X is not Y", it is clear enough that we expect X to be Y
>     (if it were not clear to somebody who read "X is not Y" that we
>     want X to be Y, then "X is not Y, but it should be" may needed,
>     but "X is not Y" is clear enough).
>
> So you think "$1 is either missing or empty" is better without "but
> should not be" added to the end?  Am I reading you correctly?
>
> I think this takes us back pretty much to square one ;-) but that is
> also fine.
>
> But the above argument depends on an untold assumption.  The message
> "X is not Y" must be clearly understood as a complaint, not a mere
> statement of a fact.  I am not sure if that is the case.
>
> Instead of "X is not Y, but it should be", the way to clarify these
> messages may be to say "error: X is not Y", perhaps?

That is exactly what came to my mind when I was later re-thinking what I had written.

Show 15 quoted lines
>> While I looked at it: there is another `test -s` in test_grep () that
>> perhaps could be fixed the same way:
>>
>> 	if test -s "$last_arg"
>> 	then
>> 		cat >&4 "$last_arg"
>> 	else
>> 		echo >&4 "<File '$last_arg' is empty>"
>> 	fi
>
> If you are worried about "test -s" failing because "$last_arg" does
> not exist, then you are worried too much.  We upfront guard the
> test_grep() helper with "test -f" of the same file and diagnoses the
> lack of the file as a bug in the test.  And we do not assume gremlins
> removing random files while we are running tests.
Yes, thank you for clarification and sorry for the noise.
Dirk
Rubén Justo· Mar 2, 2024, 16:38 UTC · re: Eric Sunshine · lore

Re: [PATCH] test-lib-functions: simplify `test_file_not_empty` failure message

On Fri, Mar 01, 2024 at 03:49:22PM -0500, Eric Sunshine wrote:
Show 23 quoted lines
> From: Eric Sunshine <sunshine@sunshineco.com>
> 
> The function `test_file_not_empty` asserts that a file exists and is not
> empty. When the assertion fails, it complains:
> 
>     'foo' is not a non-empty file.
> 
> which is difficult to interpret due to the double-negative. To make it
> easier to understand the problem, simplify the message by dropping the
> double-negative and stating the problem more directly:
> 
>     'foo' is empty but should not be
> 
> (The full-stop is also dropped from the message to reflect the style of
> messages issued by other `test_path_*` functions.)
> 
> Note: Technically, the revised message is slightly less accurate since
> the function asserts both that the file exists and that it is non-empty,
> but the new message talks only about the emptiness of the file, not
> whether it exists. A more accurate message might be "'foo' is empty but
> should not be (or doesn't exist)", but that's unnecessarily long-winded
> and adds little information that the test author couldn't discover by
> noticing the file's absence.

To improve the accuracy of the message, I wonder if it is worth doing what we do in test_must_be_empty:

	test_must_be_empty () {
		test "$#" -ne 1 && BUG "1 param"
		test_path_is_file "$1" &&
		if test -s "$1"
		then
			echo "'$1' is not empty, it contains:"
			cat "$1"
			return 1
		fi
	}
Perhaps:
Show changes to t/test-lib-functions.sh +2 −1
diff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh
index b5eaf7fdc1..5b5ee0dc1d 100644
--- a/t/test-lib-functions.sh
+++ b/t/test-lib-functions.sh
@@ -989,9 +989,10 @@ test_dir_is_empty () {
 # Check if the file exists and has a size greater than zero
 test_file_not_empty () {
        test "$#" = 2 && BUG "2 param"
+       test_path_is_file "$1" &&
        if ! test -s "$1"
        then
-               echo "'$1' is not a non-empty file."
+		echo "'$1' is empty but should not be"
                false
        fi
 }

> 
> Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
> ---
> 
> This is a tangential follow-up to the discussion at [1].
> 
> [1]: https://lore.kernel.org/git/CAPig+cQ+JNBwydUq0CsTZGs8mHs3L3fJDuSosd+-WdKwWWw=gg@mail.gmail.com/
> 
>  t/test-lib-functions.sh | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh
> index b5eaf7fdc1..9e97b324c5 100644
> --- a/t/test-lib-functions.sh
> +++ b/t/test-lib-functions.sh
> @@ -991,7 +991,7 @@ test_file_not_empty () {
>  	test "$#" = 2 && BUG "2 param"
>  	if ! test -s "$1"
>  	then
> -		echo "'$1' is not a non-empty file."
> +		echo "'$1' is empty but should not be"
>  		false
>  	fi
>  }
> -- 
> 2.44.0
>
Junio C Hamano· Mar 2, 2024, 18:08 UTC · re: Rubén Justo · lore

Re: [PATCH] test-lib-functions: simplify `test_file_not_empty` failure message

Rubén Justo <rjusto@gmail.com> writes:
Show 18 quoted lines
> To improve the accuracy of the message, I wonder if it is worth doing
> ...
> diff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh
> index b5eaf7fdc1..5b5ee0dc1d 100644
> --- a/t/test-lib-functions.sh
> +++ b/t/test-lib-functions.sh
> @@ -989,9 +989,10 @@ test_dir_is_empty () {
>  # Check if the file exists and has a size greater than zero
>  test_file_not_empty () {
>         test "$#" = 2 && BUG "2 param"
> +       test_path_is_file "$1" &&
>         if ! test -s "$1"
>         then
> -               echo "'$1' is not a non-empty file."
> +		echo "'$1' is empty but should not be"
>                 false
>         fi
>  }

Simple and effective to remove the need to have to worry about the "missing" case. The "but should not be" part may still be subject to discussion, but I do not have a strong opinion there.

← back to recent threads