git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: t5545: reduced test coverage

From
Ramsay Jones <ramsay@ramsayjones.plus.com>
Date
May 17, 2017, 20:48 UTC
Message-ID
<17124bdc-c6cb-5f1a-5021-7c92ad7e669c@ramsayjones.plus.com>
In-Reply-To
<xmqqinl0xga6.fsf@gitster.mtv.corp.google.com>
On 17/05/17 05:23, Junio C Hamano wrote:
Show 25 quoted lines
> Junio C Hamano <gitster@pobox.com> writes:
> 
>> Junio C Hamano <gitster@pobox.com> writes:
>>
>>> It appears to me that only a few tests in the entire script wants to
>>> work with HTTP server, so perhaps moving them to the end, together
>>> with the inclusion of lib-httpd and start_httpd that 438fc684 ("push
>>> options: pass push options to the transport helper", 2017-02-08)
>>> introduced, may be sufficient?
>>
>> Probably not, as lib-httpd, when it gives up, does the
>> "skip_all=message; test_done" thing, which makes test_done
>> to trigger this:
>>
>> 	# Maybe print SKIP message
>> 	if test -n "$skip_all" && test $test_count -gt 0
>> 	then
>> 		error "Can't use skip_all after running some tests"
>> 	fi
>>
>> So a bit more work is needed, than just moving things around, I am
>> afraid...
> 
> And now I am visiting bf4b7219 ("test-lib.sh: Add check for invalid
> use of 'skip_all' facility", 2012-09-01), which is yours ;-)
Indeed. :-D
Show 9 quoted lines
> I am wondering what the fallouts would be if we did the following to
> test-lib.  We used to say "1..0 # SKIP <why>" when we skip
> everything, which is still kept, so that prove can notice why things
> are skipped.
> 
> When we abort early but after executing any test, we used to error
> out, but instead we pretend that the test finished right there, as
> it seems that we are not allowed to say "1..4 # SKIP <why>" after
> running 4 tests.

Yes, I had this exact 'Huh?' moment back in 2012. I remember spending quite a long time searching 'TAP specification' documents, in order to get an answer to this. It was so long ago (and I can't find the links to those documents now), but I have a vague recollection that prove's behaviour is correct in allowing '1..0 <skip message>', while throwing a fit at '1..n <any message>' (for n > 0).

I would probably have simply split the test file into two, but ...
Show 41 quoted lines
>  t/test-lib.sh | 21 +++++++++++++--------
>  1 file changed, 13 insertions(+), 8 deletions(-)
> 
> diff --git a/t/test-lib.sh b/t/test-lib.sh
> index 13b5696822..46f29cf112 100644
> --- a/t/test-lib.sh
> +++ b/t/test-lib.sh
> @@ -745,20 +745,25 @@ test_done () {
>  	fi
>  	case "$test_failure" in
>  	0)
> -		# Maybe print SKIP message
> -		if test -n "$skip_all" && test $test_count -gt 0
> -		then
> -			error "Can't use skip_all after running some tests"
> -		fi
> -		test -z "$skip_all" || skip_all=" # SKIP $skip_all"
> -
>  		if test $test_external_has_tap -eq 0
>  		then
>  			if test $test_remaining -gt 0
>  			then
>  				say_color pass "# passed all $msg"
>  			fi
> -			say "1..$test_count$skip_all"
> +
> +			# Maybe print SKIP message
> +			test -z "$skip_all" || skip_all="# SKIP $skip_all"
> +			case "$test_count" in
> +			0)			    
> +				say "1..$test_count${skip_all:+ $skip_all}"
> +				;;
> +			*)
> +				test -z "$skip_all" ||
> +				say_color warn "$skip_all"
> +				say "1..$test_count"
> +				;;
> +			esac
>  		fi
>  
>  		test -d "$remove_trash" &&
... this looks good to me. (tested on Linux and cygwin).
Thanks!

ATB, Ramsay Jones

Previous: Junio C HamanoNext: Junio C Hamano
Message 6 of 7 in “t5545: reduced test coverage”
  1. Ramsay JonesMay 16, 2017
  2. Junio C HamanoMay 17, 2017
  3. t5545: enhance test coverage when no http server is installedStefan Beller, May 17, 2017
  4. Junio C HamanoMay 17, 2017
  5. Junio C HamanoMay 17, 2017
  6. Ramsay JonesMay 17, 2017
  7. Junio C HamanoMay 18, 2017

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.