Re: [PATCH 1/7] completion: make the 'basic' test more tester-friendly

From: Felipe Contreras <felipe.contreras@gmail.com>
Date: Nov 18, 2012, 09:39 UTC
Message-ID: <CAMP44s1zibHCFCm_=E+WXmTpX+xvpLWGbcpf32QuWsWAa+Y5uA@mail.gmail.com>
In-Reply-To: <1353150353-29874-2-git-send-email-szeder@ira.uka.de>

On Sat, Nov 17, 2012 at 12:05 PM, SZEDER Gábor <szeder@ira.uka.de> wrote:

Show 52 quoted lines

> The 'basic' test uses 'grep -q' to filter the resulting possible
> completion words while looking for the presence or absence of certain
> git commands, and relies on grep's exit status to indicate a failure.
>
> This works fine as long as there are no errors.  However, in case of a
> failure it doesn't give any indication whatsoever about the reason of
> the failure, i.e. which condition failed.
>
> To make testers' life easier provide some output about the failed
> condition: store the results of the filtering in a file and compare
> its contents to the expected results by the good old test_cmp helper.
> However, to actually get output from test_cmp in case of an error we
> must make sure that test_cmp is always executed.  Since in case of an
> error grep's exit code aborts the test immediately, before running the
> subsequent test_cmp, do the filtering using sed instead of grep.
>
> Signed-off-by: SZEDER Gábor <szeder@ira.uka.de>
> ---
>  t/t9902-completion.sh | 16 ++++++++++++----
>  1 file changed, 12 insertions(+), 4 deletions(-)
>
> diff --git a/t/t9902-completion.sh b/t/t9902-completion.sh
> index 8fa025f9..b56759f7 100755
> --- a/t/t9902-completion.sh
> +++ b/t/t9902-completion.sh
> @@ -158,14 +158,22 @@ test_expect_success '__gitcomp - suffix' '
>  test_expect_success 'basic' '
>         run_completion "git \"\"" &&
>         # built-in
> -       grep -q "^add \$" out &&
> +       echo "add " >expected &&
> +       sed -n "/^add \$/p" out >out2 &&
> +       test_cmp expected out2 &&
>         # script
> -       grep -q "^filter-branch \$" out &&
> +       echo "filter-branch " >expected &&
> +       sed -n "/^filter-branch \$/p" out >out2 &&
> +       test_cmp expected out2 &&
>         # plumbing
> -       ! grep -q "^ls-files \$" out &&
> +       >expected &&
> +       sed -n "/^ls-files \$/p" out >out2 &&
> +       test_cmp expected out2 &&
>
>         run_completion "git f" &&
> -       ! grep -q -v "^f" out
> +       >expected &&
> +       sed -n "/^[^f]/p" out >out2 &&
> +       test_cmp expected out2
>  '
>
>  test_expect_success 'double dash "git" itself' '

It's not very useful to see a single failure without context. Maybe this:

--- a/t/t9902-completion.sh
+++ b/t/t9902-completion.sh
@@ -191,18 +191,20 @@ test_expect_success '__gitcomp_nl - doesnt fail
because of invalid variable name

 test_expect_success 'basic' '
        run_completion "git " &&
+       (
        # built-in
-       echo "add " >expected &&
-       sed -n "/^add \$/p" out >out2 &&
-       test_cmp expected out2 &&
+       sed -n "/^add $/p" out &&
        # script
-       echo "filter-branch " >expected &&
-       sed -n "/^filter-branch \$/p" out >out2 &&
-       test_cmp expected out2 &&
+       sed -n "/^filter-branch $/p" out &&
        # plumbing
-       >expected &&
-       sed -n "/^ls-files \$/p" out >out2 &&
-       test_cmp expected out2 &&
+       sed -n "/^ls-files $/p" out
+       ) > actual &&
+
+       sed -e 's/Z$//' > expected <<-EOF &&
+       add Z
+       filter-branch Z
+       EOF
+       test_cmp expected actual &&

        run_completion "git f" &&
        >expected &&

-- 
Felipe Contreras