Re: [PATCH v2 1/2] test-lib: add a function to compare an expection with stdout from a command

From: Eric Sunshine <sunshine@sunshineco.com>
Date: Apr 17, 2016, 03:07 UTC
Message-ID: <CAPig+cSOuFygsScGn_Nu0_d8mvRik1hQJuanrb-Nvw3ozyt7JQ@mail.gmail.com>
In-Reply-To: <1460823230-45692-2-git-send-email-rappazzo@gmail.com>

On Sat, Apr 16, 2016 at 12:13 PM, Michael Rappazzo <rappazzo@gmail.com> wrote:

> test-lib: add a function to compare an expection with stdout from a command

Rather long subject. Perhaps:

    test-lib: add convenience function to check command output

> test_stdout accepts an expection and a command to execute.  It will execute
> the command and then compare the stdout from that command to an expectation.
> If the expectation is not met, a mock diff output is written to stderr.

I wonder if this deserves more flexibility by accepting a comparison operator, such as = and !=, similar to test_line_count()? Although, I suppose such functionality could be added later if deemed useful.

> Based-on-a-patch-by: Jeff King <peff@peff.net>

Since Peff wrote the actual code[1], it might be worthwhile to give him authorship by prepending the commit message with a "From: Jeff King <peff@peff.net>" header.

Show 16 quoted lines

> Signed-off-by: Michael Rappazzo <rappazzo@gmail.com>
> ---
> diff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh
> @@ -941,3 +941,37 @@ mingw_read_file_strip_cr_ () {
> +#      test_stdout is a helper function to compare expected output with
> +#      the standard output of a command execution
> +#
> +#      Args:
> +#              1: The expected output
> +#              2: The command to run
> +#
> +#      You can use it like:
> +#
> +#      test_expect_success 'foo works' '
> +#              test_cmp "This is expected" cmd_to_run arg1 arg2 ... argN
> +#      '

test_cmp?

> +#      The output when there is a mismatch mimics diff output, but this
> +#      can break down for a multi-line result

Hmm, considering that $(...) collapses each whitespace run (including newlines) down to a single space, I don't see how you could get a multi-line result.

By the way, either the documentation should mention this limitation ("not possible to check multi-line output") or the implementation should be upgraded to support it.

Show 14 quoted lines

> +test_stdout () {
> +       expect=$1
> +       shift
> +       if ! actual=$("$@")
> +       then
> +               echo "test_stdout: command failed: '$*'" >&2
> +               return 1
> +       fi
> +       if test "$expect" != "$actual"
> +       then
> +               echo "test_stdout: unexpected output for '$*'" >&2
> +               echo "@@ -N +N @@" >&2
> +               echo "-$expect" >&2
> +               echo "+$actual" >&2

This faux diff output is quite a bit more noisy than the simple error message emitted by Peff's original[1] and it doesn't provide any additional useful information, so it doesn't feel like an improvement. Adding quotes around $expect and $actual in Peff's error message would probably be an improvement, though.

> +               return 1
> +       fi
> +       return 0
> +}

[1]: http://article.gmane.org/gmane.comp.version-control.git/291475