Re: [PATCH v2] test_cmp: diagnose incorrect arguments

From: Jeff King <peff@peff.net>
Date: Oct 16, 2020, 00:17 UTC
Message-ID: <20201016001704.GA2937048@coredump.intra.peff.net>
In-Reply-To: <20200809174209.15466-1-sunshine@sunshineco.com>

On Sun, Aug 09, 2020 at 01:42:09PM -0400, Eric Sunshine wrote:

Show 10 quoted lines

> Under normal circumstances, if a test author misspells a filename passed
> to test_cmp(), the error is quickly discovered when the test fails
> unexpectedly due to test_cmp() being unable to find the file. However,
> if the test is expected to fail, as with test_expect_failure(), a
> misspelled filename as argument to test_cmp() will go unnoticed since
> the test will indeed fail, but for the wrong reason. Make it easier for
> test authors to discover such problems early by sanity-checking the
> arguments to test_cmp(). To avoid penalizing all clients of test_cmp()
> in the general case, only check for missing files if the comparison
> fails.

This patch caused some interesting confusion for me today.

I was looking at the patch from [1] which causes a test failure (and I wanted to see where it failed, etc). And I got:

  $ ./t5601-clone.sh
  ok 1 - setup
  ok 2 - clone with excess parameters (1)
  ok 3 - clone with excess parameters (2)
  ok 4 - output from clone
  ok 5 - clone does not keep pack
  ok 6 - clone checks out files
  ok 7 - clone respects GIT_WORK_TREE
  error: bug in the test script: test_cmp 'r2/.git/HEAD' missing

which was somewhat unhelpful (or at least less helpful than a regular test failure). The test in question does this:

	test_cmp r0/.git/HEAD r2/.git/HEAD &&

and expects to fail if an earlier step didn't correctly create r2. Is it a bug or misuse of test_cmp for it to do so? I could see an argument that it is, but I'm also not sure if there's a convenient alternative. The best I could come up with is:

  test_path_is_file r2/.git/HEAD &&
  test_cmp r0/.git/HEAD r2/.git/HEAD

which isn't that great.

-Peff

[1] https://lore.kernel.org/git/20200130102933.GE840531@coredump.intra.peff.net/