git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 1/1] t7406: avoid failures solely due to timing issues

From
Johannes Schindelin via GitGitGadget <gitgitgadget@gmail.com>
Date
Jul 23, 2018, 13:39 UTC
Message-ID
<78f7a5e4baae162d1dff18701a25903b304adf9e.1532353179.git.gitgitgadget@gmail.com>
In-Reply-To
<pull.12.git.gitgitgadget@gmail.com>
From: Johannes Schindelin <johannes.schindelin@gmx.de>

Regression tests are automated tests which try to ensure a specific behavior. The idea is: if the test case fails, the behavior indicated in the test case's title regressed.

If a regression test that fails, even occasionally, for any reason other than to indicate the particular regression(s) it tries to catch, it is less useful than when it really only fails when there is a bug in the (non-test) code that needs to be fixed.

In the instance of the test case "submodule update --init --recursive from subdirectory" of the script t7406-submodule-update.sh, the exact output of a recursive clone is compared with a pre-generated one. And this is a racy test because the structure of the submodules only guarantees a *partial* order. The 'none' and the 'rebasing' submodules *can* be cloned in any order, which means that a mismatch with the hard-coded order does not necessarily indicate a bug in the tested code.

See for example: https://git-for-windows.visualstudio.com/git/_build/results?buildId=14035&view=logs

To prevent such false positives from unnecessarily costing time when investigating test failures, let's take the exact order of the lines out of the equation by sorting them before comparing them.

This test script seems not to have any more test cases that try to verify any specific order in which recursive clones process the submodules, therefore this is the only test case that is changed in this manner.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 t/t7406-submodule-update.sh | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)
diff --git a/t/t7406-submodule-update.sh b/t/t7406-submodule-update.sh
index 9e0d31700..4937ebb67 100755
--- a/t/t7406-submodule-update.sh
+++ b/t/t7406-submodule-update.sh
@@ -115,17 +115,17 @@ Submodule path '../super/submodule': checked out '$submodulesha1'
 EOF
 
 cat <<EOF >expect2
+Cloning into '$pwd/recursivesuper/super/merging'...
+Cloning into '$pwd/recursivesuper/super/none'...
+Cloning into '$pwd/recursivesuper/super/rebasing'...
+Cloning into '$pwd/recursivesuper/super/submodule'...
 Submodule 'merging' ($pwd/merging) registered for path '../super/merging'
 Submodule 'none' ($pwd/none) registered for path '../super/none'
 Submodule 'rebasing' ($pwd/rebasing) registered for path '../super/rebasing'
 Submodule 'submodule' ($pwd/submodule) registered for path '../super/submodule'
-Cloning into '$pwd/recursivesuper/super/merging'...
 done.
-Cloning into '$pwd/recursivesuper/super/none'...
 done.
-Cloning into '$pwd/recursivesuper/super/rebasing'...
 done.
-Cloning into '$pwd/recursivesuper/super/submodule'...
 done.
 EOF
 
@@ -137,7 +137,8 @@ test_expect_success 'submodule update --init --recursive from subdirectory' '
 	 git submodule update --init --recursive ../super >../../actual 2>../../actual2
 	) &&
 	test_i18ncmp expect actual &&
-	test_i18ncmp expect2 actual2
+	sort actual2 >actual2.sorted &&
+	test_i18ncmp expect2 actual2.sorted
 '
 
 cat <<EOF >expect2
-- 
gitgitgadget
Previous: Johannes Schindelin via GitGitGadgetNext: Junio C Hamano
Message 2 of 7 in “t7406: avoid failures solely due to timing issues”
  1. 0/1 t7406: avoid failures solely due to timing issuesJohannes Schindelin via GitGitGadget, Jul 23, 2018
  2. 1/1 t7406: avoid failures solely due to timing issuesJohannes Schindelin via GitGitGadget, Jul 23, 2018
  3. Junio C HamanoJul 23, 2018
  4. Stefan BellerJul 23, 2018
  5. Junio C HamanoJul 23, 2018
  6. Junio C HamanoJul 23, 2018
  7. Torsten BögershausenJul 24, 2018

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.