threads / patch / 61686

patch, 3 partsreduce test suite dependency on network

Subject: [PATCH 0/3] reduce test suite dependency on network

## tl;dr

4 messages between Jun 26, 2024 and Jun 26, 2024. Diffs are folded; open one to read it.

replies: 3people: 1as markdown or json

Jeff King· Jun 26, 2024, 20:53 UTC · lore

I'm in a remote cabin with spotty internet this week, and I noticed several delays while running "make test". The culprit was DNS request packets being dropped due to a passing rainstorm, but not enough that the OS decided the network was down (the DNS resolver was on the other end of a VPN). So I was waiting for DNS timeouts.

This is kind of an exotic situation, but it seems to me that the test suite should avoid making network requests in general, just for the sake of reproducibility. And avoiding timeouts is a nice bonus.

So here are a few small fixes, after which running "make test" requires zero DNS lookups (and presumably zero external network at all, though of course we hit localhost/127.0.0.1 for various http, etc, tests).

  [1/3]: t5553: use local url for invalid fetch
  [2/3]: t5551: do not confirm that bogus url cannot be used
  [3/3]: t/lib-bundle-uri: use local fake bundle URLs
 t/lib-bundle-uri-protocol.sh | 4 ++--
 t/t5551-http-fetch-smart.sh  | 1 -
 t/t5553-set-upstream.sh      | 8 ++++----
 3 files changed, 6 insertions(+), 7 deletions(-)
-Peff
Jeff King· Jun 26, 2024, 20:54 UTC · re: Jeff King · lore

[PATCH 1/3] t5553: use local url for invalid fetch

We test how "fetch --set-upstream" behaves when given an invalid URL, using the bogus URL "http://nosuchdomain.example.com". But finding out that it is invalid requires an actual DNS lookup.

Reduce our dependency on external factors by using an invalid local filesystem URL, which works just as well for our purposes.

Signed-off-by: Jeff King <peff@peff.net>
---
 t/t5553-set-upstream.sh | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
Show changes to t/t5553-set-upstream.sh +4 −4
diff --git a/t/t5553-set-upstream.sh b/t/t5553-set-upstream.sh
index 48050162c2..70e3376d31 100755
--- a/t/t5553-set-upstream.sh
+++ b/t/t5553-set-upstream.sh
@@ -73,10 +73,10 @@ test_expect_success 'fetch --set-upstream main:other does not set the branch oth
 	check_config_missing other2
 '
 
-test_expect_success 'fetch --set-upstream http://nosuchdomain.example.com fails with invalid url' '
+test_expect_success 'fetch --set-upstream ./does-not-exist fails with invalid url' '
 	# main explicitly not cleared, we check that it is not touched from previous value
 	clear_config other other2 &&
-	test_must_fail git fetch --set-upstream http://nosuchdomain.example.com &&
+	test_must_fail git fetch --set-upstream ./does-not-exist &&
 	check_config main upstream refs/heads/other &&
 	check_config_missing other &&
 	check_config_missing other2
@@ -143,10 +143,10 @@ test_expect_success 'pull --set-upstream upstream tag does not set the tag' '
 	check_config_missing three
 '
 
-test_expect_success 'pull --set-upstream http://nosuchdomain.example.com fails with invalid url' '
+test_expect_success 'pull --set-upstream ./does-not-exist fails with invalid url' '
 	# main explicitly not cleared, we check that it is not touched from previous value
 	clear_config other other2 three &&
-	test_must_fail git pull --set-upstream http://nosuchdomain.example.com &&
+	test_must_fail git pull --set-upstream ./does-not-exist &&
 	check_config main upstream refs/heads/other &&
 	check_config_missing other &&
 	check_config_missing other2 &&
-- 
2.45.2.1058.g2ff574fec9
Jeff King· Jun 26, 2024, 20:55 UTC · re: Jeff King · lore

[PATCH 2/3] t5551: do not confirm that bogus url cannot be used

t5551 tries to access a URL with a bogus hostname and confirms that http.curloptResolve lets us use this otherwise unresolvable name.

Before doing so, though, we confirm that trying to access the bogus hostname without http.curloptResolve fails as expected. This isn't testing Git at all, but is confirming the test's assumptions. That's often a good thing to do, but in this case it means that we'll actually try to resolve the external name. Even though it's unlikely that "gitbogusexamplehost.invalid" would ever resolve, the DNS lookup itself may take time.

It's probably reasonable to just assume that this obviously-bogus name would not actually resolve in practice, which lets us reduce our test suite's dependency on the outside world.

Signed-off-by: Jeff King <peff@peff.net>
---
 t/t5551-http-fetch-smart.sh | 1 -
 1 file changed, 1 deletion(-)
Show changes to t/t5551-http-fetch-smart.sh +0 −1
diff --git a/t/t5551-http-fetch-smart.sh b/t/t5551-http-fetch-smart.sh
index a623a1058c..7b5ab0eae1 100755
--- a/t/t5551-http-fetch-smart.sh
+++ b/t/t5551-http-fetch-smart.sh
@@ -643,7 +643,6 @@ test_expect_success 'clone empty SHA-256 repository with protocol v0' '
 test_expect_success 'passing hostname resolution information works' '
 	BOGUS_HOST=gitbogusexamplehost.invalid &&
 	BOGUS_HTTPD_URL=$HTTPD_PROTO://$BOGUS_HOST:$LIB_HTTPD_PORT &&
-	test_must_fail git ls-remote "$BOGUS_HTTPD_URL/smart/repo.git" >/dev/null &&
 	git -c "http.curloptResolve=$BOGUS_HOST:$LIB_HTTPD_PORT:127.0.0.1" ls-remote "$BOGUS_HTTPD_URL/smart/repo.git" >/dev/null
 '
 
-- 
2.45.2.1058.g2ff574fec9
Jeff King· Jun 26, 2024, 20:57 UTC · re: Jeff King · lore

[PATCH 3/3] t/lib-bundle-uri: use local fake bundle URLs

A few of the bundle URI tests point config at a fake bundle; they care only that the client has been configured with _some_ bundle, but it doesn't have to actually contain objects.

For the file:// tests, we use "$BUNDLE_URI_REPO_URI/fake.bdl", a non-existent file inside the actual remote repo. But for git:// and http:// tests, we use "https://example.com/fake.bdl". This works OK in practice, but it means we actually make a request to example.com (which returns a placeholder HTML response). That can be annoying when running the test suite on a spotty network (it doesn't produce a wrong result, since we expect it to fail, but it may introduce delays).

We can reduce our dependency on the outside world by using a local URL. It would work to just do "file://$PWD/fake.bdl" here, since the bundle code does not care about the actual location. But in the long run I suspect we may have more restrictions on which protocols can be passed around as bundle URIs. So instead, let's stick with the file:// repo's pattern and just point to a bogus name based on the remote repo's URL.

For http this makes perfect sense; we'll make a request to the local http server and find that there's nothing there. For git:// it's a little weird, as you wouldn't normally access a bundle file over git:// at all. But it's probably the most reasonable guess we can make for now, and anybody who tightens protocol selection later will know better what's the best path forward.

Signed-off-by: Jeff King <peff@peff.net>
---
 t/lib-bundle-uri-protocol.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
Show changes to t/lib-bundle-uri-protocol.sh +2 −2
diff --git a/t/lib-bundle-uri-protocol.sh b/t/lib-bundle-uri-protocol.sh
index a4a1af8d02..de09b6b02e 100644
--- a/t/lib-bundle-uri-protocol.sh
+++ b/t/lib-bundle-uri-protocol.sh
@@ -18,15 +18,15 @@ git)
 	start_git_daemon --export-all --enable=receive-pack
 	BUNDLE_URI_PARENT="$GIT_DAEMON_DOCUMENT_ROOT_PATH/parent"
 	BUNDLE_URI_REPO_URI="$GIT_DAEMON_URL/parent"
-	BUNDLE_URI_BUNDLE_URI="https://example.com/fake.bdl"
+	BUNDLE_URI_BUNDLE_URI="$BUNDLE_URI_REPO_URI/fake.bdl"
 	test_set_prereq BUNDLE_URI_GIT
 	;;
 http)
 	. "$TEST_DIRECTORY"/lib-httpd.sh
 	start_httpd
 	BUNDLE_URI_PARENT="$HTTPD_DOCUMENT_ROOT_PATH/http_parent"
 	BUNDLE_URI_REPO_URI="$HTTPD_URL/smart/http_parent"
-	BUNDLE_URI_BUNDLE_URI="https://example.com/fake.bdl"
+	BUNDLE_URI_BUNDLE_URI="$BUNDLE_URI_REPO_URL/fake.bdl"
 	test_set_prereq BUNDLE_URI_HTTP
 	;;
 *)
-- 
2.45.2.1058.g2ff574fec9

← back to recent threads