git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 05/11] remote-curl: fix parsing of detached SHA256 heads

From
Patrick Steinhardt <ps@pks.im>
Date
Apr 19, 2024, 09:51 UTC
Message-ID
<32a429fb60e4a7e80afdbcf9f874d2b2d921badd.1713519789.git.ps@pks.im>
In-Reply-To
<cover.1713519789.git.ps@pks.im>

The dumb HTTP transport tries to read the remote HEAD reference by downloading the "HEAD" file and then parsing it via `http_fetch_ref()`. This function will either parse the file as an object ID in case it is exactly `the_hash_algo->hexsz` long, or otherwise it will check whether the reference starts with "ref :" and parse it as a symbolic ref.

This is broken when parsing detached HEADs of a remote SHA256 repository because we never update `the_hash_algo` to the discovered remote object hash. Consequently, `the_hash_algo` will always be the fallback SHA1 hash algorithm, which will cause us to fail parsing HEAD altogteher when it contains a SHA256 object ID.

Fix this issue by setting up `the_hash_algo` via `repo_set_hash_algo()`. While at it, let's make the expected SHA1 fallback explicit in our code, which also addresses an upcoming issue where we are going to remove the SHA1 fallback for `the_hash_algo`.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 remote-curl.c              | 19 ++++++++++++++++++-
 t/t5550-http-fetch-dumb.sh | 15 +++++++++++++++
 2 files changed, 33 insertions(+), 1 deletion(-)
diff --git a/remote-curl.c b/remote-curl.c
index 0b6d7815fd..004b707fdf 100644
--- a/remote-curl.c
+++ b/remote-curl.c
@@ -266,12 +266,23 @@ static struct ref *parse_git_refs(struct discovery *heads, int for_push)
 	return list;
 }
 
+/*
+ * Try to detect the hash algorithm used by the remote repository when using
+ * the dumb HTTP transport. As dumb transports cannot tell us the object hash
+ * directly have to derive it from the advertised ref lengths.
+ */
 static const struct git_hash_algo *detect_hash_algo(struct discovery *heads)
 {
 	const char *p = memchr(heads->buf, '\t', heads->len);
 	int algo;
+
+	/*
+	 * In case the remote has no refs we have no way to reliably determine
+	 * the object hash used by that repository. In that case we simply fall
+	 * back to SHA1, which may or may not be correct.
+	 */
 	if (!p)
-		return the_hash_algo;
+		return &hash_algos[GIT_HASH_SHA1];
 
 	algo = hash_algo_by_length((p - heads->buf) / 2);
 	if (algo == GIT_HASH_UNKNOWN)
@@ -295,6 +306,12 @@ static struct ref *parse_info_refs(struct discovery *heads)
 		    "is this a git repository?",
 		    transport_anonymize_url(url.buf));
 
+	/*
+	 * Set the repository's hash algo to whatever we have just detected.
+	 * This ensures that we can correctly parse the remote references.
+	 */
+	repo_set_hash_algo(the_repository, hash_algo_by_ptr(options.hash_algo));
+
 	data = heads->buf;
 	start = NULL;
 	mid = data;
diff --git a/t/t5550-http-fetch-dumb.sh b/t/t5550-http-fetch-dumb.sh
index 4c3b32785d..5f16cbc58d 100755
--- a/t/t5550-http-fetch-dumb.sh
+++ b/t/t5550-http-fetch-dumb.sh
@@ -55,6 +55,21 @@ test_expect_success 'list refs from outside any repository' '
 	test_cmp expect actual
 '
 
+
+test_expect_success 'list detached HEAD from outside any repository' '
+	git clone --mirror "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" \
+		"$HTTPD_DOCUMENT_ROOT_PATH/repo-detached.git" &&
+	git -C "$HTTPD_DOCUMENT_ROOT_PATH/repo-detached.git" \
+		update-ref --no-deref HEAD refs/heads/main &&
+	git -C "$HTTPD_DOCUMENT_ROOT_PATH/repo-detached.git" update-server-info &&
+	cat >expect <<-EOF &&
+	$(git rev-parse main)	HEAD
+	$(git rev-parse main)	refs/heads/main
+	EOF
+	nongit git ls-remote "$HTTPD_URL/dumb/repo-detached.git" >actual &&
+	test_cmp expect actual
+'
+
 test_expect_success 'create password-protected repository' '
 	mkdir -p "$HTTPD_DOCUMENT_ROOT_PATH/auth/dumb/" &&
 	cp -Rf "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" \
-- 
2.44.GIT
Previous: Patrick SteinhardtNext: Patrick Steinhardt
Message 13 of 67 in “Stop relying on SHA1 fallback for `the_hash_algo`”
  1. 00/11 Stop relying on SHA1 fallback for `the_hash_algo`Patrick Steinhardt, Apr 19, 2024
  2. 01/11 path: harden validation of HEAD with non-standard hashesPatrick Steinhardt, Apr 19, 2024
  3. brian m. carlsonApr 19, 2024
  4. Patrick SteinhardtApr 22, 2024
  5. Junio C HamanoApr 22, 2024
  6. Patrick SteinhardtApr 23, 2024
  7. Junio C HamanoApr 23, 2024
  8. 02/11 parse-options-cb: only abbreviate hashes when hash algo is knownPatrick Steinhardt, Apr 19, 2024
  9. Justin ToblerApr 23, 2024
  10. 03/11 attr: don't recompute default attribute sourcePatrick Steinhardt, Apr 19, 2024
  11. Justin ToblerApr 23, 2024
  12. 04/11 attr: fix BUG() when parsing attrs outside of repoPatrick Steinhardt, Apr 19, 2024
  13. 05/11 remote-curl: fix parsing of detached SHA256 headsPatrick Steinhardt, Apr 19, 2024
  14. 06/11 builtin/rev-parse: allow shortening to more than 40 hex charactersPatrick Steinhardt, Apr 19, 2024
  15. 07/11 builtin/blame: don't access potentially unitialized `the_hash_algo`Patrick Steinhardt, Apr 19, 2024
  16. 08/11 builtin/bundle: abort "verify" early when there is no repositoryPatrick Steinhardt, Apr 19, 2024
  17. 09/11 builtin/diff: explicitly set hash algo when there is no repoPatrick Steinhardt, Apr 19, 2024
  18. Junio C HamanoApr 22, 2024
  19. 10/11 builtin/shortlog: don't set up revisions without repoPatrick Steinhardt, Apr 19, 2024
  20. Justin ToblerApr 23, 2024
  21. 11/11 repository: stop setting SHA1 as the default object hashPatrick Steinhardt, Apr 19, 2024
  22. brian m. carlsonApr 19, 2024
  23. Junio C HamanoApr 19, 2024
  24. Patrick SteinhardtApr 22, 2024
  25. 00/12 Stop relying on SHA1 fallback for `the_hash_algo`Patrick Steinhardt, Apr 23, 2024
  26. 01/12 path: harden validation of HEAD with non-standard hashesPatrick Steinhardt, Apr 23, 2024
  27. 02/12 path: move `validate_headref()` to its only userPatrick Steinhardt, Apr 23, 2024
  28. 03/12 parse-options-cb: only abbreviate hashes when hash algo is knownPatrick Steinhardt, Apr 23, 2024
  29. 04/12 attr: don't recompute default attribute sourcePatrick Steinhardt, Apr 23, 2024
  30. 05/12 attr: fix BUG() when parsing attrs outside of repoPatrick Steinhardt, Apr 23, 2024
  31. 06/12 remote-curl: fix parsing of detached SHA256 headsPatrick Steinhardt, Apr 23, 2024
  32. 07/12 builtin/rev-parse: allow shortening to more than 40 hex charactersPatrick Steinhardt, Apr 23, 2024
  33. 08/12 builtin/blame: don't access potentially unitialized `the_hash_algo`Patrick Steinhardt, Apr 23, 2024
  34. 09/12 builtin/bundle: abort "verify" early when there is no repositoryPatrick Steinhardt, Apr 23, 2024
  35. 10/12 builtin/diff: explicitly set hash algo when there is no repoPatrick Steinhardt, Apr 23, 2024
  36. 11/12 builtin/shortlog: don't set up revisions without repoPatrick Steinhardt, Apr 23, 2024
  37. 12/12 repository: stop setting SHA1 as the default object hashPatrick Steinhardt, Apr 23, 2024
  38. Junio C HamanoApr 27, 2024
  39. Patrick SteinhardtApr 29, 2024
  40. 00/13 Stop relying on SHA1 fallback for `the_hash_algo`Patrick Steinhardt, Apr 29, 2024
  41. 01/13 path: harden validation of HEAD with non-standard hashesPatrick Steinhardt, Apr 29, 2024
  42. 02/13 path: move `validate_headref()` to its only userPatrick Steinhardt, Apr 29, 2024
  43. 03/13 parse-options-cb: only abbreviate hashes when hash algo is knownPatrick Steinhardt, Apr 29, 2024
  44. 04/13 attr: don't recompute default attribute sourcePatrick Steinhardt, Apr 29, 2024
  45. 05/13 attr: fix BUG() when parsing attrs outside of repoPatrick Steinhardt, Apr 29, 2024
  46. 06/13 remote-curl: fix parsing of detached SHA256 headsPatrick Steinhardt, Apr 29, 2024
  47. 07/13 builtin/rev-parse: allow shortening to more than 40 hex charactersPatrick Steinhardt, Apr 29, 2024
  48. 08/13 builtin/blame: don't access potentially unitialized `the_hash_algo`Patrick Steinhardt, Apr 29, 2024
  49. 09/13 builtin/bundle: abort "verify" early when there is no repositoryPatrick Steinhardt, Apr 29, 2024
  50. 10/13 builtin/diff: explicitly set hash algo when there is no repoPatrick Steinhardt, Apr 29, 2024
  51. 11/13 builtin/shortlog: don't set up revisions without repoPatrick Steinhardt, Apr 29, 2024
  52. 12/13 oss-fuzz/commit-graph: set up hash algorithmPatrick Steinhardt, Apr 29, 2024
  53. 13/13 repository: stop setting SHA1 as the default object hashPatrick Steinhardt, Apr 29, 2024
  54. 00/13 Stop relying on SHA1 fallback for `the_hash_algo`Patrick Steinhardt, May 7, 2024
  55. 01/13 path: harden validation of HEAD with non-standard hashesPatrick Steinhardt, May 7, 2024
  56. 02/13 path: move `validate_headref()` to its only userPatrick Steinhardt, May 7, 2024
  57. 03/13 parse-options-cb: only abbreviate hashes when hash algo is knownPatrick Steinhardt, May 7, 2024
  58. 04/13 attr: don't recompute default attribute sourcePatrick Steinhardt, May 7, 2024
  59. 05/13 attr: fix BUG() when parsing attrs outside of repoPatrick Steinhardt, May 7, 2024
  60. 06/13 remote-curl: fix parsing of detached SHA256 headsPatrick Steinhardt, May 7, 2024
  61. 07/13 builtin/rev-parse: allow shortening to more than 40 hex charactersPatrick Steinhardt, May 7, 2024
  62. 08/13 builtin/blame: don't access potentially unitialized `the_hash_algo`Patrick Steinhardt, May 7, 2024
  63. 09/13 builtin/bundle: abort "verify" early when there is no repositoryPatrick Steinhardt, May 7, 2024
  64. 10/13 builtin/diff: explicitly set hash algo when there is no repoPatrick Steinhardt, May 7, 2024
  65. 11/13 builtin/shortlog: don't set up revisions without repoPatrick Steinhardt, May 7, 2024
  66. 12/13 oss-fuzz/commit-graph: set up hash algorithmPatrick Steinhardt, May 7, 2024
  67. 13/13 repository: stop setting SHA1 as the default object hashPatrick Steinhardt, May 7, 2024

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.