git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v3 19/19] remote-curl: always parse incoming refs

From
Jeff King <peff@peff.net>
Date
Feb 20, 2013, 20:07 UTC
Message-ID
<20130220200719.GS25647@sigill.intra.peff.net>
In-Reply-To
<20130220195147.GA25332@sigill.intra.peff.net>

When remote-curl receives a list of refs from a server, it keeps the whole buffer intact. When we get a "list" command, we feed the result to get_remote_heads, and when we get a "fetch" or "push" command, we feed it to fetch-pack or send-pack, respectively.

If the HTTP response from the server is truncated for any reason, we will get an incomplete ref advertisement. If we then feed this incomplete list to fetch-pack, one of a few things may happen:

  1. If the truncation is in a packet header, fetch-pack
     will notice the bogus line and complain.
  2. If the truncation is inside a packet, fetch-pack will
     keep waiting for us to send the rest of the packet,
     which we never will.
  3. If the truncation is at a packet boundary, fetch-pack
     will keep waiting for us to send the next packet, which
     we never will.

As a result, fetch-pack hangs, waiting for input. However, remote-curl believes it has sent all of the advertisement, and therefore waits for fetch-pack to speak. The two processes end up in a deadlock.

We do notice the broken ref list if we feed it to get_remote_heads. So if git asks the helper to do a "list" followed by a "fetch", we are safe; we'll abort during the list operation, which parses the refs.

This patch teaches remote-curl to always parse and save the incoming ref list when we read the ref advertisement from a server. That means that we will always verify and abort before even running fetch-pack (or send-pack) when reading a corrupted list, even if we do not run the "list" command explicitly.

Since we save the result, in the common case of running "list" then "fetch", we do not do any extra parsing at all. In the case of just a "fetch", we do an extra round of parsing, but only once.

Note also that the "fetch" case will now also initialize server_capabilities from the remote (in remote-curl; we already would do so inside fetch-pack). Doing "list+fetch" already does this. It doesn't actually matter now, but the new behavior is arguably more correct, should remote-curl ever start caring about the server's capability list.

Signed-off-by: Jeff King <peff@peff.net>
---
 remote-curl.c | 22 +++++++++++++---------
 1 file changed, 13 insertions(+), 9 deletions(-)
diff --git a/remote-curl.c b/remote-curl.c
index 856decc..3d2b194 100644
--- a/remote-curl.c
+++ b/remote-curl.c
@@ -76,6 +76,7 @@ struct discovery {
 	char *buf_alloc;
 	char *buf;
 	size_t len;
+	struct ref *refs;
 	unsigned proto_git : 1;
 };
 static struct discovery *last_discovery;
@@ -145,11 +146,12 @@ static void free_discovery(struct discovery *d)
 		if (d == last_discovery)
 			last_discovery = NULL;
 		free(d->buf_alloc);
+		free_refs(d->refs);
 		free(d);
 	}
 }
 
-static struct discovery* discover_refs(const char *service)
+static struct discovery* discover_refs(const char *service, int for_push)
 {
 	struct strbuf exp = STRBUF_INIT;
 	struct strbuf type = STRBUF_INIT;
@@ -221,6 +223,11 @@ static struct discovery* discover_refs(const char *service)
 		last->proto_git = 1;
 	}
 
+	if (last->proto_git)
+		last->refs = parse_git_refs(last, for_push);
+	else
+		last->refs = parse_info_refs(last);
+
 	free(refs_url);
 	strbuf_release(&exp);
 	strbuf_release(&type);
@@ -234,13 +241,11 @@ static struct ref *get_refs(int for_push)
 	struct discovery *heads;
 
 	if (for_push)
-		heads = discover_refs("git-receive-pack");
+		heads = discover_refs("git-receive-pack", for_push);
 	else
-		heads = discover_refs("git-upload-pack");
+		heads = discover_refs("git-upload-pack", for_push);
 
-	if (heads->proto_git)
-		return parse_git_refs(heads, for_push);
-	return parse_info_refs(heads);
+	return heads->refs;
 }
 
 static void output_refs(struct ref *refs)
@@ -254,7 +259,6 @@ static void output_refs(struct ref *refs)
 	}
 	printf("\n");
 	fflush(stdout);
-	free_refs(refs);
 }
 
 struct rpc_state {
@@ -670,7 +674,7 @@ static int fetch(int nr_heads, struct ref **to_fetch)
 
 static int fetch(int nr_heads, struct ref **to_fetch)
 {
-	struct discovery *d = discover_refs("git-upload-pack");
+	struct discovery *d = discover_refs("git-upload-pack", 0);
 	if (d->proto_git)
 		return fetch_git(d, nr_heads, to_fetch);
 	else
@@ -789,7 +793,7 @@ static int push(int nr_spec, char **specs)
 
 static int push(int nr_spec, char **specs)
 {
-	struct discovery *heads = discover_refs("git-receive-pack");
+	struct discovery *heads = discover_refs("git-receive-pack", 1);
 	int ret;
 
 	if (heads->proto_git)
-- 
1.8.2.rc0.9.g352092c
Previous: Jeff King
Message 40 of 40 in “[PATCHv3 0/19] pkt-line cleanups and fixes”
  1. Jeff KingFeb 20, 2013
  2. 01/19 upload-pack: use get_sha1_hex to parse "shallow" linesJeff King, Feb 20, 2013
  3. 02/19 upload-pack: do not add duplicate objects to shallow listJeff King, Feb 20, 2013
  4. 03/19 upload-pack: remove packet debugging harnessJeff King, Feb 20, 2013
  5. 04/19 fetch-pack: fix out-of-bounds buffer offset in get_ackJeff King, Feb 20, 2013
  6. 05/19 send-pack: prefer prefixcmp over memcmp in receive_statusJeff King, Feb 20, 2013
  7. 06/19 upload-archive: do not copy repo nameJeff King, Feb 20, 2013
  8. 07/19 upload-archive: use argv_array to store client argumentsJeff King, Feb 20, 2013
  9. 08/19 write_or_die: raise SIGPIPE when we get EPIPEJeff King, Feb 20, 2013
  10. Jonathan NiederFeb 20, 2013
  11. Jeff KingFeb 20, 2013
  12. Jonathan NiederFeb 20, 2013
  13. Jeff KingFeb 20, 2013
  14. Jonathan NiederFeb 20, 2013
  15. Jeff KingFeb 20, 2013
  16. Junio C HamanoFeb 20, 2013
  17. [BUG] MSVC: error box when interrupting `gitlog` by quitting lessMarat Radchenko, Mar 28, 2014
  18. Marat RadchenkoMar 28, 2014
  19. Jeff KingMar 28, 2014
  20. Marat RadchenkoMar 28, 2014
  21. Jeff KingMar 28, 2014
  22. Johannes SixtMar 28, 2014
  23. MSVC: link in invalidcontinue.obj for better POSIX compatibilityMarat Radchenko, Mar 28, 2014
  24. Junio C HamanoMar 28, 2014
  25. Marat RadchenkoMar 28, 2014
  26. Junio C HamanoMar 28, 2014
  27. MSVC: link in invalidcontinue.obj for better POSIX compatibilityMarat Radchenko, Mar 28, 2014
  28. Junio C HamanoMar 28, 2014
  29. 09/19 pkt-line: move a misplaced commentJeff King, Feb 20, 2013
  30. 10/19 pkt-line: drop safe_write functionJeff King, Feb 20, 2013
  31. 11/19 pkt-line: provide a generic reading function with optionsJeff King, Feb 20, 2013
  32. 12/19 pkt-line: teach packet_read_line to chomp newlinesJeff King, Feb 20, 2013
  33. 13/19 pkt-line: move LARGE_PACKET_MAX definition from sidebandJeff King, Feb 20, 2013
  34. 14/19 pkt-line: provide a LARGE_PACKET_MAX static bufferJeff King, Feb 20, 2013
  35. 15/19 pkt-line: share buffer/descriptor reading implementationJeff King, Feb 20, 2013
  36. Eric SunshineFeb 22, 2013
  37. 16/19 teach get_remote_heads to read from a memory bufferJeff King, Feb 20, 2013
  38. 17/19 remote-curl: pass buffer straight to get_remote_headsJeff King, Feb 20, 2013
  39. 18/19 remote-curl: move ref-parsing code up in fileJeff King, Feb 20, 2013
  40. 19/19 remote-curl: always parse incoming refsJeff King, Feb 20, 2013

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.