git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 05/12] http: simplify parsing of remote objects/info/packs

From
René Scharfe <l.s.r@web.de>
Date
Apr 5, 2019, 10:41 UTC
Message-ID
<83129937-dcd0-f16e-c8aa-97eceec9769a@web.de>
In-Reply-To
<20190404232704.GE21839@sigill.intra.peff.net>
Am 05.04.2019 um 01:27 schrieb Jeff King:
Show 8 quoted lines
> We can use skip_prefix() and parse_oid_hex() to continuously increment
> our pointer, rather than dealing with magic numbers. This also fixes a
> few small shortcomings:
>
>   - if we see a 'P' line that does not match our expectations, we'll
>     leave our "i" counter in the middle of the line. So we'll parse:
>     "P P P pack-1234.pack" as if there was just one "P" which was not
>     intentional (though probably not that harmful).

How so? The default case, which we'd fall through to, skips the rest of such a line, doesn't it?

Show 69 quoted lines
>
>   - if we see a line with the right prefix, suffix, and length, i.e.
>     matching /P pack-.{40}.pack\n/, we'll interpret the middle part as
>     hex without checking if it could be parsed. This could lead to us
>     looking at uninitialized garbage in the hash array. In practice this
>     means we'll just make a garbage request to the server which will
>     fail, though it's interesting that a malicious server could convince
>     us to leak 40 bytes of uninitialized stack to them.
>
>   - the current code is picky about seeing a newline at the end of file,
>     but we can easily be more liberal
>
> Signed-off-by: Jeff King <peff@peff.net>
> ---
>  http.c | 35 ++++++++++++++---------------------
>  1 file changed, 14 insertions(+), 21 deletions(-)
>
> diff --git a/http.c b/http.c
> index a32ad36ddf..2ef47bc779 100644
> --- a/http.c
> +++ b/http.c
> @@ -2147,11 +2147,11 @@ static int fetch_and_setup_pack_index(struct packed_git **packs_head,
>  int http_get_info_packs(const char *base_url, struct packed_git **packs_head)
>  {
>  	struct http_get_options options = {0};
> -	int ret = 0, i = 0;
> -	char *url, *data;
> +	int ret = 0;
> +	char *url;
> +	const char *data;
>  	struct strbuf buf = STRBUF_INIT;
> -	unsigned char hash[GIT_MAX_RAWSZ];
> -	const unsigned hexsz = the_hash_algo->hexsz;
> +	struct object_id oid;
>
>  	end_url_with_slash(&buf, base_url);
>  	strbuf_addstr(&buf, "objects/info/packs");
> @@ -2163,24 +2163,17 @@ int http_get_info_packs(const char *base_url, struct packed_git **packs_head)
>  		goto cleanup;
>
>  	data = buf.buf;
> -	while (i < buf.len) {
> -		switch (data[i]) {
> -		case 'P':
> -			i++;
> -			if (i + hexsz + 12 <= buf.len &&
> -			    starts_with(data + i, " pack-") &&
> -			    starts_with(data + i + hexsz + 6, ".pack\n")) {
> -				get_sha1_hex(data + i + 6, hash);
> -				fetch_and_setup_pack_index(packs_head, hash,
> -						      base_url);
> -				i += hexsz + 11;
> -				break;
> -			}
> -		default:
> -			while (i < buf.len && data[i] != '\n')
> -				i++;
> +	while (*data) {
> +		if (skip_prefix(data, "P pack-", &data) &&
> +		    !parse_oid_hex(data, &oid, &data) &&
> +		    skip_prefix(data, ".pack", &data) &&
> +		    (*data == '\n' || *data == '\0')) {
> +			fetch_and_setup_pack_index(packs_head, oid.hash, base_url);
> +		} else {
> +			data = strchrnul(data, '\n');
>  		}
> -		i++;
> +		if (*data)
> +			data++; /* skip past newline */
So much simpler, *and* converted to object_id -- I like it!

Parsing "P" and "pack-" together crosses logical token boundaries, but that I don't mind it here.

René
Previous: Jeff KingNext: Jeff King
Message 15 of 48 in “a rabbit hole of update-server-info fixes”
  1. 0/12 a rabbit hole of update-server-info fixesJeff King, Apr 4, 2019
  2. 01/12 t5319: fix bogus cat-file argumentJeff King, Apr 4, 2019
  3. Ramsay JonesApr 5, 2019
  4. Jeff KingApr 5, 2019
  5. Jeff KingApr 5, 2019
  6. 02/12 t5319: drop useless --buffer from cat-fileJeff King, Apr 4, 2019
  7. 03/12 packfile: factor out .pack to .idx name conversionJeff King, Apr 4, 2019
  8. 04/12 packfile: check midx coverage with .idx rather than .packJeff King, Apr 4, 2019
  9. René ScharfeApr 5, 2019
  10. Jeff KingApr 5, 2019
  11. Jeff KingApr 5, 2019
  12. SZEDER GáborApr 5, 2019
  13. Jeff KingApr 5, 2019
  14. 05/12 http: simplify parsing of remote objects/info/packsJeff King, Apr 4, 2019
  15. René ScharfeApr 5, 2019
  16. Jeff KingApr 5, 2019
  17. René ScharfeApr 5, 2019
  18. 06/12 server-info: fix blind pointer arithmeticJeff King, Apr 4, 2019
  19. 07/12 server-info: simplify cleanup in parse_pack_def()Jeff King, Apr 4, 2019
  20. 08/12 server-info: use strbuf to read old info/packs fileJeff King, Apr 4, 2019
  21. 09/12 server-info: drop nr_alloc struct memberJeff King, Apr 4, 2019
  22. 10/12 packfile.h: drop extern from function declarationsJeff King, Apr 4, 2019
  23. 11/12 server-info: drop objdirlen pointer arithmeticJeff King, Apr 4, 2019
  24. 12/12 update_info_refs(): drop unused force parameterJeff King, Apr 4, 2019
  25. Junio C HamanoApr 5, 2019
  26. 0/13 a rabbit hole of update-server-info (and midx!) fixesJeff King, Apr 5, 2019
  27. 01/13 packfile.h: drop extern from function declarationsJeff King, Apr 5, 2019
  28. Ramsay JonesApr 5, 2019
  29. Jeff KingApr 5, 2019
  30. Junio C HamanoApr 8, 2019
  31. Jeff KingApr 8, 2019
  32. Junio C HamanoApr 9, 2019
  33. Jeff KingApr 9, 2019
  34. 02/13 pack-revindex: open index if necessaryJeff King, Apr 5, 2019
  35. 03/13 t5319: fix bogus cat-file argumentJeff King, Apr 5, 2019
  36. 04/13 t5319: drop useless --buffer from cat-fileJeff King, Apr 5, 2019
  37. 05/13 midx: check both pack and index names for containmentJeff King, Apr 5, 2019
  38. René ScharfeApr 5, 2019
  39. 06/13 packfile: fix pack basename computationJeff King, Apr 5, 2019
  40. 07/13 http: simplify parsing of remote objects/info/packsJeff King, Apr 5, 2019
  41. 08/13 server-info: fix blind pointer arithmeticJeff King, Apr 5, 2019
  42. 09/13 server-info: simplify cleanup in parse_pack_def()Jeff King, Apr 5, 2019
  43. Jeff KingApr 5, 2019
  44. 10/13 server-info: use strbuf to read old info/packs fileJeff King, Apr 5, 2019
  45. 11/13 server-info: drop nr_alloc struct memberJeff King, Apr 5, 2019
  46. 12/13 server-info: drop objdirlen pointer arithmeticJeff King, Apr 5, 2019
  47. 13/13 update_info_refs(): drop unused force parameterJeff King, Apr 5, 2019
  48. Jeff KingApr 5, 2019

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.