git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v7 0/2] fetch-pack: redact packfile urls in traces

From
Ivan Frade via GitGitGadget <gitgitgadget@gmail.com>
Date
Nov 10, 2021, 23:51 UTC
Message-ID
<pull.1052.v7.git.1636588289.gitgitgadget@gmail.com>
In-Reply-To
<pull.1052.v6.git.1635532975.gitgitgadget@gmail.com>
Changes since v6:
 * Use specific hash sizes instead of hexsz
 * Remove unnecessary env vars in tests
 * Added comment on bit toggle
Changes since v5:
 * Use hexsz instead of hardcoded hash sizes
Changes since v4:
 * Use "uri" instead of "url"
 * Look specifically for a line with packfile-uri format (instead of for a
   URL in general)
 * Limit the redacting to the packfile-uri section in do_fetch_pack_v2
 * Use "%.*s" instead of duplicating parts of the string to print
Changes since v3:
 * Enable redacting URLs for all sections
 * Redact only URL path (it was until the end of line)
 * Redact URL in die() with more friendly message
 * Update doc to mention that packfile URIs are also redacted.
Changes since v2:
 * Redact only the path of the URL
 * Test are now strict, validating the exact line expected in the log
Changes since v1:
 * Removed non-POSIX flags in tests
 * More accurate regex for the non-encrypted packfile line
 * Dropped documentation change
 * Dropped redacting the die message in http-fetch
Ivan Frade (2):
  fetch-pack: redact packfile urls in traces
  http-fetch: redact url on die() message
 Documentation/git.txt  |  5 +++--
 fetch-pack.c           |  5 +++++
 http-fetch.c           | 14 ++++++++++--
 pkt-line.c             | 40 ++++++++++++++++++++++++++++++++-
 pkt-line.h             |  1 +
 t/t5702-protocol-v2.sh | 51 ++++++++++++++++++++++++++++++++++++++++++
 6 files changed, 111 insertions(+), 5 deletions(-)
base-commit: 88d915a634b449147855041d44875322de2b286d
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1052%2Fifradeo%2Fredact-packfile-uri-v7
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1052/ifradeo/redact-packfile-uri-v7
Pull-Request: https://github.com/gitgitgadget/git/pull/1052
Range-diff vs v6:
 1:  a6098f98946 ! 1:  bbfdc346ede fetch-pack: redact packfile urls in traces
     @@ fetch-pack.c: static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,
      +				reader.options |= PACKET_READ_REDACT_URI_PATH;
       			if (process_section_header(&reader, "packfile-uris", 1))
       				receive_packfile_uris(&reader, &packfile_uris);
     ++			/* We don't expect more URIs. Reset to avoid expensive URI check. */
      +			reader.options &= ~PACKET_READ_REDACT_URI_PATH;
      +
       			process_section_header(&reader, "packfile", 0);
     @@ pkt-line.c: int packet_length(const char lenbuf_hex[4])
      +	buffer += 1;
      +
      +	len = strspn(buffer, "0123456789abcdefABCDEF");
     -+	if (len != (int)the_hash_algo->hexsz || buffer[len] != ' ')
     ++	/* size of SHA1 and SHA256 hash */
     ++	if (!(len == 40 || len == 64) || buffer[len] != ' ')
      +		return NULL; /* required "<hash>SP" not seen */
      +
      +	path = strstr(buffer + len + 1, URI_MARK);
     @@ t/t5702-protocol-v2.sh: test_expect_success 'packfile-uri with transfer.fsckobje
      +		"uploadpack.blobpackfileuri" \
      +		"$(cat objh) $(cat packh) $HTTPD_URL/dumb/mypack-$(cat packh).pack" &&
      +
     -+	GIT_TRACE=1 GIT_TRACE_PACKET="$(pwd)/log" GIT_TEST_SIDEBAND_ALL=1 \
     ++	GIT_TRACE_PACKET="$(pwd)/log" \
      +	git -c protocol.version=2 \
      +		-c fetch.uriprotocols=http,https \
      +		clone "$HTTPD_URL/smart/http_parent" http_child &&
     @@ t/t5702-protocol-v2.sh: test_expect_success 'packfile-uri with transfer.fsckobje
      +		"uploadpack.blobpackfileuri" \
      +		"$(cat objh) $(cat packh) $HTTPD_URL/dumb/mypack-$(cat packh).pack" &&
      +
     -+	GIT_TRACE=1 GIT_TRACE_PACKET="$(pwd)/log" GIT_TEST_SIDEBAND_ALL=1 \
     ++	GIT_TRACE_PACKET="$(pwd)/log" \
      +	GIT_TRACE_REDACT=0 \
      +	git -c protocol.version=2 \
      +		-c fetch.uriprotocols=http,https \
 2:  38859ae7b7d = 2:  3b210735bc8 http-fetch: redact url on die() message
-- 
gitgitgadget
Previous: Jonathan TanNext: Ivan Frade via GitGitGadget
Message 40 of 43 in “fetch-pack: redact packfile urls in traces”
  1. 0/2 fetch-pack: redact packfile urls in tracesIvan Frade via GitGitGadget, Oct 8, 2021
  2. 1/2 fetch-pack: redact packfile urls in tracesIvan Frade via GitGitGadget, Oct 8, 2021
  3. Ævar Arnfjörð BjarmasonOct 8, 2021
  4. Ivan FradeOct 8, 2021
  5. 2/2 Documentation: packfile-uri hash can be longer than 40 hex charsIvan Frade via GitGitGadget, Oct 8, 2021
  6. Ævar Arnfjörð BjarmasonOct 8, 2021
  7. fetch-pack: redact packfile urls in tracesIvan Frade via GitGitGadget, Oct 9, 2021
  8. Junio C HamanoOct 11, 2021
  9. Ivan FradeOct 26, 2021
  10. fetch-pack: redact packfile urls in tracesIvan Frade via GitGitGadget, Oct 19, 2021
  11. Ævar Arnfjörð BjarmasonOct 20, 2021
  12. 0/2 fetch-pack: redact packfile urls in tracesIvan Frade via GitGitGadget, Oct 26, 2021
  13. 1/2 fetch-pack: redact packfile urls in tracesIvan Frade via GitGitGadget, Oct 26, 2021
  14. Junio C HamanoOct 28, 2021
  15. Ivan FradeOct 28, 2021
  16. Junio C HamanoOct 28, 2021
  17. 2/2 http-fetch: redact url on die() messageIvan Frade via GitGitGadget, Oct 26, 2021
  18. Ævar Arnfjörð BjarmasonOct 28, 2021
  19. Eric SunshineOct 28, 2021
  20. Ivan FradeOct 28, 2021
  21. Ivan FradeOct 28, 2021
  22. Junio C HamanoOct 29, 2021
  23. Ævar Arnfjörð BjarmasonNov 9, 2021
  24. 0/2 fetch-pack: redact packfile urls in tracesIvan Frade via GitGitGadget, Oct 28, 2021
  25. 1/2 fetch-pack: redact packfile urls in tracesIvan Frade via GitGitGadget, Oct 28, 2021
  26. Junio C HamanoOct 28, 2021
  27. Ivan FradeOct 29, 2021
  28. Junio C HamanoOct 29, 2021
  29. Jonathan TanNov 8, 2021
  30. 2/2 http-fetch: redact url on die() messageIvan Frade via GitGitGadget, Oct 28, 2021
  31. 0/2 fetch-pack: redact packfile urls in tracesIvan Frade via GitGitGadget, Oct 29, 2021
  32. 1/2 fetch-pack: redact packfile urls in tracesIvan Frade via GitGitGadget, Oct 29, 2021
  33. Jonathan TanNov 8, 2021
  34. Ævar Arnfjörð BjarmasonNov 9, 2021
  35. Ivan FradeNov 10, 2021
  36. Ævar Arnfjörð BjarmasonNov 11, 2021
  37. Ivan FradeNov 10, 2021
  38. 2/2 http-fetch: redact url on die() messageIvan Frade via GitGitGadget, Oct 29, 2021
  39. Jonathan TanNov 8, 2021
  40. 0/2 fetch-pack: redact packfile urls in tracesIvan Frade via GitGitGadget, Nov 10, 2021
  41. 1/2 fetch-pack: redact packfile urls in tracesIvan Frade via GitGitGadget, Nov 10, 2021
  42. 2/2 http-fetch: redact url on die() messageIvan Frade via GitGitGadget, Nov 10, 2021
  43. Junio C HamanoNov 12, 2021

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.