git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 07/28] commit: avoid leaking already-saved buffer

From
Jeff King <peff@peff.net>
Date
Sep 24, 2024, 21:54 UTC
Message-ID
<20240924215434.GG1143820@coredump.intra.peff.net>
In-Reply-To
<20240924214930.GA1143523@coredump.intra.peff.net>

When we parse a commit via repo_parse_commit_internal(), if save_commit_buffer is set we'll stuff the buffer of the object contents into a cache, overwriting any previous value.

This can result in a leak of that previously cached value, though it's rare in practice. If we have a value in the cache it would have come from a previous parse, and during that parse we'd set the object.parsed flag, causing any subsequent parse attempts to exit without doing any work.

But it's possible to "unparse" a commit, which we do when registering a commit graft. And since shallow fetches are implemented using grafts, the leak is triggered in practice by t5539.

There are a number of possible ways to address this:
  1. the unparsing function could clear the cached commit buffer, too. I
     think this would work for the case I found, but I'm not sure if
     there are other ways to end up in the same state (an unparsed
     commit with an entry in the commit buffer cache).
  2. when we parse, we could check the buffer cache and prefer it to
     reading the contents from the object database. In theory the
     contents of a particular sha1 are immutable, but the code in
     question is violating the immutability with grafts. So this
     approach makes me a bit nervous, although I think it would work in
     practice (the grafts are applied to what we parse, but we still
     retain the original contents).
  3. We could realize the cache is already populated and discard its
     contents before overwriting. It's possible some other code could be
     holding on to a pointer to the old cache entry (and we'd introduce
     a use-after-free), but I think the risk of that is relatively low.
  4. The reverse of (3): when the cache is populated, don't bother
     saving our new copy. This is perhaps a little weird, since we'll
     have just populated the commit struct based on a different buffer.
     But the two buffers should be the same, even in the presence of
     grafts (as in (2) above).

I went with option 4. It addresses the leak directly and doesn't carry any risk of breaking other assumptions. And it's the same technique used by parse_object_buffer() for this situation, though I'm not sure when it would even come up there. The extra safety has been there since bd1e17e245 (Make "parse_object()" also fill in commit message buffer data., 2005-05-25).

This lets us mark t5539 as leak-free.
Signed-off-by: Jeff King <peff@peff.net>
---
 commit.c                      | 3 ++-
 t/t5539-fetch-http-shallow.sh | 1 +
 2 files changed, 3 insertions(+), 1 deletion(-)
diff --git a/commit.c b/commit.c
index 3a54e4db0d..cc03a93036 100644
--- a/commit.c
+++ b/commit.c
@@ -595,7 +595,8 @@ int repo_parse_commit_internal(struct repository *r,
 	}
 
 	ret = parse_commit_buffer(r, item, buffer, size, 0);
-	if (save_commit_buffer && !ret) {
+	if (save_commit_buffer && !ret &&
+	    !get_cached_commit_buffer(r, item, NULL)) {
 		set_commit_buffer(r, item, buffer, size);
 		return 0;
 	}
diff --git a/t/t5539-fetch-http-shallow.sh b/t/t5539-fetch-http-shallow.sh
index 3ea75d34ca..82fe09d0a9 100755
--- a/t/t5539-fetch-http-shallow.sh
+++ b/t/t5539-fetch-http-shallow.sh
@@ -5,6 +5,7 @@ test_description='fetch/clone from a shallow clone over http'
 GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main
 export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME
 
+TEST_PASSES_SANITIZE_LEAK=true
 . ./test-lib.sh
 . "$TEST_DIRECTORY"/lib-httpd.sh
 start_httpd
-- 
2.46.2.1011.gf1f9323e02
Previous: Jeff KingNext: Patrick Steinhardt
Message 14 of 49 in “leak fixes for http fetch/push”
  1. 0/28 leak fixes for http fetch/pushJeff King, Sep 24, 2024
  2. 01/28 http-fetch: clear leaking git-index-pack(1) argumentsJeff King, Sep 24, 2024
  3. 02/28 shallow: fix leak when unregistering last shallow rootJeff King, Sep 24, 2024
  4. 03/28 fetch-pack: fix leaking sought refsJeff King, Sep 24, 2024
  5. René ScharfeSep 25, 2024
  6. Patrick SteinhardtSep 26, 2024
  7. 04/28 connect: clear child process before freeing in diagnostic modeJeff King, Sep 24, 2024
  8. Patrick SteinhardtSep 26, 2024
  9. 05/28 fetch-pack: free object filter before exitingJeff King, Sep 24, 2024
  10. Patrick SteinhardtSep 26, 2024
  11. 06/28 fetch-pack, send-pack: clean up shallow oid arrayJeff King, Sep 24, 2024
  12. Patrick SteinhardtSep 26, 2024
  13. Jeff KingSep 27, 2024
  14. 07/28 commit: avoid leaking already-saved bufferJeff King, Sep 24, 2024
  15. Patrick SteinhardtSep 26, 2024
  16. 08/28 send-pack: free cas options before exitJeff King, Sep 24, 2024
  17. Patrick SteinhardtSep 26, 2024
  18. Jeff KingSep 27, 2024
  19. 09/28 transport-helper: fix strbuf leak in push_refs_with_push()Jeff King, Sep 24, 2024
  20. Patrick SteinhardtSep 26, 2024
  21. Jeff KingSep 27, 2024
  22. 10/28 fetch: free "raw" string when shrinking refspecJeff King, Sep 24, 2024
  23. 11/28 fetch-pack: clear pack lockfiles listJeff King, Sep 24, 2024
  24. 12/28 transport-helper: fix leak of dummy refs_listJeff King, Sep 24, 2024
  25. 13/28 http: fix leak when redacting cookies from curl traceJeff King, Sep 24, 2024
  26. 14/28 http: fix leak of http_object_request structJeff King, Sep 24, 2024
  27. Patrick SteinhardtSep 26, 2024
  28. Jeff KingSep 27, 2024
  29. 15/28 http: call git_inflate_end() when releasing http_object_requestJeff King, Sep 24, 2024
  30. Patrick SteinhardtSep 26, 2024
  31. Jeff KingSep 27, 2024
  32. 16/28 http: stop leaking buffer in http_get_info_packs()Jeff King, Sep 24, 2024
  33. 17/28 remote-curl: free HEAD ref with free_one_ref()Jeff King, Sep 24, 2024
  34. 18/28 http-walker: free fake packed_git listJeff King, Sep 24, 2024
  35. 19/28 http-push: clear refspecs before exitingJeff King, Sep 24, 2024
  36. 20/28 http-push: free repo->url stringJeff King, Sep 24, 2024
  37. Patrick SteinhardtSep 26, 2024
  38. Jeff KingSep 27, 2024
  39. 21/28 http-push: free curl header listsJeff King, Sep 24, 2024
  40. Patrick SteinhardtSep 26, 2024
  41. 22/28 http-push: free transfer_request dest fieldJeff King, Sep 24, 2024
  42. 23/28 http-push: free transfer_request strbufJeff King, Sep 24, 2024
  43. 24/28 http-push: free remote_ls_ctx.dentry_nameJeff King, Sep 24, 2024
  44. 25/28 http-push: free xml_ctx.cdata after useJeff King, Sep 24, 2024
  45. 26/28 http-push: clean up objects listJeff King, Sep 24, 2024
  46. 27/28 http-push: clean up loose request when falling back to packedJeff King, Sep 24, 2024
  47. 28/28 http-push: clean up local_refs at exitJeff King, Sep 24, 2024
  48. Patrick SteinhardtSep 26, 2024
  49. Jeff KingSep 27, 2024

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.