git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 07/28] commit: avoid leaking already-saved buffer

From
Patrick Steinhardt <ps@pks.im>
Date
Sep 26, 2024, 13:50 UTC
Message-ID
<ZvVmjbMujO1h2sQp@pks.im>
In-Reply-To
<20240924215434.GG1143820@coredump.intra.peff.net>
On Tue, Sep 24, 2024 at 05:54:34PM -0400, Jeff King wrote:
> When we parse a commit via repo_parse_commit_internal(), if
> save_commit_buffer is set we'll stuff the buffer of the object contents
> into a cache, overwriting any previous value.

Interesting. I saw some cases that I think could be caused by this, but couldn't make much sense of them.

Show 13 quoted lines
> This can result in a leak of that previously cached value, though it's
> rare in practice. If we have a value in the cache it would have come
> from a previous parse, and during that parse we'd set the object.parsed
> flag, causing any subsequent parse attempts to exit without doing any
> work.
> 
> But it's possible to "unparse" a commit, which we do when registering a
> commit graft. And since shallow fetches are implemented using grafts,
> the leak is triggered in practice by t5539.
> 
> There are a number of possible ways to address this:
> 
>   1. the unparsing function could clear the cached commit buffer, too. I
s/the/The/
Show 5 quoted lines
>      think this would work for the case I found, but I'm not sure if
>      there are other ways to end up in the same state (an unparsed
>      commit with an entry in the commit buffer cache).
> 
>   2. when we parse, we could check the buffer cache and prefer it to
s/when/When/
Show 24 quoted lines
>      reading the contents from the object database. In theory the
>      contents of a particular sha1 are immutable, but the code in
>      question is violating the immutability with grafts. So this
>      approach makes me a bit nervous, although I think it would work in
>      practice (the grafts are applied to what we parse, but we still
>      retain the original contents).
> 
>   3. We could realize the cache is already populated and discard its
>      contents before overwriting. It's possible some other code could be
>      holding on to a pointer to the old cache entry (and we'd introduce
>      a use-after-free), but I think the risk of that is relatively low.
> 
>   4. The reverse of (3): when the cache is populated, don't bother
>      saving our new copy. This is perhaps a little weird, since we'll
>      have just populated the commit struct based on a different buffer.
>      But the two buffers should be the same, even in the presence of
>      grafts (as in (2) above).
> 
> I went with option 4. It addresses the leak directly and doesn't carry
> any risk of breaking other assumptions. And it's the same technique used
> by parse_object_buffer() for this situation, though I'm not sure when it
> would even come up there. The extra safety has been there since
> bd1e17e245 (Make "parse_object()" also fill in commit message buffer
> data., 2005-05-25).

Okay. This feels a bit weird indeed, but the fact that we already use the same strategy in other places makes me feel way safer.

Show 22 quoted lines
> This lets us mark t5539 as leak-free.
> 
> Signed-off-by: Jeff King <peff@peff.net>
> ---
>  commit.c                      | 3 ++-
>  t/t5539-fetch-http-shallow.sh | 1 +
>  2 files changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/commit.c b/commit.c
> index 3a54e4db0d..cc03a93036 100644
> --- a/commit.c
> +++ b/commit.c
> @@ -595,7 +595,8 @@ int repo_parse_commit_internal(struct repository *r,
>  	}
>  
>  	ret = parse_commit_buffer(r, item, buffer, size, 0);
> -	if (save_commit_buffer && !ret) {
> +	if (save_commit_buffer && !ret &&
> +	    !get_cached_commit_buffer(r, item, NULL)) {
>  		set_commit_buffer(r, item, buffer, size);
>  		return 0;
>  	}
And the fix is trivial.
Patrick
Previous: Jeff KingNext: Jeff King
Message 15 of 49 in “leak fixes for http fetch/push”
  1. 0/28 leak fixes for http fetch/pushJeff King, Sep 24, 2024
  2. 01/28 http-fetch: clear leaking git-index-pack(1) argumentsJeff King, Sep 24, 2024
  3. 02/28 shallow: fix leak when unregistering last shallow rootJeff King, Sep 24, 2024
  4. 03/28 fetch-pack: fix leaking sought refsJeff King, Sep 24, 2024
  5. René ScharfeSep 25, 2024
  6. Patrick SteinhardtSep 26, 2024
  7. 04/28 connect: clear child process before freeing in diagnostic modeJeff King, Sep 24, 2024
  8. Patrick SteinhardtSep 26, 2024
  9. 05/28 fetch-pack: free object filter before exitingJeff King, Sep 24, 2024
  10. Patrick SteinhardtSep 26, 2024
  11. 06/28 fetch-pack, send-pack: clean up shallow oid arrayJeff King, Sep 24, 2024
  12. Patrick SteinhardtSep 26, 2024
  13. Jeff KingSep 27, 2024
  14. 07/28 commit: avoid leaking already-saved bufferJeff King, Sep 24, 2024
  15. Patrick SteinhardtSep 26, 2024
  16. 08/28 send-pack: free cas options before exitJeff King, Sep 24, 2024
  17. Patrick SteinhardtSep 26, 2024
  18. Jeff KingSep 27, 2024
  19. 09/28 transport-helper: fix strbuf leak in push_refs_with_push()Jeff King, Sep 24, 2024
  20. Patrick SteinhardtSep 26, 2024
  21. Jeff KingSep 27, 2024
  22. 10/28 fetch: free "raw" string when shrinking refspecJeff King, Sep 24, 2024
  23. 11/28 fetch-pack: clear pack lockfiles listJeff King, Sep 24, 2024
  24. 12/28 transport-helper: fix leak of dummy refs_listJeff King, Sep 24, 2024
  25. 13/28 http: fix leak when redacting cookies from curl traceJeff King, Sep 24, 2024
  26. 14/28 http: fix leak of http_object_request structJeff King, Sep 24, 2024
  27. Patrick SteinhardtSep 26, 2024
  28. Jeff KingSep 27, 2024
  29. 15/28 http: call git_inflate_end() when releasing http_object_requestJeff King, Sep 24, 2024
  30. Patrick SteinhardtSep 26, 2024
  31. Jeff KingSep 27, 2024
  32. 16/28 http: stop leaking buffer in http_get_info_packs()Jeff King, Sep 24, 2024
  33. 17/28 remote-curl: free HEAD ref with free_one_ref()Jeff King, Sep 24, 2024
  34. 18/28 http-walker: free fake packed_git listJeff King, Sep 24, 2024
  35. 19/28 http-push: clear refspecs before exitingJeff King, Sep 24, 2024
  36. 20/28 http-push: free repo->url stringJeff King, Sep 24, 2024
  37. Patrick SteinhardtSep 26, 2024
  38. Jeff KingSep 27, 2024
  39. 21/28 http-push: free curl header listsJeff King, Sep 24, 2024
  40. Patrick SteinhardtSep 26, 2024
  41. 22/28 http-push: free transfer_request dest fieldJeff King, Sep 24, 2024
  42. 23/28 http-push: free transfer_request strbufJeff King, Sep 24, 2024
  43. 24/28 http-push: free remote_ls_ctx.dentry_nameJeff King, Sep 24, 2024
  44. 25/28 http-push: free xml_ctx.cdata after useJeff King, Sep 24, 2024
  45. 26/28 http-push: clean up objects listJeff King, Sep 24, 2024
  46. 27/28 http-push: clean up loose request when falling back to packedJeff King, Sep 24, 2024
  47. 28/28 http-push: clean up local_refs at exitJeff King, Sep 24, 2024
  48. Patrick SteinhardtSep 26, 2024
  49. Jeff KingSep 27, 2024

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.