git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 18/28] http-walker: free fake packed_git list

From
Jeff King <peff@peff.net>
Date
Sep 24, 2024, 22:04 UTC
Message-ID
<20240924220412.GR1143820@coredump.intra.peff.net>
In-Reply-To
<20240924214930.GA1143523@coredump.intra.peff.net>

The dumb-http walker code creates a "fake" packed_git list representing packs we've downloaded from the remote (I call it "fake" because generally that struct is only used and managed by the local repository struct). But during our cleanup phase we don't touch those at all, causing a leak.

There's no support here from the rest of the object-database API, as these structs are not meant to be freed, except when closing the object store completely. But we can see that raw_object_store_clear() just calls free() on them, and that's enough here to fix the leak.

I also added a call to close_pack() before each. In the regular code this happens via close_object_store(), which we do as part of raw_object_store_clear(). This is necessary to prevent leaking mmap'd data (like the pack idx) or descriptors. The leak-checker won't catch either of these itself, but I did confirm with some hacky warning() calls and running t5550 that it's easy to leak at least index data.

This is all much more intimate with the packed_git struct than I'd like, but I think fixing it would be a pretty big refactor. And it's just not worth it for dumb-http code which is rarely used these days. If we can silence the leak-checker without creating too much hassle, we should just do that.

This lets us mark t5550 as leak-free.
Signed-off-by: Jeff King <peff@peff.net>
---
 http-walker.c              | 10 ++++++++++
 t/t5550-http-fetch-dumb.sh |  1 +
 2 files changed, 11 insertions(+)
diff --git a/http-walker.c b/http-walker.c
index 9c1e5c37e6..fb2d86d5e7 100644
--- a/http-walker.c
+++ b/http-walker.c
@@ -579,8 +579,18 @@ static void cleanup(struct walker *walker)
 	if (data) {
 		alt = data->alt;
 		while (alt) {
+			struct packed_git *pack;
+
 			alt_next = alt->next;
 
+			pack = alt->packs;
+			while (pack) {
+				struct packed_git *pack_next = pack->next;
+				close_pack(pack);
+				free(pack);
+				pack = pack_next;
+			}
+
 			free(alt->base);
 			free(alt);
 
diff --git a/t/t5550-http-fetch-dumb.sh b/t/t5550-http-fetch-dumb.sh
index ea8e48f627..58189c9f7d 100755
--- a/t/t5550-http-fetch-dumb.sh
+++ b/t/t5550-http-fetch-dumb.sh
@@ -4,6 +4,7 @@ test_description='test dumb fetching over http via static file'
 GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main
 export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME
 
+TEST_PASSES_SANITIZE_LEAK=true
 . ./test-lib.sh
 
 if test_have_prereq !REFFILES
-- 
2.46.2.1011.gf1f9323e02
Previous: Jeff KingNext: Jeff King
Message 34 of 49 in “leak fixes for http fetch/push”
  1. 0/28 leak fixes for http fetch/pushJeff King, Sep 24, 2024
  2. 01/28 http-fetch: clear leaking git-index-pack(1) argumentsJeff King, Sep 24, 2024
  3. 02/28 shallow: fix leak when unregistering last shallow rootJeff King, Sep 24, 2024
  4. 03/28 fetch-pack: fix leaking sought refsJeff King, Sep 24, 2024
  5. René ScharfeSep 25, 2024
  6. Patrick SteinhardtSep 26, 2024
  7. 04/28 connect: clear child process before freeing in diagnostic modeJeff King, Sep 24, 2024
  8. Patrick SteinhardtSep 26, 2024
  9. 05/28 fetch-pack: free object filter before exitingJeff King, Sep 24, 2024
  10. Patrick SteinhardtSep 26, 2024
  11. 06/28 fetch-pack, send-pack: clean up shallow oid arrayJeff King, Sep 24, 2024
  12. Patrick SteinhardtSep 26, 2024
  13. Jeff KingSep 27, 2024
  14. 07/28 commit: avoid leaking already-saved bufferJeff King, Sep 24, 2024
  15. Patrick SteinhardtSep 26, 2024
  16. 08/28 send-pack: free cas options before exitJeff King, Sep 24, 2024
  17. Patrick SteinhardtSep 26, 2024
  18. Jeff KingSep 27, 2024
  19. 09/28 transport-helper: fix strbuf leak in push_refs_with_push()Jeff King, Sep 24, 2024
  20. Patrick SteinhardtSep 26, 2024
  21. Jeff KingSep 27, 2024
  22. 10/28 fetch: free "raw" string when shrinking refspecJeff King, Sep 24, 2024
  23. 11/28 fetch-pack: clear pack lockfiles listJeff King, Sep 24, 2024
  24. 12/28 transport-helper: fix leak of dummy refs_listJeff King, Sep 24, 2024
  25. 13/28 http: fix leak when redacting cookies from curl traceJeff King, Sep 24, 2024
  26. 14/28 http: fix leak of http_object_request structJeff King, Sep 24, 2024
  27. Patrick SteinhardtSep 26, 2024
  28. Jeff KingSep 27, 2024
  29. 15/28 http: call git_inflate_end() when releasing http_object_requestJeff King, Sep 24, 2024
  30. Patrick SteinhardtSep 26, 2024
  31. Jeff KingSep 27, 2024
  32. 16/28 http: stop leaking buffer in http_get_info_packs()Jeff King, Sep 24, 2024
  33. 17/28 remote-curl: free HEAD ref with free_one_ref()Jeff King, Sep 24, 2024
  34. 18/28 http-walker: free fake packed_git listJeff King, Sep 24, 2024
  35. 19/28 http-push: clear refspecs before exitingJeff King, Sep 24, 2024
  36. 20/28 http-push: free repo->url stringJeff King, Sep 24, 2024
  37. Patrick SteinhardtSep 26, 2024
  38. Jeff KingSep 27, 2024
  39. 21/28 http-push: free curl header listsJeff King, Sep 24, 2024
  40. Patrick SteinhardtSep 26, 2024
  41. 22/28 http-push: free transfer_request dest fieldJeff King, Sep 24, 2024
  42. 23/28 http-push: free transfer_request strbufJeff King, Sep 24, 2024
  43. 24/28 http-push: free remote_ls_ctx.dentry_nameJeff King, Sep 24, 2024
  44. 25/28 http-push: free xml_ctx.cdata after useJeff King, Sep 24, 2024
  45. 26/28 http-push: clean up objects listJeff King, Sep 24, 2024
  46. 27/28 http-push: clean up loose request when falling back to packedJeff King, Sep 24, 2024
  47. 28/28 http-push: clean up local_refs at exitJeff King, Sep 24, 2024
  48. Patrick SteinhardtSep 26, 2024
  49. Jeff KingSep 27, 2024

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.