threads / patch / 59769

patch, 2 partsa few format-patch leak fixes

Subject: [PATCH 0/2] a few format-patch leak fixes

## tl;dr

6 messages between May 19, 2023 and May 19, 2023. Diffs are folded; open one to read it.

replies: 5people: 3as markdown or json

Jeff King· May 19, 2023, 00:02 UTC · lore

I got bit earlier by the leak-checker complaining about an old leak on jc/send-email-pre-process-fix, so I started to poke at the code. I see that you since silenced it via 20bd08aefb (t9001: mark the script as no longer leak checker clean, 2023-05-17).

I think that's an OK solution for the release period. But since I dug into the actual leak, I didn't want those fixes to go to waste. :)

So these can be applied at our leisure post-release, though I think the first one could also be taken independently, and is enough to fix t9001 (obviously we could also take them both, but the first one is the less risky of the two, since we're in an -rc period).

  [1/2]: format-patch: free rev.message_id when exiting
  [2/2]: format-patch: free elements of rev.ref_message_ids list
 builtin/log.c         | 15 ++++++++-------
 t/t9001-send-email.sh |  2 +-
 2 files changed, 9 insertions(+), 8 deletions(-)
-Peff
Jeff King· May 19, 2023, 00:03 UTC · re: Jeff King · lore

[PATCH 1/2] format-patch: free rev.message_id when exiting

We may allocate a message-id string via gen_message_id(), but we never free it, causing a small leak. This can be demonstrated by running t9001 with a leak-checking build. The offending test is the one touched by 3ece9bf0f9 (send-email: clear the $message_id after validation, 2023-05-17), but the leak is much older than that. The test was simply unlucky enough to trigger the leaking code path for the first time.

We can fix this by freeing the string at the end of the function. We can also re-mark the test script as leak-free, effectively reverting 20bd08aefb (t9001: mark the script as no longer leak checker clean, 2023-05-17).

Signed-off-by: Jeff King <peff@peff.net>
---
 builtin/log.c         | 1 +
 t/t9001-send-email.sh | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)
Show changes to 2 files +2 −1

builtin/log.c, t/t9001-send-email.sh

diff --git a/builtin/log.c b/builtin/log.c
index 7d19578963..ab74760386 100644
--- a/builtin/log.c
+++ b/builtin/log.c
@@ -2415,6 +2415,7 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 	strbuf_release(&rdiff_title);
 	strbuf_release(&sprefix);
 	free(to_free);
+	free(rev.message_id);
 	if (rev.ref_message_ids)
 		string_list_clear(rev.ref_message_ids, 0);
 	free(rev.ref_message_ids);
diff --git a/t/t9001-send-email.sh b/t/t9001-send-email.sh
index 2051103226..8d49eff91a 100755
--- a/t/t9001-send-email.sh
+++ b/t/t9001-send-email.sh
@@ -4,7 +4,7 @@ test_description='git send-email'
 GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main
 export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME
 
-# no longer TEST_PASSES_SANITIZE_LEAK=true - format-patch --thread leaks
+TEST_PASSES_SANITIZE_LEAK=true
 . ./test-lib.sh
 
 # May be altered later in the test
-- 
2.41.0.rc0.359.g22664e20e7
Jeff King· May 19, 2023, 00:05 UTC · re: Jeff King · lore

[PATCH 2/2] format-patch: free elements of rev.ref_message_ids list

When we are showing multiple patches with format-patch, we have to repeatedly overwrite the rev.message_id field. We take care to avoid leaking the old value by either freeing it, or adding it to ref_message_ids, a string list of ids to reference in subsequent messages.

But unfortunately we do leak the value via that string list. We try to clear the string list, courtesy of 89f45cf4eb (format-patch: don't leak "extra_headers" or "ref_message_ids", 2022-04-13). But since it was initialized as "nodup", the string list doesn't realize it owns the strings, and it leaks them.

We have two options here:
  1. Continue to init with "nodup", but then tweak the value of
     ref_message_ids.strdup_strings just before clearing.
  2. Init with "dup", but use "append_nodup" when transferring ownership
     of strings to the list. Clearing just works.

I picked the second here, as I think it calls attention to the tricky part (transferring ownership via the nodup call).

There's one other related fix we have to do, though. We also insert the result of clean_message_id() into the list. This _sometimes_ allocates and sometimes does not, depending on whether we have to remove cruft from the end of the string. Let's teach it to consistently return an allocated string, so that the caller knows it must be freed.

There's no new test here, as the leak can already be see in t4014.44 (as well as others in that script). We can't mark all of t4014 as leak-free, though, as there are other unrelated leaks that it triggers.

Signed-off-by: Jeff King <peff@peff.net>
---
 builtin/log.c | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)
Show changes to builtin/log.c +7 −7
diff --git a/builtin/log.c b/builtin/log.c
index ab74760386..fee5834c0f 100644
--- a/builtin/log.c
+++ b/builtin/log.c
@@ -1401,7 +1401,7 @@ static void make_cover_letter(struct rev_info *rev, int use_separate_file,
 	}
 }
 
-static const char *clean_message_id(const char *msg_id)
+static char *clean_message_id(const char *msg_id)
 {
 	char ch;
 	const char *a, *z, *m;
@@ -1419,7 +1419,7 @@ static const char *clean_message_id(const char *msg_id)
 	if (!z)
 		die(_("insane in-reply-to: %s"), msg_id);
 	if (++z == m)
-		return a;
+		return xstrdup(a);
 	return xmemdupz(a, z - a);
 }
 
@@ -2305,11 +2305,11 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 
 	if (in_reply_to || thread || cover_letter) {
 		rev.ref_message_ids = xmalloc(sizeof(*rev.ref_message_ids));
-		string_list_init_nodup(rev.ref_message_ids);
+		string_list_init_dup(rev.ref_message_ids);
 	}
 	if (in_reply_to) {
-		const char *msgid = clean_message_id(in_reply_to);
-		string_list_append(rev.ref_message_ids, msgid);
+		char *msgid = clean_message_id(in_reply_to);
+		string_list_append_nodup(rev.ref_message_ids, msgid);
 	}
 	rev.numbered_files = just_numbers;
 	rev.patch_suffix = fmt_patch_suffix;
@@ -2365,8 +2365,8 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 				    && (!cover_letter || rev.nr > 1))
 					free(rev.message_id);
 				else
-					string_list_append(rev.ref_message_ids,
-							   rev.message_id);
+					string_list_append_nodup(rev.ref_message_ids,
+								 rev.message_id);
 			}
 			gen_message_id(&rev, oid_to_hex(&commit->object.oid));
 		}
-- 
2.41.0.rc0.359.g22664e20e7
Junio C Hamano· May 19, 2023, 01:32 UTC · re: Jeff King · lore

Re: [PATCH 2/2] format-patch: free elements of rev.ref_message_ids list

Jeff King <peff@peff.net> writes:
Show 22 quoted lines
> When we are showing multiple patches with format-patch, we have to
> repeatedly overwrite the rev.message_id field. We take care to avoid
> leaking the old value by either freeing it, or adding it to
> ref_message_ids, a string list of ids to reference in subsequent
> messages.
>
> But unfortunately we do leak the value via that string list. We try
> to clear the string list, courtesy of 89f45cf4eb (format-patch: don't
> leak "extra_headers" or "ref_message_ids", 2022-04-13). But since it was
> initialized as "nodup", the string list doesn't realize it owns the
> strings, and it leaks them.
>
> We have two options here:
>
>   1. Continue to init with "nodup", but then tweak the value of
>      ref_message_ids.strdup_strings just before clearing.
>
>   2. Init with "dup", but use "append_nodup" when transferring ownership
>      of strings to the list. Clearing just works.
>
> I picked the second here, as I think it calls attention to the tricky
> part (transferring ownership via the nodup call).

This is pretty much what I had in mind wheh I wrote the log message for the follow-up "Yikes, for now let's mark the test no longer sanitizer clean" patch. Very pleased to see a clean-up I punted materialize while I was looking the other way ;-)

Show 5 quoted lines
> There's one other related fix we have to do, though. We also insert the
> result of clean_message_id() into the list. This _sometimes_ allocates
> and sometimes does not, depending on whether we have to remove cruft
> from the end of the string. Let's teach it to consistently return an
> allocated string, so that the caller knows it must be freed.
Yes!
Thanks.
Konstantin Khomoutov· May 19, 2023, 14:54 UTC · re: Jeff King · lore

Re: [PATCH 2/2] format-patch: free elements of rev.ref_message_ids list

On Thu, May 18, 2023 at 08:05:43PM -0400, Jeff King wrote:
[...]
> There's no new test here, as the leak can already be see in t4014.44 (as
> well as others in that script). We can't mark all of t4014 as leak-free,
> though, as there are other unrelated leaks that it triggers.
Probably s/be see/be seen/.
[...]
Junio C Hamano· May 19, 2023, 16:56 UTC · re: Konstantin Khomoutov · lore

Re: [PATCH 2/2] format-patch: free elements of rev.ref_message_ids list

Konstantin Khomoutov <kostix@bswap.ru> writes:
Show 9 quoted lines
> On Thu, May 18, 2023 at 08:05:43PM -0400, Jeff King wrote:
>
> [...]
>
>> There's no new test here, as the leak can already be see in t4014.44 (as
>> well as others in that script). We can't mark all of t4014 as leak-free,
>> though, as there are other unrelated leaks that it triggers.
>
> Probably s/be see/be seen/.

Indeed. Will locally amend. Thanks.

← back to recent threads