git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 2/9] ref-filter: avoid extra copies of payload/signature

From
Jeff King <peff@peff.net>
Date
Sep 9, 2024, 23:12 UTC
Message-ID
<20240909231228.GB921834@coredump.intra.peff.net>
In-Reply-To
<20240909230758.GA921697@coredump.intra.peff.net>

When we know we're going to show the subject or body of a tag or commit, we call find_subpos(), which returns pointers and lengths for the three parts: subject, body, signature.

Oddly, the function finds the signature twice: once by calling parse_signature() at the start, which copies the signature into a separate strbuf, and then again by calling parse_signed_buffer() after we've parsed past the subject.

This is due to 482c119186 (gpg-interface: improve interface for parsing tags, 2021-02-11) and 88bce0e24c (ref-filter: hoist signature parsing, 2021-02-11). The idea is that in a multi-hash world, tag signatures may appear in the header, rather than at the end of the body, in which case we need to extract them into a separate buffer.

But parse_signature() would never find such a buffer! It only looks for signature lines (like "-----BEGIN PGP") at the start of each line, without any header keyword. So this code will never find anything except the usual in-body signature.

And the extra code has two downsides:
  1. We spend time copying the payload and signature into strbufs. That
     might even be useful if we ended up with a NUL-terminated copy of
     the payload data, but we throw it away immediately. And the
     signature, since it comes at the end of the message, is already its
     own NUL-terminated buffer.
     The overhead isn't huge, but I measured a pretty consistent 1-2%
     speedup running "git for-each-ref --format='%(subject)'" with this
     patch on a clone of linux.git.
  2. The output of find_subpos() is a set of three ptr/len combinations,
     but only two of them point into the original buffer. This makes the
     interface confusing: you can't do pointer comparisons between them,
     and you have to remember to free the signature buffer. Since
     there's only one caller, it's not too bad in practice, but it did
     bite me while working on the next patch (and simplifying it will
     pave the way for that).

In the long run we might have to go back to something like this approach, if we do have multi-hash header signatures. But I would argue that the extra buffer should kick in only for a header signature, and be passed out of find_subpos() separately.

Signed-off-by: Jeff King <peff@peff.net>
---
This should produce no behavior change.

It does seem funny to me that we do this signature parsing for commits as well as tags, even through the former would have an in-header signature. So arguably we are wrongly cutting in-body signatures from commits, and not showing their signatures with %(contents:signature). (But note that %(signature) is its own thing and does handle commits correctly).

So I don't know if we'd want to fix that or not, but I left it as-is in this series. And as I said above, if we did want to go that way, I think we'd still want to build it on top of this, so that find_subpos() returns the in-header and in-body signatures separately.

 ref-filter.c | 11 +++--------
 1 file changed, 3 insertions(+), 8 deletions(-)
diff --git a/ref-filter.c b/ref-filter.c
index b6c6c10127..0f5513ba7e 100644
--- a/ref-filter.c
+++ b/ref-filter.c
@@ -1833,16 +1833,10 @@ static void find_subpos(const char *buf,
 			size_t *nonsiglen,
 			const char **sig, size_t *siglen)
 {
-	struct strbuf payload = STRBUF_INIT;
-	struct strbuf signature = STRBUF_INIT;
 	const char *eol;
 	const char *end = buf + strlen(buf);
 	const char *sigstart;
 
-	/* parse signature first; we might not even have a subject line */
-	parse_signature(buf, end - buf, &payload, &signature);
-	strbuf_release(&payload);
-
 	/* skip past header until we hit empty line */
 	while (*buf && *buf != '\n') {
 		eol = strchrnul(buf, '\n');
@@ -1853,8 +1847,10 @@ static void find_subpos(const char *buf,
 	/* skip any empty lines */
 	while (*buf == '\n')
 		buf++;
-	*sig = strbuf_detach(&signature, siglen);
+	/* parse signature first; we might not even have a subject line */
 	sigstart = buf + parse_signed_buffer(buf, strlen(buf));
+	*sig = sigstart;
+	*siglen = end - *sig;
 
 	/* subject is first non-empty line */
 	*sub = buf;
@@ -2021,7 +2017,6 @@ static void grab_sub_body_contents(struct atom_value *val, int deref, struct exp
 			v->s = xstrdup(subpos);
 
 	}
-	free((void *)sigpos);
 }
 
 /*
-- 
2.46.0.867.gf99b2b8e0f
Previous: Jeff KingNext: Patrick Steinhardt
Message 3 of 27 in “ref-filter %(trailer) fixes”
  1. 0/9 ref-filter %(trailer) fixesJeff King, Sep 9, 2024
  2. 1/9 t6300: drop newline from wrapped test titleJeff King, Sep 9, 2024
  3. 2/9 ref-filter: avoid extra copies of payload/signatureJeff King, Sep 9, 2024
  4. Patrick SteinhardtSep 10, 2024
  5. Jeff KingSep 10, 2024
  6. 3/9 ref-filter: strip signature when parsing tag trailersJeff King, Sep 9, 2024
  7. Patrick SteinhardtSep 10, 2024
  8. Jeff KingSep 10, 2024
  9. 4/9 ref-filter: drop useless cast in trailers_atom_parser()Jeff King, Sep 9, 2024
  10. 5/9 ref-filter: store ref_trailer_buf data per-atomJeff King, Sep 9, 2024
  11. Patrick SteinhardtSep 10, 2024
  12. 6/9 ref-filter: fix leak of %(trailers) "argbuf"Jeff King, Sep 9, 2024
  13. Patrick SteinhardtSep 10, 2024
  14. Jeff KingSep 10, 2024
  15. 7/9 ref-filter: fix leak with %(describe) argumentsJeff King, Sep 9, 2024
  16. 8/9 ref-filter: fix leak when formatting %(push:remoteref)Jeff King, Sep 9, 2024
  17. Patrick SteinhardtSep 10, 2024
  18. 9/9 ref-filter: add ref_format_clear() functionJeff King, Sep 9, 2024
  19. Patrick SteinhardtSep 10, 2024
  20. Jeff KingSep 10, 2024
  21. 10/9 ref-filter: fix leak with unterminated %(if) atomsPatrick Steinhardt, Sep 10, 2024
  22. Jeff KingSep 10, 2024
  23. Junio C HamanoSep 10, 2024
  24. Patrick SteinhardtSep 12, 2024
  25. Jeff KingSep 12, 2024
  26. Patrick SteinhardtSep 12, 2024
  27. Junio C HamanoSep 12, 2024

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.