git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v3 03/10] trailer: teach iterator about non-trailer lines

From
LGLinus Arver via GitGitGadget <gitgitgadget@gmail.com>
Date
Apr 26, 2024, 00:26 UTC
Message-ID
<9077d5a315d0d7272266856bf75a75b0a24df91d.1714091170.git.gitgitgadget@gmail.com>
In-Reply-To
<pull.1696.v3.git.1714091170.gitgitgadget@gmail.com>
From: Linus Arver <linusa@google.com>

Previously the iterator did not iterate over non-trailer lines. This was somewhat unfortunate, because trailer blocks could have non-trailer lines in them since 146245063e (trailer: allow non-trailers in trailer block, 2016-10-21), which was before the iterator was created in f0939a0eb1 (trailer: add interface for iterating over commit trailers, 2020-09-27).

So if trailer API users wanted to iterate over all lines in a trailer block (including non-trailer lines), they could not use the iterator and were forced to use the lower-level trailer_info struct directly (which provides a raw string array that includes all lines in the trailer block).

Change the iterator's behavior so that we also iterate over non-trailer lines, instead of skipping over them. The new "raw" member of the iterator allows API users to access previously inaccessible non-trailer lines. Reword the variable "trailer" to just "line" because this variable can now hold both trailer lines _and_ non-trailer lines.

The new "raw" member is important because anyone currently not using the iterator is using trailer_info's raw string array directly to access lines to check what the combined key + value looks like. If we didn't provide a "raw" member here, iterator users would have to re-construct the unparsed line by concatenating the key and value back together again --- which places an undue burden for iterator users.

The next commit demonstrates the use of the iterator in sequencer.c as an example of where "raw" will be useful, so that it can start using the iterator.

For the existing use of the iterator in builtin/shortlog.c, we don't have to change the code there because that code does

    trailer_iterator_init(&iter, body);
    while (trailer_iterator_advance(&iter)) {
        const char *value = iter.val.buf;
        if (!string_list_has_string(&log->trailers, iter.key.buf))
            continue;
        ...
and the
        if (!string_list_has_string(&log->trailers, iter.key.buf))

condition already skips over non-trailer lines (iter.key.buf is empty for non-trailer lines, making the comparison still work even with this commit).

Rename "num_expected_trailers" to "num_expected_objects" in t/unit-tests/t-trailer.c because the items we iterate over now include non-trailer lines.

Signed-off-by: Linus Arver <linusa@google.com>
---
 t/unit-tests/t-trailer.c | 16 +++++++++++-----
 trailer.c                | 12 +++++-------
 trailer.h                |  8 ++++++++
 3 files changed, 24 insertions(+), 12 deletions(-)
diff --git a/t/unit-tests/t-trailer.c b/t/unit-tests/t-trailer.c
index c1f897235c7..262e2838273 100644
--- a/t/unit-tests/t-trailer.c
+++ b/t/unit-tests/t-trailer.c
@@ -1,7 +1,7 @@
 #include "test-lib.h"
 #include "trailer.h"
 
-static void t_trailer_iterator(const char *msg, size_t num_expected_trailers)
+static void t_trailer_iterator(const char *msg, size_t num_expected_objects)
 {
 	struct trailer_iterator iter;
 	size_t i = 0;
@@ -11,7 +11,7 @@ static void t_trailer_iterator(const char *msg, size_t num_expected_trailers)
 		i++;
 	trailer_iterator_release(&iter);
 
-	check_uint(i, ==, num_expected_trailers);
+	check_uint(i, ==, num_expected_objects);
 }
 
 static void run_t_trailer_iterator(void)
@@ -19,7 +19,7 @@ static void run_t_trailer_iterator(void)
 	static struct test_cases {
 		const char *name;
 		const char *msg;
-		size_t num_expected_trailers;
+		size_t num_expected_objects;
 	} tc[] = {
 		{
 			"empty input",
@@ -119,7 +119,13 @@ static void run_t_trailer_iterator(void)
 			"not a trailer line\n"
 			"not a trailer line\n"
 			"Signed-off-by: x\n",
-			1
+			/*
+			 * Even though there is only really 1 real "trailer"
+			 * (Signed-off-by), we still have 4 trailer objects
+			 * because we still want to iterate through the entire
+			 * block.
+			 */
+			4
 		},
 		{
 			"with non-trailer lines (one too many) in trailer block",
@@ -162,7 +168,7 @@ static void run_t_trailer_iterator(void)
 
 	for (int i = 0; i < sizeof(tc) / sizeof(tc[0]); i++) {
 		TEST(t_trailer_iterator(tc[i].msg,
-					tc[i].num_expected_trailers),
+					tc[i].num_expected_objects),
 		     "%s", tc[i].name);
 	}
 }
diff --git a/trailer.c b/trailer.c
index 3e4dab9c065..4700c441442 100644
--- a/trailer.c
+++ b/trailer.c
@@ -1146,17 +1146,15 @@ void trailer_iterator_init(struct trailer_iterator *iter, const char *msg)
 
 int trailer_iterator_advance(struct trailer_iterator *iter)
 {
-	while (iter->internal.cur < iter->internal.info.trailer_nr) {
-		char *trailer = iter->internal.info.trailers[iter->internal.cur++];
-		int separator_pos = find_separator(trailer, separators);
-
-		if (separator_pos < 1)
-			continue; /* not a real trailer */
+	if (iter->internal.cur < iter->internal.info.trailer_nr) {
+		char *line = iter->internal.info.trailers[iter->internal.cur++];
+		int separator_pos = find_separator(line, separators);
 
+		iter->raw = line;
 		strbuf_reset(&iter->key);
 		strbuf_reset(&iter->val);
 		parse_trailer(&iter->key, &iter->val, NULL,
-			      trailer, separator_pos);
+			      line, separator_pos);
 		/* Always unfold values during iteration. */
 		unfold_value(&iter->val);
 		return 1;
diff --git a/trailer.h b/trailer.h
index 9f42aa75994..ebafa3657e4 100644
--- a/trailer.h
+++ b/trailer.h
@@ -125,6 +125,14 @@ void format_trailers_from_commit(const struct process_trailer_options *,
  *   trailer_iterator_release(&iter);
  */
 struct trailer_iterator {
+	/*
+	 * Raw line (e.g., "foo: bar baz") before being parsed as a trailer
+	 * key/val pair as part of a trailer block. A trailer block can be
+	 * either 100% trailer lines, or mixed in with non-trailer lines (in
+	 * which case at least 25% must be trailer lines).
+	 */
+	const char *raw;
+
 	struct strbuf key;
 	struct strbuf val;
 
-- 
gitgitgadget
Previous: Linus ArverNext: Christian Couder
Message 36 of 66 in “Make trailer_info struct private (plus sequencer cleanup)”
  1. 0/6 Make trailer_info struct private (plus sequencer cleanup)Linus Arver via GitGitGadget, Mar 16, 2024
  2. 1/6 trailer: teach iterator about non-trailer linesLinus Arver via GitGitGadget, Mar 16, 2024
  3. 2/6 sequencer: use the trailer iteratorLinus Arver via GitGitGadget, Mar 16, 2024
  4. 3/6 interpret-trailers: access trailer_info with new helpersLinus Arver via GitGitGadget, Mar 16, 2024
  5. 4/6 trailer: make parse_trailers() return trailer_info pointerLinus Arver via GitGitGadget, Mar 16, 2024
  6. 5/6 trailer: make trailer_info struct privateLinus Arver via GitGitGadget, Mar 16, 2024
  7. 6/6 trailer: retire trailer_info_get() from APILinus Arver via GitGitGadget, Mar 16, 2024
  8. Junio C HamanoMar 16, 2024
  9. Junio C HamanoMar 26, 2024
  10. Linus ArverApr 19, 2024
  11. 0/8 Make trailer_info struct private (plus sequencer cleanup)Linus Arver via GitGitGadget, Apr 19, 2024
  12. 1/8 Makefile: sort UNIT_TEST_PROGRAMSLinus Arver via GitGitGadget, Apr 19, 2024
  13. 2/8 trailer: add unit tests for trailer iteratorLinus Arver via GitGitGadget, Apr 19, 2024
  14. Linus ArverApr 19, 2024
  15. Linus ArverApr 19, 2024
  16. Junio C HamanoApr 19, 2024
  17. Linus ArverApr 20, 2024
  18. 3/8 trailer: teach iterator about non-trailer linesLinus Arver via GitGitGadget, Apr 19, 2024
  19. 4/8 sequencer: use the trailer iteratorLinus Arver via GitGitGadget, Apr 19, 2024
  20. Junio C HamanoApr 23, 2024
  21. 5/8 interpret-trailers: access trailer_info with new helpersLinus Arver via GitGitGadget, Apr 19, 2024
  22. 7/8 trailer: make trailer_info struct privateLinus Arver via GitGitGadget, Apr 19, 2024
  23. Junio C HamanoApr 23, 2024
  24. Linus ArverApr 25, 2024
  25. 6/8 trailer: make parse_trailers() return trailer_info pointerLinus Arver via GitGitGadget, Apr 19, 2024
  26. Junio C HamanoApr 23, 2024
  27. 8/8 trailer: retire trailer_info_get() from APILinus Arver via GitGitGadget, Apr 19, 2024
  28. Junio C HamanoApr 23, 2024
  29. Junio C HamanoApr 24, 2024
  30. 00/10 Make trailer_info struct private (plus sequencer cleanup)Linus Arver via GitGitGadget, Apr 26, 2024
  31. 01/10 Makefile: sort UNIT_TEST_PROGRAMSLinus Arver via GitGitGadget, Apr 26, 2024
  32. 02/10 trailer: add unit tests for trailer iteratorLinus Arver via GitGitGadget, Apr 26, 2024
  33. Christian CouderApr 26, 2024
  34. Junio C HamanoApr 26, 2024
  35. Linus ArverApr 26, 2024
  36. 03/10 trailer: teach iterator about non-trailer linesLinus Arver via GitGitGadget, Apr 26, 2024
  37. Christian CouderApr 27, 2024
  38. Linus ArverApr 30, 2024
  39. Linus ArverApr 30, 2024
  40. 04/10 sequencer: use the trailer iteratorLinus Arver via GitGitGadget, Apr 26, 2024
  41. 05/10 interpret-trailers: access trailer_info with new helpersLinus Arver via GitGitGadget, Apr 26, 2024
  42. 06/10 trailer: make parse_trailers() return trailer_info pointerLinus Arver via GitGitGadget, Apr 26, 2024
  43. 07/10 trailer: make trailer_info struct privateLinus Arver via GitGitGadget, Apr 26, 2024
  44. 08/10 trailer: retire trailer_info_get() from APILinus Arver via GitGitGadget, Apr 26, 2024
  45. 10/10 trailer unit tests: inspect iterator contentsLinus Arver via GitGitGadget, Apr 26, 2024
  46. 09/10 trailer: document parse_trailers() usageLinus Arver via GitGitGadget, Apr 26, 2024
  47. Christian CouderApr 27, 2024
  48. 00/10 Make trailer_info struct private (plus sequencer cleanup)Linus Arver via GitGitGadget, May 2, 2024
  49. 01/10 Makefile: sort UNIT_TEST_PROGRAMSLinus Arver via GitGitGadget, May 2, 2024
  50. 02/10 trailer: add unit tests for trailer iteratorLinus Arver via GitGitGadget, May 2, 2024
  51. Junio C HamanoMay 2, 2024
  52. 03/10 trailer: teach iterator about non-trailer linesLinus Arver via GitGitGadget, May 2, 2024
  53. Phillip WoodMay 4, 2024
  54. Linus ArverMay 5, 2024
  55. Phillip WoodMay 5, 2024
  56. Linus ArverMay 9, 2024
  57. Phillip WoodMay 13, 2024
  58. Phillip WoodMay 13, 2024
  59. 04/10 sequencer: use the trailer iteratorLinus Arver via GitGitGadget, May 2, 2024
  60. 05/10 interpret-trailers: access trailer_info with new helpersLinus Arver via GitGitGadget, May 2, 2024
  61. 06/10 trailer: make parse_trailers() return trailer_info pointerLinus Arver via GitGitGadget, May 2, 2024
  62. 07/10 trailer: make trailer_info struct privateLinus Arver via GitGitGadget, May 2, 2024
  63. 08/10 trailer: retire trailer_info_get() from APILinus Arver via GitGitGadget, May 2, 2024
  64. 09/10 trailer: document parse_trailers() usageLinus Arver via GitGitGadget, May 2, 2024
  65. 10/10 trailer unit tests: inspect iterator contentsLinus Arver via GitGitGadget, May 2, 2024
  66. Junio C HamanoMay 2, 2024

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.