git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v3 4/6] rev-list: support delimiting objects with NUL bytes

From
Justin Tobler <jltobler@gmail.com>
Date
Mar 13, 2025, 23:57 UTC
Message-ID
<20250313235747.9583-5-jltobler@gmail.com>
In-Reply-To
<20250313235747.9583-1-jltobler@gmail.com>

When walking objects, git-rev-list(1) prints each object entry on a separate line. Some options, such as `--objects`, may print additional information about tree and blob object on the same line in the form:

        $ git rev-list --objects <rev>
        <tree/blob oid> SP [<path>] LF

Note that in this form the SP is appended regardless of whether the tree or blob object has path information available. Paths containing a newline are also truncated at the newline.

Introduce the `-z` option for git-rev-list(1) which reformats the output to use NUL-delimiters between objects and associated info in the following form:

        $ git rev-list -z --objects <rev>
        <oid> NUL [path=<path> NUL]

In this form, the start of each record is signaled by an OID entry that is all hexidecimal and does not contain any '='. Additional path info from `--objects` is appended to the record as a token/value pair `path=<path>` as-is without any truncation.

In this mode, revision and pathspec arguments provided on stdin with the `--stdin` option are also separated by a NUL byte instead of being newline delimited.

For now, the `--objects` and `--stdin` flag are the only options that can be used in combination with `-z`. In a subsequent commit, NUL-delimited support for other options is added. Other options that do not make sense with be used in combination with `-z` are rejected.

Signed-off-by: Justin Tobler <jltobler@gmail.com>
---
 Documentation/rev-list-options.adoc | 23 ++++++++++++++++++
 builtin/rev-list.c                  | 36 +++++++++++++++++++++++++----
 t/t6000-rev-list-misc.sh            | 35 ++++++++++++++++++++++++++++
 t/t6017-rev-list-stdin.sh           |  9 ++++++++
 4 files changed, 98 insertions(+), 5 deletions(-)
diff --git a/Documentation/rev-list-options.adoc b/Documentation/rev-list-options.adoc
index 785c0786e0..14d82fdfbf 100644
--- a/Documentation/rev-list-options.adoc
+++ b/Documentation/rev-list-options.adoc
@@ -361,6 +361,29 @@ ifdef::git-rev-list[]
 --progress=<header>::
 	Show progress reports on stderr as objects are considered. The
 	`<header>` text will be printed with each progress update.
+
+-z::
+	Instead of being newline-delimited, each outputted object and its
+	accompanying metadata is delimited using NUL bytes. In this mode, when
+	the `--stdin` option is provided, revision and pathspec arguments on
+	stdin are also delimited using a NUL byte. Output is printed in the
+	following form:
++
+-----------------------------------------------------------------------
+<OID> NUL [<token>=<value> NUL]...
+-----------------------------------------------------------------------
++
+Additional object metadata, such as object paths, is printed using the
+`<token>=<value>` form. Token values are printed as-is without any
+encoding/truncation. An OID entry never contains a '=' character and thus
+is used to signal the start of a new object record. Examples:
++
+-----------------------------------------------------------------------
+<OID> NUL
+<OID> NUL path=<path> NUL
+-----------------------------------------------------------------------
++
+This mode is only compatible with the `--objects` output option.
 endif::git-rev-list[]
 
 History Simplification
diff --git a/builtin/rev-list.c b/builtin/rev-list.c
index 04d9c893b5..f048500679 100644
--- a/builtin/rev-list.c
+++ b/builtin/rev-list.c
@@ -65,6 +65,7 @@ static const char rev_list_usage[] =
 "    --abbrev-commit\n"
 "    --left-right\n"
 "    --count\n"
+"    -z\n"
 "  special purpose:\n"
 "    --bisect\n"
 "    --bisect-vars\n"
@@ -97,6 +98,9 @@ static int arg_show_object_names = 1;
 
 #define DEFAULT_OIDSET_SIZE     (16*1024)
 
+static char line_term = '\n';
+static char info_term = ' ';
+
 static int show_disk_usage;
 static off_t total_disk_usage;
 static int human_readable;
@@ -264,7 +268,7 @@ static void show_commit(struct commit *commit, void *data)
 	if (revs->commit_format == CMIT_FMT_ONELINE)
 		putchar(' ');
 	else if (revs->include_header)
-		putchar('\n');
+		putchar(line_term);
 
 	if (revs->verbose_header) {
 		struct strbuf buf = STRBUF_INIT;
@@ -361,12 +365,16 @@ static void show_object(struct object *obj, const char *name, void *cb_data)
 	printf("%s", oid_to_hex(&obj->oid));
 
 	if (arg_show_object_names) {
-		putchar(' ');
-		for (const char *p = name; *p && *p != '\n'; p++)
-			putchar(*p);
+		if (line_term) {
+			putchar(info_term);
+			for (const char *p = name; *p && *p != '\n'; p++)
+				putchar(*p);
+		} else if (*name) {
+			printf("%cpath=%s", info_term, name);
+		}
 	}
 
-	putchar('\n');
+	putchar(line_term);
 }
 
 static void show_edge(struct commit *commit)
@@ -642,6 +650,10 @@ int cmd_rev_list(int argc,
 			revs.exclude_promisor_objects = 1;
 		} else if (skip_prefix(arg, "--missing=", &arg)) {
 			parse_missing_action_value(arg);
+		} else if (!strcmp(arg, "-z")) {
+			s_r_opt.nul_delim_stdin = 1;
+			line_term = '\0';
+			info_term = '\0';
 		}
 	}
 
@@ -757,6 +769,20 @@ int cmd_rev_list(int argc,
 		usage(rev_list_usage);
 
 	}
+
+	/*
+	 * Reject options currently incompatible with -z. For some options, this
+	 * is not an inherent limitation and support may be implemented in the
+	 * future.
+	 */
+	if (!line_term) {
+		if (revs.graph || revs.verbose_header || show_disk_usage ||
+		    info.show_timestamp || info.header_prefix || bisect_list ||
+		    use_bitmap_index || revs.edge_hint || revs.left_right ||
+		    revs.cherry_mark || arg_missing_action || revs.boundary)
+			die(_("-z option used with unsupported option"));
+	}
+
 	if (revs.commit_format != CMIT_FMT_USERFORMAT)
 		revs.include_header = 1;
 	if (revs.commit_format != CMIT_FMT_UNSPECIFIED) {
diff --git a/t/t6000-rev-list-misc.sh b/t/t6000-rev-list-misc.sh
index 6289a2e8b0..dfbbc0aee6 100755
--- a/t/t6000-rev-list-misc.sh
+++ b/t/t6000-rev-list-misc.sh
@@ -182,4 +182,39 @@ test_expect_success 'rev-list --unpacked' '
 	test_cmp expect actual
 '
 
+test_expect_success 'rev-list -z' '
+	test_when_finished rm -rf repo &&
+
+	git init repo &&
+	test_commit -C repo 1 &&
+	test_commit -C repo 2 &&
+
+	oid1=$(git -C repo rev-parse HEAD) &&
+	oid2=$(git -C repo rev-parse HEAD~) &&
+
+	printf "%s\0%s\0" "$oid1" "$oid2" >expect &&
+	git -C repo rev-list -z HEAD >actual &&
+
+	test_cmp expect actual
+'
+
+test_expect_success 'rev-list -z --objects' '
+	test_when_finished rm -rf repo &&
+
+	git init repo &&
+	test_commit -C repo 1 &&
+	test_commit -C repo 2 &&
+
+	oid1=$(git -C repo rev-parse HEAD:1.t) &&
+	oid2=$(git -C repo rev-parse HEAD:2.t) &&
+	path1=1.t &&
+	path2=2.t &&
+
+	printf "%s\0path=%s\0%s\0path=%s\0" "$oid1" "$path1" "$oid2" "$path2" \
+		>expect &&
+	git -C repo rev-list -z --objects HEAD:1.t HEAD:2.t >actual &&
+
+	test_cmp expect actual
+'
+
 test_done
diff --git a/t/t6017-rev-list-stdin.sh b/t/t6017-rev-list-stdin.sh
index 4821b90e74..362a8b126a 100755
--- a/t/t6017-rev-list-stdin.sh
+++ b/t/t6017-rev-list-stdin.sh
@@ -148,4 +148,13 @@ test_expect_success '--not via stdin does not influence revisions from command l
 	test_cmp expect actual
 '
 
+test_expect_success 'NUL-delimited stdin' '
+	printf "%s\0%s\0%s\0" "HEAD" "--" "file-1" > input &&
+
+	git rev-list -z --objects HEAD -- file-1 >expect &&
+	git rev-list -z --objects --stdin <input >actual &&
+
+	test_cmp expect actual
+'
+
 test_done

-- 
2.49.0.rc2
Previous: Justin ToblerNext: Christian Couder
Message 50 of 60 in “rev-list: introduce NUL-delimited output mode”
  1. 0/4 rev-list: introduce NUL-delimited output modeJustin Tobler, Mar 10, 2025
  2. 1/4 rev-list: inline `show_object_with_name()` in `show_object()`Justin Tobler, Mar 10, 2025
  3. Junio C HamanoMar 10, 2025
  4. 2/4 rev-list: refactor early option parsingJustin Tobler, Mar 10, 2025
  5. Junio C HamanoMar 10, 2025
  6. Justin ToblerMar 12, 2025
  7. 3/4 rev-list: support delimiting objects with NUL bytesJustin Tobler, Mar 10, 2025
  8. Junio C HamanoMar 10, 2025
  9. Justin ToblerMar 12, 2025
  10. Patrick SteinhardtMar 12, 2025
  11. Justin ToblerMar 12, 2025
  12. 4/4 rev-list: support NUL-delimited --missing optionJustin Tobler, Mar 10, 2025
  13. Junio C HamanoMar 10, 2025
  14. Junio C HamanoMar 10, 2025
  15. Justin ToblerMar 11, 2025
  16. Justin ToblerMar 11, 2025
  17. Junio C HamanoMar 11, 2025
  18. Patrick SteinhardtMar 12, 2025
  19. Justin ToblerMar 12, 2025
  20. D. Ben KnobleMar 10, 2025
  21. Justin ToblerMar 11, 2025
  22. Jeff KingMar 11, 2025
  23. Patrick SteinhardtMar 12, 2025
  24. Junio C HamanoMar 12, 2025
  25. Patrick SteinhardtMar 13, 2025
  26. Justin ToblerMar 12, 2025
  27. Jeff KingMar 13, 2025
  28. Justin ToblerMar 13, 2025
  29. Jeff KingMar 14, 2025
  30. Junio C HamanoMar 14, 2025
  31. Jeff KingMar 14, 2025
  32. Justin ToblerMar 14, 2025
  33. Junio C HamanoMar 14, 2025
  34. Justin ToblerMar 19, 2025
  35. 0/6 rev-list: introduce NUL-delimited output modeJustin Tobler, Mar 13, 2025
  36. 1/6 rev-list: inline `show_object_with_name()` in `show_object()`Justin Tobler, Mar 13, 2025
  37. 2/6 rev-list: refactor early option parsingJustin Tobler, Mar 13, 2025
  38. 3/6 revision: support NUL-delimited --stdin modeJustin Tobler, Mar 13, 2025
  39. 4/6 rev-list: support delimiting objects with NUL bytesJustin Tobler, Mar 13, 2025
  40. Patrick SteinhardtMar 13, 2025
  41. Justin ToblerMar 13, 2025
  42. 5/6 rev-list: support NUL-delimited --boundary optionJustin Tobler, Mar 13, 2025
  43. 6/6 rev-list: support NUL-delimited --missing optionJustin Tobler, Mar 13, 2025
  44. Patrick SteinhardtMar 13, 2025
  45. Justin ToblerMar 13, 2025
  46. 0/6 rev-list: introduce NUL-delimited output modeJustin Tobler, Mar 13, 2025
  47. 1/6 rev-list: inline `show_object_with_name()` in `show_object()`Justin Tobler, Mar 13, 2025
  48. 2/6 rev-list: refactor early option parsingJustin Tobler, Mar 13, 2025
  49. 3/6 revision: support NUL-delimited --stdin modeJustin Tobler, Mar 13, 2025
  50. 4/6 rev-list: support delimiting objects with NUL bytesJustin Tobler, Mar 13, 2025
  51. Christian CouderMar 19, 2025
  52. Justin ToblerMar 19, 2025
  53. 5/6 rev-list: support NUL-delimited --boundary optionJustin Tobler, Mar 13, 2025
  54. 6/6 rev-list: support NUL-delimited --missing optionJustin Tobler, Mar 13, 2025
  55. 0/5 rev-list: introduce NUL-delimited output modeJustin Tobler, Mar 19, 2025
  56. 1/5 rev-list: inline `show_object_with_name()` in `show_object()`Justin Tobler, Mar 19, 2025
  57. 2/5 rev-list: refactor early option parsingJustin Tobler, Mar 19, 2025
  58. 3/5 rev-list: support delimiting objects with NUL bytesJustin Tobler, Mar 19, 2025
  59. 4/5 rev-list: support NUL-delimited --boundary optionJustin Tobler, Mar 19, 2025
  60. 5/5 rev-list: support NUL-delimited --missing optionJustin Tobler, Mar 19, 2025

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.