{"thread":{"id":"63089","subject":"[PATCH 0/4] rev-list: introduce NUL-delimited output mode","startedAt":"2025-03-10T19:32:12Z","lastAt":"2025-03-19T18:37:58Z","messageCount":60,"participants":["Justin Tobler","Junio C Hamano","D. Ben Knoble","Jeff King","Patrick Steinhardt","Christian Couder"],"isPatch":true,"patchVersion":1,"patchTotal":4},"messages":[{"id":"513932","messageId":"20250310192829.661692-1-jltobler@gmail.com","threadId":"63089","inReplyTo":null,"subject":"[PATCH 0/4] rev-list: introduce NUL-delimited output mode","fromName":"Justin Tobler","fromEmail":"jltobler@gmail.com","sentAt":"2025-03-10T19:28:25Z","receivedAt":"2025-03-10T19:32:12Z","isPatch":true,"sender":{"key":"jltobler@gmail.com","avatar":"https://avatars.githubusercontent.com/u/53454972?v=4"},"body":"When walking objects, git-rev-list(1) prints each object entry on a\nseparate line in the form:\n\n LF\n\nSome options, such as `--objects`, may print additional information\nabout the object on the same line:\n\n SP [] LF\n\nIn this mode, if the object path contains a newline it is truncated at\nthe newline.\n\nWhen the `--missing={print,print-info}` option is provided, information\nabout any missing objects encountered during the object walk are also\nprinted in the form:\n\n ? [SP =]... LF\n\nwhere values containing LF or SP are printed in a token specific fashion\nso that the resulting encoded value does not contain either of these two\nproblematic bytes. For example, missing object paths are quoted in the C\nstyle so they contain LF or SP.\n\nTo make machine parsing easier, this series introduces a NUL-delimited\noutput mode for git-rev-list(1) via a `-z` option following a suggestion\nfrom Junio in a previous thread[1]. In this mode, instead of LF, each\nobject is delimited with two NUL bytes and any object metadata is\nseparated with a single NUL byte. Examples:\n\n NUL NUL\n [NUL ] NUL NUL\n ? [NUL =]... NUL NUL\n\nIn this mode, path and value info are printed as-is without any special\nencoding or truncation.\n\nFor now this series only adds support for use with the `--objects` and\n`--missing` options. Usage of `-z` with other options is rejected, so it\ncan potentially be added in the future.\n\nOne idea I had, but did not implement in this version, was to also use\nthe `=` format for regular non-missing object info while\nin the NUL-delimited mode. I could see this being a bit more flexible\ninstead of relying strictly on order. Interested if anyone has thoughts\non this. :)\n\nThis series is structured as follows:\n\n - Patches 1 and 2 do some minor preparatory refactors.\n\n - Patch 3 adds the `-z` option to git-rev-list(1) to print\n objects in a NUL-delimited fashion. Printed object paths with\n the `--objects` option are also handled.\n\n - Patch 4 teaches the `--missing` option how to print info in a\n NUL-delimited fashion.\n\nThanks for taking a look,\n-Justin\n\n[1]: \n\nJustin Tobler (4):\n rev-list: inline `show_object_with_name()` in `show_object()`\n rev-list: refactor early option parsing\n rev-list: support delimiting objects with NUL bytes\n rev-list: support NUL-delimited --missing option\n\n Documentation/rev-list-options.adoc | 26 +++++++++\n builtin/rev-list.c | 86 ++++++++++++++++++++++-------\n revision.c | 8 ---\n revision.h | 2 -\n t/t6000-rev-list-misc.sh | 34 ++++++++++++\n t/t6022-rev-list-missing.sh | 30 ++++++++++\n 6 files changed, 155 insertions(+), 31 deletions(-)\n\n\nbase-commit: 87a0bdbf0f72b7561f3cd50636eee33dcb7dbcc3\n-- \n2.49.0.rc2\n\n"},{"id":"513933","messageId":"20250310192829.661692-2-jltobler@gmail.com","threadId":"63089","inReplyTo":"20250310192829.661692-1-jltobler@gmail.com","subject":"[PATCH 1/4] rev-list: inline `show_object_with_name()` in `show_object()`","fromName":"Justin Tobler","fromEmail":"jltobler@gmail.com","sentAt":"2025-03-10T19:28:26Z","receivedAt":"2025-03-10T19:32:13Z","isPatch":true,"sender":{"key":"jltobler@gmail.com","avatar":"https://avatars.githubusercontent.com/u/53454972?v=4"},"body":"The `show_object_with_name()` function only has a single call site.\nInline call to `show_object_with_name()` in `show_object()` so the\nexplicit function can be cleaned up and live closer to where it is used.\nWhile at it, factor out the code that prints the OID and newline for\nboth objects with and without a name. In a subsequent commit,\n`show_object()` is modified to support printing object information in a\nNUL-delimited format.\n\nSigned-off-by: Justin Tobler \n---\n builtin/rev-list.c | 13 +++++++++----\n revision.c | 8 --------\n revision.h | 2 --\n 3 files changed, 9 insertions(+), 14 deletions(-)\n\ndiff --git a/builtin/rev-list.c b/builtin/rev-list.c\nindex bb26bee0d4..dcd079c16c 100644\n--- a/builtin/rev-list.c\n+++ b/builtin/rev-list.c\n@@ -357,10 +357,15 @@ static void show_object(struct object *obj, const char *name, void *cb_data)\n \t\treturn;\n \t}\n \n-\tif (arg_show_object_names)\n-\t\tshow_object_with_name(stdout, obj, name);\n-\telse\n-\t\tprintf(\"%s\\n\", oid_to_hex(&obj->oid));\n+\tprintf(\"%s\", oid_to_hex(&obj->oid));\n+\n+\tif (arg_show_object_names) {\n+\t\tputchar(' ');\n+\t\tfor (const char *p = name; *p && *p != '\\n'; p++)\n+\t\t\tputchar(*p);\n+\t}\n+\n+\tputchar('\\n');\n }\n \n static void show_edge(struct commit *commit)\ndiff --git a/revision.c b/revision.c\nindex c4390f0938..0eaebe4478 100644\n--- a/revision.c\n+++ b/revision.c\n@@ -59,14 +59,6 @@ implement_shared_commit_slab(revision_sources, char *);\n \n static inline int want_ancestry(const struct rev_info *revs);\n \n-void show_object_with_name(FILE *out, struct object *obj, const char *name)\n-{\n-\tfprintf(out, \"%s \", oid_to_hex(&obj->oid));\n-\tfor (const char *p = name; *p && *p != '\\n'; p++)\n-\t\tfputc(*p, out);\n-\tfputc('\\n', out);\n-}\n-\n static void mark_blob_uninteresting(struct blob *blob)\n {\n \tif (!blob)\ndiff --git a/revision.h b/revision.h\nindex 71e984c452..21c6a69899 100644\n--- a/revision.h\n+++ b/revision.h\n@@ -489,8 +489,6 @@ void mark_parents_uninteresting(struct rev_info *revs, struct commit *commit);\n void mark_tree_uninteresting(struct repository *r, struct tree *tree);\n void mark_trees_uninteresting_sparse(struct repository *r, struct oidset *trees);\n \n-void show_object_with_name(FILE *, struct object *, const char *);\n-\n /**\n * Helpers to check if a reference should be excluded.\n */\n-- \n2.49.0.rc2\n\n"},{"id":"513934","messageId":"20250310192829.661692-3-jltobler@gmail.com","threadId":"63089","inReplyTo":"20250310192829.661692-1-jltobler@gmail.com","subject":"[PATCH 2/4] rev-list: refactor early option parsing","fromName":"Justin Tobler","fromEmail":"jltobler@gmail.com","sentAt":"2025-03-10T19:28:27Z","receivedAt":"2025-03-10T19:32:14Z","isPatch":true,"sender":{"key":"jltobler@gmail.com","avatar":"https://avatars.githubusercontent.com/u/53454972?v=4"},"body":"Before invoking `setup_revisions()`, the `--missing` and\n`--exclude-promisor-objects` options are parsed early. In a subsequent\ncommit, another option is added that must be parsed early.\n\nRefactor the code to parse both options in a single early pass.\n\nSigned-off-by: Justin Tobler \n---\n builtin/rev-list.c | 17 +++++++----------\n 1 file changed, 7 insertions(+), 10 deletions(-)\n\ndiff --git a/builtin/rev-list.c b/builtin/rev-list.c\nindex dcd079c16c..04d9c893b5 100644\n--- a/builtin/rev-list.c\n+++ b/builtin/rev-list.c\n@@ -16,6 +16,7 @@\n #include \"object-file.h\"\n #include \"object-store-ll.h\"\n #include \"pack-bitmap.h\"\n+#include \"parse-options.h\"\n #include \"log-tree.h\"\n #include \"graph.h\"\n #include \"bisect.h\"\n@@ -639,19 +640,15 @@ int cmd_rev_list(int argc,\n \t\tif (!strcmp(arg, \"--exclude-promisor-objects\")) {\n \t\t\tfetch_if_missing = 0;\n \t\t\trevs.exclude_promisor_objects = 1;\n-\t\t\tbreak;\n-\t\t}\n-\t}\n-\tfor (i = 1; i < argc; i++) {\n-\t\tconst char *arg = argv[i];\n-\t\tif (skip_prefix(arg, \"--missing=\", &arg)) {\n-\t\t\tif (revs.exclude_promisor_objects)\n-\t\t\t\tdie(_(\"options '%s' and '%s' cannot be used together\"), \"--exclude-promisor-objects\", \"--missing\");\n-\t\t\tif (parse_missing_action_value(arg))\n-\t\t\t\tbreak;\n+\t\t} else if (skip_prefix(arg, \"--missing=\", &arg)) {\n+\t\t\tparse_missing_action_value(arg);\n \t\t}\n \t}\n \n+\tdie_for_incompatible_opt2(revs.exclude_promisor_objects,\n+\t\t\t\t \"--exclude_promisor_objects\",\n+\t\t\t\t arg_missing_action, \"--missing\");\n+\n \tif (arg_missing_action)\n \t\trevs.do_not_die_on_missing_objects = 1;\n \n-- \n2.49.0.rc2\n\n"},{"id":"513935","messageId":"20250310192829.661692-4-jltobler@gmail.com","threadId":"63089","inReplyTo":"20250310192829.661692-1-jltobler@gmail.com","subject":"[PATCH 3/4] rev-list: support delimiting objects with NUL bytes","fromName":"Justin Tobler","fromEmail":"jltobler@gmail.com","sentAt":"2025-03-10T19:28:28Z","receivedAt":"2025-03-10T19:32:16Z","isPatch":true,"sender":{"key":"jltobler@gmail.com","avatar":"https://avatars.githubusercontent.com/u/53454972?v=4"},"body":"When walking objects, git-rev-list(1) prints each object entry on a\nseparate line. Some options, such as `--objects`, may print additional\ninformation about tree and blob object on the same line in the form:\n\n $ git rev-list --objects \n SP [] LF\n\nNote that in this form the SP is appended regardless of whether the tree\nor blob object has path information available. Paths containing a\nnewline are also truncated at the newline.\n\nIntroduce the `-z` option for git-rev-list(1) which reformats the output\nto use NUL-delimiters between objects and associated info. Each object\nline uses two NUL bytes to indicate the end of an object entry and a\nsingle NUL byte to delimit between object information in the following\nform:\n\n $ git rev-list -z --objects \n [NUL ] NUL NUL\n\nFor now, the `--objects` flag is the only option that can be used in\ncombination with `-z`. In this mode, the object path is not truncated at\nnewlines. In a subsequent commit, NUL-delimiter support for other\noptions is added. Other options that do not make sense with be used in\ncombination with `-z` are rejected.\n\nSigned-off-by: Justin Tobler \n---\n Documentation/rev-list-options.adoc | 18 +++++++++++++\n builtin/rev-list.c | 39 +++++++++++++++++++++++++----\n t/t6000-rev-list-misc.sh | 34 +++++++++++++++++++++++++\n 3 files changed, 86 insertions(+), 5 deletions(-)\n\ndiff --git a/Documentation/rev-list-options.adoc b/Documentation/rev-list-options.adoc\nindex 785c0786e0..d21016d657 100644\n--- a/Documentation/rev-list-options.adoc\n+++ b/Documentation/rev-list-options.adoc\n@@ -361,6 +361,24 @@ ifdef::git-rev-list[]\n --progress=
::\n \tShow progress reports on stderr as objects are considered. The\n \t`
` text will be printed with each progress update.\n+\n+-z::\n+\tInstead of being newline-delimited, each outputted object is delimited\n+\twith two NUL bytes in the following form:\n++\n+-----------------------------------------------------------------------\n+ NUL NUL\n+-----------------------------------------------------------------------\n++\n+When the `--objects` option is also present, available object name information\n+is printed in the following form without any truncation for object names\n+containing newline characters:\n++\n+-----------------------------------------------------------------------\n+ [NUL ] NUL NUL\n+-----------------------------------------------------------------------\n++\n+This option is only compatible with `--objects`.\n endif::git-rev-list[]\n \n History Simplification\ndiff --git a/builtin/rev-list.c b/builtin/rev-list.c\nindex 04d9c893b5..86b3ce5806 100644\n--- a/builtin/rev-list.c\n+++ b/builtin/rev-list.c\n@@ -65,6 +65,7 @@ static const char rev_list_usage[] =\n \" --abbrev-commit\\n\"\n \" --left-right\\n\"\n \" --count\\n\"\n+\" -z\\n\"\n \" special purpose:\\n\"\n \" --bisect\\n\"\n \" --bisect-vars\\n\"\n@@ -97,10 +98,23 @@ static int arg_show_object_names = 1;\n \n #define DEFAULT_OIDSET_SIZE (16*1024)\n \n+static int nul_delim;\n static int show_disk_usage;\n static off_t total_disk_usage;\n static int human_readable;\n \n+static void print_object_term(int nul_delim)\n+{\n+\tchar line_sep = '\\n';\n+\n+\tif (nul_delim)\n+\t\tline_sep = '\\0';\n+\n+\tputchar(line_sep);\n+\tif (nul_delim)\n+\t\tputchar(line_sep);\n+}\n+\n static off_t get_object_disk_usage(struct object *obj)\n {\n \toff_t size;\n@@ -264,7 +278,7 @@ static void show_commit(struct commit *commit, void *data)\n \tif (revs->commit_format == CMIT_FMT_ONELINE)\n \t\tputchar(' ');\n \telse if (revs->include_header)\n-\t\tputchar('\\n');\n+\t\tprint_object_term(nul_delim);\n \n \tif (revs->verbose_header) {\n \t\tstruct strbuf buf = STRBUF_INIT;\n@@ -361,12 +375,17 @@ static void show_object(struct object *obj, const char *name, void *cb_data)\n \tprintf(\"%s\", oid_to_hex(&obj->oid));\n \n \tif (arg_show_object_names) {\n-\t\tputchar(' ');\n-\t\tfor (const char *p = name; *p && *p != '\\n'; p++)\n-\t\t\tputchar(*p);\n+\t\tif (nul_delim && *name) {\n+\t\t\tputchar('\\0');\n+\t\t\tprintf(\"%s\", name);\n+\t\t} else if (!nul_delim) {\n+\t\t\tputchar(' ');\n+\t\t\tfor (const char *p = name; *p && *p != '\\n'; p++)\n+\t\t\t\tputchar(*p);\n+\t\t}\n \t}\n \n-\tputchar('\\n');\n+\tprint_object_term(nul_delim);\n }\n \n static void show_edge(struct commit *commit)\n@@ -642,6 +661,8 @@ int cmd_rev_list(int argc,\n \t\t\trevs.exclude_promisor_objects = 1;\n \t\t} else if (skip_prefix(arg, \"--missing=\", &arg)) {\n \t\t\tparse_missing_action_value(arg);\n+\t\t} else if (!strcmp(arg, \"-z\")) {\n+\t\t\tnul_delim = 1;\n \t\t}\n \t}\n \n@@ -757,6 +778,14 @@ int cmd_rev_list(int argc,\n \t\tusage(rev_list_usage);\n \n \t}\n+\n+\tif (nul_delim) {\n+\t\tif (revs.graph || revs.verbose_header || show_disk_usage ||\n+\t\t info.show_timestamp || info.header_prefix || bisect_list ||\n+\t\t use_bitmap_index || revs.edge_hint || arg_missing_action)\n+\t\t\tdie(_(\"-z option used with unsupported option\"));\n+\t}\n+\n \tif (revs.commit_format != CMIT_FMT_USERFORMAT)\n \t\trevs.include_header = 1;\n \tif (revs.commit_format != CMIT_FMT_UNSPECIFIED) {\ndiff --git a/t/t6000-rev-list-misc.sh b/t/t6000-rev-list-misc.sh\nindex 6289a2e8b0..25c2f2f238 100755\n--- a/t/t6000-rev-list-misc.sh\n+++ b/t/t6000-rev-list-misc.sh\n@@ -182,4 +182,38 @@ test_expect_success 'rev-list --unpacked' '\n \ttest_cmp expect actual\n '\n \n+test_expect_success 'rev-list -z' '\n+\ttest_when_finished rm -rf repo &&\n+\n+\tgit init repo &&\n+\ttest_commit -C repo 1 &&\n+\ttest_commit -C repo 2 &&\n+\n+\toid1=$(git -C repo rev-parse HEAD) &&\n+\toid2=$(git -C repo rev-parse HEAD~) &&\n+\n+\tprintf \"%s\\0\\0%s\\0\\0\" \"$oid1\" \"$oid2\" >expect &&\n+\tgit -C repo rev-list -z HEAD >actual &&\n+\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'rev-list -z --objects' '\n+\ttest_when_finished rm -rf repo &&\n+\n+\tgit init repo &&\n+\ttest_commit -C repo 1 &&\n+\ttest_commit -C repo 2 &&\n+\n+\toid1=$(git -C repo rev-parse HEAD:1.t) &&\n+\toid2=$(git -C repo rev-parse HEAD:2.t) &&\n+\tpath1=1.t &&\n+\tpath2=2.t &&\n+\n+\tprintf \"%s\\0%s\\0\\0%s\\0%s\\0\\0\" \"$oid1\" \"$path1\" \"$oid2\" \"$path2\" >expect &&\n+\tgit -C repo rev-list -z --objects HEAD:1.t HEAD:2.t >actual &&\n+\n+\ttest_cmp expect actual\n+'\n+\n test_done\n-- \n2.49.0.rc2\n\n"},{"id":"513936","messageId":"20250310192829.661692-5-jltobler@gmail.com","threadId":"63089","inReplyTo":"20250310192829.661692-1-jltobler@gmail.com","subject":"[PATCH 4/4] rev-list: support NUL-delimited --missing option","fromName":"Justin Tobler","fromEmail":"jltobler@gmail.com","sentAt":"2025-03-10T19:28:29Z","receivedAt":"2025-03-10T19:32:17Z","isPatch":true,"sender":{"key":"jltobler@gmail.com","avatar":"https://avatars.githubusercontent.com/u/53454972?v=4"},"body":"The `--missing={print,print-info}` option for git-rev-list(1) prints\nmissing objects found while performing the revision walk. Add support\nfor printing missing objects in a NUL-delimited format when the `-z`\noption is enabled.\n\n $ git rev-list -z --missing=print-info \n NUL NUL\n ? [NUL =]... NUL NUL\n\nIn this mode, values containing special characters or spaces are printed\nas-is without being escaped or quoted.\n\nSigned-off-by: Justin Tobler \n---\n Documentation/rev-list-options.adoc | 10 +++++++++-\n builtin/rev-list.c | 27 +++++++++++++++++++-------\n t/t6022-rev-list-missing.sh | 30 +++++++++++++++++++++++++++++\n 3 files changed, 59 insertions(+), 8 deletions(-)\n\ndiff --git a/Documentation/rev-list-options.adoc b/Documentation/rev-list-options.adoc\nindex d21016d657..48648b7507 100644\n--- a/Documentation/rev-list-options.adoc\n+++ b/Documentation/rev-list-options.adoc\n@@ -378,7 +378,15 @@ containing newline characters:\n [NUL ] NUL NUL\n -----------------------------------------------------------------------\n +\n-This option is only compatible with `--objects`.\n+When the `--missing` option is provided, missing objects are printed in the\n+following form where value is printed as-is without any token specific\n+encoding:\n++\n+-----------------------------------------------------------------------\n+? [NUL =]... NUL NUL\n+-----------------------------------------------------------------------\n++\n+This option is only compatible with `--objects` and `--missing`.\n endif::git-rev-list[]\n \n History Simplification\ndiff --git a/builtin/rev-list.c b/builtin/rev-list.c\nindex 86b3ce5806..5bbc4a787e 100644\n--- a/builtin/rev-list.c\n+++ b/builtin/rev-list.c\n@@ -145,25 +145,38 @@ static void print_missing_object(struct missing_objects_map_entry *entry,\n \t\t\t\t int print_missing_info)\n {\n \tstruct strbuf sb = STRBUF_INIT;\n+\tchar info_sep = ' ';\n+\n+\tif (nul_delim)\n+\t\tinfo_sep = '\\0';\n+\n+\tprintf(\"?%s\", oid_to_hex(&entry->entry.oid));\n \n \tif (!print_missing_info) {\n-\t\tprintf(\"?%s\\n\", oid_to_hex(&entry->entry.oid));\n+\t\tprint_object_term(nul_delim);\n \t\treturn;\n \t}\n \n \tif (entry->path && *entry->path) {\n \t\tstruct strbuf path = STRBUF_INIT;\n \n-\t\tstrbuf_addstr(&sb, \" path=\");\n-\t\tquote_path(entry->path, NULL, &path, QUOTE_PATH_QUOTE_SP);\n-\t\tstrbuf_addbuf(&sb, &path);\n+\t\tstrbuf_addf(&sb, \"%cpath=\", info_sep);\n+\n+\t\tif (nul_delim) {\n+\t\t\tstrbuf_addstr(&sb, entry->path);\n+\t\t} else {\n+\t\t\tquote_path(entry->path, NULL, &path, QUOTE_PATH_QUOTE_SP);\n+\t\t\tstrbuf_addbuf(&sb, &path);\n+\t\t}\n \n \t\tstrbuf_release(&path);\n \t}\n \tif (entry->type)\n-\t\tstrbuf_addf(&sb, \" type=%s\", type_name(entry->type));\n+\t\tstrbuf_addf(&sb, \"%ctype=%s\", info_sep, type_name(entry->type));\n+\n+\tfwrite(sb.buf, sizeof(char), sb.len, stdout);\n+\tprint_object_term(nul_delim);\n \n-\tprintf(\"?%s%s\\n\", oid_to_hex(&entry->entry.oid), sb.buf);\n \tstrbuf_release(&sb);\n }\n \n@@ -782,7 +795,7 @@ int cmd_rev_list(int argc,\n \tif (nul_delim) {\n \t\tif (revs.graph || revs.verbose_header || show_disk_usage ||\n \t\t info.show_timestamp || info.header_prefix || bisect_list ||\n-\t\t use_bitmap_index || revs.edge_hint || arg_missing_action)\n+\t\t use_bitmap_index || revs.edge_hint)\n \t\t\tdie(_(\"-z option used with unsupported option\"));\n \t}\n \ndiff --git a/t/t6022-rev-list-missing.sh b/t/t6022-rev-list-missing.sh\nindex 3e2790d4c8..3ae25e4cfb 100755\n--- a/t/t6022-rev-list-missing.sh\n+++ b/t/t6022-rev-list-missing.sh\n@@ -198,4 +198,34 @@ do\n \t'\n done\n \n+test_expect_success \"-z nul-delimited --missing\" '\n+\ttest_when_finished rm -rf repo &&\n+\n+\tgit init repo &&\n+\t(\n+\t\tcd repo &&\n+\t\tgit commit --allow-empty -m first &&\n+\n+\t\tpath=\"foo bar\" &&\n+\t\techo foobar >\"$path\" &&\n+\t\tgit add -A &&\n+\t\tgit commit -m second &&\n+\n+\t\toid=$(git rev-parse \"HEAD:$path\") &&\n+\t\ttype=\"$(git cat-file -t $oid)\" &&\n+\n+\t\tobj_path=\".git/objects/$(test_oid_to_path $oid)\" &&\n+\n+\t\tgit rev-list -z --objects --no-object-names \\\n+\t\t\tHEAD ^\"$oid\" >expect &&\n+\t\tprintf \"?%s\\0path=%s\\0type=%s\\0\\0\" \"$oid\" \"$path\" \"$type\" >>expect &&\n+\n+\t\tmv \"$obj_path\" \"$obj_path.hidden\" &&\n+\t\tgit rev-list -z --objects --no-object-names \\\n+\t\t\t--missing=print-info HEAD >actual &&\n+\n+\t\ttest_cmp expect actual\n+\t)\n+'\n+\n test_done\n-- \n2.49.0.rc2\n\n"},{"id":"513937","messageId":"xmqq34fk7hb7.fsf@gitster.g","threadId":"63089","inReplyTo":"20250310192829.661692-1-jltobler@gmail.com","subject":"Re: [PATCH 0/4] rev-list: introduce NUL-delimited output mode","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-03-10T20:37:48Z","receivedAt":"2025-03-10T20:37:51Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Justin Tobler writes:\n\n> ? [SP =]... LF\n>\n> where values containing LF or SP are printed in a token specific fashion\n> so that the resulting encoded value does not contain either of these two\n> problematic bytes. For example, missing object paths are quoted in the C\n> style so they contain LF or SP.\n\n\"so\" -> \"when\"???\n\n> To make machine parsing easier, this series introduces a NUL-delimited\n> output mode for git-rev-list(1) via a `-z` option following a suggestion\n> from Junio in a previous thread[1]. In this mode, instead of LF, each\n> object is delimited with two NUL bytes and any object metadata is\n> separated with a single NUL byte. Examples:\n>\n> NUL NUL\n> [NUL ] NUL NUL\n\nWhy do we need double-NUL in the above two cases?\n\n> ? [NUL =]... NUL NUL\n\nThis one I understand; we could do without double-NUL and take the\nlack of \"=\" in the token after NUL termination as the sign that the\nprevious record ended, though, to avoid double-NUL while keeping the\nformat extensible.\n\nAs this topic is designing essentially a new and machine parseable\nformat, we could even unify all three formats into one. For example,\nthe format could be like this:\n\n\t NUL [= NUL]...\n\nwhere\n\n (1) A record ends when a new record begins.\n\n (2) The beginning of a new record is signaled by that is all\n hexadecimal and does not have any '=' in it.\n\n (3) The traditional \"rev-list --objects\" output that gives path in\n addition to the object name uses \"path\" as the name,\n i.e. such a record looks like \" NUL path= NUL\".\n\n (4) The traditional \"rev-list --missing\" output loses the leading\n \"?\"; it is replaced by \"missing\" as the name, i.e. such\n a record may look like \" NUL missing=yes NUL...\" together\n with other \"= NUL\" pairs appended as needed at\n the end.\n\nHmm?\n"},{"id":"513938","messageId":"xmqqy0xc623q.fsf@gitster.g","threadId":"63089","inReplyTo":"20250310192829.661692-2-jltobler@gmail.com","subject":"Re: [PATCH 1/4] rev-list: inline `show_object_with_name()` in `show_object()`","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-03-10T20:51:37Z","receivedAt":"2025-03-10T20:51:39Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Justin Tobler writes:\n\n> The `show_object_with_name()` function only has a single call site.\n> Inline call to `show_object_with_name()` in `show_object()` so the\n> explicit function can be cleaned up and live closer to where it is used.\n> While at it, factor out the code that prints the OID and newline for\n> both objects with and without a name. In a subsequent commit,\n> `show_object()` is modified to support printing object information in a\n> NUL-delimited format.\n>\n> Signed-off-by: Justin Tobler \n> ---\n> builtin/rev-list.c | 13 +++++++++----\n> revision.c | 8 --------\n> revision.h | 2 --\n> 3 files changed, 9 insertions(+), 14 deletions(-)\n>\n> diff --git a/builtin/rev-list.c b/builtin/rev-list.c\n> index bb26bee0d4..dcd079c16c 100644\n> --- a/builtin/rev-list.c\n> +++ b/builtin/rev-list.c\n> @@ -357,10 +357,15 @@ static void show_object(struct object *obj, const char *name, void *cb_data)\n> \t\treturn;\n> \t}\n> \n> -\tif (arg_show_object_names)\n> -\t\tshow_object_with_name(stdout, obj, name);\n> -\telse\n> -\t\tprintf(\"%s\\n\", oid_to_hex(&obj->oid));\n> +\tprintf(\"%s\", oid_to_hex(&obj->oid));\n> +\n> +\tif (arg_show_object_names) {\n> +\t\tputchar(' ');\n> +\t\tfor (const char *p = name; *p && *p != '\\n'; p++)\n> +\t\t\tputchar(*p);\n> +\t}\n> +\n> +\tputchar('\\n');\n> }\n\nMakes sense.\n"},{"id":"513940","messageId":"xmqqr03461z7.fsf@gitster.g","threadId":"63089","inReplyTo":"20250310192829.661692-3-jltobler@gmail.com","subject":"Re: [PATCH 2/4] rev-list: refactor early option parsing","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-03-10T20:54:20Z","receivedAt":"2025-03-10T20:54:23Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Justin Tobler writes:\n\n> @@ -639,19 +640,15 @@ int cmd_rev_list(int argc,\n> \t\tif (!strcmp(arg, \"--exclude-promisor-objects\")) {\n> \t\t\tfetch_if_missing = 0;\n> \t\t\trevs.exclude_promisor_objects = 1;\n> -\t\t\tbreak;\n> -\t\t}\n> -\t}\n> -\tfor (i = 1; i < argc; i++) {\n> -\t\tconst char *arg = argv[i];\n> -\t\tif (skip_prefix(arg, \"--missing=\", &arg)) {\n> -\t\t\tif (revs.exclude_promisor_objects)\n> -\t\t\t\tdie(_(\"options '%s' and '%s' cannot be used together\"), \"--exclude-promisor-objects\", \"--missing\");\n> -\t\t\tif (parse_missing_action_value(arg))\n> -\t\t\t\tbreak;\n> +\t\t} else if (skip_prefix(arg, \"--missing=\", &arg)) {\n> +\t\t\tparse_missing_action_value(arg);\n> \t\t}\n> \t}\n\nThere is a huge NEEDSWORK comment that essentially says that the\nabove two loops that this patch combines into one is fundamentally\nbroken. I suspect that the remaining two patches in this series\nwould punt and not improve them, but offhand I am not sure if this\nchange is making it harder to fix them properly easier or harder.\n"},{"id":"513941","messageId":"xmqqmsds61pw.fsf@gitster.g","threadId":"63089","inReplyTo":"20250310192829.661692-4-jltobler@gmail.com","subject":"Re: [PATCH 3/4] rev-list: support delimiting objects with NUL bytes","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-03-10T20:59:55Z","receivedAt":"2025-03-10T20:59:58Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Justin Tobler writes:\n\n> +-z::\n> +\tInstead of being newline-delimited, each outputted object is delimited\n> +\twith two NUL bytes in the following form:\n> ++\n> +-----------------------------------------------------------------------\n> + NUL NUL\n> +-----------------------------------------------------------------------\n> ++\n> +When the `--objects` option is also present, available object name information\n> +is printed in the following form without any truncation for object names\n> +containing newline characters:\n> ++\n> +-----------------------------------------------------------------------\n> + [NUL ] NUL NUL\n> +-----------------------------------------------------------------------\n> ++\n> +This option is only compatible with `--objects`.\n> endif::git-rev-list[]\n\nAs I said, I do not think we strongly want double-NUL and would\nprefer to do away with a single NUL if possible.\n\n> +static int nul_delim;\n> static int show_disk_usage;\n> static off_t total_disk_usage;\n> static int human_readable;\n> \n> +static void print_object_term(int nul_delim)\n> +{\n> +\tchar line_sep = '\\n';\n> +\n> +\tif (nul_delim)\n> +\t\tline_sep = '\\0';\n> +\n> +\tputchar(line_sep);\n> +\tif (nul_delim)\n> +\t\tputchar(line_sep);\n> +}\n\nThis looks, to put it mildly, strange. The concept of the line\ndelimiter byte (which can take any single byte) is wider than having\na NUL as the line delimiter byte. Why would we even want both?\n\nIOW, wouldn't it make more sense to have line_delim as the global\n(or per-invocation parameter to this function) and have\nprint_object_term() just use it? If you want to make it behave\ndifferently only when line-delimter is NUL (which I do not\nrecommend), you can switch on the value of line_delimiter being NUL.\nSo I do not see a merit in having two separate variables (except for\nconfusing future readers).\n\n"},{"id":"513943","messageId":"xmqqfrjk61bb.fsf@gitster.g","threadId":"63089","inReplyTo":"xmqq34fk7hb7.fsf@gitster.g","subject":"Re: [PATCH 0/4] rev-list: introduce NUL-delimited output mode","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-03-10T21:08:40Z","receivedAt":"2025-03-10T21:08:43Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Junio C Hamano writes:\n\n> As this topic is designing essentially a new and machine parseable\n> format, we could even unify all three formats into one. For example,\n> the format could be like this:\n>\n> \t NUL [= NUL]...\n>\n> where\n\n(0) \"rev-list\" that gives only a sequence of \"\" for commit-ish,\n as well as \"rev-list --boundary\", would fall out as a natural\n consequence. Bog-standard \"list of commits\" would see a\n sequence of \" NUL\", while a boundary object would see\n \" NUL boundary=yes NUL\".\n\n> (1) A record ends when a new record begins.\n>\n> (2) The beginning of a new record is signaled by that is all\n> hexadecimal and does not have any '=' in it.\n>\n> (3) The traditional \"rev-list --objects\" output that gives path in\n> addition to the object name uses \"path\" as the name,\n> i.e. such a record looks like \" NUL path= NUL\".\n>\n> (4) The traditional \"rev-list --missing\" output loses the leading\n> \"?\"; it is replaced by \"missing\" as the name, i.e. such\n> a record may look like \" NUL missing=yes NUL...\" together\n> with other \"= NUL\" pairs appended as needed at\n> the end.\n\n"},{"id":"513951","messageId":"CALnO6CBh_pYiU0p6pSndJ8rdDiykqBkRgosyrO1QhaQ55BKrkQ@mail.gmail.com","threadId":"63089","inReplyTo":"20250310192829.661692-1-jltobler@gmail.com","subject":"Re: [PATCH 0/4] rev-list: introduce NUL-delimited output mode","fromName":"D. Ben Knoble","fromEmail":"ben.knoble@gmail.com","sentAt":"2025-03-10T22:38:54Z","receivedAt":"2025-03-10T22:39:08Z","isPatch":true,"sender":{"key":"ben.knoble@gmail.com","avatar":"https://avatars.githubusercontent.com/u/22802209?v=4"},"body":"On Mon, Mar 10, 2025 at 3:32 PM Justin Tobler wrote:\n>\n> When walking objects, git-rev-list(1) prints each object entry on a\n> separate line in the form:\n>\n> LF\n>\n> Some options, such as `--objects`, may print additional information\n> about the object on the same line:\n>\n> SP [] LF\n>\n> In this mode, if the object path contains a newline it is truncated at\n> the newline.\n>\n> When the `--missing={print,print-info}` option is provided, information\n> about any missing objects encountered during the object walk are also\n> printed in the form:\n>\n> ? [SP =]... LF\n>\n> where values containing LF or SP are printed in a token specific fashion\n> so that the resulting encoded value does not contain either of these two\n> problematic bytes. For example, missing object paths are quoted in the C\n> style so they contain LF or SP.\n>\n> To make machine parsing easier, this series introduces a NUL-delimited\n> output mode for git-rev-list(1) via a `-z` option following a suggestion\n> from Junio in a previous thread[1]. In this mode, instead of LF, each\n> object is delimited with two NUL bytes and any object metadata is\n> separated with a single NUL byte. Examples:\n>\n> NUL NUL\n> [NUL ] NUL NUL\n> ? [NUL =]... NUL NUL\n>\n> In this mode, path and value info are printed as-is without any special\n> encoding or truncation.\n>\n> For now this series only adds support for use with the `--objects` and\n> `--missing` options. Usage of `-z` with other options is rejected, so it\n> can potentially be added in the future.\n>\n> One idea I had, but did not implement in this version, was to also use\n> the `=` format for regular non-missing object info while\n> in the NUL-delimited mode. I could see this being a bit more flexible\n> instead of relying strictly on order. Interested if anyone has thoughts\n> on this. :)\n\nWithout taking a deeper look, I think token=value has the benefit of\nbeing self-describing at the cost of more output bytes (which might\nmatter over the wire, for example). Generally I like the idea;\nsometimes I find it troublesome having to parse prose manuals for the\nspecifics of output formats like field order, especially when I end up\ncoding a parser for the format. If the field order doesn’t matter to\nthe consumer, then perhaps using ordered fields AWK-style is\ninappropriately terse?\n\nOTOH, the -z format is for machines, and they don’t need human labels\n;) [I think token labels would be a great parser-writing and debugging\naid]\n\nBest,\nBen\n\n>\n> This series is structured as follows:\n>\n> - Patches 1 and 2 do some minor preparatory refactors.\n>\n> - Patch 3 adds the `-z` option to git-rev-list(1) to print\n> objects in a NUL-delimited fashion. Printed object paths with\n> the `--objects` option are also handled.\n>\n> - Patch 4 teaches the `--missing` option how to print info in a\n> NUL-delimited fashion.\n>\n> Thanks for taking a look,\n> -Justin\n>\n> [1]: \n>\n> Justin Tobler (4):\n> rev-list: inline `show_object_with_name()` in `show_object()`\n> rev-list: refactor early option parsing\n> rev-list: support delimiting objects with NUL bytes\n> rev-list: support NUL-delimited --missing option\n>\n> Documentation/rev-list-options.adoc | 26 +++++++++\n> builtin/rev-list.c | 86 ++++++++++++++++++++++-------\n> revision.c | 8 ---\n> revision.h | 2 -\n> t/t6000-rev-list-misc.sh | 34 ++++++++++++\n> t/t6022-rev-list-missing.sh | 30 ++++++++++\n> 6 files changed, 155 insertions(+), 31 deletions(-)\n>\n>\n> base-commit: 87a0bdbf0f72b7561f3cd50636eee33dcb7dbcc3\n> --\n> 2.49.0.rc2\n>\n>\n\n\n-- \nD. Ben Knoble\n"},{"id":"514010","messageId":"gc4fnq2s57jle5zig4iz5fp3hu6juylp4g3dberzqjfuaszxl3@hucjve73hcbe","threadId":"63089","inReplyTo":"CALnO6CBh_pYiU0p6pSndJ8rdDiykqBkRgosyrO1QhaQ55BKrkQ@mail.gmail.com","subject":"Re: [PATCH 0/4] rev-list: introduce NUL-delimited output mode","fromName":"Justin Tobler","fromEmail":"jltobler@gmail.com","sentAt":"2025-03-11T22:59:23Z","receivedAt":"2025-03-11T23:02:51Z","isPatch":true,"sender":{"key":"jltobler@gmail.com","avatar":"https://avatars.githubusercontent.com/u/53454972?v=4"},"body":"On 25/03/10 06:38PM, D. Ben Knoble wrote:\n> On Mon, Mar 10, 2025 at 3:32 PM Justin Tobler wrote:\n> >\n> > When walking objects, git-rev-list(1) prints each object entry on a\n> > separate line in the form:\n> >\n> > LF\n> >\n> > Some options, such as `--objects`, may print additional information\n> > about the object on the same line:\n> >\n> > SP [] LF\n> >\n> > In this mode, if the object path contains a newline it is truncated at\n> > the newline.\n> >\n> > When the `--missing={print,print-info}` option is provided, information\n> > about any missing objects encountered during the object walk are also\n> > printed in the form:\n> >\n> > ? [SP =]... LF\n> >\n> > where values containing LF or SP are printed in a token specific fashion\n> > so that the resulting encoded value does not contain either of these two\n> > problematic bytes. For example, missing object paths are quoted in the C\n> > style so they contain LF or SP.\n> >\n> > To make machine parsing easier, this series introduces a NUL-delimited\n> > output mode for git-rev-list(1) via a `-z` option following a suggestion\n> > from Junio in a previous thread[1]. In this mode, instead of LF, each\n> > object is delimited with two NUL bytes and any object metadata is\n> > separated with a single NUL byte. Examples:\n> >\n> > NUL NUL\n> > [NUL ] NUL NUL\n> > ? [NUL =]... NUL NUL\n> >\n> > In this mode, path and value info are printed as-is without any special\n> > encoding or truncation.\n> >\n> > For now this series only adds support for use with the `--objects` and\n> > `--missing` options. Usage of `-z` with other options is rejected, so it\n> > can potentially be added in the future.\n> >\n> > One idea I had, but did not implement in this version, was to also use\n> > the `=` format for regular non-missing object info while\n> > in the NUL-delimited mode. I could see this being a bit more flexible\n> > instead of relying strictly on order. Interested if anyone has thoughts\n> > on this. :)\n> \n> Without taking a deeper look, I think token=value has the benefit of\n> being self-describing at the cost of more output bytes (which might\n> matter over the wire, for example). Generally I like the idea;\n> sometimes I find it troublesome having to parse prose manuals for the\n> specifics of output formats like field order, especially when I end up\n> coding a parser for the format. If the field order doesn’t matter to\n> the consumer, then perhaps using ordered fields AWK-style is\n> inappropriately terse?\n> \n> OTOH, the -z format is for machines, and they don’t need human labels\n> ;) [I think token labels would be a great parser-writing and debugging\n> aid]\n\nOne of the challenges with parsing git-rev-list(1) is all the various\nforms it can take based on the options provided. For example:\n\n $ git rev-list --timestamp --objects --parents \n\n timestamp SP [SP ] LF (commit)\n SP [] LF (tree/blob)\n\nRelying strictly on order can be a bit tricky to parse due to how the\noutput format can change even line to line. So even for machine parsing,\nlabels may help simplify things if all object records follow something\nalong the lines of:\n\n NUL [= NUL]...\n\nAs you mentioned, this could potentially also be useful for users since\nthe attributes would be self-describing. This series is currently\nfocussed on the machine parsing side, but I think support for this mode\nin a human-readable format could be added via a separate option in the\nfuture.\n\n-Justin\n"},{"id":"514013","messageId":"nfrfyg56nq7m2bcko4v4tsfdklrafpkybc6uhqgo25swdeebh3@a7aneoylg46u","threadId":"63089","inReplyTo":"xmqq34fk7hb7.fsf@gitster.g","subject":"Re: [PATCH 0/4] rev-list: introduce NUL-delimited output mode","fromName":"Justin Tobler","fromEmail":"jltobler@gmail.com","sentAt":"2025-03-11T23:19:45Z","receivedAt":"2025-03-11T23:23:14Z","isPatch":true,"sender":{"key":"jltobler@gmail.com","avatar":"https://avatars.githubusercontent.com/u/53454972?v=4"},"body":"On 25/03/10 01:37PM, Junio C Hamano wrote:\n> Justin Tobler writes:\n> > To make machine parsing easier, this series introduces a NUL-delimited\n> > output mode for git-rev-list(1) via a `-z` option following a suggestion\n> > from Junio in a previous thread[1]. In this mode, instead of LF, each\n> > object is delimited with two NUL bytes and any object metadata is\n> > separated with a single NUL byte. Examples:\n> >\n> > NUL NUL\n> > [NUL ] NUL NUL\n> \n> Why do we need double-NUL in the above two cases?\n\nIn the ` [NUL ] NUL NUL` case, it would technically be\npossible for an object path to match an OID. The use of two NUL bytes\nsignals when the object record ends.\n\nWithout someother mechanism to know when a record starts/stops, even the\n` NUL NUL` case would need the two trailing NUL bytes to avoid\nbeing considered a potential path.\n\nIf the output format would not result in any additional object metadata\nbeing appended, we could use a single NUL byte to delimit between\nobjects in this case, but always using two NUL bytes allowed for a more\nconsistent format.\n\n> \n> > ? [NUL =]... NUL NUL\n> \n> This one I understand; we could do without double-NUL and take the\n> lack of \"=\" in the token after NUL termination as the sign that the\n> previous record ended, though, to avoid double-NUL while keeping the\n> format extensible.\n> \n> As this topic is designing essentially a new and machine parseable\n> format, we could even unify all three formats into one. For example,\n> the format could be like this:\n> \n> \t NUL [= NUL]...\n\nI was also considering something similar. This format could allow other\nobject metadata like `--timestamp` to be supported in the future with a\nmore flexible format. In the next version I'll implement a unified\nformat here.\n\n> \n> where\n> \n> (1) A record ends when a new record begins.\n> \n> (2) The beginning of a new record is signaled by that is all\n> hexadecimal and does not have any '=' in it.\n\nI think this is a good idea. By always appending printed object metadata\nin the form `=`, we know that any entry without '=' must\nbe the start of a new record. This removes the need for the two NUL\nbytes to indicate the end of a record.\n\nI'll use only a single NUL byte to delimit in the next version.\n\n> \n> (3) The traditional \"rev-list --objects\" output that gives path in\n> addition to the object name uses \"path\" as the name,\n> i.e. such a record looks like \" NUL path= NUL\".\n> \n> (4) The traditional \"rev-list --missing\" output loses the leading\n> \"?\"; it is replaced by \"missing\" as the name, i.e. such\n> a record may look like \" NUL missing=yes NUL...\" together\n> with other \"= NUL\" pairs appended as needed at\n> the end.\n\nI think this is good. Instead of prefixing missing OIDs with '?', we can\njust append another token/value pair `missing=yes`.\n\nThanks,\n-Justin\n"},{"id":"514014","messageId":"72x6ku4jylvsnqjapqlfidq6m4yvpzu5gzpweg47phzgjg35fg@laabmtehkes4","threadId":"63089","inReplyTo":"xmqqfrjk61bb.fsf@gitster.g","subject":"Re: [PATCH 0/4] rev-list: introduce NUL-delimited output mode","fromName":"Justin Tobler","fromEmail":"jltobler@gmail.com","sentAt":"2025-03-11T23:24:28Z","receivedAt":"2025-03-11T23:27:56Z","isPatch":true,"sender":{"key":"jltobler@gmail.com","avatar":"https://avatars.githubusercontent.com/u/53454972?v=4"},"body":"On 25/03/10 02:08PM, Junio C Hamano wrote:\n> Junio C Hamano writes:\n> \n> > As this topic is designing essentially a new and machine parseable\n> > format, we could even unify all three formats into one. For example,\n> > the format could be like this:\n> >\n> > \t NUL [= NUL]...\n> >\n> > where\n> \n> (0) \"rev-list\" that gives only a sequence of \"\" for commit-ish,\n> as well as \"rev-list --boundary\", would fall out as a natural\n> consequence. Bog-standard \"list of commits\" would see a\n> sequence of \" NUL\", while a boundary object would see\n> \" NUL boundary=yes NUL\".\n\nI had not considered handling the `--boundary` option. It looks like\nboundary objects are printed as part of `show_commit()`, so I can adapt\nthe handling and do something similar to missing objects:\n\n $ git rev-list -z --boundary \n NUL boundary=yes NUL\n\nThis would remain consistent with the unified format.\n\n-Justin\n"},{"id":"514017","messageId":"xmqq1pv32kvp.fsf@gitster.g","threadId":"63089","inReplyTo":"nfrfyg56nq7m2bcko4v4tsfdklrafpkybc6uhqgo25swdeebh3@a7aneoylg46u","subject":"Re: [PATCH 0/4] rev-list: introduce NUL-delimited output mode","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-03-11T23:44:10Z","receivedAt":"2025-03-11T23:44:12Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Justin Tobler writes:\n\n>> (4) The traditional \"rev-list --missing\" output loses the leading\n>> \"?\"; it is replaced by \"missing\" as the name, i.e. such\n>> a record may look like \" NUL missing=yes NUL...\" together\n>> with other \"= NUL\" pairs appended as needed at\n>> the end.\n>\n> I think this is good. Instead of prefixing missing OIDs with '?', we can\n> just append another token/value pair `missing=yes`.\n\nAnd we may want to avoid excessive bloat of the output that is not\nprimarily meant for human consumption, in which case perhaps a\ncommon things like \"missing\" and \"path\" can be given a shorter\ntoken, perhaps like \"m=yes\" and \"p=Makefile\"?\n"},{"id":"514018","messageId":"20250311235720.GA73755@coredump.intra.peff.net","threadId":"63089","inReplyTo":"20250310192829.661692-1-jltobler@gmail.com","subject":"Re: [PATCH 0/4] rev-list: introduce NUL-delimited output mode","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2025-03-11T23:57:20Z","receivedAt":"2025-03-11T23:57:22Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Mar 10, 2025 at 02:28:25PM -0500, Justin Tobler wrote:\n\n> To make machine parsing easier, this series introduces a NUL-delimited\n> output mode for git-rev-list(1) via a `-z` option following a suggestion\n> from Junio in a previous thread[1]. In this mode, instead of LF, each\n> object is delimited with two NUL bytes and any object metadata is\n> separated with a single NUL byte. Examples:\n> \n> NUL NUL\n> [NUL ] NUL NUL\n> ? [NUL =]... NUL NUL\n> \n> In this mode, path and value info are printed as-is without any special\n> encoding or truncation.\n\nI think this is a good direction, but I have two compatibility\nquestions:\n\n 1. What should \"git rev-list -z --stdin\" do? In most other programs\n with a \"-z\" option it affects both input and output. I don't\n particularly care about this case myself, but it will be hard to\n change later. So we probably want to decide now.\n\n 2. I was a little surprised that rev-list already takes a \"-z\" option,\n but it doesn't seem to do anything. I guess it's probably picked up\n via diff_opt_parse(), though (I think) you can't actually trigger a\n diff via rev-list itself. So even though this is a change in\n behavior, probably nobody would have been using it until now?\n\n If it is possible to see some effect from \"-z\" now (I didn't dig\n very far), then it may be better to continue to let the diff\n options parser handle it, and simply pick the result out of\n revs.diffopt.line_termination. As your patch 3 is written, I think\n the diff code probably doesn't see it anymore at all.\n\n-Peff\n"},{"id":"514026","messageId":"Z9E5qXQ2v4lu9hom@pks.im","threadId":"63089","inReplyTo":"xmqq1pv32kvp.fsf@gitster.g","subject":"Re: [PATCH 0/4] rev-list: introduce NUL-delimited output mode","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-03-12T07:37:13Z","receivedAt":"2025-03-12T07:37:21Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Tue, Mar 11, 2025 at 04:44:10PM -0700, Junio C Hamano wrote:\n> Justin Tobler writes:\n> \n> >> (4) The traditional \"rev-list --missing\" output loses the leading\n> >> \"?\"; it is replaced by \"missing\" as the name, i.e. such\n> >> a record may look like \" NUL missing=yes NUL...\" together\n> >> with other \"= NUL\" pairs appended as needed at\n> >> the end.\n> >\n> > I think this is good. Instead of prefixing missing OIDs with '?', we can\n> > just append another token/value pair `missing=yes`.\n> \n> And we may want to avoid excessive bloat of the output that is not\n> primarily meant for human consumption, in which case perhaps a\n> common things like \"missing\" and \"path\" can be given a shorter\n> token, perhaps like \"m=yes\" and \"p=Makefile\"?\n\nI would prefer to keep longer paths here:\n\n - While the output typically shouldn't be seen by a human, the code\n handling it both on the printing and on the receiving side would.\n And there it helps the reader quite a bit to get additional context\n rather than cryptic single-letter abbreviations that one would have\n to always look up.\n\n - By keeping with long names we also avoid \"letter squatting\" when two\n attributes of an object start with the same letter.\n\n - I doubt that the couple of extra characters is really going to\n matter much given that most of the time will most likely be spent\n reading and decompressing the objects from disk anyway.\n\nPatrick\n"},{"id":"514027","messageId":"Z9E60LG6ExSJzmuo@pks.im","threadId":"63089","inReplyTo":"20250311235720.GA73755@coredump.intra.peff.net","subject":"Re: [PATCH 0/4] rev-list: introduce NUL-delimited output mode","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-03-12T07:42:08Z","receivedAt":"2025-03-12T07:42:12Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Tue, Mar 11, 2025 at 07:57:20PM -0400, Jeff King wrote:\n> On Mon, Mar 10, 2025 at 02:28:25PM -0500, Justin Tobler wrote:\n> \n> > To make machine parsing easier, this series introduces a NUL-delimited\n> > output mode for git-rev-list(1) via a `-z` option following a suggestion\n> > from Junio in a previous thread[1]. In this mode, instead of LF, each\n> > object is delimited with two NUL bytes and any object metadata is\n> > separated with a single NUL byte. Examples:\n> > \n> > NUL NUL\n> > [NUL ] NUL NUL\n> > ? [NUL =]... NUL NUL\n> > \n> > In this mode, path and value info are printed as-is without any special\n> > encoding or truncation.\n> \n> I think this is a good direction, but I have two compatibility\n> questions:\n> \n> 1. What should \"git rev-list -z --stdin\" do? In most other programs\n> with a \"-z\" option it affects both input and output. I don't\n> particularly care about this case myself, but it will be hard to\n> change later. So we probably want to decide now.\n\nI would lean into the direction of making \"-z\" change the format both\nfor stdin and stdout. That's what we do in most cases, and in those\ncases where we didn't we came to regret it (git-cat-file(1)).\n\nPatrick\n"},{"id":"514028","messageId":"Z9E81I52prH-ROab@pks.im","threadId":"63089","inReplyTo":"20250310192829.661692-4-jltobler@gmail.com","subject":"Re: [PATCH 3/4] rev-list: support delimiting objects with NUL bytes","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-03-12T07:50:44Z","receivedAt":"2025-03-12T07:50:49Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Mon, Mar 10, 2025 at 02:28:28PM -0500, Justin Tobler wrote:\n> diff --git a/builtin/rev-list.c b/builtin/rev-list.c\n> index 04d9c893b5..86b3ce5806 100644\n> --- a/builtin/rev-list.c\n> +++ b/builtin/rev-list.c\n> @@ -757,6 +778,14 @@ int cmd_rev_list(int argc,\n> \t\tusage(rev_list_usage);\n> \n> \t}\n> +\n> +\tif (nul_delim) {\n> +\t\tif (revs.graph || revs.verbose_header || show_disk_usage ||\n> +\t\t info.show_timestamp || info.header_prefix || bisect_list ||\n> +\t\t use_bitmap_index || revs.edge_hint || arg_missing_action)\n> +\t\t\tdie(_(\"-z option used with unsupported option\"));\n> +\t}\n> +\n\nNot sure whether it's worth it, but do we maybe want to add a comment\nhere that mentions that this isn't an inherent limitation, but rather\nthat the initial implementation simply didn't implement compatibility\nwith these options? This would explicitly keep the door open for any\nfuture improvements in this area.\n\nPatrick\n"},{"id":"514070","messageId":"xmqqfrji1bvy.fsf@gitster.g","threadId":"63089","inReplyTo":"Z9E60LG6ExSJzmuo@pks.im","subject":"Re: [PATCH 0/4] rev-list: introduce NUL-delimited output mode","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-03-12T15:56:01Z","receivedAt":"2025-03-12T15:56:04Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Patrick Steinhardt writes:\n\n> On Tue, Mar 11, 2025 at 07:57:20PM -0400, Jeff King wrote:\n>> On Mon, Mar 10, 2025 at 02:28:25PM -0500, Justin Tobler wrote:\n>> \n>> > To make machine parsing easier, this series introduces a NUL-delimited\n>> > output mode for git-rev-list(1) via a `-z` option following a suggestion\n>> > from Junio in a previous thread[1]. In this mode, instead of LF, each\n>> > object is delimited with two NUL bytes and any object metadata is\n>> > separated with a single NUL byte. Examples:\n>> > \n>> > NUL NUL\n>> > [NUL ] NUL NUL\n>> > ? [NUL =]... NUL NUL\n>> > \n>> > In this mode, path and value info are printed as-is without any special\n>> > encoding or truncation.\n>> \n>> I think this is a good direction, but I have two compatibility\n>> questions:\n>> \n>> 1. What should \"git rev-list -z --stdin\" do? In most other programs\n>> with a \"-z\" option it affects both input and output. I don't\n>> particularly care about this case myself, but it will be hard to\n>> change later. So we probably want to decide now.\n>\n> I would lean into the direction of making \"-z\" change the format both\n> for stdin and stdout. That's what we do in most cases, and in those\n> cases where we didn't we came to regret it (git-cat-file(1)).\n\nI've seen \"-Z\", in addition to \"-z\", used to differentiate between\ninput and output in some commands. If we are not going to do that,\nI agree that making \"-z\" to affect both input and output is less\nsurprising than having to remember which side is still text.\n\n"},{"id":"514121","messageId":"zozymxxh4s2qbh2cknqyy7z4ktgppm7i62bhkgupnij4enkq5n@6v2icwdt77nc","threadId":"63089","inReplyTo":"xmqqr03461z7.fsf@gitster.g","subject":"Re: [PATCH 2/4] rev-list: refactor early option parsing","fromName":"Justin Tobler","fromEmail":"jltobler@gmail.com","sentAt":"2025-03-12T21:39:07Z","receivedAt":"2025-03-12T21:42:37Z","isPatch":true,"sender":{"key":"jltobler@gmail.com","avatar":"https://avatars.githubusercontent.com/u/53454972?v=4"},"body":"On 25/03/10 01:54PM, Junio C Hamano wrote:\n> Justin Tobler writes:\n> \n> > @@ -639,19 +640,15 @@ int cmd_rev_list(int argc,\n> > \t\tif (!strcmp(arg, \"--exclude-promisor-objects\")) {\n> > \t\t\tfetch_if_missing = 0;\n> > \t\t\trevs.exclude_promisor_objects = 1;\n> > -\t\t\tbreak;\n> > -\t\t}\n> > -\t}\n> > -\tfor (i = 1; i < argc; i++) {\n> > -\t\tconst char *arg = argv[i];\n> > -\t\tif (skip_prefix(arg, \"--missing=\", &arg)) {\n> > -\t\t\tif (revs.exclude_promisor_objects)\n> > -\t\t\t\tdie(_(\"options '%s' and '%s' cannot be used together\"), \"--exclude-promisor-objects\", \"--missing\");\n> > -\t\t\tif (parse_missing_action_value(arg))\n> > -\t\t\t\tbreak;\n> > +\t\t} else if (skip_prefix(arg, \"--missing=\", &arg)) {\n> > +\t\t\tparse_missing_action_value(arg);\n> > \t\t}\n> > \t}\n> \n> There is a huge NEEDSWORK comment that essentially says that the\n> above two loops that this patch combines into one is fundamentally\n> broken. I suspect that the remaining two patches in this series\n> would punt and not improve them, but offhand I am not sure if this\n> change is making it harder to fix them properly easier or harder.\n\nMy understanding here is that `setup_revisions()` does not provide a\nmechanism to reject certain individual or combinations of parsed options\nthat do not make sense for a command. As you mentioned, this patch is\njust punting on that issue, but I don't think it is really making the\nproblem any harder to be resolved in the future.\n\nIn this case, the `-z` option is parsed early to avoid is being handled\nby the diff options parsing in `setup_revisions()`. From the perspective\nof git-rev-list(1), I don't think there is any reason to be parsing diff\noptions at all. We could introduce an option to disable diff options\nparsing in `setup_revisions()`, but now that we also want to modify\nstdin argument parsing to be NUL-delimited, we would still have to parse\nthe -z option early in order to configure `setup_revisions()`\nappropriately.\n\nAnother way we could potentially resolve these issues would be to\noptionally separate the argument/option parsing logic out of\n`setup_revisions()` and expose it directly. That could give the caller\nmore control over how options are handled before fully setting up the\n`rev_info` structure. I suspect this would be non-trivial though, so I\nthink it would preferrable to keep the current approach for now as it\nisn't making the problem harder to fix IMO.\n\n-Justin\n"},{"id":"514122","messageId":"rrc53bb5433gkdoboubhjrfk4pr6uaykiugiqghwbyhwtigaws@dyzt2buwbasl","threadId":"63089","inReplyTo":"xmqqmsds61pw.fsf@gitster.g","subject":"Re: [PATCH 3/4] rev-list: support delimiting objects with NUL bytes","fromName":"Justin Tobler","fromEmail":"jltobler@gmail.com","sentAt":"2025-03-12T21:39:15Z","receivedAt":"2025-03-12T21:42:45Z","isPatch":true,"sender":{"key":"jltobler@gmail.com","avatar":"https://avatars.githubusercontent.com/u/53454972?v=4"},"body":"On 25/03/10 01:59PM, Junio C Hamano wrote:\n> > +static int nul_delim;\n> > static int show_disk_usage;\n> > static off_t total_disk_usage;\n> > static int human_readable;\n> > \n> > +static void print_object_term(int nul_delim)\n> > +{\n> > +\tchar line_sep = '\\n';\n> > +\n> > +\tif (nul_delim)\n> > +\t\tline_sep = '\\0';\n> > +\n> > +\tputchar(line_sep);\n> > +\tif (nul_delim)\n> > +\t\tputchar(line_sep);\n> > +}\n> \n> This looks, to put it mildly, strange. The concept of the line\n> delimiter byte (which can take any single byte) is wider than having\n> a NUL as the line delimiter byte. Why would we even want both?\n\nThis is a fair point.\n\nThere are scerarios where the printed object format varies more than\njust the delimiter used. For example, in the normal output mode printed\nobject paths are truncated if they contain a newline. In the\nNUL-delimited mode we want to print the complete path. Similarly,\nmissing object paths are c-quoted if they contain SP or LF characters.\nThese should also be printed as-is when in the NUL-delimited mode.\n\nDue to branching behavior, I initially thought it would be easier to\nfollow if there was separate variable that signaled printing behavior,\nbut as you mentioned, we could also just use the global to hold the line\nterminator being used and check if it is set to NUL when there is\nconditional behavior.\n\n> IOW, wouldn't it make more sense to have line_delim as the global\n> (or per-invocation parameter to this function) and have\n> print_object_term() just use it? If you want to make it behave\n> differently only when line-delimter is NUL (which I do not\n> recommend), you can switch on the value of line_delimiter being NUL.\n> So I do not see a merit in having two separate variables (except for\n> confusing future readers).\n\nIn the next version, I'll use a global to hold the configured line\ndelimiter. I think it also makes sense to have a separate global\nvariable for the metadata delimiter since in normal mode, it is usually\na SP character. When the -z option is detected, we can set both of these\nto a NUL byte.\n\n-Justin\n"},{"id":"514125","messageId":"rm6fvk3biqgk22qeuh2ur6xywrkiiofugtnndpuh4lnmqb4ivl@hvc4vfodb2wm","threadId":"63089","inReplyTo":"Z9E81I52prH-ROab@pks.im","subject":"Re: [PATCH 3/4] rev-list: support delimiting objects with NUL bytes","fromName":"Justin Tobler","fromEmail":"jltobler@gmail.com","sentAt":"2025-03-12T21:41:11Z","receivedAt":"2025-03-12T21:44:40Z","isPatch":true,"sender":{"key":"jltobler@gmail.com","avatar":"https://avatars.githubusercontent.com/u/53454972?v=4"},"body":"On 25/03/12 08:50AM, Patrick Steinhardt wrote:\n> On Mon, Mar 10, 2025 at 02:28:28PM -0500, Justin Tobler wrote:\n> > diff --git a/builtin/rev-list.c b/builtin/rev-list.c\n> > index 04d9c893b5..86b3ce5806 100644\n> > --- a/builtin/rev-list.c\n> > +++ b/builtin/rev-list.c\n> > @@ -757,6 +778,14 @@ int cmd_rev_list(int argc,\n> > \t\tusage(rev_list_usage);\n> > \n> > \t}\n> > +\n> > +\tif (nul_delim) {\n> > +\t\tif (revs.graph || revs.verbose_header || show_disk_usage ||\n> > +\t\t info.show_timestamp || info.header_prefix || bisect_list ||\n> > +\t\t use_bitmap_index || revs.edge_hint || arg_missing_action)\n> > +\t\t\tdie(_(\"-z option used with unsupported option\"));\n> > +\t}\n> > +\n> \n> Not sure whether it's worth it, but do we maybe want to add a comment\n> here that mentions that this isn't an inherent limitation, but rather\n> that the initial implementation simply didn't implement compatibility\n> with these options? This would explicitly keep the door open for any\n> future improvements in this area.\n\nThat's fair. I'll mention this is a comment that way we know support\nNUL-delimited mode support can be added for some of the options in the\nfuture.\n\nThanks\n-Justin\n"},{"id":"514127","messageId":"pwovu5vmheonndcyq7hmpdzagengq4p5dxsyjshygjcdt45g7g@g7gebobbrygu","threadId":"63089","inReplyTo":"Z9E5qXQ2v4lu9hom@pks.im","subject":"Re: [PATCH 0/4] rev-list: introduce NUL-delimited output mode","fromName":"Justin Tobler","fromEmail":"jltobler@gmail.com","sentAt":"2025-03-12T21:45:47Z","receivedAt":"2025-03-12T21:49:17Z","isPatch":true,"sender":{"key":"jltobler@gmail.com","avatar":"https://avatars.githubusercontent.com/u/53454972?v=4"},"body":"On 25/03/12 08:37AM, Patrick Steinhardt wrote:\n> On Tue, Mar 11, 2025 at 04:44:10PM -0700, Junio C Hamano wrote:\n> > Justin Tobler writes:\n> > \n> > >> (4) The traditional \"rev-list --missing\" output loses the leading\n> > >> \"?\"; it is replaced by \"missing\" as the name, i.e. such\n> > >> a record may look like \" NUL missing=yes NUL...\" together\n> > >> with other \"= NUL\" pairs appended as needed at\n> > >> the end.\n> > >\n> > > I think this is good. Instead of prefixing missing OIDs with '?', we can\n> > > just append another token/value pair `missing=yes`.\n> > \n> > And we may want to avoid excessive bloat of the output that is not\n> > primarily meant for human consumption, in which case perhaps a\n> > common things like \"missing\" and \"path\" can be given a shorter\n> > token, perhaps like \"m=yes\" and \"p=Makefile\"?\n> \n> I would prefer to keep longer paths here:\n> \n> - While the output typically shouldn't be seen by a human, the code\n> handling it both on the printing and on the receiving side would.\n> And there it helps the reader quite a bit to get additional context\n> rather than cryptic single-letter abbreviations that one would have\n> to always look up.\n> \n> - By keeping with long names we also avoid \"letter squatting\" when two\n> attributes of an object start with the same letter.\n> \n> - I doubt that the couple of extra characters is really going to\n> matter much given that most of the time will most likely be spent\n> reading and decompressing the objects from disk anyway.\n\nI also would prefer to keep the longer token names here. I don't think\nthe extra bytes should be much of a concern and it allows us to avoid\nhaving to conditionally change the printed token name when in the\nNUL-delimited mode for missing object metadata.\n\n-Justin\n"},{"id":"514132","messageId":"sm5xvh7bipyrlpw45bptlctbuhey7v3qktpskal5dzexomt34f@wgfziyhcf3t6","threadId":"63089","inReplyTo":"20250311235720.GA73755@coredump.intra.peff.net","subject":"Re: [PATCH 0/4] rev-list: introduce NUL-delimited output mode","fromName":"Justin Tobler","fromEmail":"jltobler@gmail.com","sentAt":"2025-03-12T22:09:41Z","receivedAt":"2025-03-12T22:13:10Z","isPatch":true,"sender":{"key":"jltobler@gmail.com","avatar":"https://avatars.githubusercontent.com/u/53454972?v=4"},"body":"On 25/03/11 07:57PM, Jeff King wrote:\n> On Mon, Mar 10, 2025 at 02:28:25PM -0500, Justin Tobler wrote:\n> \n> > To make machine parsing easier, this series introduces a NUL-delimited\n> > output mode for git-rev-list(1) via a `-z` option following a suggestion\n> > from Junio in a previous thread[1]. In this mode, instead of LF, each\n> > object is delimited with two NUL bytes and any object metadata is\n> > separated with a single NUL byte. Examples:\n> > \n> > NUL NUL\n> > [NUL ] NUL NUL\n> > ? [NUL =]... NUL NUL\n> > \n> > In this mode, path and value info are printed as-is without any special\n> > encoding or truncation.\n> \n> I think this is a good direction, but I have two compatibility\n> questions:\n> \n> 1. What should \"git rev-list -z --stdin\" do? In most other programs\n> with a \"-z\" option it affects both input and output. I don't\n> particularly care about this case myself, but it will be hard to\n> change later. So we probably want to decide now.\n\nAs others suggested in this thread, in the next version I'll make\nrevision and pathspec parsing on stdin also be NUL-delimited when -z is\nused.\n\n> 2. I was a little surprised that rev-list already takes a \"-z\" option,\n> but it doesn't seem to do anything. I guess it's probably picked up\n> via diff_opt_parse(), though (I think) you can't actually trigger a\n> diff via rev-list itself. So even though this is a change in\n> behavior, probably nobody would have been using it until now?\n\nThis is correct. Technically git-rev-list(1) accepts all the common diff\noptions because it invokes `setup_revisions()`. From my understanding it\nisn't possible to trigger diffs so I think parsing diff options is\nunnecessary to begin with. As it also isn't a documented option, I\nfigured -z being an accepted option for git-rev-list(1) is largely an\nunintended consequence of it using `setup_revisions()` and should be\nfine to use.\n\n> If it is possible to see some effect from \"-z\" now (I didn't dig\n> very far), then it may be better to continue to let the diff\n> options parser handle it, and simply pick the result out of\n> revs.diffopt.line_termination. As your patch 3 is written, I think\n> the diff code probably doesn't see it anymore at all.\n\nAs currently implemented, the early parsing of -z doesn't effect the\ndiff options parsing in `setup_revisions()`. The early parsing doesn't\nremove the option and thus it continues to be set in the diff options.\n\nFurthermore, revision and pathspec argument parsing is all handled in\n`setup_revisions()` so if we want to NUL-delimit arguments parsed on\nstdin with -z, we would still need to parse the option early anyway. I\nthink it should be fine to leave the early -z option parsing as-is.\n\nThanks,\n-Justin\n"},{"id":"514138","messageId":"20250313001706.3390502-1-jltobler@gmail.com","threadId":"63089","inReplyTo":"20250310192829.661692-1-jltobler@gmail.com","subject":"[PATCH v2 0/6] rev-list: introduce NUL-delimited output mode","fromName":"Justin Tobler","fromEmail":"jltobler@gmail.com","sentAt":"2025-03-13T00:17:00Z","receivedAt":"2025-03-13T00:21:14Z","isPatch":true,"sender":{"key":"jltobler@gmail.com","avatar":"https://avatars.githubusercontent.com/u/53454972?v=4"},"body":"When walking objects, git-rev-list(1) prints each object entry on a\nseparate line in the form:\n\n LF\n\nSome options, such as `--objects`, may print additional information\nabout the object on the same line:\n\n SP [] LF\n\nIn this mode, if the object path contains a newline it is truncated at\nthe newline.\n\nThe `--boundary` option also modifies output by prefixing boundary\nobjects with `-`:\n\n - LF\n\nWhen the `--missing={print,print-info}` option is provided, information\nabout any missing objects encountered during the object walk are also\nprinted in the form:\n\n ? [SP =]... LF\n\nwhere values containing LF or SP are printed in a token specific fashion\nso that the resulting encoded value does not contain either of these two\nproblematic bytes. For example, missing object paths are quoted in the C\nstyle when they contain LF or SP.\n\nTo make machine parsing easier, this series introduces a NUL-delimited\noutput mode for git-rev-list(1) via a `-z` option. In this mode, the\noutput format for object records is unified such that each object and\nits accompanying metadata is formatted without relying on object\nmetadata order. This format follows the existing `=` used\nby the `--missing` option to represent object metadata in the form:\n\n NUL [= NUL]...\n\n # Examples\n LF -> NUL\n SP LF -> NUL path= NUL\n - LF -> NUL boundary=yes NUL\n ? [SP =]... -> NUL missing=yes NUL [= NUL]...\n\nNote that token value info is printed as-is without any special encoding\nor truncation. Prefixes such as '-' and '?' are dropped in favor using a\ntoken/value pair to signal the same information.\n\nWhile in this mode, if the `--sdtin` option is used, revision and\npathspec arguments read from stdin are separated with a NUL byte instead\nof being newline delimited.\n\nFor now this series only adds support for use with the `--objects`,\n`--boundary` and `--missing` output options. Usage of `-z` with other\noptions is rejected, so it can potentially be added in the future.\n\nThis series is structured as follows:\n\n - Patches 1 and 2 do some minor preparatory refactors.\n\n - Patch 3 modifies stdin argument parsing handled by\n `setup_revisions()` to support NUL-delimited arguments.\n\n - Patch 4 adds the `-z` option to git-rev-list(1) to print\n objects in a NUL-delimited fashion. Arguments parsed on stdin\n while in the mode are also NUL-delimited.\n\n - Patch 5 teaches the `--boundary` option how to print info in a\n NUL-delimited fashino using the unified output format.\n\n - Patch 6 teaches the `--missing` option how to print info in a\n NUL-delimited fashion using the unified output format.\n\nChanges since V1:\n\n - Use unified output format with `=` pairs for\n all object metadata.\n\n - Add support for the `--boundary` option in NUL-delimited mode.\n\n - Add support for NUL-delimited stdin argument parsing in\n NUL-delimited mode.\n\n - Instead of using two NUL bytes to delimit between object\n records, a single NUL byte is used. Now that object metadata\n is always in the form `=`, we know a new object\n record starts when there is an OID entry which will not\n contain '='.\n\nThanks for taking a look,\n-Justin\n\nJustin Tobler (6):\n rev-list: inline `show_object_with_name()` in `show_object()`\n rev-list: refactor early option parsing\n revision: support NUL-delimited --stdin mode\n rev-list: support delimiting objects with NUL bytes\n rev-list: support NUL-delimited --boundary option\n rev-list: support NUL-delimited --missing option\n\n Documentation/rev-list-options.adoc | 26 ++++++++\n builtin/rev-list.c | 92 ++++++++++++++++++++++-------\n revision.c | 27 ++++-----\n revision.h | 5 +-\n t/t6000-rev-list-misc.sh | 51 ++++++++++++++++\n t/t6017-rev-list-stdin.sh | 9 +++\n t/t6022-rev-list-missing.sh | 31 ++++++++++\n 7 files changed, 200 insertions(+), 41 deletions(-)\n\nRange-diff against v1:\n1: d2eded3ac7 = 1: d2eded3ac7 rev-list: inline `show_object_with_name()` in `show_object()`\n2: 03cd08c859 = 2: 03cd08c859 rev-list: refactor early option parsing\n-: ---------- > 3: 803a49933a revision: support NUL-delimited --stdin mode\n3: 41c5cb7737 ! 4: d3b3c4ef89 rev-list: support delimiting objects with NUL bytes\n @@ Commit message\n newline are also truncated at the newline.\n \n Introduce the `-z` option for git-rev-list(1) which reformats the output\n - to use NUL-delimiters between objects and associated info. Each object\n - line uses two NUL bytes to indicate the end of an object entry and a\n - single NUL byte to delimit between object information in the following\n - form:\n + to use NUL-delimiters between objects and associated info in the\n + following form:\n \n $ git rev-list -z --objects \n - [NUL ] NUL NUL\n + NUL [path= NUL]\n \n - For now, the `--objects` flag is the only option that can be used in\n - combination with `-z`. In this mode, the object path is not truncated at\n - newlines. In a subsequent commit, NUL-delimiter support for other\n - options is added. Other options that do not make sense with be used in\n - combination with `-z` are rejected.\n + In this form, the start of each record is signaled by an OID entry that\n + is all hexidecimal and does not contain any '='. Additional path info\n + from `--objects` is appended to the record as a token/value pair\n + `path=` as-is without any truncation.\n +\n + In this mode, revision and pathspec arguments provided on stdin with the\n + `--stdin` option are also separated by a NUL byte instead of being\n + newline delimited.\n +\n + For now, the `--objects` and `--stdin` flag are the only options that\n + can be used in combination with `-z`. In a subsequent commit,\n + NUL-delimited support for other options is added. Other options that do\n + not make sense with be used in combination with `-z` are rejected.\n \n Signed-off-by: Justin Tobler \n \n @@ Documentation/rev-list-options.adoc: ifdef::git-rev-list[]\n \t`
` text will be printed with each progress update.\n +\n +-z::\n -+\tInstead of being newline-delimited, each outputted object is delimited\n -+\twith two NUL bytes in the following form:\n ++\tInstead of being newline-delimited, each outputted object and its\n ++\taccompanying metadata is delimited using NUL bytes in the following\n ++\tform:\n ++\n +-----------------------------------------------------------------------\n -+ NUL NUL\n ++ NUL [= NUL]...\n +-----------------------------------------------------------------------\n ++\n -+When the `--objects` option is also present, available object name information\n -+is printed in the following form without any truncation for object names\n -+containing newline characters:\n ++Additional object metadata, such as object paths, is printed using the\n ++`=` form. Token values are printed as-is without any\n ++encoding/truncation. An OID entry never contains a '=' character and thus\n ++is used to signal the start of a new object record. Examples:\n ++\n +-----------------------------------------------------------------------\n -+ [NUL ] NUL NUL\n ++ NUL\n ++ NUL path= NUL\n +-----------------------------------------------------------------------\n ++\n -+This option is only compatible with `--objects`.\n ++This mode is only compatible with the `--objects` output option. Also, revision\n ++and pathspec argument parsing on stdin with the `--stdin` option is NUL byte\n ++delimited instead of using newlines while in this mode.\n endif::git-rev-list[]\n \n History Simplification\n @@ builtin/rev-list.c: static int arg_show_object_names = 1;\n \n #define DEFAULT_OIDSET_SIZE (16*1024)\n \n -+static int nul_delim;\n ++static char line_term = '\\n';\n ++static char info_term = ' ';\n ++\n static int show_disk_usage;\n static off_t total_disk_usage;\n static int human_readable;\n - \n -+static void print_object_term(int nul_delim)\n -+{\n -+\tchar line_sep = '\\n';\n -+\n -+\tif (nul_delim)\n -+\t\tline_sep = '\\0';\n -+\n -+\tputchar(line_sep);\n -+\tif (nul_delim)\n -+\t\tputchar(line_sep);\n -+}\n -+\n - static off_t get_object_disk_usage(struct object *obj)\n - {\n - \toff_t size;\n @@ builtin/rev-list.c: static void show_commit(struct commit *commit, void *data)\n \tif (revs->commit_format == CMIT_FMT_ONELINE)\n \t\tputchar(' ');\n \telse if (revs->include_header)\n -\t\tputchar('\\n');\n -+\t\tprint_object_term(nul_delim);\n ++\t\tputchar(line_term);\n \n \tif (revs->verbose_header) {\n \t\tstruct strbuf buf = STRBUF_INIT;\n @@ builtin/rev-list.c: static void show_object(struct object *obj, const char *name\n -\t\tputchar(' ');\n -\t\tfor (const char *p = name; *p && *p != '\\n'; p++)\n -\t\t\tputchar(*p);\n -+\t\tif (nul_delim && *name) {\n -+\t\t\tputchar('\\0');\n -+\t\t\tprintf(\"%s\", name);\n -+\t\t} else if (!nul_delim) {\n -+\t\t\tputchar(' ');\n ++\t\tif (line_term) {\n ++\t\t\tputchar(info_term);\n +\t\t\tfor (const char *p = name; *p && *p != '\\n'; p++)\n +\t\t\t\tputchar(*p);\n ++\t\t} else if (*name) {\n ++\t\t\tprintf(\"%cpath=%s\", info_term, name);\n +\t\t}\n \t}\n \n -\tputchar('\\n');\n -+\tprint_object_term(nul_delim);\n ++\tputchar(line_term);\n }\n \n static void show_edge(struct commit *commit)\n @@ builtin/rev-list.c: int cmd_rev_list(int argc,\n \t\t} else if (skip_prefix(arg, \"--missing=\", &arg)) {\n \t\t\tparse_missing_action_value(arg);\n +\t\t} else if (!strcmp(arg, \"-z\")) {\n -+\t\t\tnul_delim = 1;\n ++\t\t\ts_r_opt.nul_delim_stdin = 1;\n ++\t\t\tline_term = '\\0';\n ++\t\t\tinfo_term = '\\0';\n \t\t}\n \t}\n \n @@ builtin/rev-list.c: int cmd_rev_list(int argc,\n \n \t}\n +\n -+\tif (nul_delim) {\n ++\t/*\n ++\t * Reject options currently incompatible with -z. For some options, this\n ++\t * is not an inherent limitation and support may be implemented in the\n ++\t * future.\n ++\t */\n ++\tif (!line_term) {\n +\t\tif (revs.graph || revs.verbose_header || show_disk_usage ||\n +\t\t info.show_timestamp || info.header_prefix || bisect_list ||\n -+\t\t use_bitmap_index || revs.edge_hint || arg_missing_action)\n ++\t\t use_bitmap_index || revs.edge_hint || revs.left_right ||\n ++\t\t revs.cherry_mark || arg_missing_action || revs.boundary)\n +\t\t\tdie(_(\"-z option used with unsupported option\"));\n +\t}\n +\n @@ t/t6000-rev-list-misc.sh: test_expect_success 'rev-list --unpacked' '\n +\toid1=$(git -C repo rev-parse HEAD) &&\n +\toid2=$(git -C repo rev-parse HEAD~) &&\n +\n -+\tprintf \"%s\\0\\0%s\\0\\0\" \"$oid1\" \"$oid2\" >expect &&\n ++\tprintf \"%s\\0%s\\0\" \"$oid1\" \"$oid2\" >expect &&\n +\tgit -C repo rev-list -z HEAD >actual &&\n +\n +\ttest_cmp expect actual\n @@ t/t6000-rev-list-misc.sh: test_expect_success 'rev-list --unpacked' '\n +\tpath1=1.t &&\n +\tpath2=2.t &&\n +\n -+\tprintf \"%s\\0%s\\0\\0%s\\0%s\\0\\0\" \"$oid1\" \"$path1\" \"$oid2\" \"$path2\" >expect &&\n ++\tprintf \"%s\\0path=%s\\0%s\\0path=%s\\0\" \"$oid1\" \"$path1\" \"$oid2\" \"$path2\" \\\n ++\t\t>expect &&\n +\tgit -C repo rev-list -z --objects HEAD:1.t HEAD:2.t >actual &&\n +\n +\ttest_cmp expect actual\n +'\n +\n test_done\n +\n + ## t/t6017-rev-list-stdin.sh ##\n +@@ t/t6017-rev-list-stdin.sh: test_expect_success '--not via stdin does not influence revisions from command l\n + \ttest_cmp expect actual\n + '\n + \n ++test_expect_success 'NUL-delimited stdin' '\n ++\tprintf \"%s\\0%s\\0%s\\0\" \"HEAD\" \"--\" \"file-1\" > input &&\n ++\n ++\tgit rev-list -z --objects HEAD -- file-1 >expect &&\n ++\tgit rev-list -z --objects --stdin actual &&\n ++\n ++\ttest_cmp expect actual\n ++'\n ++\n + test_done\n-: ---------- > 5: 5e4fc41976 rev-list: support NUL-delimited --boundary option\n4: 007adbac25 ! 6: 7744966514 rev-list: support NUL-delimited --missing option\n @@ Commit message\n rev-list: support NUL-delimited --missing option\n \n The `--missing={print,print-info}` option for git-rev-list(1) prints\n - missing objects found while performing the revision walk. Add support\n - for printing missing objects in a NUL-delimited format when the `-z`\n - option is enabled.\n + missing objects found while performing the object walk in the form:\n +\n + $ git rev-list --missing=print-info \n + ? [SP =]... LF\n +\n + Add support for printing missing objects in a NUL-delimited format when\n + the `-z` option is enabled.\n \n $ git rev-list -z --missing=print-info \n - NUL NUL\n - ? [NUL =]... NUL NUL\n + NUL missing=yes NUL [= NUL]...\n \n In this mode, values containing special characters or spaces are printed\n - as-is without being escaped or quoted.\n + as-is without being escaped or quoted. Instead of prefixing the missing\n + OID with '?', a separate `missing=yes` token/value pair is appended.\n \n Signed-off-by: Justin Tobler \n \n ## Documentation/rev-list-options.adoc ##\n -@@ Documentation/rev-list-options.adoc: containing newline characters:\n - [NUL ] NUL NUL\n +@@ Documentation/rev-list-options.adoc: ifdef::git-rev-list[]\n + NUL [= NUL]...\n + -----------------------------------------------------------------------\n + +\n +-Additional object metadata, such as object paths or boundary objects, is\n +-printed using the `=` form. Token values are printed as-is\n ++Additional object metadata, such as object paths or boundary/missing objects,\n ++is printed using the `=` form. Token values are printed as-is\n + without any encoding/truncation. An OID entry never contains a '=' character\n + and thus is used to signal the start of a new object record. Examples:\n + +\n +@@ Documentation/rev-list-options.adoc: and thus is used to signal the start of a new object record. Examples:\n + NUL\n + NUL path= NUL\n + NUL boundary=yes NUL\n ++ NUL missing=yes NUL [= NUL]...\n -----------------------------------------------------------------------\n +\n --This option is only compatible with `--objects`.\n -+When the `--missing` option is provided, missing objects are printed in the\n -+following form where value is printed as-is without any token specific\n -+encoding:\n -++\n -+-----------------------------------------------------------------------\n -+? [NUL =]... NUL NUL\n -+-----------------------------------------------------------------------\n -++\n -+This option is only compatible with `--objects` and `--missing`.\n +-This mode is only compatible with the `--objects` and `--boundary` output\n +-options. Also, revision and pathspec argument parsing on stdin with the\n +-`--stdin` option is NUL byte delimited instead of using newlines while in this\n +-mode.\n ++This mode is only compatible with the `--objects`, `--boundary`, and\n ++`--missing` output options. Also, revision and pathspec argument parsing on\n ++stdin with the `--stdin` option is NUL byte delimited instead of using newlines\n ++while in this mode.\n endif::git-rev-list[]\n \n History Simplification\n \n ## builtin/rev-list.c ##\n @@ builtin/rev-list.c: static void print_missing_object(struct missing_objects_map_entry *entry,\n - \t\t\t\t int print_missing_info)\n {\n \tstruct strbuf sb = STRBUF_INIT;\n -+\tchar info_sep = ' ';\n + \n ++\tif (line_term)\n ++\t\tputchar('?');\n +\n -+\tif (nul_delim)\n -+\t\tinfo_sep = '\\0';\n ++\tprintf(\"%s\", oid_to_hex(&entry->entry.oid));\n ++\n ++\tif (!line_term)\n ++\t\tprintf(\"%cmissing=yes\", info_term);\n +\n -+\tprintf(\"?%s\", oid_to_hex(&entry->entry.oid));\n - \n \tif (!print_missing_info) {\n -\t\tprintf(\"?%s\\n\", oid_to_hex(&entry->entry.oid));\n -+\t\tprint_object_term(nul_delim);\n ++\t\tputchar(line_term);\n \t\treturn;\n \t}\n \n @@ builtin/rev-list.c: static void print_missing_object(struct missing_objects_map_\n -\t\tstrbuf_addstr(&sb, \" path=\");\n -\t\tquote_path(entry->path, NULL, &path, QUOTE_PATH_QUOTE_SP);\n -\t\tstrbuf_addbuf(&sb, &path);\n -+\t\tstrbuf_addf(&sb, \"%cpath=\", info_sep);\n ++\t\tstrbuf_addf(&sb, \"%cpath=\", info_term);\n +\n -+\t\tif (nul_delim) {\n -+\t\t\tstrbuf_addstr(&sb, entry->path);\n -+\t\t} else {\n ++\t\tif (line_term) {\n +\t\t\tquote_path(entry->path, NULL, &path, QUOTE_PATH_QUOTE_SP);\n +\t\t\tstrbuf_addbuf(&sb, &path);\n ++\t\t} else {\n ++\t\t\tstrbuf_addstr(&sb, entry->path);\n +\t\t}\n \n \t\tstrbuf_release(&path);\n \t}\n \tif (entry->type)\n -\t\tstrbuf_addf(&sb, \" type=%s\", type_name(entry->type));\n -+\t\tstrbuf_addf(&sb, \"%ctype=%s\", info_sep, type_name(entry->type));\n ++\t\tstrbuf_addf(&sb, \"%ctype=%s\", info_term, type_name(entry->type));\n +\n +\tfwrite(sb.buf, sizeof(char), sb.len, stdout);\n -+\tprint_object_term(nul_delim);\n ++\tputchar(line_term);\n \n -\tprintf(\"?%s%s\\n\", oid_to_hex(&entry->entry.oid), sb.buf);\n \tstrbuf_release(&sb);\n }\n \n @@ builtin/rev-list.c: int cmd_rev_list(int argc,\n - \tif (nul_delim) {\n \t\tif (revs.graph || revs.verbose_header || show_disk_usage ||\n \t\t info.show_timestamp || info.header_prefix || bisect_list ||\n --\t\t use_bitmap_index || revs.edge_hint || arg_missing_action)\n -+\t\t use_bitmap_index || revs.edge_hint)\n + \t\t use_bitmap_index || revs.edge_hint || revs.left_right ||\n +-\t\t revs.cherry_mark || arg_missing_action)\n ++\t\t revs.cherry_mark)\n \t\t\tdie(_(\"-z option used with unsupported option\"));\n \t}\n \n @@ t/t6022-rev-list-missing.sh: do\n +\n +\t\tgit rev-list -z --objects --no-object-names \\\n +\t\t\tHEAD ^\"$oid\" >expect &&\n -+\t\tprintf \"?%s\\0path=%s\\0type=%s\\0\\0\" \"$oid\" \"$path\" \"$type\" >>expect &&\n ++\t\tprintf \"%s\\0missing=yes\\0path=%s\\0type=%s\\0\" \"$oid\" \"$path\" \\\n ++\t\t\t\"$type\" >>expect &&\n +\n +\t\tmv \"$obj_path\" \"$obj_path.hidden\" &&\n +\t\tgit rev-list -z --objects --no-object-names \\\n\nbase-commit: 87a0bdbf0f72b7561f3cd50636eee33dcb7dbcc3\n-- \n2.49.0.rc2\n\n"},{"id":"514139","messageId":"20250313001706.3390502-2-jltobler@gmail.com","threadId":"63089","inReplyTo":"20250313001706.3390502-1-jltobler@gmail.com","subject":"[PATCH v2 1/6] rev-list: inline `show_object_with_name()` in `show_object()`","fromName":"Justin Tobler","fromEmail":"jltobler@gmail.com","sentAt":"2025-03-13T00:17:01Z","receivedAt":"2025-03-13T00:21:14Z","isPatch":true,"sender":{"key":"jltobler@gmail.com","avatar":"https://avatars.githubusercontent.com/u/53454972?v=4"},"body":"The `show_object_with_name()` function only has a single call site.\nInline call to `show_object_with_name()` in `show_object()` so the\nexplicit function can be cleaned up and live closer to where it is used.\nWhile at it, factor out the code that prints the OID and newline for\nboth objects with and without a name. In a subsequent commit,\n`show_object()` is modified to support printing object information in a\nNUL-delimited format.\n\nSigned-off-by: Justin Tobler \n---\n builtin/rev-list.c | 13 +++++++++----\n revision.c | 8 --------\n revision.h | 2 --\n 3 files changed, 9 insertions(+), 14 deletions(-)\n\ndiff --git a/builtin/rev-list.c b/builtin/rev-list.c\nindex bb26bee0d4..dcd079c16c 100644\n--- a/builtin/rev-list.c\n+++ b/builtin/rev-list.c\n@@ -357,10 +357,15 @@ static void show_object(struct object *obj, const char *name, void *cb_data)\n \t\treturn;\n \t}\n \n-\tif (arg_show_object_names)\n-\t\tshow_object_with_name(stdout, obj, name);\n-\telse\n-\t\tprintf(\"%s\\n\", oid_to_hex(&obj->oid));\n+\tprintf(\"%s\", oid_to_hex(&obj->oid));\n+\n+\tif (arg_show_object_names) {\n+\t\tputchar(' ');\n+\t\tfor (const char *p = name; *p && *p != '\\n'; p++)\n+\t\t\tputchar(*p);\n+\t}\n+\n+\tputchar('\\n');\n }\n \n static void show_edge(struct commit *commit)\ndiff --git a/revision.c b/revision.c\nindex c4390f0938..0eaebe4478 100644\n--- a/revision.c\n+++ b/revision.c\n@@ -59,14 +59,6 @@ implement_shared_commit_slab(revision_sources, char *);\n \n static inline int want_ancestry(const struct rev_info *revs);\n \n-void show_object_with_name(FILE *out, struct object *obj, const char *name)\n-{\n-\tfprintf(out, \"%s \", oid_to_hex(&obj->oid));\n-\tfor (const char *p = name; *p && *p != '\\n'; p++)\n-\t\tfputc(*p, out);\n-\tfputc('\\n', out);\n-}\n-\n static void mark_blob_uninteresting(struct blob *blob)\n {\n \tif (!blob)\ndiff --git a/revision.h b/revision.h\nindex 71e984c452..21c6a69899 100644\n--- a/revision.h\n+++ b/revision.h\n@@ -489,8 +489,6 @@ void mark_parents_uninteresting(struct rev_info *revs, struct commit *commit);\n void mark_tree_uninteresting(struct repository *r, struct tree *tree);\n void mark_trees_uninteresting_sparse(struct repository *r, struct oidset *trees);\n \n-void show_object_with_name(FILE *, struct object *, const char *);\n-\n /**\n * Helpers to check if a reference should be excluded.\n */\n-- \n2.49.0.rc2\n\n"},{"id":"514140","messageId":"20250313001706.3390502-3-jltobler@gmail.com","threadId":"63089","inReplyTo":"20250313001706.3390502-1-jltobler@gmail.com","subject":"[PATCH v2 2/6] rev-list: refactor early option parsing","fromName":"Justin Tobler","fromEmail":"jltobler@gmail.com","sentAt":"2025-03-13T00:17:02Z","receivedAt":"2025-03-13T00:21:16Z","isPatch":true,"sender":{"key":"jltobler@gmail.com","avatar":"https://avatars.githubusercontent.com/u/53454972?v=4"},"body":"Before invoking `setup_revisions()`, the `--missing` and\n`--exclude-promisor-objects` options are parsed early. In a subsequent\ncommit, another option is added that must be parsed early.\n\nRefactor the code to parse both options in a single early pass.\n\nSigned-off-by: Justin Tobler \n---\n builtin/rev-list.c | 17 +++++++----------\n 1 file changed, 7 insertions(+), 10 deletions(-)\n\ndiff --git a/builtin/rev-list.c b/builtin/rev-list.c\nindex dcd079c16c..04d9c893b5 100644\n--- a/builtin/rev-list.c\n+++ b/builtin/rev-list.c\n@@ -16,6 +16,7 @@\n #include \"object-file.h\"\n #include \"object-store-ll.h\"\n #include \"pack-bitmap.h\"\n+#include \"parse-options.h\"\n #include \"log-tree.h\"\n #include \"graph.h\"\n #include \"bisect.h\"\n@@ -639,19 +640,15 @@ int cmd_rev_list(int argc,\n \t\tif (!strcmp(arg, \"--exclude-promisor-objects\")) {\n \t\t\tfetch_if_missing = 0;\n \t\t\trevs.exclude_promisor_objects = 1;\n-\t\t\tbreak;\n-\t\t}\n-\t}\n-\tfor (i = 1; i < argc; i++) {\n-\t\tconst char *arg = argv[i];\n-\t\tif (skip_prefix(arg, \"--missing=\", &arg)) {\n-\t\t\tif (revs.exclude_promisor_objects)\n-\t\t\t\tdie(_(\"options '%s' and '%s' cannot be used together\"), \"--exclude-promisor-objects\", \"--missing\");\n-\t\t\tif (parse_missing_action_value(arg))\n-\t\t\t\tbreak;\n+\t\t} else if (skip_prefix(arg, \"--missing=\", &arg)) {\n+\t\t\tparse_missing_action_value(arg);\n \t\t}\n \t}\n \n+\tdie_for_incompatible_opt2(revs.exclude_promisor_objects,\n+\t\t\t\t \"--exclude_promisor_objects\",\n+\t\t\t\t arg_missing_action, \"--missing\");\n+\n \tif (arg_missing_action)\n \t\trevs.do_not_die_on_missing_objects = 1;\n \n-- \n2.49.0.rc2\n\n"},{"id":"514141","messageId":"20250313001706.3390502-4-jltobler@gmail.com","threadId":"63089","inReplyTo":"20250313001706.3390502-1-jltobler@gmail.com","subject":"[PATCH v2 3/6] revision: support NUL-delimited --stdin mode","fromName":"Justin Tobler","fromEmail":"jltobler@gmail.com","sentAt":"2025-03-13T00:17:03Z","receivedAt":"2025-03-13T00:21:16Z","isPatch":true,"sender":{"key":"jltobler@gmail.com","avatar":"https://avatars.githubusercontent.com/u/53454972?v=4"},"body":"When `setup_revisions()` parses the `--stdin` option, revision and\npathspec arguments are read from stdin. Each line of input is handled as\na separate argument.\n\nIntroduce the `nul_delim_stdin` field to `setup_revision_opt` that, when\nenabled, uses a NUL byte to delimit between stdin arguments instead of\nnewline.\n\nSigned-off-by: Justin Tobler \n---\n revision.c | 19 +++++++++++--------\n revision.h | 3 ++-\n 2 files changed, 13 insertions(+), 9 deletions(-)\n\ndiff --git a/revision.c b/revision.c\nindex 0eaebe4478..5de6309830 100644\n--- a/revision.c\n+++ b/revision.c\n@@ -2275,10 +2275,10 @@ int handle_revision_arg(const char *arg, struct rev_info *revs, int flags, unsig\n \treturn ret;\n }\n \n-static void read_pathspec_from_stdin(struct strbuf *sb,\n-\t\t\t\t struct strvec *prune)\n+static void read_pathspec_from_stdin(struct strbuf *sb, struct strvec *prune,\n+\t\t\t\t int line_term)\n {\n-\twhile (strbuf_getline(sb, stdin) != EOF)\n+\twhile (strbuf_getdelim_strip_crlf(sb, stdin, line_term) != EOF)\n \t\tstrvec_push(prune, sb->buf);\n }\n \n@@ -2905,8 +2905,8 @@ static int handle_revision_pseudo_opt(struct rev_info *revs,\n \treturn 1;\n }\n \n-static void read_revisions_from_stdin(struct rev_info *revs,\n-\t\t\t\t struct strvec *prune)\n+static void read_revisions_from_stdin(struct rev_info *revs, struct strvec *prune,\n+\t\t\t\t int line_term)\n {\n \tstruct strbuf sb;\n \tint seen_dashdash = 0;\n@@ -2918,7 +2918,7 @@ static void read_revisions_from_stdin(struct rev_info *revs,\n \twarn_on_object_refname_ambiguity = 0;\n \n \tstrbuf_init(&sb, 1000);\n-\twhile (strbuf_getline(&sb, stdin) != EOF) {\n+\twhile (strbuf_getdelim_strip_crlf(&sb, stdin, line_term) != EOF) {\n \t\tif (!sb.len)\n \t\t\tbreak;\n \n@@ -2946,7 +2946,7 @@ static void read_revisions_from_stdin(struct rev_info *revs,\n \t\t\tdie(\"bad revision '%s'\", sb.buf);\n \t}\n \tif (seen_dashdash)\n-\t\tread_pathspec_from_stdin(&sb, prune);\n+\t\tread_pathspec_from_stdin(&sb, prune, line_term);\n \n \tstrbuf_release(&sb);\n \twarn_on_object_refname_ambiguity = save_warning;\n@@ -3019,13 +3019,16 @@ int setup_revisions(int argc, const char **argv, struct rev_info *revs, struct s\n \t\t\t}\n \n \t\t\tif (!strcmp(arg, \"--stdin\")) {\n+\t\t\t\tint term = opt && opt->nul_delim_stdin ? '\\0' : '\\n';\n+\n \t\t\t\tif (revs->disable_stdin) {\n \t\t\t\t\targv[left++] = arg;\n \t\t\t\t\tcontinue;\n \t\t\t\t}\n \t\t\t\tif (revs->read_from_stdin++)\n \t\t\t\t\tdie(\"--stdin given twice?\");\n-\t\t\t\tread_revisions_from_stdin(revs, &prune_data);\n+\t\t\t\tread_revisions_from_stdin(revs, &prune_data,\n+\t\t\t\t\t\t\t term);\n \t\t\t\tcontinue;\n \t\t\t}\n \ndiff --git a/revision.h b/revision.h\nindex 21c6a69899..0e680c3667 100644\n--- a/revision.h\n+++ b/revision.h\n@@ -439,7 +439,8 @@ struct setup_revision_opt {\n \tvoid (*tweak)(struct rev_info *);\n \tunsigned int\tassume_dashdash:1,\n \t\t\tallow_exclude_promisor_objects:1,\n-\t\t\tfree_removed_argv_elements:1;\n+\t\t\tfree_removed_argv_elements:1,\n+\t\t\tnul_delim_stdin:1;\n \tunsigned revarg_opt;\n };\n int setup_revisions(int argc, const char **argv, struct rev_info *revs,\n-- \n2.49.0.rc2\n\n"},{"id":"514142","messageId":"20250313001706.3390502-5-jltobler@gmail.com","threadId":"63089","inReplyTo":"20250313001706.3390502-1-jltobler@gmail.com","subject":"[PATCH v2 4/6] rev-list: support delimiting objects with NUL bytes","fromName":"Justin Tobler","fromEmail":"jltobler@gmail.com","sentAt":"2025-03-13T00:17:04Z","receivedAt":"2025-03-13T00:21:18Z","isPatch":true,"sender":{"key":"jltobler@gmail.com","avatar":"https://avatars.githubusercontent.com/u/53454972?v=4"},"body":"When walking objects, git-rev-list(1) prints each object entry on a\nseparate line. Some options, such as `--objects`, may print additional\ninformation about tree and blob object on the same line in the form:\n\n $ git rev-list --objects \n SP [] LF\n\nNote that in this form the SP is appended regardless of whether the tree\nor blob object has path information available. Paths containing a\nnewline are also truncated at the newline.\n\nIntroduce the `-z` option for git-rev-list(1) which reformats the output\nto use NUL-delimiters between objects and associated info in the\nfollowing form:\n\n $ git rev-list -z --objects \n NUL [path= NUL]\n\nIn this form, the start of each record is signaled by an OID entry that\nis all hexidecimal and does not contain any '='. Additional path info\nfrom `--objects` is appended to the record as a token/value pair\n`path=` as-is without any truncation.\n\nIn this mode, revision and pathspec arguments provided on stdin with the\n`--stdin` option are also separated by a NUL byte instead of being\nnewline delimited.\n\nFor now, the `--objects` and `--stdin` flag are the only options that\ncan be used in combination with `-z`. In a subsequent commit,\nNUL-delimited support for other options is added. Other options that do\nnot make sense with be used in combination with `-z` are rejected.\n\nSigned-off-by: Justin Tobler \n---\n Documentation/rev-list-options.adoc | 23 ++++++++++++++++++\n builtin/rev-list.c | 36 +++++++++++++++++++++++++----\n t/t6000-rev-list-misc.sh | 35 ++++++++++++++++++++++++++++\n t/t6017-rev-list-stdin.sh | 9 ++++++++\n 4 files changed, 98 insertions(+), 5 deletions(-)\n\ndiff --git a/Documentation/rev-list-options.adoc b/Documentation/rev-list-options.adoc\nindex 785c0786e0..166d3cd19e 100644\n--- a/Documentation/rev-list-options.adoc\n+++ b/Documentation/rev-list-options.adoc\n@@ -361,6 +361,29 @@ ifdef::git-rev-list[]\n --progress=
::\n \tShow progress reports on stderr as objects are considered. The\n \t`
` text will be printed with each progress update.\n+\n+-z::\n+\tInstead of being newline-delimited, each outputted object and its\n+\taccompanying metadata is delimited using NUL bytes in the following\n+\tform:\n++\n+-----------------------------------------------------------------------\n+ NUL [= NUL]...\n+-----------------------------------------------------------------------\n++\n+Additional object metadata, such as object paths, is printed using the\n+`=` form. Token values are printed as-is without any\n+encoding/truncation. An OID entry never contains a '=' character and thus\n+is used to signal the start of a new object record. Examples:\n++\n+-----------------------------------------------------------------------\n+ NUL\n+ NUL path= NUL\n+-----------------------------------------------------------------------\n++\n+This mode is only compatible with the `--objects` output option. Also, revision\n+and pathspec argument parsing on stdin with the `--stdin` option is NUL byte\n+delimited instead of using newlines while in this mode.\n endif::git-rev-list[]\n \n History Simplification\ndiff --git a/builtin/rev-list.c b/builtin/rev-list.c\nindex 04d9c893b5..f048500679 100644\n--- a/builtin/rev-list.c\n+++ b/builtin/rev-list.c\n@@ -65,6 +65,7 @@ static const char rev_list_usage[] =\n \" --abbrev-commit\\n\"\n \" --left-right\\n\"\n \" --count\\n\"\n+\" -z\\n\"\n \" special purpose:\\n\"\n \" --bisect\\n\"\n \" --bisect-vars\\n\"\n@@ -97,6 +98,9 @@ static int arg_show_object_names = 1;\n \n #define DEFAULT_OIDSET_SIZE (16*1024)\n \n+static char line_term = '\\n';\n+static char info_term = ' ';\n+\n static int show_disk_usage;\n static off_t total_disk_usage;\n static int human_readable;\n@@ -264,7 +268,7 @@ static void show_commit(struct commit *commit, void *data)\n \tif (revs->commit_format == CMIT_FMT_ONELINE)\n \t\tputchar(' ');\n \telse if (revs->include_header)\n-\t\tputchar('\\n');\n+\t\tputchar(line_term);\n \n \tif (revs->verbose_header) {\n \t\tstruct strbuf buf = STRBUF_INIT;\n@@ -361,12 +365,16 @@ static void show_object(struct object *obj, const char *name, void *cb_data)\n \tprintf(\"%s\", oid_to_hex(&obj->oid));\n \n \tif (arg_show_object_names) {\n-\t\tputchar(' ');\n-\t\tfor (const char *p = name; *p && *p != '\\n'; p++)\n-\t\t\tputchar(*p);\n+\t\tif (line_term) {\n+\t\t\tputchar(info_term);\n+\t\t\tfor (const char *p = name; *p && *p != '\\n'; p++)\n+\t\t\t\tputchar(*p);\n+\t\t} else if (*name) {\n+\t\t\tprintf(\"%cpath=%s\", info_term, name);\n+\t\t}\n \t}\n \n-\tputchar('\\n');\n+\tputchar(line_term);\n }\n \n static void show_edge(struct commit *commit)\n@@ -642,6 +650,10 @@ int cmd_rev_list(int argc,\n \t\t\trevs.exclude_promisor_objects = 1;\n \t\t} else if (skip_prefix(arg, \"--missing=\", &arg)) {\n \t\t\tparse_missing_action_value(arg);\n+\t\t} else if (!strcmp(arg, \"-z\")) {\n+\t\t\ts_r_opt.nul_delim_stdin = 1;\n+\t\t\tline_term = '\\0';\n+\t\t\tinfo_term = '\\0';\n \t\t}\n \t}\n \n@@ -757,6 +769,20 @@ int cmd_rev_list(int argc,\n \t\tusage(rev_list_usage);\n \n \t}\n+\n+\t/*\n+\t * Reject options currently incompatible with -z. For some options, this\n+\t * is not an inherent limitation and support may be implemented in the\n+\t * future.\n+\t */\n+\tif (!line_term) {\n+\t\tif (revs.graph || revs.verbose_header || show_disk_usage ||\n+\t\t info.show_timestamp || info.header_prefix || bisect_list ||\n+\t\t use_bitmap_index || revs.edge_hint || revs.left_right ||\n+\t\t revs.cherry_mark || arg_missing_action || revs.boundary)\n+\t\t\tdie(_(\"-z option used with unsupported option\"));\n+\t}\n+\n \tif (revs.commit_format != CMIT_FMT_USERFORMAT)\n \t\trevs.include_header = 1;\n \tif (revs.commit_format != CMIT_FMT_UNSPECIFIED) {\ndiff --git a/t/t6000-rev-list-misc.sh b/t/t6000-rev-list-misc.sh\nindex 6289a2e8b0..dfbbc0aee6 100755\n--- a/t/t6000-rev-list-misc.sh\n+++ b/t/t6000-rev-list-misc.sh\n@@ -182,4 +182,39 @@ test_expect_success 'rev-list --unpacked' '\n \ttest_cmp expect actual\n '\n \n+test_expect_success 'rev-list -z' '\n+\ttest_when_finished rm -rf repo &&\n+\n+\tgit init repo &&\n+\ttest_commit -C repo 1 &&\n+\ttest_commit -C repo 2 &&\n+\n+\toid1=$(git -C repo rev-parse HEAD) &&\n+\toid2=$(git -C repo rev-parse HEAD~) &&\n+\n+\tprintf \"%s\\0%s\\0\" \"$oid1\" \"$oid2\" >expect &&\n+\tgit -C repo rev-list -z HEAD >actual &&\n+\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'rev-list -z --objects' '\n+\ttest_when_finished rm -rf repo &&\n+\n+\tgit init repo &&\n+\ttest_commit -C repo 1 &&\n+\ttest_commit -C repo 2 &&\n+\n+\toid1=$(git -C repo rev-parse HEAD:1.t) &&\n+\toid2=$(git -C repo rev-parse HEAD:2.t) &&\n+\tpath1=1.t &&\n+\tpath2=2.t &&\n+\n+\tprintf \"%s\\0path=%s\\0%s\\0path=%s\\0\" \"$oid1\" \"$path1\" \"$oid2\" \"$path2\" \\\n+\t\t>expect &&\n+\tgit -C repo rev-list -z --objects HEAD:1.t HEAD:2.t >actual &&\n+\n+\ttest_cmp expect actual\n+'\n+\n test_done\ndiff --git a/t/t6017-rev-list-stdin.sh b/t/t6017-rev-list-stdin.sh\nindex 4821b90e74..362a8b126a 100755\n--- a/t/t6017-rev-list-stdin.sh\n+++ b/t/t6017-rev-list-stdin.sh\n@@ -148,4 +148,13 @@ test_expect_success '--not via stdin does not influence revisions from command l\n \ttest_cmp expect actual\n '\n \n+test_expect_success 'NUL-delimited stdin' '\n+\tprintf \"%s\\0%s\\0%s\\0\" \"HEAD\" \"--\" \"file-1\" > input &&\n+\n+\tgit rev-list -z --objects HEAD -- file-1 >expect &&\n+\tgit rev-list -z --objects --stdin actual &&\n+\n+\ttest_cmp expect actual\n+'\n+\n test_done\n-- \n2.49.0.rc2\n\n"},{"id":"514143","messageId":"20250313001706.3390502-6-jltobler@gmail.com","threadId":"63089","inReplyTo":"20250313001706.3390502-1-jltobler@gmail.com","subject":"[PATCH v2 5/6] rev-list: support NUL-delimited --boundary option","fromName":"Justin Tobler","fromEmail":"jltobler@gmail.com","sentAt":"2025-03-13T00:17:05Z","receivedAt":"2025-03-13T00:21:18Z","isPatch":true,"sender":{"key":"jltobler@gmail.com","avatar":"https://avatars.githubusercontent.com/u/53454972?v=4"},"body":"The `--boundary` option for git-rev-list(1) prints boundary objects\nfound while performing the object walk in the form:\n\n $ git rev-list --boundary \n - LF\n\nAdd support for printing boundary objects in a NUL-delimited format when\nthe `-z` option is enabled.\n\n $ git rev-list -z --boundary \n NUL boundary=yes NUL\n\nIn this mode, instead of prefixing the boundary OID with '-', a separate\n`boundary=yes` token/value pair is appended.\n\nSigned-off-by: Justin Tobler \n---\n Documentation/rev-list-options.adoc | 16 +++++++++-------\n builtin/rev-list.c | 9 +++++++--\n t/t6000-rev-list-misc.sh | 16 ++++++++++++++++\n 3 files changed, 32 insertions(+), 9 deletions(-)\n\ndiff --git a/Documentation/rev-list-options.adoc b/Documentation/rev-list-options.adoc\nindex 166d3cd19e..d400d76cf2 100644\n--- a/Documentation/rev-list-options.adoc\n+++ b/Documentation/rev-list-options.adoc\n@@ -371,19 +371,21 @@ ifdef::git-rev-list[]\n NUL [= NUL]...\n -----------------------------------------------------------------------\n +\n-Additional object metadata, such as object paths, is printed using the\n-`=` form. Token values are printed as-is without any\n-encoding/truncation. An OID entry never contains a '=' character and thus\n-is used to signal the start of a new object record. Examples:\n+Additional object metadata, such as object paths or boundary objects, is\n+printed using the `=` form. Token values are printed as-is\n+without any encoding/truncation. An OID entry never contains a '=' character\n+and thus is used to signal the start of a new object record. Examples:\n +\n -----------------------------------------------------------------------\n NUL\n NUL path= NUL\n+ NUL boundary=yes NUL\n -----------------------------------------------------------------------\n +\n-This mode is only compatible with the `--objects` output option. Also, revision\n-and pathspec argument parsing on stdin with the `--stdin` option is NUL byte\n-delimited instead of using newlines while in this mode.\n+This mode is only compatible with the `--objects` and `--boundary` output\n+options. Also, revision and pathspec argument parsing on stdin with the\n+`--stdin` option is NUL byte delimited instead of using newlines while in this\n+mode.\n endif::git-rev-list[]\n \n History Simplification\ndiff --git a/builtin/rev-list.c b/builtin/rev-list.c\nindex f048500679..7c6d4b25b0 100644\n--- a/builtin/rev-list.c\n+++ b/builtin/rev-list.c\n@@ -240,13 +240,18 @@ static void show_commit(struct commit *commit, void *data)\n \t\tfputs(info->header_prefix, stdout);\n \n \tif (revs->include_header) {\n-\t\tif (!revs->graph)\n+\t\tif (!revs->graph && line_term)\n \t\t\tfputs(get_revision_mark(revs, commit), stdout);\n \t\tif (revs->abbrev_commit && revs->abbrev)\n \t\t\tfputs(repo_find_unique_abbrev(the_repository, &commit->object.oid, revs->abbrev),\n \t\t\t stdout);\n \t\telse\n \t\t\tfputs(oid_to_hex(&commit->object.oid), stdout);\n+\n+\t\tif (!line_term) {\n+\t\t\tif (commit->object.flags & BOUNDARY)\n+\t\t\t\tprintf(\"%cboundary=yes\", info_term);\n+\t\t}\n \t}\n \tif (revs->print_parents) {\n \t\tstruct commit_list *parents = commit->parents;\n@@ -779,7 +784,7 @@ int cmd_rev_list(int argc,\n \t\tif (revs.graph || revs.verbose_header || show_disk_usage ||\n \t\t info.show_timestamp || info.header_prefix || bisect_list ||\n \t\t use_bitmap_index || revs.edge_hint || revs.left_right ||\n-\t\t revs.cherry_mark || arg_missing_action || revs.boundary)\n+\t\t revs.cherry_mark || arg_missing_action)\n \t\t\tdie(_(\"-z option used with unsupported option\"));\n \t}\n \ndiff --git a/t/t6000-rev-list-misc.sh b/t/t6000-rev-list-misc.sh\nindex dfbbc0aee6..349bf5ec3d 100755\n--- a/t/t6000-rev-list-misc.sh\n+++ b/t/t6000-rev-list-misc.sh\n@@ -217,4 +217,20 @@ test_expect_success 'rev-list -z --objects' '\n \ttest_cmp expect actual\n '\n \n+test_expect_success 'rev-list -z --boundary' '\n+\ttest_when_finished rm -rf repo &&\n+\n+\tgit init repo &&\n+\ttest_commit -C repo 1 &&\n+\ttest_commit -C repo 2 &&\n+\n+\toid1=$(git -C repo rev-parse HEAD) &&\n+\toid2=$(git -C repo rev-parse HEAD~) &&\n+\n+\tprintf \"%s\\0%s\\0boundary=yes\\0\" \"$oid1\" \"$oid2\" >expect &&\n+\tgit -C repo rev-list -z --boundary HEAD~.. >actual &&\n+\n+\ttest_cmp expect actual\n+'\n+\n test_done\n-- \n2.49.0.rc2\n\n"},{"id":"514144","messageId":"20250313001706.3390502-7-jltobler@gmail.com","threadId":"63089","inReplyTo":"20250313001706.3390502-1-jltobler@gmail.com","subject":"[PATCH v2 6/6] rev-list: support NUL-delimited --missing option","fromName":"Justin Tobler","fromEmail":"jltobler@gmail.com","sentAt":"2025-03-13T00:17:06Z","receivedAt":"2025-03-13T00:21:19Z","isPatch":true,"sender":{"key":"jltobler@gmail.com","avatar":"https://avatars.githubusercontent.com/u/53454972?v=4"},"body":"The `--missing={print,print-info}` option for git-rev-list(1) prints\nmissing objects found while performing the object walk in the form:\n\n $ git rev-list --missing=print-info \n ? [SP =]... LF\n\nAdd support for printing missing objects in a NUL-delimited format when\nthe `-z` option is enabled.\n\n $ git rev-list -z --missing=print-info \n NUL missing=yes NUL [= NUL]...\n\nIn this mode, values containing special characters or spaces are printed\nas-is without being escaped or quoted. Instead of prefixing the missing\nOID with '?', a separate `missing=yes` token/value pair is appended.\n\nSigned-off-by: Justin Tobler \n---\n Documentation/rev-list-options.adoc | 13 ++++++------\n builtin/rev-list.c | 29 ++++++++++++++++++++-------\n t/t6022-rev-list-missing.sh | 31 +++++++++++++++++++++++++++++\n 3 files changed, 60 insertions(+), 13 deletions(-)\n\ndiff --git a/Documentation/rev-list-options.adoc b/Documentation/rev-list-options.adoc\nindex d400d76cf2..145ded5c78 100644\n--- a/Documentation/rev-list-options.adoc\n+++ b/Documentation/rev-list-options.adoc\n@@ -371,8 +371,8 @@ ifdef::git-rev-list[]\n NUL [= NUL]...\n -----------------------------------------------------------------------\n +\n-Additional object metadata, such as object paths or boundary objects, is\n-printed using the `=` form. Token values are printed as-is\n+Additional object metadata, such as object paths or boundary/missing objects,\n+is printed using the `=` form. Token values are printed as-is\n without any encoding/truncation. An OID entry never contains a '=' character\n and thus is used to signal the start of a new object record. Examples:\n +\n@@ -380,12 +380,13 @@ and thus is used to signal the start of a new object record. Examples:\n NUL\n NUL path= NUL\n NUL boundary=yes NUL\n+ NUL missing=yes NUL [= NUL]...\n -----------------------------------------------------------------------\n +\n-This mode is only compatible with the `--objects` and `--boundary` output\n-options. Also, revision and pathspec argument parsing on stdin with the\n-`--stdin` option is NUL byte delimited instead of using newlines while in this\n-mode.\n+This mode is only compatible with the `--objects`, `--boundary`, and\n+`--missing` output options. Also, revision and pathspec argument parsing on\n+stdin with the `--stdin` option is NUL byte delimited instead of using newlines\n+while in this mode.\n endif::git-rev-list[]\n \n History Simplification\ndiff --git a/builtin/rev-list.c b/builtin/rev-list.c\nindex 7c6d4b25b0..d7b4dd48ff 100644\n--- a/builtin/rev-list.c\n+++ b/builtin/rev-list.c\n@@ -136,24 +136,39 @@ static void print_missing_object(struct missing_objects_map_entry *entry,\n {\n \tstruct strbuf sb = STRBUF_INIT;\n \n+\tif (line_term)\n+\t\tputchar('?');\n+\n+\tprintf(\"%s\", oid_to_hex(&entry->entry.oid));\n+\n+\tif (!line_term)\n+\t\tprintf(\"%cmissing=yes\", info_term);\n+\n \tif (!print_missing_info) {\n-\t\tprintf(\"?%s\\n\", oid_to_hex(&entry->entry.oid));\n+\t\tputchar(line_term);\n \t\treturn;\n \t}\n \n \tif (entry->path && *entry->path) {\n \t\tstruct strbuf path = STRBUF_INIT;\n \n-\t\tstrbuf_addstr(&sb, \" path=\");\n-\t\tquote_path(entry->path, NULL, &path, QUOTE_PATH_QUOTE_SP);\n-\t\tstrbuf_addbuf(&sb, &path);\n+\t\tstrbuf_addf(&sb, \"%cpath=\", info_term);\n+\n+\t\tif (line_term) {\n+\t\t\tquote_path(entry->path, NULL, &path, QUOTE_PATH_QUOTE_SP);\n+\t\t\tstrbuf_addbuf(&sb, &path);\n+\t\t} else {\n+\t\t\tstrbuf_addstr(&sb, entry->path);\n+\t\t}\n \n \t\tstrbuf_release(&path);\n \t}\n \tif (entry->type)\n-\t\tstrbuf_addf(&sb, \" type=%s\", type_name(entry->type));\n+\t\tstrbuf_addf(&sb, \"%ctype=%s\", info_term, type_name(entry->type));\n+\n+\tfwrite(sb.buf, sizeof(char), sb.len, stdout);\n+\tputchar(line_term);\n \n-\tprintf(\"?%s%s\\n\", oid_to_hex(&entry->entry.oid), sb.buf);\n \tstrbuf_release(&sb);\n }\n \n@@ -784,7 +799,7 @@ int cmd_rev_list(int argc,\n \t\tif (revs.graph || revs.verbose_header || show_disk_usage ||\n \t\t info.show_timestamp || info.header_prefix || bisect_list ||\n \t\t use_bitmap_index || revs.edge_hint || revs.left_right ||\n-\t\t revs.cherry_mark || arg_missing_action)\n+\t\t revs.cherry_mark)\n \t\t\tdie(_(\"-z option used with unsupported option\"));\n \t}\n \ndiff --git a/t/t6022-rev-list-missing.sh b/t/t6022-rev-list-missing.sh\nindex 3e2790d4c8..08e92dd002 100755\n--- a/t/t6022-rev-list-missing.sh\n+++ b/t/t6022-rev-list-missing.sh\n@@ -198,4 +198,35 @@ do\n \t'\n done\n \n+test_expect_success \"-z nul-delimited --missing\" '\n+\ttest_when_finished rm -rf repo &&\n+\n+\tgit init repo &&\n+\t(\n+\t\tcd repo &&\n+\t\tgit commit --allow-empty -m first &&\n+\n+\t\tpath=\"foo bar\" &&\n+\t\techo foobar >\"$path\" &&\n+\t\tgit add -A &&\n+\t\tgit commit -m second &&\n+\n+\t\toid=$(git rev-parse \"HEAD:$path\") &&\n+\t\ttype=\"$(git cat-file -t $oid)\" &&\n+\n+\t\tobj_path=\".git/objects/$(test_oid_to_path $oid)\" &&\n+\n+\t\tgit rev-list -z --objects --no-object-names \\\n+\t\t\tHEAD ^\"$oid\" >expect &&\n+\t\tprintf \"%s\\0missing=yes\\0path=%s\\0type=%s\\0\" \"$oid\" \"$path\" \\\n+\t\t\t\"$type\" >>expect &&\n+\n+\t\tmv \"$obj_path\" \"$obj_path.hidden\" &&\n+\t\tgit rev-list -z --objects --no-object-names \\\n+\t\t\t--missing=print-info HEAD >actual &&\n+\n+\t\ttest_cmp expect actual\n+\t)\n+'\n+\n test_done\n-- \n2.49.0.rc2\n\n"},{"id":"514156","messageId":"20250313053329.GC94015@coredump.intra.peff.net","threadId":"63089","inReplyTo":"sm5xvh7bipyrlpw45bptlctbuhey7v3qktpskal5dzexomt34f@wgfziyhcf3t6","subject":"Re: [PATCH 0/4] rev-list: introduce NUL-delimited output mode","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2025-03-13T05:33:29Z","receivedAt":"2025-03-13T05:33:30Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Mar 12, 2025 at 05:09:41PM -0500, Justin Tobler wrote:\n\n> > If it is possible to see some effect from \"-z\" now (I didn't dig\n> > very far), then it may be better to continue to let the diff\n> > options parser handle it, and simply pick the result out of\n> > revs.diffopt.line_termination. As your patch 3 is written, I think\n> > the diff code probably doesn't see it anymore at all.\n> \n> As currently implemented, the early parsing of -z doesn't effect the\n> diff options parsing in `setup_revisions()`. The early parsing doesn't\n> remove the option and thus it continues to be set in the diff options.\n\nAh, OK. From the diff context I didn't realize it was not in the main\noption parsing loop. That makes sense.\n\n> Furthermore, revision and pathspec argument parsing is all handled in\n> `setup_revisions()` so if we want to NUL-delimit arguments parsed on\n> stdin with -z, we would still need to parse the option early anyway. I\n> think it should be fine to leave the early -z option parsing as-is.\n\nMakes sense. And I guess we might not want to have setup_revisions() do\nthat handling of \"-z\" for input, as that would make:\n\n git log --stdin --raw -z\n\nbehave differently (since it does not currently change stdin handling,\nonly the diff output). Though that does mean that these two commands\nwill behave differently:\n\n git log --stdin -z\n git rev-list --stdin -z\n\nwhich seems...not great. My earlier suggestion to tie \"-z\" to stdin\nhandling was for consistency with other tools like grep. But if we\nalready have cases where \"-z\" is only for output, maybe it is better to\nstay consistent with other parts of git. I.e., I was worried about us\npainting ourselves into a corner with your patches, but we may have\nalready done so years ago. ;)\n\n-Peff\n"},{"id":"514166","messageId":"Z9KNQ8XliWrrYgAT@pks.im","threadId":"63089","inReplyTo":"xmqqfrji1bvy.fsf@gitster.g","subject":"Re: [PATCH 0/4] rev-list: introduce NUL-delimited output mode","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-03-13T07:46:11Z","receivedAt":"2025-03-13T07:46:20Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Wed, Mar 12, 2025 at 08:56:01AM -0700, Junio C Hamano wrote:\n> Patrick Steinhardt writes:\n> \n> > On Tue, Mar 11, 2025 at 07:57:20PM -0400, Jeff King wrote:\n> >> On Mon, Mar 10, 2025 at 02:28:25PM -0500, Justin Tobler wrote:\n> >> \n> >> > To make machine parsing easier, this series introduces a NUL-delimited\n> >> > output mode for git-rev-list(1) via a `-z` option following a suggestion\n> >> > from Junio in a previous thread[1]. In this mode, instead of LF, each\n> >> > object is delimited with two NUL bytes and any object metadata is\n> >> > separated with a single NUL byte. Examples:\n> >> > \n> >> > NUL NUL\n> >> > [NUL ] NUL NUL\n> >> > ? [NUL =]... NUL NUL\n> >> > \n> >> > In this mode, path and value info are printed as-is without any special\n> >> > encoding or truncation.\n> >> \n> >> I think this is a good direction, but I have two compatibility\n> >> questions:\n> >> \n> >> 1. What should \"git rev-list -z --stdin\" do? In most other programs\n> >> with a \"-z\" option it affects both input and output. I don't\n> >> particularly care about this case myself, but it will be hard to\n> >> change later. So we probably want to decide now.\n> >\n> > I would lean into the direction of making \"-z\" change the format both\n> > for stdin and stdout. That's what we do in most cases, and in those\n> > cases where we didn't we came to regret it (git-cat-file(1)).\n> \n> I've seen \"-Z\", in addition to \"-z\", used to differentiate between\n> input and output in some commands. If we are not going to do that,\n> I agree that making \"-z\" to affect both input and output is less\n> surprising than having to remember which side is still text.\n\nYeah, \"-Z\" is what I meant with git-cat-file(1) above. I just think it's\nsafer to always use NUL as delimiter in machine parsing -- and while one\noften thinks initially that it may not be needed either in stdin or\nstdout, my learning is that eventually you always find an edge case\nwhere you do need NUL termination in both. So I'd rather want to push\napplication developers into the right direction and make \"-z\" adjust\nboth streams.\n\nPatrick\n"},{"id":"514179","messageId":"Z9LVxt8urwRXLLcL@pks.im","threadId":"63089","inReplyTo":"20250313001706.3390502-5-jltobler@gmail.com","subject":"Re: [PATCH v2 4/6] rev-list: support delimiting objects with NUL bytes","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-03-13T12:55:34Z","receivedAt":"2025-03-13T12:55:43Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Wed, Mar 12, 2025 at 07:17:04PM -0500, Justin Tobler wrote:\n> diff --git a/Documentation/rev-list-options.adoc b/Documentation/rev-list-options.adoc\n> index 785c0786e0..166d3cd19e 100644\n> --- a/Documentation/rev-list-options.adoc\n> +++ b/Documentation/rev-list-options.adoc\n> @@ -361,6 +361,29 @@ ifdef::git-rev-list[]\n> --progress=
::\n> \tShow progress reports on stderr as objects are considered. The\n> \t`
` text will be printed with each progress update.\n> +\n> +-z::\n> +\tInstead of being newline-delimited, each outputted object and its\n> +\taccompanying metadata is delimited using NUL bytes in the following\n> +\tform:\n> ++\n> +-----------------------------------------------------------------------\n> + NUL [= NUL]...\n> +-----------------------------------------------------------------------\n> ++\n> +Additional object metadata, such as object paths, is printed using the\n> +`=` form. Token values are printed as-is without any\n> +encoding/truncation. An OID entry never contains a '=' character and thus\n> +is used to signal the start of a new object record. Examples:\n> ++\n> +-----------------------------------------------------------------------\n> + NUL\n> + NUL path= NUL\n> +-----------------------------------------------------------------------\n> ++\n> +This mode is only compatible with the `--objects` output option. Also, revision\n> +and pathspec argument parsing on stdin with the `--stdin` option is NUL byte\n> +delimited instead of using newlines while in this mode.\n> endif::git-rev-list[]\n> \n> History Simplification\n\nI feel like this last paragraph, where we talk about `--stdin` being\nNUL-delimited, should already be mentioned in the first paragraph.\n\nPatrick\n"},{"id":"514178","messageId":"Z9LVzWX4-6KGV6S7@pks.im","threadId":"63089","inReplyTo":"20250313001706.3390502-7-jltobler@gmail.com","subject":"Re: [PATCH v2 6/6] rev-list: support NUL-delimited --missing option","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-03-13T12:55:41Z","receivedAt":"2025-03-13T12:55:44Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Wed, Mar 12, 2025 at 07:17:06PM -0500, Justin Tobler wrote:\n> diff --git a/Documentation/rev-list-options.adoc b/Documentation/rev-list-options.adoc\n> index d400d76cf2..145ded5c78 100644\n> --- a/Documentation/rev-list-options.adoc\n> +++ b/Documentation/rev-list-options.adoc\n> @@ -371,8 +371,8 @@ ifdef::git-rev-list[]\n> NUL [= NUL]...\n> -----------------------------------------------------------------------\n> +\n> -Additional object metadata, such as object paths or boundary objects, is\n> -printed using the `=` form. Token values are printed as-is\n> +Additional object metadata, such as object paths or boundary/missing objects,\n> +is printed using the `=` form. Token values are printed as-is\n> without any encoding/truncation. An OID entry never contains a '=' character\n> and thus is used to signal the start of a new object record. Examples:\n> +\n\nNit: I don't think we need to update this paragraph here as it is\nwritten as a non-exhaustive list anyway.\n\n> diff --git a/builtin/rev-list.c b/builtin/rev-list.c\n> index 7c6d4b25b0..d7b4dd48ff 100644\n> --- a/builtin/rev-list.c\n> +++ b/builtin/rev-list.c\n> @@ -136,24 +136,39 @@ static void print_missing_object(struct missing_objects_map_entry *entry,\n> {\n> \tstruct strbuf sb = STRBUF_INIT;\n> \n> +\tif (line_term)\n> +\t\tputchar('?');\n> +\n> +\tprintf(\"%s\", oid_to_hex(&entry->entry.oid));\n> +\n> +\tif (!line_term)\n> +\t\tprintf(\"%cmissing=yes\", info_term);\n> +\n> \tif (!print_missing_info) {\n> -\t\tprintf(\"?%s\\n\", oid_to_hex(&entry->entry.oid));\n> +\t\tputchar(line_term);\n> \t\treturn;\n> \t}\n> \n> \tif (entry->path && *entry->path) {\n> \t\tstruct strbuf path = STRBUF_INIT;\n\nNit: the variable and its cleanup could be moved closer to where it's\nused.\n\n> -\t\tstrbuf_addstr(&sb, \" path=\");\n> -\t\tquote_path(entry->path, NULL, &path, QUOTE_PATH_QUOTE_SP);\n> -\t\tstrbuf_addbuf(&sb, &path);\n> +\t\tstrbuf_addf(&sb, \"%cpath=\", info_term);\n> +\n> +\t\tif (line_term) {\n> +\t\t\tquote_path(entry->path, NULL, &path, QUOTE_PATH_QUOTE_SP);\n> +\t\t\tstrbuf_addbuf(&sb, &path);\n> +\t\t} else {\n> +\t\t\tstrbuf_addstr(&sb, entry->path);\n> +\t\t}\n> \n> \t\tstrbuf_release(&path);\n> \t}\n> \tif (entry->type)\n> -\t\tstrbuf_addf(&sb, \" type=%s\", type_name(entry->type));\n> +\t\tstrbuf_addf(&sb, \"%ctype=%s\", info_term, type_name(entry->type));\n> +\n> +\tfwrite(sb.buf, sizeof(char), sb.len, stdout);\n> +\tputchar(line_term);\n> \n> -\tprintf(\"?%s%s\\n\", oid_to_hex(&entry->entry.oid), sb.buf);\n> \tstrbuf_release(&sb);\n> }\n> \n\nPatrick\n"},{"id":"514195","messageId":"e6d4eld6brhris7gmov6zwn2niywmlc3ik2aqoz4bquvpzwurg@g5mlot5mvrta","threadId":"63089","inReplyTo":"Z9LVxt8urwRXLLcL@pks.im","subject":"Re: [PATCH v2 4/6] rev-list: support delimiting objects with NUL bytes","fromName":"Justin Tobler","fromEmail":"jltobler@gmail.com","sentAt":"2025-03-13T14:44:59Z","receivedAt":"2025-03-13T14:48:29Z","isPatch":true,"sender":{"key":"jltobler@gmail.com","avatar":"https://avatars.githubusercontent.com/u/53454972?v=4"},"body":"On 25/03/13 01:55PM, Patrick Steinhardt wrote:\n> On Wed, Mar 12, 2025 at 07:17:04PM -0500, Justin Tobler wrote:\n> > diff --git a/Documentation/rev-list-options.adoc b/Documentation/rev-list-options.adoc\n> > index 785c0786e0..166d3cd19e 100644\n> > --- a/Documentation/rev-list-options.adoc\n> > +++ b/Documentation/rev-list-options.adoc\n> > @@ -361,6 +361,29 @@ ifdef::git-rev-list[]\n> > --progress=
::\n> > \tShow progress reports on stderr as objects are considered. The\n> > \t`
` text will be printed with each progress update.\n> > +\n> > +-z::\n> > +\tInstead of being newline-delimited, each outputted object and its\n> > +\taccompanying metadata is delimited using NUL bytes in the following\n> > +\tform:\n> > ++\n> > +-----------------------------------------------------------------------\n> > + NUL [= NUL]...\n> > +-----------------------------------------------------------------------\n> > ++\n> > +Additional object metadata, such as object paths, is printed using the\n> > +`=` form. Token values are printed as-is without any\n> > +encoding/truncation. An OID entry never contains a '=' character and thus\n> > +is used to signal the start of a new object record. Examples:\n> > ++\n> > +-----------------------------------------------------------------------\n> > + NUL\n> > + NUL path= NUL\n> > +-----------------------------------------------------------------------\n> > ++\n> > +This mode is only compatible with the `--objects` output option. Also, revision\n> > +and pathspec argument parsing on stdin with the `--stdin` option is NUL byte\n> > +delimited instead of using newlines while in this mode.\n> > endif::git-rev-list[]\n> > \n> > History Simplification\n> \n> I feel like this last paragraph, where we talk about `--stdin` being\n> NUL-delimited, should already be mentioned in the first paragraph.\n\nThat's fair. I'll move the `--stdin` part to the beginning.\n\n-Justin\n"},{"id":"514196","messageId":"kmfjcyib7ur2qcadzjfq2gjmi5rdpdehjtb3locsw4emmnbr54@qyrnghyf2jg6","threadId":"63089","inReplyTo":"Z9LVzWX4-6KGV6S7@pks.im","subject":"Re: [PATCH v2 6/6] rev-list: support NUL-delimited --missing option","fromName":"Justin Tobler","fromEmail":"jltobler@gmail.com","sentAt":"2025-03-13T14:51:41Z","receivedAt":"2025-03-13T14:55:11Z","isPatch":true,"sender":{"key":"jltobler@gmail.com","avatar":"https://avatars.githubusercontent.com/u/53454972?v=4"},"body":"On 25/03/13 01:55PM, Patrick Steinhardt wrote:\n> On Wed, Mar 12, 2025 at 07:17:06PM -0500, Justin Tobler wrote:\n> > diff --git a/Documentation/rev-list-options.adoc b/Documentation/rev-list-options.adoc\n> > index d400d76cf2..145ded5c78 100644\n> > --- a/Documentation/rev-list-options.adoc\n> > +++ b/Documentation/rev-list-options.adoc\n> > @@ -371,8 +371,8 @@ ifdef::git-rev-list[]\n> > NUL [= NUL]...\n> > -----------------------------------------------------------------------\n> > +\n> > -Additional object metadata, such as object paths or boundary objects, is\n> > -printed using the `=` form. Token values are printed as-is\n> > +Additional object metadata, such as object paths or boundary/missing objects,\n> > +is printed using the `=` form. Token values are printed as-is\n> > without any encoding/truncation. An OID entry never contains a '=' character\n> > and thus is used to signal the start of a new object record. Examples:\n> > +\n> \n> Nit: I don't think we need to update this paragraph here as it is\n> written as a non-exhaustive list anyway.\n\nOk, I'll omit this change and only keep the added example.\n\n> > diff --git a/builtin/rev-list.c b/builtin/rev-list.c\n> > index 7c6d4b25b0..d7b4dd48ff 100644\n> > --- a/builtin/rev-list.c\n> > +++ b/builtin/rev-list.c\n> > @@ -136,24 +136,39 @@ static void print_missing_object(struct missing_objects_map_entry *entry,\n> > {\n> > \tstruct strbuf sb = STRBUF_INIT;\n> > \n> > +\tif (line_term)\n> > +\t\tputchar('?');\n> > +\n> > +\tprintf(\"%s\", oid_to_hex(&entry->entry.oid));\n> > +\n> > +\tif (!line_term)\n> > +\t\tprintf(\"%cmissing=yes\", info_term);\n> > +\n> > \tif (!print_missing_info) {\n> > -\t\tprintf(\"?%s\\n\", oid_to_hex(&entry->entry.oid));\n> > +\t\tputchar(line_term);\n> > \t\treturn;\n> > \t}\n> > \n> > \tif (entry->path && *entry->path) {\n> > \t\tstruct strbuf path = STRBUF_INIT;\n> \n> Nit: the variable and its cleanup could be moved closer to where it's\n> used.\n\nWill do. Thanks\n\n-Justin\n"},{"id":"514208","messageId":"qapd7sijizopaefpxh3slgn7g2ecwdtyhzqzufcypciyv6n3qm@turmldwzipwe","threadId":"63089","inReplyTo":"20250313053329.GC94015@coredump.intra.peff.net","subject":"Re: [PATCH 0/4] rev-list: introduce NUL-delimited output mode","fromName":"Justin Tobler","fromEmail":"jltobler@gmail.com","sentAt":"2025-03-13T16:41:56Z","receivedAt":"2025-03-13T16:45:26Z","isPatch":true,"sender":{"key":"jltobler@gmail.com","avatar":"https://avatars.githubusercontent.com/u/53454972?v=4"},"body":"On 25/03/13 01:33AM, Jeff King wrote:\n> > Furthermore, revision and pathspec argument parsing is all handled in\n> > `setup_revisions()` so if we want to NUL-delimit arguments parsed on\n> > stdin with -z, we would still need to parse the option early anyway. I\n> > think it should be fine to leave the early -z option parsing as-is.\n> \n> Makes sense. And I guess we might not want to have setup_revisions() do\n> that handling of \"-z\" for input, as that would make:\n> \n> git log --stdin --raw -z\n> \n> behave differently (since it does not currently change stdin handling,\n> only the diff output). \n\nYes, we won't want to include this '-z' parsing directly in\n`setup_revisions()` or else it would change the behavior of other\ncommands.\n\nIn version two of this series, NUL-delimited stdin handling for\n`setup_revisions()` is triggered by setting a `nul_delim_stdin` field in\n`setup_revision_opt`. This gives the `setup_revisions()` caller the\nability to control the parsing delimiter itself. \n\nOnly in git-rev-list(1) does the stdin parsing behavior change if '-z'\nis also present. The behavior stdin parsing for `git log -z --stdin`\nremains unchanged.\n\n> Though that does mean that these two commands\n> will behave differently:\n> \n> git log --stdin -z\n> git rev-list --stdin -z\n> \n> which seems...not great. My earlier suggestion to tie \"-z\" to stdin\n> handling was for consistency with other tools like grep. But if we\n> already have cases where \"-z\" is only for output, maybe it is better to\n> stay consistent with other parts of git. I.e., I was worried about us\n> painting ourselves into a corner with your patches, but we may have\n> already done so years ago. ;)\n\nI think to some extent Git is already inconsistent here. IMO it would be\npreferable for both input and output to use NUL as the delimiter when\nmachine parsing in git-rev-list(1) as that is the behavior I would\npersonally expect. I also agree with Patrick's reasoning else where in\nthis thread[1].\n\nI'm open to discuss further though :)\n\nThanks,\n-Justin\n\n[1]: \n"},{"id":"514236","messageId":"20250313235747.9583-1-jltobler@gmail.com","threadId":"63089","inReplyTo":"20250313001706.3390502-1-jltobler@gmail.com","subject":"[PATCH v3 0/6] rev-list: introduce NUL-delimited output mode","fromName":"Justin Tobler","fromEmail":"jltobler@gmail.com","sentAt":"2025-03-13T23:57:41Z","receivedAt":"2025-03-14T00:01:40Z","isPatch":true,"sender":{"key":"jltobler@gmail.com","avatar":"https://avatars.githubusercontent.com/u/53454972?v=4"},"body":"When walking objects, git-rev-list(1) prints each object entry on a\nseparate line in the form:\n\n LF\n\nSome options, such as `--objects`, may print additional information\nabout the object on the same line:\n\n SP [] LF\n\nIn this mode, if the object path contains a newline it is truncated at\nthe newline.\n\nThe `--boundary` option also modifies output by prefixing boundary\nobjects with `-`:\n\n - LF\n\nWhen the `--missing={print,print-info}` option is provided, information\nabout any missing objects encountered during the object walk are also\nprinted in the form:\n\n ? [SP =]... LF\n\nwhere values containing LF or SP are printed in a token specific fashion\nso that the resulting encoded value does not contain either of these two\nproblematic bytes. For example, missing object paths are quoted in the C\nstyle when they contain LF or SP.\n\nTo make machine parsing easier, this series introduces a NUL-delimited\noutput mode for git-rev-list(1) via a `-z` option. In this mode, the\noutput format for object records is unified such that each object and\nits accompanying metadata is formatted without relying on object\nmetadata order. This format follows the existing `=` used\nby the `--missing` option to represent object metadata in the form:\n\n NUL [= NUL]...\n\n # Examples\n LF -> NUL\n SP LF -> NUL path= NUL\n - LF -> NUL boundary=yes NUL\n ? [SP =]... -> NUL missing=yes NUL [= NUL]...\n\nNote that token value info is printed as-is without any special encoding\nor truncation. Prefixes such as '-' and '?' are dropped in favor using a\ntoken/value pair to signal the same information.\n\nWhile in this mode, if the `--sdtin` option is used, revision and\npathspec arguments read from stdin are separated with a NUL byte instead\nof being newline delimited.\n\nFor now this series only adds support for use with the `--objects`,\n`--boundary` and `--missing` output options. Usage of `-z` with other\noptions is rejected, so it can potentially be added in the future.\n\nThis series is structured as follows:\n\n - Patches 1 and 2 do some minor preparatory refactors.\n\n - Patch 3 modifies stdin argument parsing handled by\n `setup_revisions()` to support NUL-delimited arguments.\n\n - Patch 4 adds the `-z` option to git-rev-list(1) to print\n objects in a NUL-delimited fashion. Arguments parsed on stdin\n while in the mode are also NUL-delimited.\n\n - Patch 5 teaches the `--boundary` option how to print info in a\n NUL-delimited fashino using the unified output format.\n\n - Patch 6 teaches the `--missing` option how to print info in a\n NUL-delimited fashion using the unified output format.\n\nChanges since V2:\n\n - In patch 4, the documentation for the -z option now points out\n the `--stdin` behavior change earlier.\n\n - Minor code style and documentation changes in patch 6.\n\nChanges since V1:\n\n - Use unified output format with `=` pairs for\n all object metadata.\n\n - Add support for the `--boundary` option in NUL-delimited mode.\n\n - Add support for NUL-delimited stdin argument parsing in\n NUL-delimited mode.\n\n - Instead of using two NUL bytes to delimit between object\n records, a single NUL byte is used. Now that object metadata\n is always in the form `=`, we know a new object\n record starts when there is an OID entry which will not\n contain '='.\n\nThanks for taking a look,\n-Justin\n\nJustin Tobler (6):\n rev-list: inline `show_object_with_name()` in `show_object()`\n rev-list: refactor early option parsing\n revision: support NUL-delimited --stdin mode\n rev-list: support delimiting objects with NUL bytes\n rev-list: support NUL-delimited --boundary option\n rev-list: support NUL-delimited --missing option\n\n Documentation/rev-list-options.adoc | 26 ++++++++\n builtin/rev-list.c | 94 +++++++++++++++++++++--------\n revision.c | 27 ++++-----\n revision.h | 5 +-\n t/t6000-rev-list-misc.sh | 51 ++++++++++++++++\n t/t6017-rev-list-stdin.sh | 9 +++\n t/t6022-rev-list-missing.sh | 31 ++++++++++\n 7 files changed, 200 insertions(+), 43 deletions(-)\n\nRange-diff against v2:\n1: d2eded3ac7 = 1: d2eded3ac7 rev-list: inline `show_object_with_name()` in `show_object()`\n2: 03cd08c859 = 2: 03cd08c859 rev-list: refactor early option parsing\n3: 803a49933a = 3: 803a49933a revision: support NUL-delimited --stdin mode\n4: d3b3c4ef89 ! 4: 8eb7669089 rev-list: support delimiting objects with NUL bytes\n @@ Documentation/rev-list-options.adoc: ifdef::git-rev-list[]\n +\n +-z::\n +\tInstead of being newline-delimited, each outputted object and its\n -+\taccompanying metadata is delimited using NUL bytes in the following\n -+\tform:\n ++\taccompanying metadata is delimited using NUL bytes. In this mode, when\n ++\tthe `--stdin` option is provided, revision and pathspec arguments on\n ++\tstdin are also delimited using a NUL byte. Output is printed in the\n ++\tfollowing form:\n ++\n +-----------------------------------------------------------------------\n + NUL [= NUL]...\n @@ Documentation/rev-list-options.adoc: ifdef::git-rev-list[]\n + NUL path= NUL\n +-----------------------------------------------------------------------\n ++\n -+This mode is only compatible with the `--objects` output option. Also, revision\n -+and pathspec argument parsing on stdin with the `--stdin` option is NUL byte\n -+delimited instead of using newlines while in this mode.\n ++This mode is only compatible with the `--objects` output option.\n endif::git-rev-list[]\n \n History Simplification\n5: 5e4fc41976 ! 5: 591a2c7dac rev-list: support NUL-delimited --boundary option\n @@ Documentation/rev-list-options.adoc: ifdef::git-rev-list[]\n + NUL boundary=yes NUL\n -----------------------------------------------------------------------\n +\n --This mode is only compatible with the `--objects` output option. Also, revision\n --and pathspec argument parsing on stdin with the `--stdin` option is NUL byte\n --delimited instead of using newlines while in this mode.\n +-This mode is only compatible with the `--objects` output option.\n +This mode is only compatible with the `--objects` and `--boundary` output\n -+options. Also, revision and pathspec argument parsing on stdin with the\n -+`--stdin` option is NUL byte delimited instead of using newlines while in this\n -+mode.\n ++options.\n endif::git-rev-list[]\n \n History Simplification\n6: 7744966514 ! 6: 669b3b5d9f rev-list: support NUL-delimited --missing option\n @@ Commit message\n Signed-off-by: Justin Tobler \n \n ## Documentation/rev-list-options.adoc ##\n -@@ Documentation/rev-list-options.adoc: ifdef::git-rev-list[]\n - NUL [= NUL]...\n - -----------------------------------------------------------------------\n - +\n --Additional object metadata, such as object paths or boundary objects, is\n --printed using the `=` form. Token values are printed as-is\n -+Additional object metadata, such as object paths or boundary/missing objects,\n -+is printed using the `=` form. Token values are printed as-is\n - without any encoding/truncation. An OID entry never contains a '=' character\n - and thus is used to signal the start of a new object record. Examples:\n - +\n @@ Documentation/rev-list-options.adoc: and thus is used to signal the start of a new object record. Examples:\n NUL\n NUL path= NUL\n @@ Documentation/rev-list-options.adoc: and thus is used to signal the start of a n\n -----------------------------------------------------------------------\n +\n -This mode is only compatible with the `--objects` and `--boundary` output\n --options. Also, revision and pathspec argument parsing on stdin with the\n --`--stdin` option is NUL byte delimited instead of using newlines while in this\n --mode.\n +-options.\n +This mode is only compatible with the `--objects`, `--boundary`, and\n -+`--missing` output options. Also, revision and pathspec argument parsing on\n -+stdin with the `--stdin` option is NUL byte delimited instead of using newlines\n -+while in this mode.\n ++`--missing` output options.\n endif::git-rev-list[]\n \n History Simplification\n @@ builtin/rev-list.c: static void print_missing_object(struct missing_objects_map_\n \tstruct strbuf sb = STRBUF_INIT;\n \n +\tif (line_term)\n -+\t\tputchar('?');\n -+\n -+\tprintf(\"%s\", oid_to_hex(&entry->entry.oid));\n -+\n -+\tif (!line_term)\n -+\t\tprintf(\"%cmissing=yes\", info_term);\n ++\t\tprintf(\"?%s\", oid_to_hex(&entry->entry.oid));\n ++\telse\n ++\t\tprintf(\"%s%cmissing=yes\", oid_to_hex(&entry->entry.oid),\n ++\t\t info_term);\n +\n \tif (!print_missing_info) {\n -\t\tprintf(\"?%s\\n\", oid_to_hex(&entry->entry.oid));\n @@ builtin/rev-list.c: static void print_missing_object(struct missing_objects_map_\n \t}\n \n \tif (entry->path && *entry->path) {\n - \t\tstruct strbuf path = STRBUF_INIT;\n +-\t\tstruct strbuf path = STRBUF_INIT;\n ++\t\tstrbuf_addf(&sb, \"%cpath=\", info_term);\n ++\n ++\t\tif (line_term) {\n ++\t\t\tstruct strbuf path = STRBUF_INIT;\n \n -\t\tstrbuf_addstr(&sb, \" path=\");\n -\t\tquote_path(entry->path, NULL, &path, QUOTE_PATH_QUOTE_SP);\n -\t\tstrbuf_addbuf(&sb, &path);\n -+\t\tstrbuf_addf(&sb, \"%cpath=\", info_term);\n -+\n -+\t\tif (line_term) {\n +\t\t\tquote_path(entry->path, NULL, &path, QUOTE_PATH_QUOTE_SP);\n +\t\t\tstrbuf_addbuf(&sb, &path);\n + \n +-\t\tstrbuf_release(&path);\n ++\t\t\tstrbuf_release(&path);\n +\t\t} else {\n +\t\t\tstrbuf_addstr(&sb, entry->path);\n +\t\t}\n - \n - \t\tstrbuf_release(&path);\n \t}\n \tif (entry->type)\n -\t\tstrbuf_addf(&sb, \" type=%s\", type_name(entry->type));\n\nbase-commit: 87a0bdbf0f72b7561f3cd50636eee33dcb7dbcc3\n-- \n2.49.0.rc2\n\n"},{"id":"514237","messageId":"20250313235747.9583-2-jltobler@gmail.com","threadId":"63089","inReplyTo":"20250313235747.9583-1-jltobler@gmail.com","subject":"[PATCH v3 1/6] rev-list: inline `show_object_with_name()` in `show_object()`","fromName":"Justin Tobler","fromEmail":"jltobler@gmail.com","sentAt":"2025-03-13T23:57:42Z","receivedAt":"2025-03-14T00:01:42Z","isPatch":true,"sender":{"key":"jltobler@gmail.com","avatar":"https://avatars.githubusercontent.com/u/53454972?v=4"},"body":"The `show_object_with_name()` function only has a single call site.\nInline call to `show_object_with_name()` in `show_object()` so the\nexplicit function can be cleaned up and live closer to where it is used.\nWhile at it, factor out the code that prints the OID and newline for\nboth objects with and without a name. In a subsequent commit,\n`show_object()` is modified to support printing object information in a\nNUL-delimited format.\n\nSigned-off-by: Justin Tobler \n---\n builtin/rev-list.c | 13 +++++++++----\n revision.c | 8 --------\n revision.h | 2 --\n 3 files changed, 9 insertions(+), 14 deletions(-)\n\ndiff --git a/builtin/rev-list.c b/builtin/rev-list.c\nindex bb26bee0d4..dcd079c16c 100644\n--- a/builtin/rev-list.c\n+++ b/builtin/rev-list.c\n@@ -357,10 +357,15 @@ static void show_object(struct object *obj, const char *name, void *cb_data)\n \t\treturn;\n \t}\n \n-\tif (arg_show_object_names)\n-\t\tshow_object_with_name(stdout, obj, name);\n-\telse\n-\t\tprintf(\"%s\\n\", oid_to_hex(&obj->oid));\n+\tprintf(\"%s\", oid_to_hex(&obj->oid));\n+\n+\tif (arg_show_object_names) {\n+\t\tputchar(' ');\n+\t\tfor (const char *p = name; *p && *p != '\\n'; p++)\n+\t\t\tputchar(*p);\n+\t}\n+\n+\tputchar('\\n');\n }\n \n static void show_edge(struct commit *commit)\ndiff --git a/revision.c b/revision.c\nindex c4390f0938..0eaebe4478 100644\n--- a/revision.c\n+++ b/revision.c\n@@ -59,14 +59,6 @@ implement_shared_commit_slab(revision_sources, char *);\n \n static inline int want_ancestry(const struct rev_info *revs);\n \n-void show_object_with_name(FILE *out, struct object *obj, const char *name)\n-{\n-\tfprintf(out, \"%s \", oid_to_hex(&obj->oid));\n-\tfor (const char *p = name; *p && *p != '\\n'; p++)\n-\t\tfputc(*p, out);\n-\tfputc('\\n', out);\n-}\n-\n static void mark_blob_uninteresting(struct blob *blob)\n {\n \tif (!blob)\ndiff --git a/revision.h b/revision.h\nindex 71e984c452..21c6a69899 100644\n--- a/revision.h\n+++ b/revision.h\n@@ -489,8 +489,6 @@ void mark_parents_uninteresting(struct rev_info *revs, struct commit *commit);\n void mark_tree_uninteresting(struct repository *r, struct tree *tree);\n void mark_trees_uninteresting_sparse(struct repository *r, struct oidset *trees);\n \n-void show_object_with_name(FILE *, struct object *, const char *);\n-\n /**\n * Helpers to check if a reference should be excluded.\n */\n-- \n2.49.0.rc2\n\n"},{"id":"514238","messageId":"20250313235747.9583-3-jltobler@gmail.com","threadId":"63089","inReplyTo":"20250313235747.9583-1-jltobler@gmail.com","subject":"[PATCH v3 2/6] rev-list: refactor early option parsing","fromName":"Justin Tobler","fromEmail":"jltobler@gmail.com","sentAt":"2025-03-13T23:57:43Z","receivedAt":"2025-03-14T00:01:44Z","isPatch":true,"sender":{"key":"jltobler@gmail.com","avatar":"https://avatars.githubusercontent.com/u/53454972?v=4"},"body":"Before invoking `setup_revisions()`, the `--missing` and\n`--exclude-promisor-objects` options are parsed early. In a subsequent\ncommit, another option is added that must be parsed early.\n\nRefactor the code to parse both options in a single early pass.\n\nSigned-off-by: Justin Tobler \n---\n builtin/rev-list.c | 17 +++++++----------\n 1 file changed, 7 insertions(+), 10 deletions(-)\n\ndiff --git a/builtin/rev-list.c b/builtin/rev-list.c\nindex dcd079c16c..04d9c893b5 100644\n--- a/builtin/rev-list.c\n+++ b/builtin/rev-list.c\n@@ -16,6 +16,7 @@\n #include \"object-file.h\"\n #include \"object-store-ll.h\"\n #include \"pack-bitmap.h\"\n+#include \"parse-options.h\"\n #include \"log-tree.h\"\n #include \"graph.h\"\n #include \"bisect.h\"\n@@ -639,19 +640,15 @@ int cmd_rev_list(int argc,\n \t\tif (!strcmp(arg, \"--exclude-promisor-objects\")) {\n \t\t\tfetch_if_missing = 0;\n \t\t\trevs.exclude_promisor_objects = 1;\n-\t\t\tbreak;\n-\t\t}\n-\t}\n-\tfor (i = 1; i < argc; i++) {\n-\t\tconst char *arg = argv[i];\n-\t\tif (skip_prefix(arg, \"--missing=\", &arg)) {\n-\t\t\tif (revs.exclude_promisor_objects)\n-\t\t\t\tdie(_(\"options '%s' and '%s' cannot be used together\"), \"--exclude-promisor-objects\", \"--missing\");\n-\t\t\tif (parse_missing_action_value(arg))\n-\t\t\t\tbreak;\n+\t\t} else if (skip_prefix(arg, \"--missing=\", &arg)) {\n+\t\t\tparse_missing_action_value(arg);\n \t\t}\n \t}\n \n+\tdie_for_incompatible_opt2(revs.exclude_promisor_objects,\n+\t\t\t\t \"--exclude_promisor_objects\",\n+\t\t\t\t arg_missing_action, \"--missing\");\n+\n \tif (arg_missing_action)\n \t\trevs.do_not_die_on_missing_objects = 1;\n \n-- \n2.49.0.rc2\n\n"},{"id":"514239","messageId":"20250313235747.9583-4-jltobler@gmail.com","threadId":"63089","inReplyTo":"20250313235747.9583-1-jltobler@gmail.com","subject":"[PATCH v3 3/6] revision: support NUL-delimited --stdin mode","fromName":"Justin Tobler","fromEmail":"jltobler@gmail.com","sentAt":"2025-03-13T23:57:44Z","receivedAt":"2025-03-14T00:01:45Z","isPatch":true,"sender":{"key":"jltobler@gmail.com","avatar":"https://avatars.githubusercontent.com/u/53454972?v=4"},"body":"When `setup_revisions()` parses the `--stdin` option, revision and\npathspec arguments are read from stdin. Each line of input is handled as\na separate argument.\n\nIntroduce the `nul_delim_stdin` field to `setup_revision_opt` that, when\nenabled, uses a NUL byte to delimit between stdin arguments instead of\nnewline.\n\nSigned-off-by: Justin Tobler \n---\n revision.c | 19 +++++++++++--------\n revision.h | 3 ++-\n 2 files changed, 13 insertions(+), 9 deletions(-)\n\ndiff --git a/revision.c b/revision.c\nindex 0eaebe4478..5de6309830 100644\n--- a/revision.c\n+++ b/revision.c\n@@ -2275,10 +2275,10 @@ int handle_revision_arg(const char *arg, struct rev_info *revs, int flags, unsig\n \treturn ret;\n }\n \n-static void read_pathspec_from_stdin(struct strbuf *sb,\n-\t\t\t\t struct strvec *prune)\n+static void read_pathspec_from_stdin(struct strbuf *sb, struct strvec *prune,\n+\t\t\t\t int line_term)\n {\n-\twhile (strbuf_getline(sb, stdin) != EOF)\n+\twhile (strbuf_getdelim_strip_crlf(sb, stdin, line_term) != EOF)\n \t\tstrvec_push(prune, sb->buf);\n }\n \n@@ -2905,8 +2905,8 @@ static int handle_revision_pseudo_opt(struct rev_info *revs,\n \treturn 1;\n }\n \n-static void read_revisions_from_stdin(struct rev_info *revs,\n-\t\t\t\t struct strvec *prune)\n+static void read_revisions_from_stdin(struct rev_info *revs, struct strvec *prune,\n+\t\t\t\t int line_term)\n {\n \tstruct strbuf sb;\n \tint seen_dashdash = 0;\n@@ -2918,7 +2918,7 @@ static void read_revisions_from_stdin(struct rev_info *revs,\n \twarn_on_object_refname_ambiguity = 0;\n \n \tstrbuf_init(&sb, 1000);\n-\twhile (strbuf_getline(&sb, stdin) != EOF) {\n+\twhile (strbuf_getdelim_strip_crlf(&sb, stdin, line_term) != EOF) {\n \t\tif (!sb.len)\n \t\t\tbreak;\n \n@@ -2946,7 +2946,7 @@ static void read_revisions_from_stdin(struct rev_info *revs,\n \t\t\tdie(\"bad revision '%s'\", sb.buf);\n \t}\n \tif (seen_dashdash)\n-\t\tread_pathspec_from_stdin(&sb, prune);\n+\t\tread_pathspec_from_stdin(&sb, prune, line_term);\n \n \tstrbuf_release(&sb);\n \twarn_on_object_refname_ambiguity = save_warning;\n@@ -3019,13 +3019,16 @@ int setup_revisions(int argc, const char **argv, struct rev_info *revs, struct s\n \t\t\t}\n \n \t\t\tif (!strcmp(arg, \"--stdin\")) {\n+\t\t\t\tint term = opt && opt->nul_delim_stdin ? '\\0' : '\\n';\n+\n \t\t\t\tif (revs->disable_stdin) {\n \t\t\t\t\targv[left++] = arg;\n \t\t\t\t\tcontinue;\n \t\t\t\t}\n \t\t\t\tif (revs->read_from_stdin++)\n \t\t\t\t\tdie(\"--stdin given twice?\");\n-\t\t\t\tread_revisions_from_stdin(revs, &prune_data);\n+\t\t\t\tread_revisions_from_stdin(revs, &prune_data,\n+\t\t\t\t\t\t\t term);\n \t\t\t\tcontinue;\n \t\t\t}\n \ndiff --git a/revision.h b/revision.h\nindex 21c6a69899..0e680c3667 100644\n--- a/revision.h\n+++ b/revision.h\n@@ -439,7 +439,8 @@ struct setup_revision_opt {\n \tvoid (*tweak)(struct rev_info *);\n \tunsigned int\tassume_dashdash:1,\n \t\t\tallow_exclude_promisor_objects:1,\n-\t\t\tfree_removed_argv_elements:1;\n+\t\t\tfree_removed_argv_elements:1,\n+\t\t\tnul_delim_stdin:1;\n \tunsigned revarg_opt;\n };\n int setup_revisions(int argc, const char **argv, struct rev_info *revs,\n-- \n2.49.0.rc2\n\n"},{"id":"514240","messageId":"20250313235747.9583-5-jltobler@gmail.com","threadId":"63089","inReplyTo":"20250313235747.9583-1-jltobler@gmail.com","subject":"[PATCH v3 4/6] rev-list: support delimiting objects with NUL bytes","fromName":"Justin Tobler","fromEmail":"jltobler@gmail.com","sentAt":"2025-03-13T23:57:45Z","receivedAt":"2025-03-14T00:01:48Z","isPatch":true,"sender":{"key":"jltobler@gmail.com","avatar":"https://avatars.githubusercontent.com/u/53454972?v=4"},"body":"When walking objects, git-rev-list(1) prints each object entry on a\nseparate line. Some options, such as `--objects`, may print additional\ninformation about tree and blob object on the same line in the form:\n\n $ git rev-list --objects \n SP [] LF\n\nNote that in this form the SP is appended regardless of whether the tree\nor blob object has path information available. Paths containing a\nnewline are also truncated at the newline.\n\nIntroduce the `-z` option for git-rev-list(1) which reformats the output\nto use NUL-delimiters between objects and associated info in the\nfollowing form:\n\n $ git rev-list -z --objects \n NUL [path= NUL]\n\nIn this form, the start of each record is signaled by an OID entry that\nis all hexidecimal and does not contain any '='. Additional path info\nfrom `--objects` is appended to the record as a token/value pair\n`path=` as-is without any truncation.\n\nIn this mode, revision and pathspec arguments provided on stdin with the\n`--stdin` option are also separated by a NUL byte instead of being\nnewline delimited.\n\nFor now, the `--objects` and `--stdin` flag are the only options that\ncan be used in combination with `-z`. In a subsequent commit,\nNUL-delimited support for other options is added. Other options that do\nnot make sense with be used in combination with `-z` are rejected.\n\nSigned-off-by: Justin Tobler \n---\n Documentation/rev-list-options.adoc | 23 ++++++++++++++++++\n builtin/rev-list.c | 36 +++++++++++++++++++++++++----\n t/t6000-rev-list-misc.sh | 35 ++++++++++++++++++++++++++++\n t/t6017-rev-list-stdin.sh | 9 ++++++++\n 4 files changed, 98 insertions(+), 5 deletions(-)\n\ndiff --git a/Documentation/rev-list-options.adoc b/Documentation/rev-list-options.adoc\nindex 785c0786e0..14d82fdfbf 100644\n--- a/Documentation/rev-list-options.adoc\n+++ b/Documentation/rev-list-options.adoc\n@@ -361,6 +361,29 @@ ifdef::git-rev-list[]\n --progress=
::\n \tShow progress reports on stderr as objects are considered. The\n \t`
` text will be printed with each progress update.\n+\n+-z::\n+\tInstead of being newline-delimited, each outputted object and its\n+\taccompanying metadata is delimited using NUL bytes. In this mode, when\n+\tthe `--stdin` option is provided, revision and pathspec arguments on\n+\tstdin are also delimited using a NUL byte. Output is printed in the\n+\tfollowing form:\n++\n+-----------------------------------------------------------------------\n+ NUL [= NUL]...\n+-----------------------------------------------------------------------\n++\n+Additional object metadata, such as object paths, is printed using the\n+`=` form. Token values are printed as-is without any\n+encoding/truncation. An OID entry never contains a '=' character and thus\n+is used to signal the start of a new object record. Examples:\n++\n+-----------------------------------------------------------------------\n+ NUL\n+ NUL path= NUL\n+-----------------------------------------------------------------------\n++\n+This mode is only compatible with the `--objects` output option.\n endif::git-rev-list[]\n \n History Simplification\ndiff --git a/builtin/rev-list.c b/builtin/rev-list.c\nindex 04d9c893b5..f048500679 100644\n--- a/builtin/rev-list.c\n+++ b/builtin/rev-list.c\n@@ -65,6 +65,7 @@ static const char rev_list_usage[] =\n \" --abbrev-commit\\n\"\n \" --left-right\\n\"\n \" --count\\n\"\n+\" -z\\n\"\n \" special purpose:\\n\"\n \" --bisect\\n\"\n \" --bisect-vars\\n\"\n@@ -97,6 +98,9 @@ static int arg_show_object_names = 1;\n \n #define DEFAULT_OIDSET_SIZE (16*1024)\n \n+static char line_term = '\\n';\n+static char info_term = ' ';\n+\n static int show_disk_usage;\n static off_t total_disk_usage;\n static int human_readable;\n@@ -264,7 +268,7 @@ static void show_commit(struct commit *commit, void *data)\n \tif (revs->commit_format == CMIT_FMT_ONELINE)\n \t\tputchar(' ');\n \telse if (revs->include_header)\n-\t\tputchar('\\n');\n+\t\tputchar(line_term);\n \n \tif (revs->verbose_header) {\n \t\tstruct strbuf buf = STRBUF_INIT;\n@@ -361,12 +365,16 @@ static void show_object(struct object *obj, const char *name, void *cb_data)\n \tprintf(\"%s\", oid_to_hex(&obj->oid));\n \n \tif (arg_show_object_names) {\n-\t\tputchar(' ');\n-\t\tfor (const char *p = name; *p && *p != '\\n'; p++)\n-\t\t\tputchar(*p);\n+\t\tif (line_term) {\n+\t\t\tputchar(info_term);\n+\t\t\tfor (const char *p = name; *p && *p != '\\n'; p++)\n+\t\t\t\tputchar(*p);\n+\t\t} else if (*name) {\n+\t\t\tprintf(\"%cpath=%s\", info_term, name);\n+\t\t}\n \t}\n \n-\tputchar('\\n');\n+\tputchar(line_term);\n }\n \n static void show_edge(struct commit *commit)\n@@ -642,6 +650,10 @@ int cmd_rev_list(int argc,\n \t\t\trevs.exclude_promisor_objects = 1;\n \t\t} else if (skip_prefix(arg, \"--missing=\", &arg)) {\n \t\t\tparse_missing_action_value(arg);\n+\t\t} else if (!strcmp(arg, \"-z\")) {\n+\t\t\ts_r_opt.nul_delim_stdin = 1;\n+\t\t\tline_term = '\\0';\n+\t\t\tinfo_term = '\\0';\n \t\t}\n \t}\n \n@@ -757,6 +769,20 @@ int cmd_rev_list(int argc,\n \t\tusage(rev_list_usage);\n \n \t}\n+\n+\t/*\n+\t * Reject options currently incompatible with -z. For some options, this\n+\t * is not an inherent limitation and support may be implemented in the\n+\t * future.\n+\t */\n+\tif (!line_term) {\n+\t\tif (revs.graph || revs.verbose_header || show_disk_usage ||\n+\t\t info.show_timestamp || info.header_prefix || bisect_list ||\n+\t\t use_bitmap_index || revs.edge_hint || revs.left_right ||\n+\t\t revs.cherry_mark || arg_missing_action || revs.boundary)\n+\t\t\tdie(_(\"-z option used with unsupported option\"));\n+\t}\n+\n \tif (revs.commit_format != CMIT_FMT_USERFORMAT)\n \t\trevs.include_header = 1;\n \tif (revs.commit_format != CMIT_FMT_UNSPECIFIED) {\ndiff --git a/t/t6000-rev-list-misc.sh b/t/t6000-rev-list-misc.sh\nindex 6289a2e8b0..dfbbc0aee6 100755\n--- a/t/t6000-rev-list-misc.sh\n+++ b/t/t6000-rev-list-misc.sh\n@@ -182,4 +182,39 @@ test_expect_success 'rev-list --unpacked' '\n \ttest_cmp expect actual\n '\n \n+test_expect_success 'rev-list -z' '\n+\ttest_when_finished rm -rf repo &&\n+\n+\tgit init repo &&\n+\ttest_commit -C repo 1 &&\n+\ttest_commit -C repo 2 &&\n+\n+\toid1=$(git -C repo rev-parse HEAD) &&\n+\toid2=$(git -C repo rev-parse HEAD~) &&\n+\n+\tprintf \"%s\\0%s\\0\" \"$oid1\" \"$oid2\" >expect &&\n+\tgit -C repo rev-list -z HEAD >actual &&\n+\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'rev-list -z --objects' '\n+\ttest_when_finished rm -rf repo &&\n+\n+\tgit init repo &&\n+\ttest_commit -C repo 1 &&\n+\ttest_commit -C repo 2 &&\n+\n+\toid1=$(git -C repo rev-parse HEAD:1.t) &&\n+\toid2=$(git -C repo rev-parse HEAD:2.t) &&\n+\tpath1=1.t &&\n+\tpath2=2.t &&\n+\n+\tprintf \"%s\\0path=%s\\0%s\\0path=%s\\0\" \"$oid1\" \"$path1\" \"$oid2\" \"$path2\" \\\n+\t\t>expect &&\n+\tgit -C repo rev-list -z --objects HEAD:1.t HEAD:2.t >actual &&\n+\n+\ttest_cmp expect actual\n+'\n+\n test_done\ndiff --git a/t/t6017-rev-list-stdin.sh b/t/t6017-rev-list-stdin.sh\nindex 4821b90e74..362a8b126a 100755\n--- a/t/t6017-rev-list-stdin.sh\n+++ b/t/t6017-rev-list-stdin.sh\n@@ -148,4 +148,13 @@ test_expect_success '--not via stdin does not influence revisions from command l\n \ttest_cmp expect actual\n '\n \n+test_expect_success 'NUL-delimited stdin' '\n+\tprintf \"%s\\0%s\\0%s\\0\" \"HEAD\" \"--\" \"file-1\" > input &&\n+\n+\tgit rev-list -z --objects HEAD -- file-1 >expect &&\n+\tgit rev-list -z --objects --stdin actual &&\n+\n+\ttest_cmp expect actual\n+'\n+\n test_done\n-- \n2.49.0.rc2\n\n"},{"id":"514241","messageId":"20250313235747.9583-6-jltobler@gmail.com","threadId":"63089","inReplyTo":"20250313235747.9583-1-jltobler@gmail.com","subject":"[PATCH v3 5/6] rev-list: support NUL-delimited --boundary option","fromName":"Justin Tobler","fromEmail":"jltobler@gmail.com","sentAt":"2025-03-13T23:57:46Z","receivedAt":"2025-03-14T00:01:50Z","isPatch":true,"sender":{"key":"jltobler@gmail.com","avatar":"https://avatars.githubusercontent.com/u/53454972?v=4"},"body":"The `--boundary` option for git-rev-list(1) prints boundary objects\nfound while performing the object walk in the form:\n\n $ git rev-list --boundary \n - LF\n\nAdd support for printing boundary objects in a NUL-delimited format when\nthe `-z` option is enabled.\n\n $ git rev-list -z --boundary \n NUL boundary=yes NUL\n\nIn this mode, instead of prefixing the boundary OID with '-', a separate\n`boundary=yes` token/value pair is appended.\n\nSigned-off-by: Justin Tobler \n---\n Documentation/rev-list-options.adoc | 12 +++++++-----\n builtin/rev-list.c | 9 +++++++--\n t/t6000-rev-list-misc.sh | 16 ++++++++++++++++\n 3 files changed, 30 insertions(+), 7 deletions(-)\n\ndiff --git a/Documentation/rev-list-options.adoc b/Documentation/rev-list-options.adoc\nindex 14d82fdfbf..92ac31a8e8 100644\n--- a/Documentation/rev-list-options.adoc\n+++ b/Documentation/rev-list-options.adoc\n@@ -373,17 +373,19 @@ ifdef::git-rev-list[]\n NUL [= NUL]...\n -----------------------------------------------------------------------\n +\n-Additional object metadata, such as object paths, is printed using the\n-`=` form. Token values are printed as-is without any\n-encoding/truncation. An OID entry never contains a '=' character and thus\n-is used to signal the start of a new object record. Examples:\n+Additional object metadata, such as object paths or boundary objects, is\n+printed using the `=` form. Token values are printed as-is\n+without any encoding/truncation. An OID entry never contains a '=' character\n+and thus is used to signal the start of a new object record. Examples:\n +\n -----------------------------------------------------------------------\n NUL\n NUL path= NUL\n+ NUL boundary=yes NUL\n -----------------------------------------------------------------------\n +\n-This mode is only compatible with the `--objects` output option.\n+This mode is only compatible with the `--objects` and `--boundary` output\n+options.\n endif::git-rev-list[]\n \n History Simplification\ndiff --git a/builtin/rev-list.c b/builtin/rev-list.c\nindex f048500679..7c6d4b25b0 100644\n--- a/builtin/rev-list.c\n+++ b/builtin/rev-list.c\n@@ -240,13 +240,18 @@ static void show_commit(struct commit *commit, void *data)\n \t\tfputs(info->header_prefix, stdout);\n \n \tif (revs->include_header) {\n-\t\tif (!revs->graph)\n+\t\tif (!revs->graph && line_term)\n \t\t\tfputs(get_revision_mark(revs, commit), stdout);\n \t\tif (revs->abbrev_commit && revs->abbrev)\n \t\t\tfputs(repo_find_unique_abbrev(the_repository, &commit->object.oid, revs->abbrev),\n \t\t\t stdout);\n \t\telse\n \t\t\tfputs(oid_to_hex(&commit->object.oid), stdout);\n+\n+\t\tif (!line_term) {\n+\t\t\tif (commit->object.flags & BOUNDARY)\n+\t\t\t\tprintf(\"%cboundary=yes\", info_term);\n+\t\t}\n \t}\n \tif (revs->print_parents) {\n \t\tstruct commit_list *parents = commit->parents;\n@@ -779,7 +784,7 @@ int cmd_rev_list(int argc,\n \t\tif (revs.graph || revs.verbose_header || show_disk_usage ||\n \t\t info.show_timestamp || info.header_prefix || bisect_list ||\n \t\t use_bitmap_index || revs.edge_hint || revs.left_right ||\n-\t\t revs.cherry_mark || arg_missing_action || revs.boundary)\n+\t\t revs.cherry_mark || arg_missing_action)\n \t\t\tdie(_(\"-z option used with unsupported option\"));\n \t}\n \ndiff --git a/t/t6000-rev-list-misc.sh b/t/t6000-rev-list-misc.sh\nindex dfbbc0aee6..349bf5ec3d 100755\n--- a/t/t6000-rev-list-misc.sh\n+++ b/t/t6000-rev-list-misc.sh\n@@ -217,4 +217,20 @@ test_expect_success 'rev-list -z --objects' '\n \ttest_cmp expect actual\n '\n \n+test_expect_success 'rev-list -z --boundary' '\n+\ttest_when_finished rm -rf repo &&\n+\n+\tgit init repo &&\n+\ttest_commit -C repo 1 &&\n+\ttest_commit -C repo 2 &&\n+\n+\toid1=$(git -C repo rev-parse HEAD) &&\n+\toid2=$(git -C repo rev-parse HEAD~) &&\n+\n+\tprintf \"%s\\0%s\\0boundary=yes\\0\" \"$oid1\" \"$oid2\" >expect &&\n+\tgit -C repo rev-list -z --boundary HEAD~.. >actual &&\n+\n+\ttest_cmp expect actual\n+'\n+\n test_done\n-- \n2.49.0.rc2\n\n"},{"id":"514242","messageId":"20250313235747.9583-7-jltobler@gmail.com","threadId":"63089","inReplyTo":"20250313235747.9583-1-jltobler@gmail.com","subject":"[PATCH v3 6/6] rev-list: support NUL-delimited --missing option","fromName":"Justin Tobler","fromEmail":"jltobler@gmail.com","sentAt":"2025-03-13T23:57:47Z","receivedAt":"2025-03-14T00:01:54Z","isPatch":true,"sender":{"key":"jltobler@gmail.com","avatar":"https://avatars.githubusercontent.com/u/53454972?v=4"},"body":"The `--missing={print,print-info}` option for git-rev-list(1) prints\nmissing objects found while performing the object walk in the form:\n\n $ git rev-list --missing=print-info \n ? [SP =]... LF\n\nAdd support for printing missing objects in a NUL-delimited format when\nthe `-z` option is enabled.\n\n $ git rev-list -z --missing=print-info \n NUL missing=yes NUL [= NUL]...\n\nIn this mode, values containing special characters or spaces are printed\nas-is without being escaped or quoted. Instead of prefixing the missing\nOID with '?', a separate `missing=yes` token/value pair is appended.\n\nSigned-off-by: Justin Tobler \n---\n Documentation/rev-list-options.adoc | 5 +++--\n builtin/rev-list.c | 31 ++++++++++++++++++++---------\n t/t6022-rev-list-missing.sh | 31 +++++++++++++++++++++++++++++\n 3 files changed, 56 insertions(+), 11 deletions(-)\n\ndiff --git a/Documentation/rev-list-options.adoc b/Documentation/rev-list-options.adoc\nindex 92ac31a8e8..f4764b72f5 100644\n--- a/Documentation/rev-list-options.adoc\n+++ b/Documentation/rev-list-options.adoc\n@@ -382,10 +382,11 @@ and thus is used to signal the start of a new object record. Examples:\n NUL\n NUL path= NUL\n NUL boundary=yes NUL\n+ NUL missing=yes NUL [= NUL]...\n -----------------------------------------------------------------------\n +\n-This mode is only compatible with the `--objects` and `--boundary` output\n-options.\n+This mode is only compatible with the `--objects`, `--boundary`, and\n+`--missing` output options.\n endif::git-rev-list[]\n \n History Simplification\ndiff --git a/builtin/rev-list.c b/builtin/rev-list.c\nindex 7c6d4b25b0..036fcc26d5 100644\n--- a/builtin/rev-list.c\n+++ b/builtin/rev-list.c\n@@ -136,24 +136,37 @@ static void print_missing_object(struct missing_objects_map_entry *entry,\n {\n \tstruct strbuf sb = STRBUF_INIT;\n \n+\tif (line_term)\n+\t\tprintf(\"?%s\", oid_to_hex(&entry->entry.oid));\n+\telse\n+\t\tprintf(\"%s%cmissing=yes\", oid_to_hex(&entry->entry.oid),\n+\t\t info_term);\n+\n \tif (!print_missing_info) {\n-\t\tprintf(\"?%s\\n\", oid_to_hex(&entry->entry.oid));\n+\t\tputchar(line_term);\n \t\treturn;\n \t}\n \n \tif (entry->path && *entry->path) {\n-\t\tstruct strbuf path = STRBUF_INIT;\n+\t\tstrbuf_addf(&sb, \"%cpath=\", info_term);\n+\n+\t\tif (line_term) {\n+\t\t\tstruct strbuf path = STRBUF_INIT;\n \n-\t\tstrbuf_addstr(&sb, \" path=\");\n-\t\tquote_path(entry->path, NULL, &path, QUOTE_PATH_QUOTE_SP);\n-\t\tstrbuf_addbuf(&sb, &path);\n+\t\t\tquote_path(entry->path, NULL, &path, QUOTE_PATH_QUOTE_SP);\n+\t\t\tstrbuf_addbuf(&sb, &path);\n \n-\t\tstrbuf_release(&path);\n+\t\t\tstrbuf_release(&path);\n+\t\t} else {\n+\t\t\tstrbuf_addstr(&sb, entry->path);\n+\t\t}\n \t}\n \tif (entry->type)\n-\t\tstrbuf_addf(&sb, \" type=%s\", type_name(entry->type));\n+\t\tstrbuf_addf(&sb, \"%ctype=%s\", info_term, type_name(entry->type));\n+\n+\tfwrite(sb.buf, sizeof(char), sb.len, stdout);\n+\tputchar(line_term);\n \n-\tprintf(\"?%s%s\\n\", oid_to_hex(&entry->entry.oid), sb.buf);\n \tstrbuf_release(&sb);\n }\n \n@@ -784,7 +797,7 @@ int cmd_rev_list(int argc,\n \t\tif (revs.graph || revs.verbose_header || show_disk_usage ||\n \t\t info.show_timestamp || info.header_prefix || bisect_list ||\n \t\t use_bitmap_index || revs.edge_hint || revs.left_right ||\n-\t\t revs.cherry_mark || arg_missing_action)\n+\t\t revs.cherry_mark)\n \t\t\tdie(_(\"-z option used with unsupported option\"));\n \t}\n \ndiff --git a/t/t6022-rev-list-missing.sh b/t/t6022-rev-list-missing.sh\nindex 3e2790d4c8..08e92dd002 100755\n--- a/t/t6022-rev-list-missing.sh\n+++ b/t/t6022-rev-list-missing.sh\n@@ -198,4 +198,35 @@ do\n \t'\n done\n \n+test_expect_success \"-z nul-delimited --missing\" '\n+\ttest_when_finished rm -rf repo &&\n+\n+\tgit init repo &&\n+\t(\n+\t\tcd repo &&\n+\t\tgit commit --allow-empty -m first &&\n+\n+\t\tpath=\"foo bar\" &&\n+\t\techo foobar >\"$path\" &&\n+\t\tgit add -A &&\n+\t\tgit commit -m second &&\n+\n+\t\toid=$(git rev-parse \"HEAD:$path\") &&\n+\t\ttype=\"$(git cat-file -t $oid)\" &&\n+\n+\t\tobj_path=\".git/objects/$(test_oid_to_path $oid)\" &&\n+\n+\t\tgit rev-list -z --objects --no-object-names \\\n+\t\t\tHEAD ^\"$oid\" >expect &&\n+\t\tprintf \"%s\\0missing=yes\\0path=%s\\0type=%s\\0\" \"$oid\" \"$path\" \\\n+\t\t\t\"$type\" >>expect &&\n+\n+\t\tmv \"$obj_path\" \"$obj_path.hidden\" &&\n+\t\tgit rev-list -z --objects --no-object-names \\\n+\t\t\t--missing=print-info HEAD >actual &&\n+\n+\t\ttest_cmp expect actual\n+\t)\n+'\n+\n test_done\n-- \n2.49.0.rc2\n\n"},{"id":"514253","messageId":"20250314024929.GB114103@coredump.intra.peff.net","threadId":"63089","inReplyTo":"qapd7sijizopaefpxh3slgn7g2ecwdtyhzqzufcypciyv6n3qm@turmldwzipwe","subject":"Re: [PATCH 0/4] rev-list: introduce NUL-delimited output mode","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2025-03-14T02:49:29Z","receivedAt":"2025-03-14T02:49:31Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Mar 13, 2025 at 11:41:56AM -0500, Justin Tobler wrote:\n\n> > Though that does mean that these two commands\n> > will behave differently:\n> > \n> > git log --stdin -z\n> > git rev-list --stdin -z\n> > \n> > which seems...not great. My earlier suggestion to tie \"-z\" to stdin\n> > handling was for consistency with other tools like grep. But if we\n> > already have cases where \"-z\" is only for output, maybe it is better to\n> > stay consistent with other parts of git. I.e., I was worried about us\n> > painting ourselves into a corner with your patches, but we may have\n> > already done so years ago. ;)\n> \n> I think to some extent Git is already inconsistent here. IMO it would be\n> preferable for both input and output to use NUL as the delimiter when\n> machine parsing in git-rev-list(1) as that is the behavior I would\n> personally expect. I also agree with Patrick's reasoning else where in\n> this thread[1].\n> \n> I'm open to discuss further though :)\n\nNope, I don't have anything further to add. I just wanted to make sure\nmy suggestion in the earlier part of the thread was not leading us\naccidentally down a bad path. ;) If the inconsistencies have been\nconsidered and we're OK with them, that sounds good to me (and the\noption handling in your v2 does seem to faithfully implement that).\n\nThanks for all your responses.\n\n-Peff\n"},{"id":"514271","messageId":"xmqqsenfk0ks.fsf@gitster.g","threadId":"63089","inReplyTo":"20250314024929.GB114103@coredump.intra.peff.net","subject":"Re: [PATCH 0/4] rev-list: introduce NUL-delimited output mode","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-03-14T17:02:11Z","receivedAt":"2025-03-14T17:02:14Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King writes:\n\n> On Thu, Mar 13, 2025 at 11:41:56AM -0500, Justin Tobler wrote:\n>\n>> > Though that does mean that these two commands\n>> > will behave differently:\n>> > \n>> > git log --stdin -z\n>> > git rev-list --stdin -z\n>> > \n>> > which seems...not great. My earlier suggestion to tie \"-z\" to stdin\n>> > handling was for consistency with other tools like grep. But if we\n>> > already have cases where \"-z\" is only for output, maybe it is better to\n>> > stay consistent with other parts of git. I.e., I was worried about us\n>> > painting ourselves into a corner with your patches, but we may have\n>> > already done so years ago. ;)\n>> \n>> I think to some extent Git is already inconsistent here. IMO it would be\n>> preferable for both input and output to use NUL as the delimiter when\n>> machine parsing in git-rev-list(1) as that is the behavior I would\n>> personally expect. I also agree with Patrick's reasoning else where in\n>> this thread[1].\n>> \n>> I'm open to discuss further though :)\n>\n> Nope, I don't have anything further to add. I just wanted to make sure\n> my suggestion in the earlier part of the thread was not leading us\n> accidentally down a bad path. ;) If the inconsistencies have been\n> considered and we're OK with them, that sounds good to me (and the\n> option handling in your v2 does seem to faithfully implement that).\n>\n> Thanks for all your responses.\n\nThe developers who know that the revision walk infrastructure is\nshared between rev-list and log may wish that \"-z\" to mean the same\nthing for these two commands, but the end-users do not have to, no?\n\nAfter all, \"git log\" accepts \"-z\" but \"git rev-list\" does not in the\ncurrent system and nobody complained about the discrepancy so far.\n\nHaving said that, at the plumbing level, my preference is to have\ntwo independent options \"--nul-delimited-{output,input}\". It does\nnot prevent us from starting with a single \"-z\" that works as a\nshort-hand that flips both on (and is inconsistent with \"git log\" at\nthe Porcelain level), but we can make \"-z\" only for output for\nconsistency. As long as we agree on the design to allow us to\ncontrol both sides independently, starting with \"-z\" that is only\nfor output may be the best way forward.\n\n\n\n\n\n"},{"id":"514294","messageId":"20250314185914.GD578421@coredump.intra.peff.net","threadId":"63089","inReplyTo":"xmqqsenfk0ks.fsf@gitster.g","subject":"Re: [PATCH 0/4] rev-list: introduce NUL-delimited output mode","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2025-03-14T18:59:14Z","receivedAt":"2025-03-14T18:59:15Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Fri, Mar 14, 2025 at 10:02:11AM -0700, Junio C Hamano wrote:\n\n> The developers who know that the revision walk infrastructure is\n> shared between rev-list and log may wish that \"-z\" to mean the same\n> thing for these two commands, but the end-users do not have to, no?\n\nI think of them as related in a user-facing way, but it is likely that\nmy mind is poisoned by knowing too much of Git's internals. :)\n\n> After all, \"git log\" accepts \"-z\" but \"git rev-list\" does not in the\n> current system and nobody complained about the discrepancy so far.\n\nWell, rev-list _does_ take \"-z\", but it just happens that it cannot ever\ndo anything because you cannot convince it to produce a diff. But even\nknowing that is true is probably again a sign of my poisoned mind.\n\n> Having said that, at the plumbing level, my preference is to have\n> two independent options \"--nul-delimited-{output,input}\". It does\n> not prevent us from starting with a single \"-z\" that works as a\n> short-hand that flips both on (and is inconsistent with \"git log\" at\n> the Porcelain level), but we can make \"-z\" only for output for\n> consistency. As long as we agree on the design to allow us to\n> control both sides independently, starting with \"-z\" that is only\n> for output may be the best way forward.\n\nYeah, I almost suggested earlier having longer, unambiguous names. And\nthen that punts the issue from \"which functionality should be available\"\nto \"which functionality should be mapped to short-and-sweet -z\".\n\nI do think it's still worth considering what \"-z\" should do _now_,\nthough, because it will be painful/impossible to switch its behavior\nlater. And people seemed to like the \"both input and output\" direction.\nThat would leave the longer names as escape hatches. I.e., I'd expect:\n\n git rev-list -z --no-nul-delimited-input --stdin\n\nto use newlines for the input and NULs for the output.\n\n-Peff\n"},{"id":"514301","messageId":"po5vjngxt6afwdkg7bvpw5j4dhot5h4uz5yudb3o4eug4k2nqu@rwp5rpe5d3u6","threadId":"63089","inReplyTo":"20250314185914.GD578421@coredump.intra.peff.net","subject":"Re: [PATCH 0/4] rev-list: introduce NUL-delimited output mode","fromName":"Justin Tobler","fromEmail":"jltobler@gmail.com","sentAt":"2025-03-14T19:53:05Z","receivedAt":"2025-03-14T19:56:34Z","isPatch":true,"sender":{"key":"jltobler@gmail.com","avatar":"https://avatars.githubusercontent.com/u/53454972?v=4"},"body":"On 25/03/14 02:59PM, Jeff King wrote:\n> On Fri, Mar 14, 2025 at 10:02:11AM -0700, Junio C Hamano wrote:\n> > Having said that, at the plumbing level, my preference is to have\n> > two independent options \"--nul-delimited-{output,input}\". It does\n> > not prevent us from starting with a single \"-z\" that works as a\n> > short-hand that flips both on (and is inconsistent with \"git log\" at\n> > the Porcelain level), but we can make \"-z\" only for output for\n> > consistency. As long as we agree on the design to allow us to\n> > control both sides independently, starting with \"-z\" that is only\n> > for output may be the best way forward.\n> \n> Yeah, I almost suggested earlier having longer, unambiguous names. And\n> then that punts the issue from \"which functionality should be available\"\n> to \"which functionality should be mapped to short-and-sweet -z\".\n> \n> I do think it's still worth considering what \"-z\" should do _now_,\n> though, because it will be painful/impossible to switch its behavior\n> later. And people seemed to like the \"both input and output\" direction.\n> That would leave the longer names as escape hatches. I.e., I'd expect:\n> \n> git rev-list -z --no-nul-delimited-input --stdin\n> \n> to use newlines for the input and NULs for the output.\n\nIf we want to adopt less ambiguous long options names for NUL-delimited\ninput/output options as an alternative to \"-z\", maybe we could do\nsomething like:\n\n $ git rev-list --nul-delimited={all,input,output}\n\nwhere the default for the `--nul-delimited` could be both input/output. \n\nIf we want to do something along these lines, I can send another verison\nof this series where we drop \"-z\" in favor of using this option.\n\n-Justin\n"},{"id":"514322","messageId":"xmqqtt7ve2id.fsf@gitster.g","threadId":"63089","inReplyTo":"po5vjngxt6afwdkg7bvpw5j4dhot5h4uz5yudb3o4eug4k2nqu@rwp5rpe5d3u6","subject":"Re: [PATCH 0/4] rev-list: introduce NUL-delimited output mode","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-03-14T21:16:58Z","receivedAt":"2025-03-14T21:17:01Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Justin Tobler writes:\n\n> If we want to adopt less ambiguous long options names for NUL-delimited\n> input/output options as an alternative to \"-z\", maybe we could do\n> something like:\n>\n> $ git rev-list --nul-delimited={all,input,output}\n>\n> where the default for the `--nul-delimited` could be both input/output. \n\nI'd prefer not to see that route taken, as it does not look any\n\"less ambiguous\" at least to me. Making individual selections are\nalmost the same in either syntax, and the only difference is that\n--nul-delimited-input --nul-delimited-output can be independently\nchosen and given and happen to end up selecting both.\n\nBut with --nul-delimited=, you have to plan ahead and choose\n\"all\". When your script first wants NUL delimited I/O on the output\nside, you'd write \"output\". When later you want to allow it to\noptionally take NUL delimited I/O on the input side, you have to\nnotice that you have \"output\" there already and replace it with \"all\".\nIf the initial version did not have NUL-delimited output, your change\nto add support for NUL-delimited input would be different.\n\nAnd you also have to remember that it has to be spelled \"all\" and\nnot \"both\" when you replace existing \"output\".\n\nIn other words, I'd prefer to leave independent/orthogonal things as\nsuch, even if such a general design principle may make the result a\nbit more verbose, at the plumbing level.\n\nThanks.\n\n\n"},{"id":"514634","messageId":"CAP8UFD1JAkTESnEOP621uECvoFGFmQ8SRd372hPLDhNzALAN+A@mail.gmail.com","threadId":"63089","inReplyTo":"20250313235747.9583-5-jltobler@gmail.com","subject":"Re: [PATCH v3 4/6] rev-list: support delimiting objects with NUL bytes","fromName":"Christian Couder","fromEmail":"christian.couder@gmail.com","sentAt":"2025-03-19T12:35:19Z","receivedAt":"2025-03-19T12:35:33Z","isPatch":true,"sender":{"key":"christian.couder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/208954?v=4"},"body":"On Fri, Mar 14, 2025 at 1:01 AM Justin Tobler wrote:\n\n> For now, the `--objects` and `--stdin` flag are the only options that\n> can be used in combination with `-z`. In a subsequent commit,\n> NUL-delimited support for other options is added. Other options that do\n> not make sense with be used in combination with `-z` are rejected.\n\ns/with be used/when used/\n\n[...]\n\n> +test_expect_success 'rev-list -z' '\n> + test_when_finished rm -rf repo &&\n> +\n> + git init repo &&\n> + test_commit -C repo 1 &&\n> + test_commit -C repo 2 &&\n> +\n> + oid1=$(git -C repo rev-parse HEAD) &&\n> + oid2=$(git -C repo rev-parse HEAD~) &&\n\nIt seems to me that HEAD is at commit 2 and HEAD~ at commit 1 instead\nof the other way around.\n\nIt looks like there is the same issue in the test added in the next\npatch (\"[PATCH v3 5/6] rev-list: support NUL-delimited --boundary\noption\")\n\n> + printf \"%s\\0%s\\0\" \"$oid1\" \"$oid2\" >expect &&\n> + git -C repo rev-list -z HEAD >actual &&\n> +\n> + test_cmp expect actual\n> +'\n\nOtherwise the whole patch series looks good to me.\n\nThanks.\n"},{"id":"514657","messageId":"2aqtmbdpzi5lupflthm4rvoya3pqksv4sqnv5if6camw7tkzuz@ydecphmvo4dc","threadId":"63089","inReplyTo":"xmqqtt7ve2id.fsf@gitster.g","subject":"Re: [PATCH 0/4] rev-list: introduce NUL-delimited output mode","fromName":"Justin Tobler","fromEmail":"jltobler@gmail.com","sentAt":"2025-03-19T15:58:17Z","receivedAt":"2025-03-19T16:01:52Z","isPatch":true,"sender":{"key":"jltobler@gmail.com","avatar":"https://avatars.githubusercontent.com/u/53454972?v=4"},"body":"On 25/03/14 02:16PM, Junio C Hamano wrote:\n> Justin Tobler writes:\n> \n> > If we want to adopt less ambiguous long options names for NUL-delimited\n> > input/output options as an alternative to \"-z\", maybe we could do\n> > something like:\n> >\n> > $ git rev-list --nul-delimited={all,input,output}\n> >\n> > where the default for the `--nul-delimited` could be both input/output. \n> \n> I'd prefer not to see that route taken, as it does not look any\n> \"less ambiguous\" at least to me. Making individual selections are\n> almost the same in either syntax, and the only difference is that\n> --nul-delimited-input --nul-delimited-output can be independently\n> chosen and given and happen to end up selecting both.\n> \n> But with --nul-delimited=, you have to plan ahead and choose\n> \"all\". When your script first wants NUL delimited I/O on the output\n> side, you'd write \"output\". When later you want to allow it to\n> optionally take NUL delimited I/O on the input side, you have to\n> notice that you have \"output\" there already and replace it with \"all\".\n> If the initial version did not have NUL-delimited output, your change\n> to add support for NUL-delimited input would be different.\n> \n> And you also have to remember that it has to be spelled \"all\" and\n> not \"both\" when you replace existing \"output\".\n> \n> In other words, I'd prefer to leave independent/orthogonal things as\n> such, even if such a general design principle may make the result a\n> bit more verbose, at the plumbing level.\n\nThat's fair. I agree the explicitness of having two separate options is\nnice and, while more verbose, that is probably not a big deal at the\nplumbing level.\n\nIn my next version I'll return the \"-z\" option to only setting the\noutput to be NUL-delimited which would better match the log family of\ncommands. If we also want to support NUL-delimited stdin parsing, I can\nsubmit a followup series which teaches git-rev-list(1) the\n\"--nul-delimited-{input,output}\" options.\n\nOn a side note, I also noticed that git-commit(1) uses the \"--null\"\noption as an alias to \"-z\". I think it would be nice if going forward\nthere was greater consistency around the options used to control\ninput/output delimiters. Maybe \"--nul-delimited-{input,output}\" could be\nthat in the future.\n\nThanks,\n-Justin\n"},{"id":"514658","messageId":"3kjvitvmmi776obptqex7eo32qawegh6lzkih37ppkuina5ohg@loulhg5beghp","threadId":"63089","inReplyTo":"CAP8UFD1JAkTESnEOP621uECvoFGFmQ8SRd372hPLDhNzALAN+A@mail.gmail.com","subject":"Re: [PATCH v3 4/6] rev-list: support delimiting objects with NUL bytes","fromName":"Justin Tobler","fromEmail":"jltobler@gmail.com","sentAt":"2025-03-19T16:02:01Z","receivedAt":"2025-03-19T16:05:36Z","isPatch":true,"sender":{"key":"jltobler@gmail.com","avatar":"https://avatars.githubusercontent.com/u/53454972?v=4"},"body":"On 25/03/19 01:35PM, Christian Couder wrote:\n> On Fri, Mar 14, 2025 at 1:01 AM Justin Tobler wrote:\n> \n> > +test_expect_success 'rev-list -z' '\n> > + test_when_finished rm -rf repo &&\n> > +\n> > + git init repo &&\n> > + test_commit -C repo 1 &&\n> > + test_commit -C repo 2 &&\n> > +\n> > + oid1=$(git -C repo rev-parse HEAD) &&\n> > + oid2=$(git -C repo rev-parse HEAD~) &&\n> \n> It seems to me that HEAD is at commit 2 and HEAD~ at commit 1 instead\n> of the other way around.\n\nIn this case, oid1 and oid2 were ordered based on how they would show up\nin ouput, but this is somewhat confusing because its not the order they\nwere committed in.\n\nI'll change it to be in commit order instead.\n\nThanks,\n-Justin\n"},{"id":"514675","messageId":"20250319183410.1225428-1-jltobler@gmail.com","threadId":"63089","inReplyTo":"20250313235747.9583-1-jltobler@gmail.com","subject":"[PATCH v4 0/5] rev-list: introduce NUL-delimited output mode","fromName":"Justin Tobler","fromEmail":"jltobler@gmail.com","sentAt":"2025-03-19T18:34:05Z","receivedAt":"2025-03-19T18:37:53Z","isPatch":true,"sender":{"key":"jltobler@gmail.com","avatar":"https://avatars.githubusercontent.com/u/53454972?v=4"},"body":"When walking objects, git-rev-list(1) prints each object entry on a\nseparate line in the form:\n\n LF\n\nSome options, such as `--objects`, may print additional information\nabout the object on the same line:\n\n SP [] LF\n\nIn this mode, if the object path contains a newline it is truncated at\nthe newline.\n\nThe `--boundary` option also modifies output by prefixing boundary\nobjects with `-`:\n\n - LF\n\nWhen the `--missing={print,print-info}` option is provided, information\nabout any missing objects encountered during the object walk are also\nprinted in the form:\n\n ? [SP =]... LF\n\nwhere values containing LF or SP are printed in a token specific fashion\nso that the resulting encoded value does not contain either of these two\nproblematic bytes. For example, missing object paths are quoted in the C\nstyle when they contain LF or SP.\n\nTo make machine parsing easier, this series introduces a NUL-delimited\noutput mode for git-rev-list(1) via a `-z` option. In this mode, the\noutput format for object records is unified such that each object and\nits accompanying metadata is formatted without relying on object\nmetadata order. This format follows the existing `=` used\nby the `--missing` option to represent object metadata in the form:\n\n NUL [= NUL]...\n\n # Examples\n LF -> NUL\n SP LF -> NUL path= NUL\n - LF -> NUL boundary=yes NUL\n ? [SP =]... -> NUL missing=yes NUL [= NUL]...\n\nNote that token value info is printed as-is without any special encoding\nor truncation. Prefixes such as '-' and '?' are dropped in favor using a\ntoken/value pair to signal the same information.\n\nFor now this series only adds support for use with the `--objects`,\n`--boundary` and `--missing` output options. Usage of `-z` with other\noptions is rejected, so it can potentially be added in the future.\n\nThis series is structured as follows:\n\n - Patches 1 and 2 do some minor preparatory refactors.\n\n - Patch 3 adds the `-z` option to git-rev-list(1) to print\n objects in a NUL-delimited fashion.\n\n - Patch 4 teaches the `--boundary` option how to print info in a\n NUL-delimited fashino using the unified output format.\n\n - Patch 5 teaches the `--missing` option how to print info in a\n NUL-delimited fashion using the unified output format.\n\nChanges since V3:\n\n - The -z option now only makes output NUL-delimited. Input\n parsed on stdin via the `--stdin` option remains unchanged.\n This is done to remain more consistent with other log family\n commands. Support for more explicit options to control\n NUL-delimited input/ouput behavior may be added in a future\n series via `--NUL-delimited-{input,output}` options.\n\n - Changed some variable names in tests that were a little\n confusing.\n\nChanges since V2:\n\n - In patch 4, the documentation for the -z option now points out\n the `--stdin` behavior change earlier.\n\n - Minor code style and documentation changes in patch 6.\n\nChanges since V1:\n\n - Use unified output format with `=` pairs for\n all object metadata.\n\n - Add support for the `--boundary` option in NUL-delimited mode.\n\n - Add support for NUL-delimited stdin argument parsing in\n NUL-delimited mode.\n\n - Instead of using two NUL bytes to delimit between object\n records, a single NUL byte is used. Now that object metadata\n is always in the form `=`, we know a new object\n record starts when there is an OID entry which will not\n contain '='.\n\nThanks for taking a look,\n-Justin\n\nJustin Tobler (5):\n rev-list: inline `show_object_with_name()` in `show_object()`\n rev-list: refactor early option parsing\n rev-list: support delimiting objects with NUL bytes\n rev-list: support NUL-delimited --boundary option\n rev-list: support NUL-delimited --missing option\n\n Documentation/rev-list-options.adoc | 24 ++++++++\n builtin/rev-list.c | 93 +++++++++++++++++++++--------\n revision.c | 8 ---\n revision.h | 2 -\n t/t6000-rev-list-misc.sh | 51 ++++++++++++++++\n t/t6022-rev-list-missing.sh | 31 ++++++++++\n 6 files changed, 175 insertions(+), 34 deletions(-)\n\nRange-diff against v3:\n1: d2eded3ac7 = 1: d2eded3ac7 rev-list: inline `show_object_with_name()` in `show_object()`\n2: 03cd08c859 = 2: 03cd08c859 rev-list: refactor early option parsing\n3: 803a49933a < -: ---------- revision: support NUL-delimited --stdin mode\n4: 8eb7669089 ! 3: f6ee01571d rev-list: support delimiting objects with NUL bytes\n @@ Commit message\n from `--objects` is appended to the record as a token/value pair\n `path=` as-is without any truncation.\n \n - In this mode, revision and pathspec arguments provided on stdin with the\n - `--stdin` option are also separated by a NUL byte instead of being\n - newline delimited.\n -\n - For now, the `--objects` and `--stdin` flag are the only options that\n - can be used in combination with `-z`. In a subsequent commit,\n - NUL-delimited support for other options is added. Other options that do\n - not make sense with be used in combination with `-z` are rejected.\n + For now, the `--objects` flag is the only options that can be used in\n + combination with `-z`. In a subsequent commit, NUL-delimited support for\n + other options is added. Other options that do not make sense when used\n + in combination with `-z` are rejected.\n \n Signed-off-by: Justin Tobler \n \n @@ Documentation/rev-list-options.adoc: ifdef::git-rev-list[]\n +\n +-z::\n +\tInstead of being newline-delimited, each outputted object and its\n -+\taccompanying metadata is delimited using NUL bytes. In this mode, when\n -+\tthe `--stdin` option is provided, revision and pathspec arguments on\n -+\tstdin are also delimited using a NUL byte. Output is printed in the\n -+\tfollowing form:\n ++\taccompanying metadata is delimited using NUL bytes. Output is printed\n ++\tin the following form:\n ++\n +-----------------------------------------------------------------------\n + NUL [= NUL]...\n @@ builtin/rev-list.c: int cmd_rev_list(int argc,\n \t\t} else if (skip_prefix(arg, \"--missing=\", &arg)) {\n \t\t\tparse_missing_action_value(arg);\n +\t\t} else if (!strcmp(arg, \"-z\")) {\n -+\t\t\ts_r_opt.nul_delim_stdin = 1;\n +\t\t\tline_term = '\\0';\n +\t\t\tinfo_term = '\\0';\n \t\t}\n @@ t/t6000-rev-list-misc.sh: test_expect_success 'rev-list --unpacked' '\n +\ttest_commit -C repo 1 &&\n +\ttest_commit -C repo 2 &&\n +\n -+\toid1=$(git -C repo rev-parse HEAD) &&\n -+\toid2=$(git -C repo rev-parse HEAD~) &&\n ++\toid1=$(git -C repo rev-parse HEAD~) &&\n ++\toid2=$(git -C repo rev-parse HEAD) &&\n +\n -+\tprintf \"%s\\0%s\\0\" \"$oid1\" \"$oid2\" >expect &&\n ++\tprintf \"%s\\0%s\\0\" \"$oid2\" \"$oid1\" >expect &&\n +\tgit -C repo rev-list -z HEAD >actual &&\n +\n +\ttest_cmp expect actual\n @@ t/t6000-rev-list-misc.sh: test_expect_success 'rev-list --unpacked' '\n +'\n +\n test_done\n -\n - ## t/t6017-rev-list-stdin.sh ##\n -@@ t/t6017-rev-list-stdin.sh: test_expect_success '--not via stdin does not influence revisions from command l\n - \ttest_cmp expect actual\n - '\n - \n -+test_expect_success 'NUL-delimited stdin' '\n -+\tprintf \"%s\\0%s\\0%s\\0\" \"HEAD\" \"--\" \"file-1\" > input &&\n -+\n -+\tgit rev-list -z --objects HEAD -- file-1 >expect &&\n -+\tgit rev-list -z --objects --stdin actual &&\n -+\n -+\ttest_cmp expect actual\n -+'\n -+\n - test_done\n5: 591a2c7dac ! 4: ccf6bd8d35 rev-list: support NUL-delimited --boundary option\n @@ t/t6000-rev-list-misc.sh: test_expect_success 'rev-list -z --objects' '\n +\ttest_commit -C repo 1 &&\n +\ttest_commit -C repo 2 &&\n +\n -+\toid1=$(git -C repo rev-parse HEAD) &&\n -+\toid2=$(git -C repo rev-parse HEAD~) &&\n ++\toid1=$(git -C repo rev-parse HEAD~) &&\n ++\toid2=$(git -C repo rev-parse HEAD) &&\n +\n -+\tprintf \"%s\\0%s\\0boundary=yes\\0\" \"$oid1\" \"$oid2\" >expect &&\n ++\tprintf \"%s\\0%s\\0boundary=yes\\0\" \"$oid2\" \"$oid1\" >expect &&\n +\tgit -C repo rev-list -z --boundary HEAD~.. >actual &&\n +\n +\ttest_cmp expect actual\n6: 669b3b5d9f = 5: b1bd245155 rev-list: support NUL-delimited --missing option\n\nbase-commit: 87a0bdbf0f72b7561f3cd50636eee33dcb7dbcc3\n-- \n2.49.0.rc2\n\n"},{"id":"514676","messageId":"20250319183410.1225428-2-jltobler@gmail.com","threadId":"63089","inReplyTo":"20250319183410.1225428-1-jltobler@gmail.com","subject":"[PATCH v4 1/5] rev-list: inline `show_object_with_name()` in `show_object()`","fromName":"Justin Tobler","fromEmail":"jltobler@gmail.com","sentAt":"2025-03-19T18:34:06Z","receivedAt":"2025-03-19T18:37:54Z","isPatch":true,"sender":{"key":"jltobler@gmail.com","avatar":"https://avatars.githubusercontent.com/u/53454972?v=4"},"body":"The `show_object_with_name()` function only has a single call site.\nInline call to `show_object_with_name()` in `show_object()` so the\nexplicit function can be cleaned up and live closer to where it is used.\nWhile at it, factor out the code that prints the OID and newline for\nboth objects with and without a name. In a subsequent commit,\n`show_object()` is modified to support printing object information in a\nNUL-delimited format.\n\nSigned-off-by: Justin Tobler \n---\n builtin/rev-list.c | 13 +++++++++----\n revision.c | 8 --------\n revision.h | 2 --\n 3 files changed, 9 insertions(+), 14 deletions(-)\n\ndiff --git a/builtin/rev-list.c b/builtin/rev-list.c\nindex bb26bee0d4..dcd079c16c 100644\n--- a/builtin/rev-list.c\n+++ b/builtin/rev-list.c\n@@ -357,10 +357,15 @@ static void show_object(struct object *obj, const char *name, void *cb_data)\n \t\treturn;\n \t}\n \n-\tif (arg_show_object_names)\n-\t\tshow_object_with_name(stdout, obj, name);\n-\telse\n-\t\tprintf(\"%s\\n\", oid_to_hex(&obj->oid));\n+\tprintf(\"%s\", oid_to_hex(&obj->oid));\n+\n+\tif (arg_show_object_names) {\n+\t\tputchar(' ');\n+\t\tfor (const char *p = name; *p && *p != '\\n'; p++)\n+\t\t\tputchar(*p);\n+\t}\n+\n+\tputchar('\\n');\n }\n \n static void show_edge(struct commit *commit)\ndiff --git a/revision.c b/revision.c\nindex c4390f0938..0eaebe4478 100644\n--- a/revision.c\n+++ b/revision.c\n@@ -59,14 +59,6 @@ implement_shared_commit_slab(revision_sources, char *);\n \n static inline int want_ancestry(const struct rev_info *revs);\n \n-void show_object_with_name(FILE *out, struct object *obj, const char *name)\n-{\n-\tfprintf(out, \"%s \", oid_to_hex(&obj->oid));\n-\tfor (const char *p = name; *p && *p != '\\n'; p++)\n-\t\tfputc(*p, out);\n-\tfputc('\\n', out);\n-}\n-\n static void mark_blob_uninteresting(struct blob *blob)\n {\n \tif (!blob)\ndiff --git a/revision.h b/revision.h\nindex 71e984c452..21c6a69899 100644\n--- a/revision.h\n+++ b/revision.h\n@@ -489,8 +489,6 @@ void mark_parents_uninteresting(struct rev_info *revs, struct commit *commit);\n void mark_tree_uninteresting(struct repository *r, struct tree *tree);\n void mark_trees_uninteresting_sparse(struct repository *r, struct oidset *trees);\n \n-void show_object_with_name(FILE *, struct object *, const char *);\n-\n /**\n * Helpers to check if a reference should be excluded.\n */\n-- \n2.49.0.rc2\n\n"},{"id":"514677","messageId":"20250319183410.1225428-3-jltobler@gmail.com","threadId":"63089","inReplyTo":"20250319183410.1225428-1-jltobler@gmail.com","subject":"[PATCH v4 2/5] rev-list: refactor early option parsing","fromName":"Justin Tobler","fromEmail":"jltobler@gmail.com","sentAt":"2025-03-19T18:34:07Z","receivedAt":"2025-03-19T18:37:55Z","isPatch":true,"sender":{"key":"jltobler@gmail.com","avatar":"https://avatars.githubusercontent.com/u/53454972?v=4"},"body":"Before invoking `setup_revisions()`, the `--missing` and\n`--exclude-promisor-objects` options are parsed early. In a subsequent\ncommit, another option is added that must be parsed early.\n\nRefactor the code to parse both options in a single early pass.\n\nSigned-off-by: Justin Tobler \n---\n builtin/rev-list.c | 17 +++++++----------\n 1 file changed, 7 insertions(+), 10 deletions(-)\n\ndiff --git a/builtin/rev-list.c b/builtin/rev-list.c\nindex dcd079c16c..04d9c893b5 100644\n--- a/builtin/rev-list.c\n+++ b/builtin/rev-list.c\n@@ -16,6 +16,7 @@\n #include \"object-file.h\"\n #include \"object-store-ll.h\"\n #include \"pack-bitmap.h\"\n+#include \"parse-options.h\"\n #include \"log-tree.h\"\n #include \"graph.h\"\n #include \"bisect.h\"\n@@ -639,19 +640,15 @@ int cmd_rev_list(int argc,\n \t\tif (!strcmp(arg, \"--exclude-promisor-objects\")) {\n \t\t\tfetch_if_missing = 0;\n \t\t\trevs.exclude_promisor_objects = 1;\n-\t\t\tbreak;\n-\t\t}\n-\t}\n-\tfor (i = 1; i < argc; i++) {\n-\t\tconst char *arg = argv[i];\n-\t\tif (skip_prefix(arg, \"--missing=\", &arg)) {\n-\t\t\tif (revs.exclude_promisor_objects)\n-\t\t\t\tdie(_(\"options '%s' and '%s' cannot be used together\"), \"--exclude-promisor-objects\", \"--missing\");\n-\t\t\tif (parse_missing_action_value(arg))\n-\t\t\t\tbreak;\n+\t\t} else if (skip_prefix(arg, \"--missing=\", &arg)) {\n+\t\t\tparse_missing_action_value(arg);\n \t\t}\n \t}\n \n+\tdie_for_incompatible_opt2(revs.exclude_promisor_objects,\n+\t\t\t\t \"--exclude_promisor_objects\",\n+\t\t\t\t arg_missing_action, \"--missing\");\n+\n \tif (arg_missing_action)\n \t\trevs.do_not_die_on_missing_objects = 1;\n \n-- \n2.49.0.rc2\n\n"},{"id":"514678","messageId":"20250319183410.1225428-4-jltobler@gmail.com","threadId":"63089","inReplyTo":"20250319183410.1225428-1-jltobler@gmail.com","subject":"[PATCH v4 3/5] rev-list: support delimiting objects with NUL bytes","fromName":"Justin Tobler","fromEmail":"jltobler@gmail.com","sentAt":"2025-03-19T18:34:08Z","receivedAt":"2025-03-19T18:37:56Z","isPatch":true,"sender":{"key":"jltobler@gmail.com","avatar":"https://avatars.githubusercontent.com/u/53454972?v=4"},"body":"When walking objects, git-rev-list(1) prints each object entry on a\nseparate line. Some options, such as `--objects`, may print additional\ninformation about tree and blob object on the same line in the form:\n\n $ git rev-list --objects \n SP [] LF\n\nNote that in this form the SP is appended regardless of whether the tree\nor blob object has path information available. Paths containing a\nnewline are also truncated at the newline.\n\nIntroduce the `-z` option for git-rev-list(1) which reformats the output\nto use NUL-delimiters between objects and associated info in the\nfollowing form:\n\n $ git rev-list -z --objects \n NUL [path= NUL]\n\nIn this form, the start of each record is signaled by an OID entry that\nis all hexidecimal and does not contain any '='. Additional path info\nfrom `--objects` is appended to the record as a token/value pair\n`path=` as-is without any truncation.\n\nFor now, the `--objects` flag is the only options that can be used in\ncombination with `-z`. In a subsequent commit, NUL-delimited support for\nother options is added. Other options that do not make sense when used\nin combination with `-z` are rejected.\n\nSigned-off-by: Justin Tobler \n---\n Documentation/rev-list-options.adoc | 21 +++++++++++++++++\n builtin/rev-list.c | 35 ++++++++++++++++++++++++-----\n t/t6000-rev-list-misc.sh | 35 +++++++++++++++++++++++++++++\n 3 files changed, 86 insertions(+), 5 deletions(-)\n\ndiff --git a/Documentation/rev-list-options.adoc b/Documentation/rev-list-options.adoc\nindex 785c0786e0..aef83813b8 100644\n--- a/Documentation/rev-list-options.adoc\n+++ b/Documentation/rev-list-options.adoc\n@@ -361,6 +361,27 @@ ifdef::git-rev-list[]\n --progress=
::\n \tShow progress reports on stderr as objects are considered. The\n \t`
` text will be printed with each progress update.\n+\n+-z::\n+\tInstead of being newline-delimited, each outputted object and its\n+\taccompanying metadata is delimited using NUL bytes. Output is printed\n+\tin the following form:\n++\n+-----------------------------------------------------------------------\n+ NUL [= NUL]...\n+-----------------------------------------------------------------------\n++\n+Additional object metadata, such as object paths, is printed using the\n+`=` form. Token values are printed as-is without any\n+encoding/truncation. An OID entry never contains a '=' character and thus\n+is used to signal the start of a new object record. Examples:\n++\n+-----------------------------------------------------------------------\n+ NUL\n+ NUL path= NUL\n+-----------------------------------------------------------------------\n++\n+This mode is only compatible with the `--objects` output option.\n endif::git-rev-list[]\n \n History Simplification\ndiff --git a/builtin/rev-list.c b/builtin/rev-list.c\nindex 04d9c893b5..17de99d9ca 100644\n--- a/builtin/rev-list.c\n+++ b/builtin/rev-list.c\n@@ -65,6 +65,7 @@ static const char rev_list_usage[] =\n \" --abbrev-commit\\n\"\n \" --left-right\\n\"\n \" --count\\n\"\n+\" -z\\n\"\n \" special purpose:\\n\"\n \" --bisect\\n\"\n \" --bisect-vars\\n\"\n@@ -97,6 +98,9 @@ static int arg_show_object_names = 1;\n \n #define DEFAULT_OIDSET_SIZE (16*1024)\n \n+static char line_term = '\\n';\n+static char info_term = ' ';\n+\n static int show_disk_usage;\n static off_t total_disk_usage;\n static int human_readable;\n@@ -264,7 +268,7 @@ static void show_commit(struct commit *commit, void *data)\n \tif (revs->commit_format == CMIT_FMT_ONELINE)\n \t\tputchar(' ');\n \telse if (revs->include_header)\n-\t\tputchar('\\n');\n+\t\tputchar(line_term);\n \n \tif (revs->verbose_header) {\n \t\tstruct strbuf buf = STRBUF_INIT;\n@@ -361,12 +365,16 @@ static void show_object(struct object *obj, const char *name, void *cb_data)\n \tprintf(\"%s\", oid_to_hex(&obj->oid));\n \n \tif (arg_show_object_names) {\n-\t\tputchar(' ');\n-\t\tfor (const char *p = name; *p && *p != '\\n'; p++)\n-\t\t\tputchar(*p);\n+\t\tif (line_term) {\n+\t\t\tputchar(info_term);\n+\t\t\tfor (const char *p = name; *p && *p != '\\n'; p++)\n+\t\t\t\tputchar(*p);\n+\t\t} else if (*name) {\n+\t\t\tprintf(\"%cpath=%s\", info_term, name);\n+\t\t}\n \t}\n \n-\tputchar('\\n');\n+\tputchar(line_term);\n }\n \n static void show_edge(struct commit *commit)\n@@ -642,6 +650,9 @@ int cmd_rev_list(int argc,\n \t\t\trevs.exclude_promisor_objects = 1;\n \t\t} else if (skip_prefix(arg, \"--missing=\", &arg)) {\n \t\t\tparse_missing_action_value(arg);\n+\t\t} else if (!strcmp(arg, \"-z\")) {\n+\t\t\tline_term = '\\0';\n+\t\t\tinfo_term = '\\0';\n \t\t}\n \t}\n \n@@ -757,6 +768,20 @@ int cmd_rev_list(int argc,\n \t\tusage(rev_list_usage);\n \n \t}\n+\n+\t/*\n+\t * Reject options currently incompatible with -z. For some options, this\n+\t * is not an inherent limitation and support may be implemented in the\n+\t * future.\n+\t */\n+\tif (!line_term) {\n+\t\tif (revs.graph || revs.verbose_header || show_disk_usage ||\n+\t\t info.show_timestamp || info.header_prefix || bisect_list ||\n+\t\t use_bitmap_index || revs.edge_hint || revs.left_right ||\n+\t\t revs.cherry_mark || arg_missing_action || revs.boundary)\n+\t\t\tdie(_(\"-z option used with unsupported option\"));\n+\t}\n+\n \tif (revs.commit_format != CMIT_FMT_USERFORMAT)\n \t\trevs.include_header = 1;\n \tif (revs.commit_format != CMIT_FMT_UNSPECIFIED) {\ndiff --git a/t/t6000-rev-list-misc.sh b/t/t6000-rev-list-misc.sh\nindex 6289a2e8b0..886e2fc710 100755\n--- a/t/t6000-rev-list-misc.sh\n+++ b/t/t6000-rev-list-misc.sh\n@@ -182,4 +182,39 @@ test_expect_success 'rev-list --unpacked' '\n \ttest_cmp expect actual\n '\n \n+test_expect_success 'rev-list -z' '\n+\ttest_when_finished rm -rf repo &&\n+\n+\tgit init repo &&\n+\ttest_commit -C repo 1 &&\n+\ttest_commit -C repo 2 &&\n+\n+\toid1=$(git -C repo rev-parse HEAD~) &&\n+\toid2=$(git -C repo rev-parse HEAD) &&\n+\n+\tprintf \"%s\\0%s\\0\" \"$oid2\" \"$oid1\" >expect &&\n+\tgit -C repo rev-list -z HEAD >actual &&\n+\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'rev-list -z --objects' '\n+\ttest_when_finished rm -rf repo &&\n+\n+\tgit init repo &&\n+\ttest_commit -C repo 1 &&\n+\ttest_commit -C repo 2 &&\n+\n+\toid1=$(git -C repo rev-parse HEAD:1.t) &&\n+\toid2=$(git -C repo rev-parse HEAD:2.t) &&\n+\tpath1=1.t &&\n+\tpath2=2.t &&\n+\n+\tprintf \"%s\\0path=%s\\0%s\\0path=%s\\0\" \"$oid1\" \"$path1\" \"$oid2\" \"$path2\" \\\n+\t\t>expect &&\n+\tgit -C repo rev-list -z --objects HEAD:1.t HEAD:2.t >actual &&\n+\n+\ttest_cmp expect actual\n+'\n+\n test_done\n-- \n2.49.0.rc2\n\n"},{"id":"514679","messageId":"20250319183410.1225428-5-jltobler@gmail.com","threadId":"63089","inReplyTo":"20250319183410.1225428-1-jltobler@gmail.com","subject":"[PATCH v4 4/5] rev-list: support NUL-delimited --boundary option","fromName":"Justin Tobler","fromEmail":"jltobler@gmail.com","sentAt":"2025-03-19T18:34:09Z","receivedAt":"2025-03-19T18:37:57Z","isPatch":true,"sender":{"key":"jltobler@gmail.com","avatar":"https://avatars.githubusercontent.com/u/53454972?v=4"},"body":"The `--boundary` option for git-rev-list(1) prints boundary objects\nfound while performing the object walk in the form:\n\n $ git rev-list --boundary \n - LF\n\nAdd support for printing boundary objects in a NUL-delimited format when\nthe `-z` option is enabled.\n\n $ git rev-list -z --boundary \n NUL boundary=yes NUL\n\nIn this mode, instead of prefixing the boundary OID with '-', a separate\n`boundary=yes` token/value pair is appended.\n\nSigned-off-by: Justin Tobler \n---\n Documentation/rev-list-options.adoc | 12 +++++++-----\n builtin/rev-list.c | 9 +++++++--\n t/t6000-rev-list-misc.sh | 16 ++++++++++++++++\n 3 files changed, 30 insertions(+), 7 deletions(-)\n\ndiff --git a/Documentation/rev-list-options.adoc b/Documentation/rev-list-options.adoc\nindex aef83813b8..3fc9902d6b 100644\n--- a/Documentation/rev-list-options.adoc\n+++ b/Documentation/rev-list-options.adoc\n@@ -371,17 +371,19 @@ ifdef::git-rev-list[]\n NUL [= NUL]...\n -----------------------------------------------------------------------\n +\n-Additional object metadata, such as object paths, is printed using the\n-`=` form. Token values are printed as-is without any\n-encoding/truncation. An OID entry never contains a '=' character and thus\n-is used to signal the start of a new object record. Examples:\n+Additional object metadata, such as object paths or boundary objects, is\n+printed using the `=` form. Token values are printed as-is\n+without any encoding/truncation. An OID entry never contains a '=' character\n+and thus is used to signal the start of a new object record. Examples:\n +\n -----------------------------------------------------------------------\n NUL\n NUL path= NUL\n+ NUL boundary=yes NUL\n -----------------------------------------------------------------------\n +\n-This mode is only compatible with the `--objects` output option.\n+This mode is only compatible with the `--objects` and `--boundary` output\n+options.\n endif::git-rev-list[]\n \n History Simplification\ndiff --git a/builtin/rev-list.c b/builtin/rev-list.c\nindex 17de99d9ca..bcb880f109 100644\n--- a/builtin/rev-list.c\n+++ b/builtin/rev-list.c\n@@ -240,13 +240,18 @@ static void show_commit(struct commit *commit, void *data)\n \t\tfputs(info->header_prefix, stdout);\n \n \tif (revs->include_header) {\n-\t\tif (!revs->graph)\n+\t\tif (!revs->graph && line_term)\n \t\t\tfputs(get_revision_mark(revs, commit), stdout);\n \t\tif (revs->abbrev_commit && revs->abbrev)\n \t\t\tfputs(repo_find_unique_abbrev(the_repository, &commit->object.oid, revs->abbrev),\n \t\t\t stdout);\n \t\telse\n \t\t\tfputs(oid_to_hex(&commit->object.oid), stdout);\n+\n+\t\tif (!line_term) {\n+\t\t\tif (commit->object.flags & BOUNDARY)\n+\t\t\t\tprintf(\"%cboundary=yes\", info_term);\n+\t\t}\n \t}\n \tif (revs->print_parents) {\n \t\tstruct commit_list *parents = commit->parents;\n@@ -778,7 +783,7 @@ int cmd_rev_list(int argc,\n \t\tif (revs.graph || revs.verbose_header || show_disk_usage ||\n \t\t info.show_timestamp || info.header_prefix || bisect_list ||\n \t\t use_bitmap_index || revs.edge_hint || revs.left_right ||\n-\t\t revs.cherry_mark || arg_missing_action || revs.boundary)\n+\t\t revs.cherry_mark || arg_missing_action)\n \t\t\tdie(_(\"-z option used with unsupported option\"));\n \t}\n \ndiff --git a/t/t6000-rev-list-misc.sh b/t/t6000-rev-list-misc.sh\nindex 886e2fc710..33881274a4 100755\n--- a/t/t6000-rev-list-misc.sh\n+++ b/t/t6000-rev-list-misc.sh\n@@ -217,4 +217,20 @@ test_expect_success 'rev-list -z --objects' '\n \ttest_cmp expect actual\n '\n \n+test_expect_success 'rev-list -z --boundary' '\n+\ttest_when_finished rm -rf repo &&\n+\n+\tgit init repo &&\n+\ttest_commit -C repo 1 &&\n+\ttest_commit -C repo 2 &&\n+\n+\toid1=$(git -C repo rev-parse HEAD~) &&\n+\toid2=$(git -C repo rev-parse HEAD) &&\n+\n+\tprintf \"%s\\0%s\\0boundary=yes\\0\" \"$oid2\" \"$oid1\" >expect &&\n+\tgit -C repo rev-list -z --boundary HEAD~.. >actual &&\n+\n+\ttest_cmp expect actual\n+'\n+\n test_done\n-- \n2.49.0.rc2\n\n"},{"id":"514680","messageId":"20250319183410.1225428-6-jltobler@gmail.com","threadId":"63089","inReplyTo":"20250319183410.1225428-1-jltobler@gmail.com","subject":"[PATCH v4 5/5] rev-list: support NUL-delimited --missing option","fromName":"Justin Tobler","fromEmail":"jltobler@gmail.com","sentAt":"2025-03-19T18:34:10Z","receivedAt":"2025-03-19T18:37:58Z","isPatch":true,"sender":{"key":"jltobler@gmail.com","avatar":"https://avatars.githubusercontent.com/u/53454972?v=4"},"body":"The `--missing={print,print-info}` option for git-rev-list(1) prints\nmissing objects found while performing the object walk in the form:\n\n $ git rev-list --missing=print-info \n ? [SP =]... LF\n\nAdd support for printing missing objects in a NUL-delimited format when\nthe `-z` option is enabled.\n\n $ git rev-list -z --missing=print-info \n NUL missing=yes NUL [= NUL]...\n\nIn this mode, values containing special characters or spaces are printed\nas-is without being escaped or quoted. Instead of prefixing the missing\nOID with '?', a separate `missing=yes` token/value pair is appended.\n\nSigned-off-by: Justin Tobler \n---\n Documentation/rev-list-options.adoc | 5 +++--\n builtin/rev-list.c | 31 ++++++++++++++++++++---------\n t/t6022-rev-list-missing.sh | 31 +++++++++++++++++++++++++++++\n 3 files changed, 56 insertions(+), 11 deletions(-)\n\ndiff --git a/Documentation/rev-list-options.adoc b/Documentation/rev-list-options.adoc\nindex 3fc9902d6b..0e5605a85e 100644\n--- a/Documentation/rev-list-options.adoc\n+++ b/Documentation/rev-list-options.adoc\n@@ -380,10 +380,11 @@ and thus is used to signal the start of a new object record. Examples:\n NUL\n NUL path= NUL\n NUL boundary=yes NUL\n+ NUL missing=yes NUL [= NUL]...\n -----------------------------------------------------------------------\n +\n-This mode is only compatible with the `--objects` and `--boundary` output\n-options.\n+This mode is only compatible with the `--objects`, `--boundary`, and\n+`--missing` output options.\n endif::git-rev-list[]\n \n History Simplification\ndiff --git a/builtin/rev-list.c b/builtin/rev-list.c\nindex bcb880f109..e6ee3f82ee 100644\n--- a/builtin/rev-list.c\n+++ b/builtin/rev-list.c\n@@ -136,24 +136,37 @@ static void print_missing_object(struct missing_objects_map_entry *entry,\n {\n \tstruct strbuf sb = STRBUF_INIT;\n \n+\tif (line_term)\n+\t\tprintf(\"?%s\", oid_to_hex(&entry->entry.oid));\n+\telse\n+\t\tprintf(\"%s%cmissing=yes\", oid_to_hex(&entry->entry.oid),\n+\t\t info_term);\n+\n \tif (!print_missing_info) {\n-\t\tprintf(\"?%s\\n\", oid_to_hex(&entry->entry.oid));\n+\t\tputchar(line_term);\n \t\treturn;\n \t}\n \n \tif (entry->path && *entry->path) {\n-\t\tstruct strbuf path = STRBUF_INIT;\n+\t\tstrbuf_addf(&sb, \"%cpath=\", info_term);\n+\n+\t\tif (line_term) {\n+\t\t\tstruct strbuf path = STRBUF_INIT;\n \n-\t\tstrbuf_addstr(&sb, \" path=\");\n-\t\tquote_path(entry->path, NULL, &path, QUOTE_PATH_QUOTE_SP);\n-\t\tstrbuf_addbuf(&sb, &path);\n+\t\t\tquote_path(entry->path, NULL, &path, QUOTE_PATH_QUOTE_SP);\n+\t\t\tstrbuf_addbuf(&sb, &path);\n \n-\t\tstrbuf_release(&path);\n+\t\t\tstrbuf_release(&path);\n+\t\t} else {\n+\t\t\tstrbuf_addstr(&sb, entry->path);\n+\t\t}\n \t}\n \tif (entry->type)\n-\t\tstrbuf_addf(&sb, \" type=%s\", type_name(entry->type));\n+\t\tstrbuf_addf(&sb, \"%ctype=%s\", info_term, type_name(entry->type));\n+\n+\tfwrite(sb.buf, sizeof(char), sb.len, stdout);\n+\tputchar(line_term);\n \n-\tprintf(\"?%s%s\\n\", oid_to_hex(&entry->entry.oid), sb.buf);\n \tstrbuf_release(&sb);\n }\n \n@@ -783,7 +796,7 @@ int cmd_rev_list(int argc,\n \t\tif (revs.graph || revs.verbose_header || show_disk_usage ||\n \t\t info.show_timestamp || info.header_prefix || bisect_list ||\n \t\t use_bitmap_index || revs.edge_hint || revs.left_right ||\n-\t\t revs.cherry_mark || arg_missing_action)\n+\t\t revs.cherry_mark)\n \t\t\tdie(_(\"-z option used with unsupported option\"));\n \t}\n \ndiff --git a/t/t6022-rev-list-missing.sh b/t/t6022-rev-list-missing.sh\nindex 3e2790d4c8..08e92dd002 100755\n--- a/t/t6022-rev-list-missing.sh\n+++ b/t/t6022-rev-list-missing.sh\n@@ -198,4 +198,35 @@ do\n \t'\n done\n \n+test_expect_success \"-z nul-delimited --missing\" '\n+\ttest_when_finished rm -rf repo &&\n+\n+\tgit init repo &&\n+\t(\n+\t\tcd repo &&\n+\t\tgit commit --allow-empty -m first &&\n+\n+\t\tpath=\"foo bar\" &&\n+\t\techo foobar >\"$path\" &&\n+\t\tgit add -A &&\n+\t\tgit commit -m second &&\n+\n+\t\toid=$(git rev-parse \"HEAD:$path\") &&\n+\t\ttype=\"$(git cat-file -t $oid)\" &&\n+\n+\t\tobj_path=\".git/objects/$(test_oid_to_path $oid)\" &&\n+\n+\t\tgit rev-list -z --objects --no-object-names \\\n+\t\t\tHEAD ^\"$oid\" >expect &&\n+\t\tprintf \"%s\\0missing=yes\\0path=%s\\0type=%s\\0\" \"$oid\" \"$path\" \\\n+\t\t\t\"$type\" >>expect &&\n+\n+\t\tmv \"$obj_path\" \"$obj_path.hidden\" &&\n+\t\tgit rev-list -z --objects --no-object-names \\\n+\t\t\t--missing=print-info HEAD >actual &&\n+\n+\t\ttest_cmp expect actual\n+\t)\n+'\n+\n test_done\n-- \n2.49.0.rc2\n\n"}]}