git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 5/5] cat-file: Introduce new option to delimit output with NUL characters

From
Patrick Steinhardt <ps@pks.im>
Date
Jun 6, 2023, 05:00 UTC
Message-ID
<ZH69UBT_H0OtGR6l@ncase>
In-Reply-To
<9900512f-b0da-2e47-f1ab-ed51ec2c78ff@gmail.com>
On Mon, Jun 05, 2023 at 04:47:14PM +0100, Phillip Wood wrote:
Show 36 quoted lines
> > @@ -384,6 +390,11 @@ notdir SP <size> LF
> >   is printed when, during symlink resolution, a file is used as a
> >   directory name.
> >   
> > +Alternatively, when `-Z` is passed, the line feeds in any of the above examples
> > +are replaced with NUL terminators. This ensures that output will be parsable if
> > +the output itself would contain a linefeed and is thus recommended for
> > +scripting purposes.
> > +
> >   CAVEATS
> >   -------
> >   
> > diff --git a/builtin/cat-file.c b/builtin/cat-file.c
> > index 001dcb24d6..90ef407d30 100644
> > --- a/builtin/cat-file.c
> > +++ b/builtin/cat-file.c
> > @@ -492,17 +494,18 @@ static void batch_object_write(const char *obj_name,
> >   	strbuf_reset(scratch);
> >   
> >   	if (!opt->format) {
> > -		print_default_format(scratch, data);
> > +		print_default_format(scratch, data, opt);
> >   	} else {
> >   		strbuf_expand(scratch, opt->format, expand_format, data);
> > -		strbuf_addch(scratch, '\n');
> > +		strbuf_addch(scratch, opt->output_delim);
> >   	}
> >   
> >   	batch_write(opt, scratch->buf, scratch->len);
> >   
> >   	if (opt->batch_mode == BATCH_MODE_CONTENTS) {
> > +		char buf[] = {opt->output_delim};
> 
> I found this a bit confusing, I think it would be clearer just to do
> 
> 	batch_write(opt, &opt->output_delim, 1);
Agreed, that's cleaner.
Show 15 quoted lines
> >   		print_object_or_die(opt, data);
> > -		batch_write(opt, "\n", 1);
> > +		batch_write(opt, buf, 1);
> >   	}
> >   }
> 
> > @@ -920,7 +927,7 @@ int cmd_cat_file(int argc, const char **argv, const char *prefix)
> >   		N_("git cat-file (-t | -s) [--allow-unknown-type] <object>"),
> >   		N_("git cat-file (--batch | --batch-check | --batch-command) [--batch-all-objects]\n"
> >   		   "             [--buffer] [--follow-symlinks] [--unordered]\n"
> > -		   "             [--textconv | --filters] [-z]"),
> > +		   "             [--textconv | --filters] [-z] [-Z]"),
> 
> If we're recommending that people don't use '-z' then maybe we should 
> remove it from the synopsis and add OPT_HIDDEN to it below.

I might still change this depending on the conclusion Junio and I will arrive at, but for now I agree that this makes sense.

Show 40 quoted lines
> >   		N_("git cat-file (--textconv | --filters)\n"
> >   		   "             [<rev>:<path|tree-ish> | --path=<path|tree-ish> <rev>]"),
> >   		NULL
> > @@ -950,6 +957,7 @@ int cmd_cat_file(int argc, const char **argv, const char *prefix)
> >   			PARSE_OPT_OPTARG | PARSE_OPT_NONEG,
> >   			batch_option_callback),
> >   		OPT_BOOL('z', NULL, &input_nul_terminated, N_("stdin is NUL-terminated")),
> > +		OPT_BOOL('Z', NULL, &nul_terminated, N_("stdin and stdout is NUL-terminated")),
> 
> > diff --git a/t/t1006-cat-file.sh b/t/t1006-cat-file.sh
> > index 7b985cfded..d73a0be1b9 100755
> > --- a/t/t1006-cat-file.sh
> > +++ b/t/t1006-cat-file.sh
> > @@ -392,17 +393,18 @@ deadbeef
> >   
> >   "
> >   
> > -batch_output="$hello_sha1 blob $hello_size
> > -$hello_content
> > -$commit_sha1 commit $commit_size
> > -$commit_content
> > -$tag_sha1 tag $tag_size
> > -$tag_content
> > -deadbeef missing
> > - missing"
> > +printf "%s\0" \
> > +	"$hello_sha1 blob $hello_size" \
> > +	"$hello_content" \
> > +	"$commit_sha1 commit $commit_size" \
> > +	"$commit_content" \
> > +	"$tag_sha1 tag $tag_size" \
> > +	"$tag_content" \
> > +	"deadbeef missing" \
> > +	" missing" >batch_output
> 
> I think writing the expected output to a file is a good change as we 
> always use it with test_cmp. As "-z" is deprecated I think it makes 
> sense to model the expected output for "-Z" and use tr for the "-z" 
> tests as you have done here. It looks like we have good coverage of the 
> new option.

It's actually also required in order to not have to specify the expected output twice. While we could leave this as-is, translating it to be NUL terminated via `tr \n \0` doesn't work as the output contains newlines in places where we don't want to translate them to NUL delimiters. And storing the NUL-delimited string in a variable doesn't work either as shells will truncate the C strings.

Thanks for your review!
Patrick
Previous: Junio C HamanoNext: Junio C Hamano
Message 13 of 22 in “cat-file: introduce NUL-terminated output format”
  1. 0/5 cat-file: introduce NUL-terminated output formatPatrick Steinhardt, Jun 2, 2023
  2. 1/5 t1006: don't strip timestamps from expected resultsPatrick Steinhardt, Jun 2, 2023
  3. 2/5 t1006: modernize test style to use `test_cmp`Patrick Steinhardt, Jun 2, 2023
  4. 3/5 strbuf: provide CRLF-aware helper to read until a specified delimiterPatrick Steinhardt, Jun 2, 2023
  5. 4/5 cat-file: simplify reading from standard inputPatrick Steinhardt, Jun 2, 2023
  6. 5/5 cat-file: Introduce new option to delimit output with NUL charactersPatrick Steinhardt, Jun 2, 2023
  7. Phillip WoodJun 5, 2023
  8. Junio C HamanoJun 5, 2023
  9. Patrick SteinhardtJun 6, 2023
  10. Junio C HamanoJun 6, 2023
  11. Patrick SteinhardtJun 6, 2023
  12. Junio C HamanoJun 12, 2023
  13. Patrick SteinhardtJun 6, 2023
  14. Junio C HamanoJun 6, 2023
  15. Junio C HamanoJun 3, 2023
  16. 0/5 catfile: introduce NUL-terminated output formatPatrick Steinhardt, Jun 6, 2023
  17. 1/5 t1006: don't strip timestamps from expected resultsPatrick Steinhardt, Jun 6, 2023
  18. 2/5 t1006: modernize test style to use `test_cmp`Patrick Steinhardt, Jun 6, 2023
  19. 3/5 strbuf: provide CRLF-aware helper to read until a specified delimiterPatrick Steinhardt, Jun 6, 2023
  20. 4/5 cat-file: simplify reading from standard inputPatrick Steinhardt, Jun 6, 2023
  21. 5/5 cat-file: introduce option to delimit input and output with NULPatrick Steinhardt, Jun 6, 2023
  22. Junio C HamanoJun 12, 2023

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.