git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v4 3/3] cat-file: add --batch-command mode

From
Eric Sunshine <sunshine@sunshineco.com>
Date
Feb 10, 2022, 22:46 UTC
Message-ID
<CAPig+cQ6mOipiCKSOUo-d0UgoGq+YwShiw0CgcqGvRqxV5K77g@mail.gmail.com>
In-Reply-To
<6c51324a6623b62c385ec707a773c21375596584.1644465706.git.gitgitgadget@gmail.com>

On Wed, Feb 9, 2022 at 11:01 PM John Cai via GitGitGadget <gitgitgadget@gmail.com> wrote:

> Add a new flag --batch-command that accepts commands and arguments
> from stdin, similar to git-update-ref --stdin.
Some comments not offered by other reviewers...
Show 22 quoted lines
> This patch adds the basic structure for adding command which can be
> extended in the future to add more commands. It also adds the following
> two commands (on top of the flush command):
>
> contents <object> LF
> info <object> LF
>
> The contents command takes an <object> argument and prints out the object
> contents.
>
> The info command takes a <object> argument and prints out the object
> metadata.
>
> These can be used in the following way with --buffer:
>
> info <sha1> LF
> contents <sha1> LF
> contents <sha1> LF
> info <sha1> LF
> flush
> info <sha1> LF
> flush

s/<sha1>/<object>/ for consistency with the usage information earlier in the commit message, and since Git is migrating to SHA-256, and to avoid reviewer confusion as occurred earlier[1].

Also: s/flush$/flush LF/
Show 7 quoted lines
> When used without --buffer:
>
> info <sha1> LF
> contents <sha1> LF
> contents <sha1> LF
> info <sha1> LF
> info <sha1> LF
Ditto.
[1]: https://lore.kernel.org/git/CAPig+cTeqhOYTu9WBiY=LnZtt35hAp3Qa5RduC2yLut6p01_1w@mail.gmail.com/
Show 8 quoted lines
> diff --git a/Documentation/git-cat-file.txt b/Documentation/git-cat-file.txt
> @@ -96,6 +96,30 @@ OPTIONS
> +--batch-command::
> +       Enter a command mode that reads commands and arguments from stdin.
> +       May not be combined with any other options or arguments except
> +       `--textconv` or `--filters`, in which case the input lines also need to
> +       specify the path, separated by whitespace.  See the section
> +       `BATCH OUTPUT` below for details.
The SYNOPSIS probably needs an update too.

Perhaps say something like "Recognized commands include:" here before enumerating the commands themselves?

> +--
> +contents <object>::
> +       Print object contents for object reference <object>. This corresponds to
> +       the output of --batch.

s/<object>/`<object>`/ s/--batch/`--batch`/

> +info <object>::
> +       Print object info for object reference <object>. This corresponds to the
> +       output of --batch-check.

s/<object>/`<object>`/ s/--batch/`--batch-check`/

Show 6 quoted lines
> +flush::
> +       Used in --buffer mode to execute all preceding commands that were issued
> +       since the beginning or since the last flush was issued. When --buffer
> +       is used, no output will come until flush is issued. When --buffer is not
> +       used, commands are flushed each time without issuing `flush`.
> +--

s/--buffer/`--buffer`/g s/flush/`flush`/g

This says that it's legal to use `--buffer` along with `--batch-command`, but the description of `--batch-command` itself just above says that it can be combined only with `--textconv` or `--filters`. (I see you copied the problematic text from the other batch options, so they also are guilty of not mentioning `--buffer`. This series doesn't necessarily need to fix those existing documentation problems, but perhaps don't repeat the problem with newly-added text?)

The description of the `--buffer` option probably also needs to be updated to mention the new `--batch-command` option, and there may be other places in this document which should mention it, as well.

Show 34 quoted lines
> diff --git a/builtin/cat-file.c b/builtin/cat-file.c
> +static const struct parse_cmd {
> +       const char *prefix;
> +       parse_cmd_fn_t fn;
> +       unsigned takes_args;
> +} commands[] = {
> +       { "contents", parse_cmd_contents, 1},
> +       { "info", parse_cmd_info, 1},
> +};
> +
> +static void batch_objects_command(struct batch_options *opt,
> +                                   struct strbuf *output,
> +                                   struct expand_data *data)
> +{
> +       while (!strbuf_getline(&input, stdin)) {
> +               if (!input.len)
> +                       die(_("empty command in input"));
> +               if (isspace(*input.buf))
> +                       die(_("whitespace before command: '%s'"), input.buf);
> +
> +               if (skip_prefix(input.buf, "flush", &cmd_end)) {
> +                       if (!opt->buffer_output)
> +                               die(_("flush is only for --buffer mode"));
> +                       if (*cmd_end)
> +                               die(_("flush takes no arguments"));
> +
> +                       dispatch_calls(opt, output, data, queued_cmd, nr);
> +                       nr = 0;
> +                       continue;
> +               }
> +
> +               for (i = 0; i < ARRAY_SIZE(commands); i++) {
> +                       if (!skip_prefix(input.buf, commands[i].prefix, &cmd_end))
> +                               continue;

This prefix-matching is going to incorrectly match non-commands such as "contentsify <object>" and "information <object>" and then treat them as "contents fy <object>" and "info mation <object>", respectively, with undesirable results. You need to verify that there is a space or NUL at `*cmd_end` before treating `input.buf` as an actual command.

> +                       cmd = &commands[i];
> +                       if (cmd->takes_args)

What happens if `cmd->takes_arg` is true but no arguments follow the command? Should that be diagnosed as an error?

> +                               p = cmd_end + 1;

This unconditional +1 is going to make `p` point beyond the NUL character if the input is just a bare command, such as "contents" or "info" without any space or any argument...

Show 14 quoted lines
> +                       break;
> +               }
> +
> +               if (!cmd)
> +                       die(_("unknown command: '%s'"), input.buf);
> +
> +               if (!opt->buffer_output) {
> +                       cmd->fn(opt, p, output, data);
> +                       continue;
> +               }
> +
> +               ALLOC_GROW(queued_cmd, nr + 1, alloc);
> +               call.fn = cmd->fn;
> +               call.line = xstrdup_or_null(p);

... which means that xstrdup_or_null() will be copying whatever random garbage is in memory following the bare command.

Show 9 quoted lines
> +               queued_cmd[nr++] = call;
> +       }
> +
> +       if (opt->buffer_output && nr)
> +               dispatch_calls(opt, output, data, queued_cmd, nr);
> +
> +       free(queued_cmd);
> +       strbuf_release(&input);
> +}
Previous: John CaiNext: Junio C Hamano
Message 45 of 97 in “Add cat-file --batch-command flag”
  1. 0/2 Add cat-file --batch-command flagJohn Cai via GitGitGadget, Feb 3, 2022
  2. 1/2 cat-file.c: rename cmdmode to modeJohn Cai via GitGitGadget, Feb 3, 2022
  3. Junio C HamanoFeb 3, 2022
  4. Ævar Arnfjörð BjarmasonFeb 4, 2022
  5. 2/2 catfile.c: add --batch-command modeJohn Cai via GitGitGadget, Feb 3, 2022
  6. Junio C HamanoFeb 3, 2022
  7. John CaiFeb 4, 2022
  8. Phillip WoodFeb 4, 2022
  9. Eric SunshineFeb 4, 2022
  10. John CaiFeb 4, 2022
  11. Eric SunshineFeb 5, 2022
  12. Ævar Arnfjörð BjarmasonFeb 4, 2022
  13. Phillip WoodFeb 4, 2022
  14. 0/2 Add cat-file --batch-command flagJohn Cai via GitGitGadget, Feb 7, 2022
  15. 1/2 cat-file: rename cmdmode to transform_modeJohn Cai via GitGitGadget, Feb 7, 2022
  16. Junio C HamanoFeb 7, 2022
  17. 2/2 cat-file: add --batch-command modeJohn Cai via GitGitGadget, Feb 7, 2022
  18. Jonathan TanFeb 7, 2022
  19. Phillip WoodFeb 8, 2022
  20. Jonathan TanFeb 8, 2022
  21. Junio C HamanoFeb 8, 2022
  22. Jonathan TanFeb 9, 2022
  23. Junio C HamanoFeb 8, 2022
  24. Phillip WoodFeb 8, 2022
  25. 0/3 Add cat-file --batch-command flagJohn Cai via GitGitGadget, Feb 8, 2022
  26. 1/3 cat-file: rename cmdmode to transform_modeJohn Cai via GitGitGadget, Feb 8, 2022
  27. 3/3 cat-file: add --batch-command modeJohn Cai via GitGitGadget, Feb 8, 2022
  28. Junio C HamanoFeb 8, 2022
  29. 2/3 cat-file: introduce batch_command enum to replace print_contentsJohn Cai via GitGitGadget, Feb 8, 2022
  30. Junio C HamanoFeb 8, 2022
  31. Junio C HamanoFeb 9, 2022
  32. John CaiFeb 9, 2022
  33. John CaiFeb 9, 2022
  34. 0/3 Add cat-file --batch-command flagJohn Cai via GitGitGadget, Feb 10, 2022
  35. 1/3 cat-file: rename cmdmode to transform_modeJohn Cai via GitGitGadget, Feb 10, 2022
  36. 2/3 cat-file: introduce batch_mode enum to replace print_contentsJohn Cai via GitGitGadget, Feb 10, 2022
  37. Christian CouderFeb 10, 2022
  38. 3/3 cat-file: add --batch-command modeJohn Cai via GitGitGadget, Feb 10, 2022
  39. Phillip WoodFeb 10, 2022
  40. Junio C HamanoFeb 10, 2022
  41. John CaiFeb 11, 2022
  42. Junio C HamanoFeb 11, 2022
  43. John CaiFeb 11, 2022
  44. John CaiFeb 10, 2022
  45. Eric SunshineFeb 10, 2022
  46. Junio C HamanoFeb 10, 2022
  47. 0/3 Add cat-file --batch-command flagJohn Cai via GitGitGadget, Feb 11, 2022
  48. 1/3 cat-file: rename cmdmode to transform_modeJohn Cai via GitGitGadget, Feb 11, 2022
  49. 2/3 cat-file: introduce batch_mode enum to replace print_contentsJohn Cai via GitGitGadget, Feb 11, 2022
  50. 3/3 cat-file: add --batch-command modeJohn Cai via GitGitGadget, Feb 11, 2022
  51. Phillip WoodFeb 14, 2022
  52. John CaiFeb 14, 2022
  53. 0/4 Add cat-file --batch-command flagJohn Cai via GitGitGadget, Feb 14, 2022
  54. 1/4 cat-file: rename cmdmode to transform_modeJohn Cai via GitGitGadget, Feb 14, 2022
  55. 2/4 cat-file: introduce batch_mode enum to replace print_contentsJohn Cai via GitGitGadget, Feb 14, 2022
  56. 3/4 cat-file: add remove_timestamp helperJohn Cai via GitGitGadget, Feb 14, 2022
  57. 4/4 cat-file: add --batch-command modeJohn Cai via GitGitGadget, Feb 14, 2022
  58. Eric SunshineFeb 15, 2022
  59. John CaiFeb 15, 2022
  60. Eric SunshineFeb 15, 2022
  61. 0/4 Add cat-file --batch-command flagJohn Cai via GitGitGadget, Feb 16, 2022
  62. 1/4 cat-file: rename cmdmode to transform_modeJohn Cai via GitGitGadget, Feb 16, 2022
  63. 2/4 cat-file: introduce batch_mode enum to replace print_contentsJohn Cai via GitGitGadget, Feb 16, 2022
  64. 3/4 cat-file: add remove_timestamp helperJohn Cai via GitGitGadget, Feb 16, 2022
  65. 4/4 cat-file: add --batch-command modeJohn Cai via GitGitGadget, Feb 16, 2022
  66. Junio C HamanoFeb 16, 2022
  67. John CaiFeb 16, 2022
  68. Junio C HamanoFeb 16, 2022
  69. Eric SunshineFeb 16, 2022
  70. Eric SunshineFeb 16, 2022
  71. 0/4 Add cat-file --batch-command flagJohn Cai via GitGitGadget, Feb 16, 2022
  72. 2/4 cat-file: introduce batch_mode enum to replace print_contentsJohn Cai via GitGitGadget, Feb 16, 2022
  73. 1/4 cat-file: rename cmdmode to transform_modeJohn Cai via GitGitGadget, Feb 16, 2022
  74. 3/4 cat-file: add remove_timestamp helperJohn Cai via GitGitGadget, Feb 16, 2022
  75. 4/4 cat-file: add --batch-command modeJohn Cai via GitGitGadget, Feb 16, 2022
  76. Junio C HamanoFeb 16, 2022
  77. Eric SunshineFeb 16, 2022
  78. John CaiFeb 16, 2022
  79. 0/4 Add cat-file --batch-command flagJohn Cai via GitGitGadget, Feb 16, 2022
  80. 1/4 cat-file: rename cmdmode to transform_modeJohn Cai via GitGitGadget, Feb 16, 2022
  81. 2/4 cat-file: introduce batch_mode enum to replace print_contentsJohn Cai via GitGitGadget, Feb 16, 2022
  82. 3/4 cat-file: add remove_timestamp helperJohn Cai via GitGitGadget, Feb 16, 2022
  83. 4/4 cat-file: add --batch-command modeJohn Cai via GitGitGadget, Feb 16, 2022
  84. Phillip WoodFeb 18, 2022
  85. John CaiFeb 18, 2022
  86. Junio C HamanoFeb 18, 2022
  87. Junio C HamanoFeb 18, 2022
  88. 0/4 Add cat-file --batch-command flagJohn Cai via GitGitGadget, Feb 18, 2022
  89. 1/4 cat-file: rename cmdmode to transform_modeJohn Cai via GitGitGadget, Feb 18, 2022
  90. 2/4 cat-file: introduce batch_mode enum to replace print_contentsJohn Cai via GitGitGadget, Feb 18, 2022
  91. 3/4 cat-file: add remove_timestamp helperJohn Cai via GitGitGadget, Feb 18, 2022
  92. Ævar Arnfjörð BjarmasonFeb 19, 2022
  93. John CaiFeb 22, 2022
  94. 4/4 cat-file: add --batch-command modeJohn Cai via GitGitGadget, Feb 18, 2022
  95. Ævar Arnfjörð BjarmasonFeb 19, 2022
  96. Junio C HamanoFeb 18, 2022
  97. Phillip WoodFeb 22, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.