git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v3] [GSOC]trailer: pass arg as positional parameter

From
ZheNing Hu <adlternative@gmail.com>
Date
Mar 26, 2021, 13:29 UTC
Message-ID
<CAOLTT8TMFjvH1TzZ9qRtW1v7aDYypdajJxRKpmZ0TaG_xx5H0w@mail.gmail.com>
In-Reply-To
<xmqqwntuuav7.fsf@gitster.g>
Junio C Hamano <gitster@pobox.com> 于2021年3月26日周五 上午6:28写道:
Show 18 quoted lines
>
> "ZheNing Hu via GitGitGadget" <gitgitgadget@gmail.com> writes:
>
> > From: ZheNing Hu <adlternative@gmail.com>
> >
> > Original implementation of `trailer.<token>.command` use
>
> uses
>
> > `strbuf_replace` to replace $ARG in command with the <value>
> > of the trailer, but it have a problem: `strbuf_replace`
>
> has
>
> > replace the $ARG only once, If the user's trailer command
>
> replaces the $ARG only once.
>
Okay... singular and plural problems.
Show 9 quoted lines
> > have used more than one $ARG, the remaining replacement will
> > fail.
>
> "will fail" is quite vague.  It is just left unreplaced, and if the
> user expects all of them to be replaced, then the expectation and
> reality would not match, but all of that you have already said by
> "replaces the $ARG only once.", so I think this sentence should be
> removed.
>
Indeed so.
Show 19 quoted lines
> > If directly modify the implementation of the original
> > `trailer.<token>.command`, The user’s previous `'$ARG'` in
> > trailer command will not be replaced.
>
> That statement does not make much sense.  Depending on the way how
> the implementation is "directly" modified, you can fix the "replaces
> only once" problem without introducing such a problem.  Just look
> for '$ARG' in the string and replace all of them, not just the first
> one.  It's not too difficult.
>
> This confusion primarily comes from the fact that you forgot to
> explain the other problem you are fixing, I think.  Even though the
> trailer.<token>.command documentation implies that the user is
> expected to give a shell script or some sort as the command and the
> use of $ARG makes it look like a shell variable, the original
> implementation does not treat it as a shell variable at all.  And
> the textual replacement is done without making sure the value being
> replaced has characters with special meaning in the shell language.
>

Yes! The accidental injection problem caused should have been the focus of my explanation.

Show 12 quoted lines
> So existing .command may incorrectly use $ARG inside a single-quote
> pair and expect it to be replaced to a string inside a single-quote
> pair.  A malformed, or worse, malicious, value would escape out of
> the single-quote pair (remember, the '; rm -fr .' example?) and
> execute arbitrary code in an unexpected way.  The (ungrammatical)
> "if directly modify the implementation" refers to a potential way to
> fix these two problems at the same time by doing the $ARG thing
> without using textual replacement, namely, exporting the value as an
> environment variable ARG.  If that approach was taken, then, $ARG
> enclosed in a single-quote pair will no longer be replaced, which
> makes it a backward incompatible change.
>
Oh, I remeber it: terrible shell injection. Specifically, it looks like this:

$ git config trailer.sign.command "git log --author='\$ARG'" $ git interpret-trailers --trailer "sign = adl' && rm -rf ./repo/'"

now I know that should be "backward incompatible change" as you said.
Show 49 quoted lines
> But without describing what solution you are talking about, your
> three-line description does not make much sense.
>
> > So now add new config
> > "trailer.<token>.cmd", pass trailer's value as positional
> > parameter 1 to the user's command, the user can use $1 as
> > trailer's value, to implement original variable replacement.
> >
> > If the user has these two configuration: "trailer.<token>.cmd"
> > and "trailer.<token>.command", "cmd" will execute and "command"
> > will not executed.
> >
> > Original `trailer.<token>.command` can still be used until git
> > completely abandoned it.
> >
> > Signed-off-by: ZheNing Hu <adlternative@gmail.com>
>
> Let's rewrite it completely.
>
>         The `trailer.<token>.command` configuration variable
>         specifies a command (run via the shall, so it does not have
>         to be a single name of or path to the command, but can be a
>         shell script), and the first occurrence of substring $ARG is
>         replaced with the value given to the `interpret-trailer`
>         command for the token.  This has two downsides:
>
>         * The use of $ARG in the mechanism misleads the users that
>           the value is passed in the shell variable, and tempt them
>           to use $ARG more than once, but that would not work, as
>           the second and subsequent $ARG are not replaced.
>
>         * Because $ARG is textually replaced without regard to the
>           shell language syntax, even '$ARG' (inside a single-quote
>           pair), which a user would expect to stay intact, would be
>           replaced, and worse, if the value had an unmatching single
>           quote (imagine a name like "O'Connor", substituted into
>           NAME='$ARG' to make it NAME='O'Connor), it would result in
>           a broken command that is not syntactically correct (or
>           worse).
>
>         Introduce a new `trailer.<token>.cmd` configuration that
>         takes higher precedence to deprecate and eventually remove
>         `trailer.<token>.command`, which passes the value as a
>         parameter to the command.  Instead of "$ARG", the users will
>         refer to the value as positional argument, $1, in their
>         scripts.
>
> I tried to cover everything we need to tell the reviewers about this
> change with the above.  Did I miss anything?

Nothing to blame, feature of the old command, 2 problems, 1 solution, this is what the log should look like.

Show 11 quoted lines
>
> > +trailer.<token>.cmd::
> > +     Similar to 'trailer.<token>.command'. But the difference is that
> > +     `$1` is used in the command to replace the value of the trailer
> > +     instead of the original `$ARG`, which means that we can pass the
> > +     trailer value multiple times in the command.
>
> We eventually want to deprecate the .command, so we'd prefer not to
> rely on its description too much (e.g. try to find a way to say what
> you want to say without "instead of the original `$ARG`").
>

Oh! here I can’t rely on the documentation of the old `.command`, otherwise it’s not easy to delete the old documentation.

Show 23 quoted lines
>         The command specified by this configuration variable is run
>         with a single parameter, which is the <value> part of an
>         existing trailer with the same <token>.  The output from the
>         command is then used as the value for the <token> in the
>         resulting trailer.
>
> would be the replacement for the part that talks about $ARG in the
> description of trailer.<token>.command.
>
> The original description for `trailer.<token>.command` is so jumbled
> (not your failure at all) that I had a hard time to understand what
> it is trying to say (e.g. what does "as if a special <token>=<value>
> argument were added at the beginning of the command line" mean?  Is
> it making a one-shot export of environment variable to run the
> command???), so the above may need further adjustment.  Christian?
> Care to help out?
>
> > +     E.g. `git config trailer.sign.cmd "test -n \"$1\" && echo \"$1\" || true "`.
>
> An example is good.  There is a whole EXAMPLES section in this
> manual page, and the one that uses $ARG may be a good candidate to
> look at and change to use .cmd (instead of .command).
>
Okay, I will modify the paragraphs containing `.command` in these examples.
Show 62 quoted lines
> > +     If the user has these two configuration: "trailer.<token>.cmd"
> > +     and "trailer.<token>.command", "cmd" will be executed and "command"
> > +     will not be executed.
>
>         When both .cmd and .command are given for the same <token>,
>         .cmd is used and .command is ignored.
>
> > diff --git a/trailer.c b/trailer.c
> > index be4e9726421c..634d3f1ff04a 100644
> > --- a/trailer.c
> > +++ b/trailer.c
> > @@ -14,6 +14,7 @@ struct conf_info {
> >       char *name;
> >       char *key;
> >       char *command;
> > +     char *cmd;
> >       enum trailer_where where;
> >       enum trailer_if_exists if_exists;
> >       enum trailer_if_missing if_missing;
> > @@ -127,6 +128,7 @@ static void free_arg_item(struct arg_item *item)
> >       free(item->conf.name);
> >       free(item->conf.key);
> >       free(item->conf.command);
> > +     free(item->conf.cmd);
> >       free(item->token);
> >       free(item->value);
> >       free(item);
> > @@ -216,18 +218,24 @@ static int check_if_different(struct trailer_item *in_tok,
> >       return 1;
> >  }
> >
> > -static char *apply_command(const char *command, const char *arg)
> > +static char *apply_command(const char *command, const char *cmd_, const char *arg)
> >  {
> >       struct strbuf cmd = STRBUF_INIT;
> >       struct strbuf buf = STRBUF_INIT;
> >       struct child_process cp = CHILD_PROCESS_INIT;
> >       char *result;
> >
> > -     strbuf_addstr(&cmd, command);
> > -     if (arg)
> > -             strbuf_replace(&cmd, TRAILER_ARG_STRING, arg);
> > -
> > -     strvec_push(&cp.args, cmd.buf);
> > +     if (cmd_) {
> > +             strbuf_addstr(&cmd, cmd_);
> > +             strvec_push(&cp.args, cmd.buf);
> > +             if (arg)
> > +                     strvec_push(&cp.args, arg);
> > +     } else if (command) {
> > +             strbuf_addstr(&cmd, command);
> > +             strvec_push(&cp.args, cmd.buf);
> > +             if (arg)
> > +                     strbuf_replace(&cmd, TRAILER_ARG_STRING, arg);
> > +     }
>
> OK.  it is clear cmd_ takes precedence this way.
>
> Later (not as part of this patch, but a few releases down the road),
> we may want to add a warning() about using a deprecated feature when
> "else if (command)" block is taken.
>
Fine, I will keep this version of "cmd priority execution" for now.
Show 23 quoted lines
> > @@ -247,7 +255,7 @@ static char *apply_command(const char *command, const char *arg)
> >
> >  static void apply_item_command(struct trailer_item *in_tok, struct arg_item *arg_tok)
> >  {
> > -     if (arg_tok->conf.command) {
> > +     if (arg_tok->conf.command || arg_tok->conf.cmd) {
> >               const char *arg;
> >               if (arg_tok->value && arg_tok->value[0]) {
> >                       arg = arg_tok->value;
> > @@ -257,7 +265,7 @@ static void apply_item_command(struct trailer_item *in_tok, struct arg_item *arg
> >                       else
> >                               arg = xstrdup("");
> >               }
> > -             arg_tok->value = apply_command(arg_tok->conf.command, arg);
> > +             arg_tok->value = apply_command(arg_tok->conf.command, arg_tok->conf.cmd, arg);
>
> It might be cleaner to just pass arg_tok->conf to apply_command()
> and hide "cmd takes precedence over command" as an implementation
> detail of that helper function.
>
> The implementation looks as good as the original "command" with that
> change at this point.  Documentation may need a bit more polishing.
>
you're right.
> Thanks.

Thanks, Junio. You and the people in the git community are very enthusiastic, You have patiently explained these small mistakes that I made, and taught me a lot of problems that I didn't notice.

Grateful.

-- ZheNing Hu

Previous: Junio C HamanoNext: ZheNing Hu via GitGitGadget
Message 7 of 101 in “[GSOC]trailer: change $ARG to environment variable”
  1. [GSOC]trailer: change $ARG to environment variableZheNing Hu via GitGitGadget, Mar 23, 2021
  2. [GSOC]trailer: pass arg as positional parameterZheNing Hu via GitGitGadget, Mar 24, 2021
  3. Junio C HamanoMar 24, 2021
  4. ZheNing HuMar 25, 2021
  5. [GSOC]trailer: pass arg as positional parameterZheNing Hu via GitGitGadget, Mar 25, 2021
  6. Junio C HamanoMar 25, 2021
  7. ZheNing HuMar 26, 2021
  8. [GSOC]trailer: pass arg as positional parameterZheNing Hu via GitGitGadget, Mar 26, 2021
  9. Junio C HamanoMar 27, 2021
  10. Christian CouderMar 27, 2021
  11. ZheNing HuMar 28, 2021
  12. Christian CouderMar 29, 2021
  13. ZheNing HuMar 29, 2021
  14. Christian CouderMar 30, 2021
  15. ZheNing HuMar 30, 2021
  16. ZheNing HuMar 30, 2021
  17. Junio C HamanoMar 30, 2021
  18. ZheNing HuMar 31, 2021
  19. Junio C HamanoMar 31, 2021
  20. Junio C HamanoMar 31, 2021
  21. ZheNing HuApr 1, 2021
  22. Junio C HamanoApr 1, 2021
  23. ZheNing HuApr 2, 2021
  24. ZheNing HuApr 1, 2021
  25. 0/2 [GSOC]trailer: pass arg as positional parameterZheNing Hu via GitGitGadget, Mar 31, 2021
  26. 1/2 [GSOC] run-command: add shell_no_implicit_args optionZheNing Hu via GitGitGadget, Mar 31, 2021
  27. Christian CouderApr 1, 2021
  28. ZheNing HuApr 1, 2021
  29. 2/2 [GSOC]trailer: pass arg as positional parameterZheNing Hu via GitGitGadget, Mar 31, 2021
  30. Christian CouderApr 1, 2021
  31. ZheNing HuApr 1, 2021
  32. [GSOC] trailer: add new trailer.<token>.cmd config optionZheNing Hu via GitGitGadget, Apr 2, 2021
  33. Junio C HamanoApr 2, 2021
  34. ZheNing HuApr 3, 2021
  35. Junio C HamanoApr 4, 2021
  36. Christian CouderApr 3, 2021
  37. Junio C HamanoApr 4, 2021
  38. Christian CouderApr 6, 2021
  39. Christian CouderApr 6, 2021
  40. ZheNing HuApr 6, 2021
  41. Junio C HamanoApr 6, 2021
  42. Junio C HamanoApr 6, 2021
  43. ZheNing HuApr 4, 2021
  44. Christian CouderApr 4, 2021
  45. ZheNing HuApr 4, 2021
  46. Junio C HamanoApr 2, 2021
  47. ZheNing HuApr 3, 2021
  48. Junio C HamanoApr 3, 2021
  49. ZheNing HuApr 3, 2021
  50. [GSOC] trailer: add new trailer.<token>.cmd config optionZheNing Hu via GitGitGadget, Apr 4, 2021
  51. Christian CouderApr 6, 2021
  52. ZheNing HuApr 7, 2021
  53. 0/2 [GSOC] trailer: add new .cmd config optionZheNing Hu via GitGitGadget, Apr 9, 2021
  54. 1/2 [GSOC] docs: correct descript of trailer.<token>.commandZheNing Hu via GitGitGadget, Apr 9, 2021
  55. Christian CouderApr 9, 2021
  56. ZheNing HuApr 10, 2021
  57. 2/2 [GSOC] trailer: add new .cmd config optionZheNing Hu via GitGitGadget, Apr 9, 2021
  58. Christian CouderApr 9, 2021
  59. ZheNing HuApr 10, 2021
  60. Christian CouderApr 9, 2021
  61. 0/2 [GSOC] trailer: add new .cmd config optionZheNing Hu via GitGitGadget, Apr 12, 2021
  62. 1/2 [GSOC] docs: correct descript of trailer.<token>.commandZheNing Hu via GitGitGadget, Apr 12, 2021
  63. Junio C HamanoApr 12, 2021
  64. Christian CouderApr 16, 2021
  65. Junio C HamanoApr 17, 2021
  66. 2/2 [GSOC] trailer: add new .cmd config optionZheNing Hu via GitGitGadget, Apr 12, 2021
  67. Junio C HamanoApr 12, 2021
  68. Christian CouderApr 13, 2021
  69. ZheNing HuApr 13, 2021
  70. Junio C HamanoApr 13, 2021
  71. ZheNing HuApr 14, 2021
  72. Junio C HamanoApr 14, 2021
  73. ZheNing HuApr 15, 2021
  74. Junio C HamanoApr 15, 2021
  75. Christian CouderApr 16, 2021
  76. Junio C HamanoApr 13, 2021
  77. 0/2 [GSOC] trailer: add new .cmd config optionZheNing Hu via GitGitGadget, Apr 16, 2021
  78. 2/2 [GSOC] trailer: add new .cmd config optionZheNing Hu via GitGitGadget, Apr 16, 2021
  79. Junio C HamanoApr 16, 2021
  80. Junio C HamanoApr 16, 2021
  81. Junio C HamanoApr 16, 2021
  82. Junio C HamanoApr 17, 2021
  83. Junio C HamanoApr 17, 2021
  84. ZheNing HuApr 17, 2021
  85. Junio C HamanoApr 17, 2021
  86. 1/2 [GSOC] docs: correct descript of trailer.<token>.commandZheNing Hu via GitGitGadget, Apr 16, 2021
  87. Junio C HamanoApr 16, 2021
  88. 0/2 [GSOC] trailer: add new .cmd config optionZheNing Hu via GitGitGadget, Apr 17, 2021
  89. 1/2 [GSOC] docs: correct description of .commandZheNing Hu via GitGitGadget, Apr 17, 2021
  90. 2/2 [GSOC] trailer: add new .cmd config optionZheNing Hu via GitGitGadget, Apr 17, 2021
  91. Junio C HamanoApr 17, 2021
  92. ZheNing HuApr 18, 2021
  93. Junio C HamanoApr 21, 2021
  94. ZheNing HuApr 21, 2021
  95. Junio C HamanoApr 21, 2021
  96. ZheNing HuApr 22, 2021
  97. Junio C HamanoApr 27, 2021
  98. ZheNing HuApr 27, 2021
  99. 0/2 [GSOC] trailer: add new .cmd config optionZheNing Hu via GitGitGadget, May 3, 2021
  100. 2/2 [GSOC] trailer: add new .cmd config optionZheNing Hu via GitGitGadget, May 3, 2021
  101. 1/2 [GSOC] docs: correct descript of trailer.<token>.commandZheNing Hu via GitGitGadget, May 3, 2021

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.