git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2 2/6] run-command: prepare command before forking

From
BWBrandon Williams <bmwill@google.com>
Date
Apr 13, 2017, 22:41 UTC
Message-ID
<20170413224129.GB115420@google.com>
In-Reply-To
<20170413211447.GC10084@aiede.mtv.corp.google.com>
On 04/13, Jonathan Nieder wrote:
Show 37 quoted lines
> Hi,
> 
> Brandon Williams wrote:
> 
> > In order to avoid allocation between 'fork()' and 'exec()' the argv
> > array used in the exec call is prepared prior to forking the process.
> 
> nit: s/(the argv array.*) is prepared/prepare \1/
> 
> Git's commit messages are in the imperative mood, as if they are
> ordering the code or the computer to do something.
> 
> More importantly, the commit message is a good place to explain some
> of the motivation behind the patch so that people can understand what
> the patch is for by reading it without having to dig into mailing list
> archives and get the discussion there.
> 
> E.g. this could say
> 
> - that grep tests are intermittently failing in configurations using
>   some versions of tcmalloc
> 
> - that the cause is interaction between fork and threads: malloc holds
>   a lock that those versions of tcmalloc doesn't release in a
>   pthread_atfork handler
> 
> - that according to [1] we need to only call async-signal-safe
>   operations between fork and exec.  Using malloc to build the argv
>   array isn't async-signal-safe
> 
> [1] http://pubs.opengroup.org/onlinepubs/009695399/functions/fork.html
> 
> > In addition to this, the function used to exec is changed from
> > 'execvp()' to 'execv()' as the (p) variant of exec has the potential to
> > call malloc during the path resolution it performs.
> 
> *puzzled* is execvp actually allowed to call malloc?
It could possible as it isn't async-signal-safe.
> 
> Could this part go in a separate patch?  That would make it easier to
> review.
I'll break this conversion out to a different patch.
Show 60 quoted lines
> 
> [...]
> > +++ b/run-command.c
> [...]
> > +	/*
> > +	 * If there are no '/' characters in the command then perform a path
> > +	 * lookup and use the resolved path as the command to exec.  If there
> > +	 * are no '/' characters or if the command wasn't found in the path,
> > +	 * have exec attempt to invoke the command directly.
> > +	 */
> > +	if (!strchr(out->argv[0], '/')) {
> > +		char *program = locate_in_PATH(out->argv[0]);
> > +		if (program) {
> > +			free((char *)out->argv[0]);
> > +			out->argv[0] = program;
> > +		}
> > +	}
> 
> This does one half of what execvp does but leaves out the other half.
> http://pubs.opengroup.org/onlinepubs/009695399/functions/exec.html
> explains:
> 
>   There are two distinct ways in which the contents of the process
>   image file may cause the execution to fail, distinguished by the
>   setting of errno to either [ENOEXEC] or [EINVAL] (see the ERRORS
>   section). In the cases where the other members of the exec family of
>   functions would fail and set errno to [ENOEXEC], the execlp() and
>   execvp() functions shall execute a command interpreter and the
>   environment of the executed command shall be as if the process
>   invoked the sh utility using execl() as follows:
> 
>   execl(<shell path>, arg0, file, arg1, ..., (char *)0);
> 
> I think this is what the first patch in the series was about.  Do we
> want to drop that support?
> 
> I think we need to keep it, since it is easy for authors of e.g.
> credential helpers to accidentally rely on it.
> 
> [...]
> > @@ -437,12 +458,9 @@ int start_command(struct child_process *cmd)
> >  					unsetenv(*cmd->env);
> >  			}
> >  		}
> > -		if (cmd->git_cmd)
> > -			execv_git_cmd(cmd->argv);
> > -		else if (cmd->use_shell)
> > -			execv_shell_cmd(cmd->argv);
> > -		else
> > -			sane_execvp(cmd->argv[0], (char *const*) cmd->argv);
> > +
> > +		execv(argv.argv[0], (char *const *) argv.argv);
> 
> What happens in the case sane_execvp was trying to handle?  Does
> prepare_cmd need error handling for when the command isn't found?
> 
> Sorry this got so fussy.
> 
> Thanks and hope that helps,
> Jonathan
-- 
Brandon Williams
Previous: Jonathan NiederNext: Brandon Williams
Message 34 of 140 in “forking and threading”
  1. 0/5 forking and threadingBrandon Williams, Apr 10, 2017
  2. 1/5 run-command: convert sane_execvp to sane_execvpeBrandon Williams, Apr 10, 2017
  3. Brandon WilliamsApr 12, 2017
  4. 2/5 run-command: prepare argv before forkingBrandon Williams, Apr 10, 2017
  5. 5/5 run-command: add note about forking and threadingBrandon Williams, Apr 10, 2017
  6. Jonathan NiederApr 11, 2017
  7. Eric WongApr 11, 2017
  8. Jonathan NiederApr 11, 2017
  9. Brandon WilliamsApr 11, 2017
  10. Eric WongApr 11, 2017
  11. 4/5 run-command: prepare child environment before forkingBrandon Williams, Apr 10, 2017
  12. Jonathan NiederApr 11, 2017
  13. Brandon WilliamsApr 11, 2017
  14. Jonathan NiederApr 11, 2017
  15. 3/5 run-command: allocate child_err before forkingBrandon Williams, Apr 10, 2017
  16. 6/5 run-command: avoid potential dangers in forked childEric Wong, Apr 11, 2017
  17. Brandon WilliamsApr 11, 2017
  18. Eric WongApr 11, 2017
  19. Brandon WilliamsApr 11, 2017
  20. Jonathan NiederApr 11, 2017
  21. Brandon WilliamsApr 11, 2017
  22. 0/6 forking and threadingBrandon Williams, Apr 13, 2017
  23. 4/6 run-command: don't die in child when duping /dev/nullBrandon Williams, Apr 13, 2017
  24. Eric WongApr 13, 2017
  25. Brandon WilliamsApr 13, 2017
  26. 1/6 t5550: use write_script to generate post-update hookBrandon Williams, Apr 13, 2017
  27. Jonathan NiederApr 13, 2017
  28. Eric WongApr 13, 2017
  29. Brandon WilliamsApr 13, 2017
  30. Eric WongApr 13, 2017
  31. 5/6 run-command: eliminate calls to error handling functions in childBrandon Williams, Apr 13, 2017
  32. 2/6 run-command: prepare command before forkingBrandon Williams, Apr 13, 2017
  33. Jonathan NiederApr 13, 2017
  34. Brandon WilliamsApr 13, 2017
  35. 6/6 run-command: add note about forking and threadingBrandon Williams, Apr 13, 2017
  36. 3/6 run-command: prepare child environment before forkingBrandon Williams, Apr 13, 2017
  37. Jonathan NiederApr 13, 2017
  38. Brandon WilliamsApr 13, 2017
  39. 7/6 run-command: block signals between fork and execveEric Wong, Apr 13, 2017
  40. Brandon WilliamsApr 13, 2017
  41. Brandon WilliamsApr 14, 2017
  42. Eric WongApr 14, 2017
  43. Eric WongApr 14, 2017
  44. 00/10 forking and threadingBrandon Williams, Apr 14, 2017
  45. 01/10 t5550: use write_script to generate post-update hookBrandon Williams, Apr 14, 2017
  46. 02/10 t0061: run_command executes scripts without a #! lineBrandon Williams, Apr 14, 2017
  47. 03/10 run-command: prepare command before forkingBrandon Williams, Apr 14, 2017
  48. 07/10 run-command: eliminate calls to error handling functions in childBrandon Williams, Apr 14, 2017
  49. Eric WongApr 14, 2017
  50. Brandon WilliamsApr 14, 2017
  51. 05/10 run-command: prepare child environment before forkingBrandon Williams, Apr 14, 2017
  52. 09/10 run-command: add note about forking and threadingBrandon Williams, Apr 14, 2017
  53. 08/10 run-command: handle dup2 and close errors in childBrandon Williams, Apr 14, 2017
  54. 10/10 run-command: block signals between fork and execveBrandon Williams, Apr 14, 2017
  55. Brandon WilliamsApr 14, 2017
  56. Eric WongApr 14, 2017
  57. 06/10 run-command: don't die in child when duping /dev/nullBrandon Williams, Apr 14, 2017
  58. Eric WongApr 14, 2017
  59. Brandon WilliamsApr 14, 2017
  60. 04/10 run-command: use the async-signal-safe execv instead of execvpBrandon Williams, Apr 14, 2017
  61. 00/10 forking and threadingBrandon Williams, Apr 17, 2017
  62. 01/10 t5550: use write_script to generate post-update hookBrandon Williams, Apr 17, 2017
  63. 03/10 run-command: prepare command before forkingBrandon Williams, Apr 17, 2017
  64. 02/10 t0061: run_command executes scripts without a #! lineBrandon Williams, Apr 17, 2017
  65. 04/10 run-command: use the async-signal-safe execv instead of execvpBrandon Williams, Apr 17, 2017
  66. 07/10 run-command: eliminate calls to error handling functions in childBrandon Williams, Apr 17, 2017
  67. 06/10 run-command: don't die in child when duping /dev/nullBrandon Williams, Apr 17, 2017
  68. 10/10 run-command: block signals between fork and execveBrandon Williams, Apr 17, 2017
  69. 09/10 run-command: add note about forking and threadingBrandon Williams, Apr 17, 2017
  70. 05/10 run-command: prepare child environment before forkingBrandon Williams, Apr 17, 2017
  71. Eric WongApr 18, 2017
  72. Brandon WilliamsApr 18, 2017
  73. 08/10 run-command: handle dup2 and close errors in childBrandon Williams, Apr 17, 2017
  74. 00/11 forking and threadingBrandon Williams, Apr 18, 2017
  75. 01/11 t5550: use write_script to generate post-update hookBrandon Williams, Apr 18, 2017
  76. 03/11 run-command: prepare command before forkingBrandon Williams, Apr 18, 2017
  77. 02/11 t0061: run_command executes scripts without a #! lineBrandon Williams, Apr 18, 2017
  78. Johannes SixtApr 19, 2017
  79. Johannes SixtApr 19, 2017
  80. Brandon WilliamsApr 19, 2017
  81. Johannes SixtApr 19, 2017
  82. Johannes SchindelinApr 20, 2017
  83. Brandon WilliamsApr 20, 2017
  84. Johannes SchindelinApr 20, 2017
  85. Brandon WilliamsApr 20, 2017
  86. 05/11 string-list: add string_list_remove functionBrandon Williams, Apr 18, 2017
  87. Stefan BellerApr 18, 2017
  88. Brandon WilliamsApr 18, 2017
  89. Stefan BellerApr 18, 2017
  90. 07/11 run-command: don't die in child when duping /dev/nullBrandon Williams, Apr 18, 2017
  91. 04/11 run-command: use the async-signal-safe execv instead of execvpBrandon Williams, Apr 18, 2017
  92. 08/11 run-command: eliminate calls to error handling functions in childBrandon Williams, Apr 18, 2017
  93. 10/11 run-command: add note about forking and threadingBrandon Williams, Apr 18, 2017
  94. 09/11 run-command: handle dup2 and close errors in childBrandon Williams, Apr 18, 2017
  95. 11/11 run-command: block signals between fork and execveBrandon Williams, Apr 18, 2017
  96. Johannes SixtApr 19, 2017
  97. Eric WongApr 19, 2017
  98. Brandon WilliamsApr 19, 2017
  99. 06/11 run-command: prepare child environment before forkingBrandon Williams, Apr 18, 2017
  100. 00/11 forking and threadingBrandon Williams, Apr 19, 2017
  101. 01/11 t5550: use write_script to generate post-update hookBrandon Williams, Apr 19, 2017
  102. 02/11 t0061: run_command executes scripts without a #! lineBrandon Williams, Apr 19, 2017
  103. Johannes SchindelinApr 20, 2017
  104. Brandon WilliamsApr 20, 2017
  105. 04/11 run-command: use the async-signal-safe execv instead of execvpBrandon Williams, Apr 19, 2017
  106. Junio C HamanoMay 17, 2017
  107. Jeff KingMay 17, 2017
  108. Jeff KingMay 17, 2017
  109. Junio C HamanoMay 17, 2017
  110. Brandon WilliamsMay 17, 2017
  111. 05/11 string-list: add string_list_remove functionBrandon Williams, Apr 19, 2017
  112. 08/11 run-command: eliminate calls to error handling functions in childBrandon Williams, Apr 19, 2017
  113. 06/11 run-command: prepare child environment before forkingBrandon Williams, Apr 19, 2017
  114. 03/11 run-command: prepare command before forkingBrandon Williams, Apr 19, 2017
  115. 07/11 run-command: don't die in child when duping /dev/nullBrandon Williams, Apr 19, 2017
  116. 09/11 run-command: handle dup2 and close errors in childBrandon Williams, Apr 19, 2017
  117. 10/11 run-command: add note about forking and threadingBrandon Williams, Apr 19, 2017
  118. 11/11 run-command: block signals between fork and execveBrandon Williams, Apr 19, 2017
  119. Brandon WilliamsApr 24, 2017
  120. 12/11 run-command: don't try to execute directoriesBrandon Williams, Apr 24, 2017
  121. Jonathan NiederApr 25, 2017
  122. Junio C HamanoApr 25, 2017
  123. Jonathan NiederApr 25, 2017
  124. Jeff KingApr 25, 2017
  125. Junio C HamanoApr 25, 2017
  126. Jonathan NiederApr 25, 2017
  127. 1/2 exec_cmd: expose is_executable functionBrandon Williams, Apr 25, 2017
  128. 2/2 run-command: don't try to execute directoriesBrandon Williams, Apr 25, 2017
  129. Jonathan NiederApr 25, 2017
  130. Brandon WilliamsApr 25, 2017
  131. Jonathan NiederApr 25, 2017
  132. Johannes SixtApr 25, 2017
  133. Brandon WilliamsApr 25, 2017
  134. 1/2 run-command: expose is_executable functionBrandon Williams, Apr 25, 2017
  135. 2/2 run-command: restrict PATH search to executable filesBrandon Williams, Apr 25, 2017
  136. Jonathan NiederApr 25, 2017
  137. Junio C HamanoApr 26, 2017
  138. 2/2 run-command: restrict PATH search to executable filesBrandon Williams, Apr 26, 2017
  139. Junio C HamanoApr 27, 2017
  140. Jonathan NiederApr 25, 2017

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.