threads / patch / 47021

patchmerge-recursive: check GIT_MERGE_VERBOSITY only once

Subject: [PATCH] merge-recursive: check GIT_MERGE_VERBOSITY only once

## tl;dr

22 messages between Oct 24, 2017 and Oct 31, 2017. Diffs are folded; open one to read it.

replies: 21people: 6as markdown or json

Andrey Okoshkin· Oct 24, 2017, 15:27 UTC · lore

Add check of 'GIT_MERGE_VERBOSITY' environment variable only once in init_merge_options(). Consequential call of getenv() may return NULL pointer and strtol() crashes. However the stored pointer to the obtained getenv() result may be invalidated by some other getenv() call from another thread as getenv() is not thread-safe.

Signed-off-by: Andrey Okoshkin <a.okoshkin@samsung.com>
---
 merge-recursive.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
Show changes to merge-recursive.c +3 −3
diff --git a/merge-recursive.c b/merge-recursive.c
index 1494ffdb8..eaac98145 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -2163,6 +2163,7 @@ static void merge_recursive_config(struct merge_options *o)
 
 void init_merge_options(struct merge_options *o)
 {
+	const char *merge_verbosity = getenv("GIT_MERGE_VERBOSITY");
 	memset(o, 0, sizeof(struct merge_options));
 	o->verbosity = 2;
 	o->buffer_output = 1;
@@ -2171,9 +2172,8 @@ void init_merge_options(struct merge_options *o)
 	o->renormalize = 0;
 	o->detect_rename = 1;
 	merge_recursive_config(o);
-	if (getenv("GIT_MERGE_VERBOSITY"))
-		o->verbosity =
-			strtol(getenv("GIT_MERGE_VERBOSITY"), NULL, 10);
+	if (merge_verbosity)
+		o->verbosity = strtol(merge_verbosity, NULL, 10);
 	if (o->verbosity >= 5)
 		o->buffer_output = 0;
 	strbuf_init(&o->obuf, 0);
-- 
2.14.3
Stefan Beller· Oct 24, 2017, 16:28 UTC · re: Andrey Okoshkin · lore

Re: [PATCH] merge-recursive: check GIT_MERGE_VERBOSITY only once

On Tue, Oct 24, 2017 at 8:27 AM, Andrey Okoshkin <a.okoshkin@samsung.com> wrote:
Show 5 quoted lines
> Add check of 'GIT_MERGE_VERBOSITY' environment variable only once in
> init_merge_options().
> Consequential call of getenv() may return NULL pointer and strtol() crashes.
> However the stored pointer to the obtained getenv() result may be invalidated
> by some other getenv() call from another thread as getenv() is not thread-safe.
But do we have other threads running at the time?
Inspecting the four callsites:
* sequencer.c:
  The prior lines to hold the index lock suggests we're not in a threaded
  environment
* builtin/merge-recursive.c:
  In cmd_merge_recursive and we're fiddling with argv/argc, which
  suggests we in a main function, not having threads around
* builtin/am.c: fall_back_threeway called by am_run.
  (am is not threaded)
* builtin/merge:
  In try_merge_strategy called from the main function, also index locking
* builtin/checkout.c:
  merge_working_tree also locks the index.

So I think this function is never called from within a threaded environment in git.

Show 15 quoted lines
>
> Signed-off-by: Andrey Okoshkin <a.okoshkin@samsung.com>
> ---
>  merge-recursive.c | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)
>
> diff --git a/merge-recursive.c b/merge-recursive.c
> index 1494ffdb8..eaac98145 100644
> --- a/merge-recursive.c
> +++ b/merge-recursive.c
> @@ -2163,6 +2163,7 @@ static void merge_recursive_config(struct merge_options *o)
>
>  void init_merge_options(struct merge_options *o)
>  {
> +       const char *merge_verbosity = getenv("GIT_MERGE_VERBOSITY");

Despite not being in a threaded environment, I wonder if we want to minimize the time between calling getenv and the use of the result, i.e. declare merge_verbosity here, but assign it later, just before the condition?

(The compiler may shuffle stuff around anyway, so this is a moot suggestion; It gears mostly towards making the code more readable/maintainable when presenting this part of the code to the user.)

With or without this change:
Reviewed-by: Stefan Beller <sbeller@google.com>
Show 17 quoted lines
>         memset(o, 0, sizeof(struct merge_options));
>         o->verbosity = 2;
>         o->buffer_output = 1;
> @@ -2171,9 +2172,8 @@ void init_merge_options(struct merge_options *o)
>         o->renormalize = 0;
>         o->detect_rename = 1;
>         merge_recursive_config(o);
> -       if (getenv("GIT_MERGE_VERBOSITY"))
> -               o->verbosity =
> -                       strtol(getenv("GIT_MERGE_VERBOSITY"), NULL, 10);
> +       if (merge_verbosity)
> +               o->verbosity = strtol(merge_verbosity, NULL, 10);
>         if (o->verbosity >= 5)
>                 o->buffer_output = 0;
>         strbuf_init(&o->obuf, 0);
> --
> 2.14.3
Eric Sunshine· Oct 24, 2017, 16:45 UTC · re: Stefan Beller · lore

Re: [PATCH] merge-recursive: check GIT_MERGE_VERBOSITY only once

On Tue, Oct 24, 2017 at 12:28 PM, Stefan Beller <sbeller@google.com> wrote:
Show 8 quoted lines
> On Tue, Oct 24, 2017 at 8:27 AM, Andrey Okoshkin <a.okoshkin@samsung.com> wrote:
>> Add check of 'GIT_MERGE_VERBOSITY' environment variable only once in
>> init_merge_options().
>> Consequential call of getenv() may return NULL pointer and strtol() crashes.
>> However the stored pointer to the obtained getenv() result may be invalidated
>> by some other getenv() call from another thread as getenv() is not thread-safe.
>
> But do we have other threads running at the time?

I feel uncomfortable about this change, not due to lack of thread safety, but due to the distance between the getenv() invocation and its point of use. See below for more detail.

Show 15 quoted lines
>> Signed-off-by: Andrey Okoshkin <a.okoshkin@samsung.com>
>> ---
>> diff --git a/merge-recursive.c b/merge-recursive.c
>> @@ -2163,6 +2163,7 @@ static void merge_recursive_config(struct merge_options *o)
>>  void init_merge_options(struct merge_options *o)
>>  {
>> +       const char *merge_verbosity = getenv("GIT_MERGE_VERBOSITY");
>
> Despite not being in a threaded environment, I wonder if we want to
> minimize the time between  calling getenv and the use of the result,
> i.e. declare merge_verbosity here, but assign it later, just before the
> condition?
>
> With or without this change:
> Reviewed-by: Stefan Beller <sbeller@google.com>

The distance between getenv() and the point where the value is actually used is a big concern due to not knowing what is or might be going on in called functions between the two points. According to [1], the value returned by getenv() could be invalidated by another call to getenv() (or setenv() or unsetenv() or putenv()), and we don't have guarantee that we're safe from such invalidation considering that this function calls out to others. For instance, after getenv() but before the value is used, init_merge_options() calls merge_recursive_config() which calls git_config() which calls git_xmerge_config(), and so on.

For this reason, I have difficulty endorsing this change as-is.
[1]: http://pubs.opengroup.org/onlinepubs/9699919799/functions/getenv.html
Show 17 quoted lines
>>         memset(o, 0, sizeof(struct merge_options));
>>         o->verbosity = 2;
>>         o->buffer_output = 1;
>> @@ -2171,9 +2172,8 @@ void init_merge_options(struct merge_options *o)
>>         o->renormalize = 0;
>>         o->detect_rename = 1;
>>         merge_recursive_config(o);
>> -       if (getenv("GIT_MERGE_VERBOSITY"))
>> -               o->verbosity =
>> -                       strtol(getenv("GIT_MERGE_VERBOSITY"), NULL, 10);
>> +       if (merge_verbosity)
>> +               o->verbosity = strtol(merge_verbosity, NULL, 10);
>>         if (o->verbosity >= 5)
>>                 o->buffer_output = 0;
>>         strbuf_init(&o->obuf, 0);
>> --
>> 2.14.3
Martin Ågren· Oct 24, 2017, 17:11 UTC · re: Eric Sunshine · lore

Re: [PATCH] merge-recursive: check GIT_MERGE_VERBOSITY only once

On 24 October 2017 at 18:45, Eric Sunshine <sunshine@sunshineco.com> wrote:
Show 7 quoted lines
> On Tue, Oct 24, 2017 at 12:28 PM, Stefan Beller <sbeller@google.com> wrote:
>> On Tue, Oct 24, 2017 at 8:27 AM, Andrey Okoshkin <a.okoshkin@samsung.com> wrote:
>>> Add check of 'GIT_MERGE_VERBOSITY' environment variable only once in
>>> init_merge_options().
>>> Consequential call of getenv() may return NULL pointer and strtol() crashes.
>>> However the stored pointer to the obtained getenv() result may be invalidated
>>> by some other getenv() call from another thread as getenv() is not thread-safe.

I'm having trouble wrapping my head around this. Under which circumstances could the second call in the current code return NULL, but the code after your patch behave in a well-defined (and correct) way?

Show 11 quoted lines
> The distance between getenv() and the point where the value is
> actually used is a big concern due to not knowing what is or might be
> going on in called functions between the two points. According to [1],
> the value returned by getenv() could be invalidated by another call to
> getenv() (or setenv() or unsetenv() or putenv()), and we don't have
> guarantee that we're safe from such invalidation considering that this
> function calls out to others. For instance, after getenv() but before
> the value is used, init_merge_options() calls merge_recursive_config()
> which calls git_config() which calls git_xmerge_config(), and so on.
>
> For this reason, I have difficulty endorsing this change as-is.

Yeah. The call should be immediately before `merge_verbosity` is used. Then, if a compiler wants to move the call, it has to do the work and prove that it's ok.

Jeff King· Oct 24, 2017, 19:52 UTC · re: Martin Ågren · lore

Re: [PATCH] merge-recursive: check GIT_MERGE_VERBOSITY only once

On Tue, Oct 24, 2017 at 07:11:24PM +0200, Martin Ågren wrote:
Show 12 quoted lines
> On 24 October 2017 at 18:45, Eric Sunshine <sunshine@sunshineco.com> wrote:
> > On Tue, Oct 24, 2017 at 12:28 PM, Stefan Beller <sbeller@google.com> wrote:
> >> On Tue, Oct 24, 2017 at 8:27 AM, Andrey Okoshkin <a.okoshkin@samsung.com> wrote:
> >>> Add check of 'GIT_MERGE_VERBOSITY' environment variable only once in
> >>> init_merge_options().
> >>> Consequential call of getenv() may return NULL pointer and strtol() crashes.
> >>> However the stored pointer to the obtained getenv() result may be invalidated
> >>> by some other getenv() call from another thread as getenv() is not thread-safe.
> 
> I'm having trouble wrapping my head around this. Under which
> circumstances could the second call in the current code return NULL, but
> the code after your patch behave in a well-defined (and correct) way?

Yeah, it's not at all clear to me this is solving a real problem. I know Andrey mentioned playing around with fault injection in an earlier thread, so I'm wondering if there is an artificial fault being injected into the second getenv() call. Which does not seem like something that should be possible in the real world.

I definitely agree with the sentiment that as few things as possible should happen between calling getenv() and using its result. I've seen real bugs there from unexpected invalidation of the static buffer.

-Peff
Junio C Hamano· Oct 25, 2017, 01:48 UTC · re: Jeff King · lore

Re: [PATCH] merge-recursive: check GIT_MERGE_VERBOSITY only once

Jeff King <peff@peff.net> writes:
> I definitely agree with the sentiment that as few things as possible
> should happen between calling getenv() and using its result. I've seen
> real bugs there from unexpected invalidation of the static buffer.

Sure. I do not think we mind xstrdup()ing the result and freeing after we are done, though ;-)

Eric Sunshine· Oct 25, 2017, 04:07 UTC · re: Junio C Hamano · lore

Re: [PATCH] merge-recursive: check GIT_MERGE_VERBOSITY only once

On Tue, Oct 24, 2017 at 9:48 PM, Junio C Hamano <gitster@pobox.com> wrote:
Show 7 quoted lines
> Jeff King <peff@peff.net> writes:
>> I definitely agree with the sentiment that as few things as possible
>> should happen between calling getenv() and using its result. I've seen
>> real bugs there from unexpected invalidation of the static buffer.
>
> Sure.  I do not think we mind xstrdup()ing the result and freeing
> after we are done, though ;-)
More specifically, xstrdup_or_null(getenv(...)), if that route is chosen.
Jeff King· Oct 25, 2017, 07:27 UTC · re: Eric Sunshine · lore

Re: [PATCH] merge-recursive: check GIT_MERGE_VERBOSITY only once

On Wed, Oct 25, 2017 at 12:07:12AM -0400, Eric Sunshine wrote:
Show 10 quoted lines
> On Tue, Oct 24, 2017 at 9:48 PM, Junio C Hamano <gitster@pobox.com> wrote:
> > Jeff King <peff@peff.net> writes:
> >> I definitely agree with the sentiment that as few things as possible
> >> should happen between calling getenv() and using its result. I've seen
> >> real bugs there from unexpected invalidation of the static buffer.
> >
> > Sure.  I do not think we mind xstrdup()ing the result and freeing
> > after we are done, though ;-)
> 
> More specifically, xstrdup_or_null(getenv(...)), if that route is chosen.

That would be the way to do it, but I do not see thta we need to record the string at all. The current code is calling strtol on it on it immediately.

So the options are:
  1. Save the result of getenv() in a variable. If it is non-NULL, then
     immediately call strtol() on it.
  2. Do nothing. The double-call to getenv() is actually fine in the
     real world as it will return consistent results.

But the patch under discussion, which calls getenv() then expects it to be correct after a call to merge_recursive_config(), introduces a problem.

-Peff
Andrey Okoshkin· Oct 25, 2017, 11:39 UTC · re: Jeff King · lore

[PATCH v2] merge-recursive: check GIT_MERGE_VERBOSITY only once

Check 'GIT_MERGE_VERBOSITY' environment variable only once in init_merge_options(). Consequential call of getenv() may return NULL pointer. However the stored pointer to the obtained getenv() result may be invalidated by some other getenv() call as getenv() is not thread-safe.

Signed-off-by: Andrey Okoshkin <a.okoshkin@samsung.com>
Reviewed-by: Stefan Beller <sbeller@google.com>
---
Changes since the previous patch:
* no actions are taken between the merge_verbosity assignment and check.
 merge-recursive.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)
Show changes to merge-recursive.c +4 −3
diff --git a/merge-recursive.c b/merge-recursive.c
index 1494ffdb8..60084e3a0 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -2163,6 +2163,7 @@ static void merge_recursive_config(struct merge_options *o)
 
 void init_merge_options(struct merge_options *o)
 {
+	const char *merge_verbosity;
 	memset(o, 0, sizeof(struct merge_options));
 	o->verbosity = 2;
 	o->buffer_output = 1;
@@ -2171,9 +2172,9 @@ void init_merge_options(struct merge_options *o)
 	o->renormalize = 0;
 	o->detect_rename = 1;
 	merge_recursive_config(o);
-	if (getenv("GIT_MERGE_VERBOSITY"))
-		o->verbosity =
-			strtol(getenv("GIT_MERGE_VERBOSITY"), NULL, 10);
+	merge_verbosity = getenv("GIT_MERGE_VERBOSITY");
+	if (merge_verbosity)
+		o->verbosity = strtol(merge_verbosity, NULL, 10);
 	if (o->verbosity >= 5)
 		o->buffer_output = 0;
 	strbuf_init(&o->obuf, 0);
-- 
2.14.3
Eric Sunshine· Oct 25, 2017, 11:53 UTC · re: Andrey Okoshkin · lore

Re: [PATCH v2] merge-recursive: check GIT_MERGE_VERBOSITY only once

On Wed, Oct 25, 2017 at 7:39 AM, Andrey Okoshkin <a.okoshkin@samsung.com> wrote:
> Check 'GIT_MERGE_VERBOSITY' environment variable only once in
> init_merge_options().
> Consequential call of getenv() may return NULL pointer.

It would be particularly nice to have a more detailed explanation in the commit message of the potential problem this patch is trying to solve. Given the amount of discussion, thus far, surrounding such a simple patch, this cryptic warning about getenv() returning NULL upon second invocation is insufficient to explain why this patch is desirable; it merely leads to a lot of head-scratching.

> However the stored pointer to the obtained getenv() result may be invalidated
> by some other getenv() call as getenv() is not thread-safe.

This is even more cryptic, as it appears to be arguing for or against _something_ (it's not clear what) and it seems to be talking about a facet of the existing code, rather than explaining why the updated code consumes its 'merge_verbosity' value as early as possible after being assigned. Perhaps this part could be reworded something like this:

    Instead, call getenv() only once, storing its value and
    consulting it as many times as needed. This update takes care
    to consume the value returned by getenv() without any
    intervening calls to getenv(), setenv(), unsetenv(), or
    putenv(), any of which might invalidate the pointer returned
    by the initial call.
> Signed-off-by: Andrey Okoshkin <a.okoshkin@samsung.com>
> Reviewed-by: Stefan Beller <sbeller@google.com>

As this patch is semantically quite different from the original to which Stefan gave his Reviewed-by: (and which other people argued against), it might be better omit this footer and let him re-give it if he so desires.

Show 33 quoted lines
> ---
> Changes since the previous patch:
> * no actions are taken between the merge_verbosity assignment and check.
>  merge-recursive.c | 7 ++++---
>  1 file changed, 4 insertions(+), 3 deletions(-)
>
> diff --git a/merge-recursive.c b/merge-recursive.c
> index 1494ffdb8..60084e3a0 100644
> --- a/merge-recursive.c
> +++ b/merge-recursive.c
> @@ -2163,6 +2163,7 @@ static void merge_recursive_config(struct merge_options *o)
>
>  void init_merge_options(struct merge_options *o)
>  {
> +       const char *merge_verbosity;
>         memset(o, 0, sizeof(struct merge_options));
>         o->verbosity = 2;
>         o->buffer_output = 1;
> @@ -2171,9 +2172,9 @@ void init_merge_options(struct merge_options *o)
>         o->renormalize = 0;
>         o->detect_rename = 1;
>         merge_recursive_config(o);
> -       if (getenv("GIT_MERGE_VERBOSITY"))
> -               o->verbosity =
> -                       strtol(getenv("GIT_MERGE_VERBOSITY"), NULL, 10);
> +       merge_verbosity = getenv("GIT_MERGE_VERBOSITY");
> +       if (merge_verbosity)
> +               o->verbosity = strtol(merge_verbosity, NULL, 10);
>         if (o->verbosity >= 5)
>                 o->buffer_output = 0;
>         strbuf_init(&o->obuf, 0);
> --
> 2.14.3
Andrey Okoshkin· Oct 25, 2017, 12:27 UTC · re: Eric Sunshine · lore

Re: [PATCH v2] merge-recursive: check GIT_MERGE_VERBOSITY only once

Thanks, Eric, indeed it's better to change the commit message.
25.10.2017 14:53, Eric Sunshine wrote:
Show 73 quoted lines
> On Wed, Oct 25, 2017 at 7:39 AM, Andrey Okoshkin <a.okoshkin@samsung.com> wrote:
>> Check 'GIT_MERGE_VERBOSITY' environment variable only once in
>> init_merge_options().
>> Consequential call of getenv() may return NULL pointer.
> 
> It would be particularly nice to have a more detailed explanation in
> the commit message of the potential problem this patch is trying to
> solve. Given the amount of discussion, thus far, surrounding such a
> simple patch, this cryptic warning about getenv() returning NULL upon
> second invocation is insufficient to explain why this patch is
> desirable; it merely leads to a lot of head-scratching.
> 
>> However the stored pointer to the obtained getenv() result may be invalidated
>> by some other getenv() call as getenv() is not thread-safe.
> 
> This is even more cryptic, as it appears to be arguing for or against
> _something_ (it's not clear what) and it seems to be talking about a
> facet of the existing code, rather than explaining why the updated
> code consumes its 'merge_verbosity' value as early as possible after
> being assigned. Perhaps this part could be reworded something like
> this:
> 
>     Instead, call getenv() only once, storing its value and
>     consulting it as many times as needed. This update takes care
>     to consume the value returned by getenv() without any
>     intervening calls to getenv(), setenv(), unsetenv(), or
>     putenv(), any of which might invalidate the pointer returned
>     by the initial call.
> 
>> Signed-off-by: Andrey Okoshkin <a.okoshkin@samsung.com>
>> Reviewed-by: Stefan Beller <sbeller@google.com>
> 
> As this patch is semantically quite different from the original to
> which Stefan gave his Reviewed-by: (and which other people argued
> against), it might be better omit this footer and let him re-give it
> if he so desires.
> 
>> ---
>> Changes since the previous patch:
>> * no actions are taken between the merge_verbosity assignment and check.
>>  merge-recursive.c | 7 ++++---
>>  1 file changed, 4 insertions(+), 3 deletions(-)
>>
>> diff --git a/merge-recursive.c b/merge-recursive.c
>> index 1494ffdb8..60084e3a0 100644
>> --- a/merge-recursive.c
>> +++ b/merge-recursive.c
>> @@ -2163,6 +2163,7 @@ static void merge_recursive_config(struct merge_options *o)
>>
>>  void init_merge_options(struct merge_options *o)
>>  {
>> +       const char *merge_verbosity;
>>         memset(o, 0, sizeof(struct merge_options));
>>         o->verbosity = 2;
>>         o->buffer_output = 1;
>> @@ -2171,9 +2172,9 @@ void init_merge_options(struct merge_options *o)
>>         o->renormalize = 0;
>>         o->detect_rename = 1;
>>         merge_recursive_config(o);
>> -       if (getenv("GIT_MERGE_VERBOSITY"))
>> -               o->verbosity =
>> -                       strtol(getenv("GIT_MERGE_VERBOSITY"), NULL, 10);
>> +       merge_verbosity = getenv("GIT_MERGE_VERBOSITY");
>> +       if (merge_verbosity)
>> +               o->verbosity = strtol(merge_verbosity, NULL, 10);
>>         if (o->verbosity >= 5)
>>                 o->buffer_output = 0;
>>         strbuf_init(&o->obuf, 0);
>> --
>> 2.14.3
> 
> 
> 
-- 
Best regards,
Andrey Okoshkin
Andrey Okoshkin· Oct 25, 2017, 13:03 UTC · re: Eric Sunshine · lore

Re: [PATCH v3] merge-recursive: check GIT_MERGE_VERBOSITY only once

Get 'GIT_MERGE_VERBOSITY' environment variable only once in init_merge_options() and store the pointer to its value for the further check. No intervening calls to getenv(), putenv(), setenv() or unsetenv() are done between the initial getenv() call and the consequential result pass to strtol() as these environment related functions could modify the string pointer returned by the initial getenv() call.

Signed-off-by: Andrey Okoshkin <a.okoshkin@samsung.com>
---
I tried to rework the commit message.
 merge-recursive.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)
Show changes to merge-recursive.c +4 −3
diff --git a/merge-recursive.c b/merge-recursive.c
index 1494ffdb8..60084e3a0 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -2163,6 +2163,7 @@ static void merge_recursive_config(struct merge_options *o)
 
 void init_merge_options(struct merge_options *o)
 {
+	const char *merge_verbosity;
 	memset(o, 0, sizeof(struct merge_options));
 	o->verbosity = 2;
 	o->buffer_output = 1;
@@ -2171,9 +2172,9 @@ void init_merge_options(struct merge_options *o)
 	o->renormalize = 0;
 	o->detect_rename = 1;
 	merge_recursive_config(o);
-	if (getenv("GIT_MERGE_VERBOSITY"))
-		o->verbosity =
-			strtol(getenv("GIT_MERGE_VERBOSITY"), NULL, 10);
+	merge_verbosity = getenv("GIT_MERGE_VERBOSITY");
+	if (merge_verbosity)
+		o->verbosity = strtol(merge_verbosity, NULL, 10);
 	if (o->verbosity >= 5)
 		o->buffer_output = 0;
 	strbuf_init(&o->obuf, 0);
-- 
2.14.3
Stefan Beller· Oct 27, 2017, 17:29 UTC · re: Andrey Okoshkin · lore

Re: [PATCH v3] merge-recursive: check GIT_MERGE_VERBOSITY only once

On Wed, Oct 25, 2017 at 6:03 AM, Andrey Okoshkin <a.okoshkin@samsung.com> wrote:
Show 8 quoted lines
> Get 'GIT_MERGE_VERBOSITY' environment variable only once in
> init_merge_options() and store the pointer to its value for the further check.
> No intervening calls to getenv(), putenv(), setenv() or unsetenv() are done
> between the initial getenv() call and the consequential result pass to strtol()
> as these environment related functions could modify the string pointer returned
> by the initial getenv() call.
>
> Signed-off-by: Andrey Okoshkin <a.okoshkin@samsung.com>
This is
Reviewed-by: Stefan Beller <sbeller@google.com>
Andrey Okoshkin· Oct 30, 2017, 07:42 UTC · re: Stefan Beller · lore

[PATCH v4] merge-recursive: check GIT_MERGE_VERBOSITY only once

Get 'GIT_MERGE_VERBOSITY' environment variable only once in init_merge_options() and store the pointer to its value for the further check. No intervening calls to getenv(), putenv(), setenv() or unsetenv() are done between the initial getenv() call and the consequential result pass to strtol() as these environment related functions could modify the string pointer returned by the initial getenv() call.

Signed-off-by: Andrey Okoshkin <a.okoshkin@samsung.com>
Reviewed-by: Stefan Beller <sbeller@google.com>
---
Added 'reviewed-by' field.
 merge-recursive.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)
Show changes to merge-recursive.c +4 −3
diff --git a/merge-recursive.c b/merge-recursive.c
index 1494ffdb8..60084e3a0 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -2163,6 +2163,7 @@ static void merge_recursive_config(struct merge_options *o)
 
 void init_merge_options(struct merge_options *o)
 {
+	const char *merge_verbosity;
 	memset(o, 0, sizeof(struct merge_options));
 	o->verbosity = 2;
 	o->buffer_output = 1;
@@ -2171,9 +2172,9 @@ void init_merge_options(struct merge_options *o)
 	o->renormalize = 0;
 	o->detect_rename = 1;
 	merge_recursive_config(o);
-	if (getenv("GIT_MERGE_VERBOSITY"))
-		o->verbosity =
-			strtol(getenv("GIT_MERGE_VERBOSITY"), NULL, 10);
+	merge_verbosity = getenv("GIT_MERGE_VERBOSITY");
+	if (merge_verbosity)
+		o->verbosity = strtol(merge_verbosity, NULL, 10);
 	if (o->verbosity >= 5)
 		o->buffer_output = 0;
 	strbuf_init(&o->obuf, 0);
-- 
2.14.3
Junio C Hamano· Oct 31, 2017, 01:42 UTC · re: Andrey Okoshkin · lore

Re: [PATCH v4] merge-recursive: check GIT_MERGE_VERBOSITY only once

Andrey Okoshkin <a.okoshkin@samsung.com> writes:
> Get 'GIT_MERGE_VERBOSITY' environment variable only once in
> init_merge_options() and store the pointer to its value for the further check.
OK, that is "what this thing does" description.
> No intervening calls to getenv(), putenv(), setenv() or unsetenv() are done
> between the initial getenv() call and the consequential result pass to strtol()
> as these environment related functions could modify the string pointer returned
> by the initial getenv() call.

That holds true for the code before or after this patch equally. In other words, that sounds like a justification for rejecting this patch (i.e. explanation of why this change is not needed).

If we are worried about another thread calling these functions after the time we call getenv() and before the time we pass the result to strtol(), then I do not think this patch gives a better protection against such race, so I do not think that is why you are doing this.

So... why do we want to do this change?  I am puzzled.
Show 33 quoted lines
>
> Signed-off-by: Andrey Okoshkin <a.okoshkin@samsung.com>
> Reviewed-by: Stefan Beller <sbeller@google.com>
> ---
> Added 'reviewed-by' field.
>  merge-recursive.c | 7 ++++---
>  1 file changed, 4 insertions(+), 3 deletions(-)
>
> diff --git a/merge-recursive.c b/merge-recursive.c
> index 1494ffdb8..60084e3a0 100644
> --- a/merge-recursive.c
> +++ b/merge-recursive.c
> @@ -2163,6 +2163,7 @@ static void merge_recursive_config(struct merge_options *o)
>  
>  void init_merge_options(struct merge_options *o)
>  {
> +	const char *merge_verbosity;
>  	memset(o, 0, sizeof(struct merge_options));
>  	o->verbosity = 2;
>  	o->buffer_output = 1;
> @@ -2171,9 +2172,9 @@ void init_merge_options(struct merge_options *o)
>  	o->renormalize = 0;
>  	o->detect_rename = 1;
>  	merge_recursive_config(o);
> -	if (getenv("GIT_MERGE_VERBOSITY"))
> -		o->verbosity =
> -			strtol(getenv("GIT_MERGE_VERBOSITY"), NULL, 10);
> +	merge_verbosity = getenv("GIT_MERGE_VERBOSITY");
> +	if (merge_verbosity)
> +		o->verbosity = strtol(merge_verbosity, NULL, 10);
>  	if (o->verbosity >= 5)
>  		o->buffer_output = 0;
>  	strbuf_init(&o->obuf, 0);
Junio C Hamano· Oct 31, 2017, 02:26 UTC · re: Junio C Hamano · lore

Re: [PATCH v4] merge-recursive: check GIT_MERGE_VERBOSITY only once

Junio C Hamano <gitster@pobox.com> writes:
Show 10 quoted lines
> That holds true for the code before or after this patch equally.  In
> other words, that sounds like a justification for rejecting this
> patch (i.e. explanation of why this change is not needed).
>
> If we are worried about another thread calling these functions after
> the time we call getenv() and before the time we pass the result to
> strtol(), then I do not think this patch gives a better protection
> against such race, so I do not think that is why you are doing this.
>
> So... why do we want to do this change?  I am puzzled.

For the sake of fairness, I would say that the resulting code may be easier to follow and has one less instance of a constant string that the compiler cannot statically check if we made a typo. That's the only benefit in this patch as far as I can see.

The original makes a call to see if the result is NULL, and then makes the same call, expecting that we get the same result (not just that it is not NULL, but it is the same verbosity request the end user made via the environment as the one we checked earlier), and I can understand that it feels a bit redundant and ugly.

Show 32 quoted lines
>> Signed-off-by: Andrey Okoshkin <a.okoshkin@samsung.com>
>> Reviewed-by: Stefan Beller <sbeller@google.com>
>> ---
>> Added 'reviewed-by' field.
>>  merge-recursive.c | 7 ++++---
>>  1 file changed, 4 insertions(+), 3 deletions(-)
>>
>> diff --git a/merge-recursive.c b/merge-recursive.c
>> index 1494ffdb8..60084e3a0 100644
>> --- a/merge-recursive.c
>> +++ b/merge-recursive.c
>> @@ -2163,6 +2163,7 @@ static void merge_recursive_config(struct merge_options *o)
>>  
>>  void init_merge_options(struct merge_options *o)
>>  {
>> +	const char *merge_verbosity;
>>  	memset(o, 0, sizeof(struct merge_options));
>>  	o->verbosity = 2;
>>  	o->buffer_output = 1;
>> @@ -2171,9 +2172,9 @@ void init_merge_options(struct merge_options *o)
>>  	o->renormalize = 0;
>>  	o->detect_rename = 1;
>>  	merge_recursive_config(o);
>> -	if (getenv("GIT_MERGE_VERBOSITY"))
>> -		o->verbosity =
>> -			strtol(getenv("GIT_MERGE_VERBOSITY"), NULL, 10);
>> +	merge_verbosity = getenv("GIT_MERGE_VERBOSITY");
>> +	if (merge_verbosity)
>> +		o->verbosity = strtol(merge_verbosity, NULL, 10);
>>  	if (o->verbosity >= 5)
>>  		o->buffer_output = 0;
>>  	strbuf_init(&o->obuf, 0);
Andrey Okoshkin· Oct 31, 2017, 07:13 UTC · re: Junio C Hamano · lore

Re: [PATCH v4] merge-recursive: check GIT_MERGE_VERBOSITY only once

31.10.2017 05:26, Junio C Hamano wrote:
Show 12 quoted lines
> Junio C Hamano <gitster@pobox.com> writes:
> 
>> That holds true for the code before or after this patch equally.  In
>> other words, that sounds like a justification for rejecting this
>> patch (i.e. explanation of why this change is not needed).
>>
>> If we are worried about another thread calling these functions after
>> the time we call getenv() and before the time we pass the result to
>> strtol(), then I do not think this patch gives a better protection
>> against such race, so I do not think that is why you are doing this.
>>
>> So... why do we want to do this change?  I am puzzled.
I think, the main benefits are:
* Code is more readable, no duplicated calls with the same constant string
argument.
* Code is potentially safer, the second getenv() call may return another
pointer value which could be NULL (and yes, this is an arguable point as it
can be done only artificially).
Show 10 quoted lines
> For the sake of fairness, I would say that the resulting code may be
> easier to follow and has one less instance of a constant string that
> the compiler cannot statically check if we made a typo.  That's the
> only benefit in this patch as far as I can see.
> 
> The original makes a call to see if the result is NULL, and then
> makes the same call, expecting that we get the same result (not just
> that it is not NULL, but it is the same verbosity request the end
> user made via the environment as the one we checked earlier), and I
> can understand that it feels a bit redundant and ugly.

Yes, you absolutely right. I believe this patch makes code more beautiful :-)

Show 35 quoted lines
>>> Signed-off-by: Andrey Okoshkin <a.okoshkin@samsung.com>
>>> Reviewed-by: Stefan Beller <sbeller@google.com>
>>> ---
>>> Added 'reviewed-by' field.
>>>  merge-recursive.c | 7 ++++---
>>>  1 file changed, 4 insertions(+), 3 deletions(-)
>>>
>>> diff --git a/merge-recursive.c b/merge-recursive.c
>>> index 1494ffdb8..60084e3a0 100644
>>> --- a/merge-recursive.c
>>> +++ b/merge-recursive.c
>>> @@ -2163,6 +2163,7 @@ static void merge_recursive_config(struct merge_options *o)
>>>  
>>>  void init_merge_options(struct merge_options *o)
>>>  {
>>> +	const char *merge_verbosity;
>>>  	memset(o, 0, sizeof(struct merge_options));
>>>  	o->verbosity = 2;
>>>  	o->buffer_output = 1;
>>> @@ -2171,9 +2172,9 @@ void init_merge_options(struct merge_options *o)
>>>  	o->renormalize = 0;
>>>  	o->detect_rename = 1;
>>>  	merge_recursive_config(o);
>>> -	if (getenv("GIT_MERGE_VERBOSITY"))
>>> -		o->verbosity =
>>> -			strtol(getenv("GIT_MERGE_VERBOSITY"), NULL, 10);
>>> +	merge_verbosity = getenv("GIT_MERGE_VERBOSITY");
>>> +	if (merge_verbosity)
>>> +		o->verbosity = strtol(merge_verbosity, NULL, 10);
>>>  	if (o->verbosity >= 5)
>>>  		o->buffer_output = 0;
>>>  	strbuf_init(&o->obuf, 0);
> 
> 
> 
-- 
Best regards,
Andrey Okoshkin
Junio C Hamano· Oct 31, 2017, 07:20 UTC · re: Andrey Okoshkin · lore

Re: [PATCH v4] merge-recursive: check GIT_MERGE_VERBOSITY only once

Andrey Okoshkin <a.okoshkin@samsung.com> writes:
Show 19 quoted lines
> I think, the main benefits are:
> * Code is more readable, no duplicated calls with the same constant string
> argument.
> * Code is potentially safer, the second getenv() call may return another
> pointer value which could be NULL (and yes, this is an arguable point as it
> can be done only artificially).
>
>> For the sake of fairness, I would say that the resulting code may be
>> easier to follow and has one less instance of a constant string that
>> the compiler cannot statically check if we made a typo.  That's the
>> only benefit in this patch as far as I can see.
>> 
>> The original makes a call to see if the result is NULL, and then
>> makes the same call, expecting that we get the same result (not just
>> that it is not NULL, but it is the same verbosity request the end
>> user made via the environment as the one we checked earlier), and I
>> can understand that it feels a bit redundant and ugly.
>
> Yes, you absolutely right.

I am absolutely right when I say your "code is potentially safer" is total BS. The result from first getenv() call may be pointing at an invalid piece of memory by the time it is used, if you are in a situation in which not having the second getenv() matters (i.e. somebody else is also mucking with getenv() at the same time).

So please update the log message so that the patch is not sold on that basis.

Thanks.
Andrey Okoshkin· Oct 31, 2017, 09:09 UTC · re: Junio C Hamano · lore

[PATCH v5] merge-recursive: check GIT_MERGE_VERBOSITY only once

Get rid of the duplicated getenv('GIT_MERGE_VERBOSITY') calls with the same constant string argument. This makes code more readable and prevents typo in the further development.

Signed-off-by: Andrey Okoshkin <a.okoshkin@samsung.com>
Reviewed-by: Stefan Beller <sbeller@google.com>
---
Commit message is reworked according to the feedback.
 merge-recursive.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)
Show changes to merge-recursive.c +4 −3
diff --git a/merge-recursive.c b/merge-recursive.c
index 1494ffdb8..60084e3a0 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -2163,6 +2163,7 @@ static void merge_recursive_config(struct merge_options *o)
 
 void init_merge_options(struct merge_options *o)
 {
+	const char *merge_verbosity;
 	memset(o, 0, sizeof(struct merge_options));
 	o->verbosity = 2;
 	o->buffer_output = 1;
@@ -2171,9 +2172,9 @@ void init_merge_options(struct merge_options *o)
 	o->renormalize = 0;
 	o->detect_rename = 1;
 	merge_recursive_config(o);
-	if (getenv("GIT_MERGE_VERBOSITY"))
-		o->verbosity =
-			strtol(getenv("GIT_MERGE_VERBOSITY"), NULL, 10);
+	merge_verbosity = getenv("GIT_MERGE_VERBOSITY");
+	if (merge_verbosity)
+		o->verbosity = strtol(merge_verbosity, NULL, 10);
 	if (o->verbosity >= 5)
 		o->buffer_output = 0;
 	strbuf_init(&o->obuf, 0);
-- 
2.14.3
Andrey Okoshkin· Oct 25, 2017, 11:13 UTC · re: Jeff King · lore

Re: [PATCH] merge-recursive: check GIT_MERGE_VERBOSITY only once

24.10.2017 22:52, Jeff King wrote:
Show 26 quoted lines
> On Tue, Oct 24, 2017 at 07:11:24PM +0200, Martin Ågren wrote:
> 
>> On 24 October 2017 at 18:45, Eric Sunshine <sunshine@sunshineco.com> wrote:
>>> On Tue, Oct 24, 2017 at 12:28 PM, Stefan Beller <sbeller@google.com> wrote:
>>>> On Tue, Oct 24, 2017 at 8:27 AM, Andrey Okoshkin <a.okoshkin@samsung.com> wrote:
>>>>> Add check of 'GIT_MERGE_VERBOSITY' environment variable only once in
>>>>> init_merge_options().
>>>>> Consequential call of getenv() may return NULL pointer and strtol() crashes.
>>>>> However the stored pointer to the obtained getenv() result may be invalidated
>>>>> by some other getenv() call from another thread as getenv() is not thread-safe.
>>
>> I'm having trouble wrapping my head around this. Under which
>> circumstances could the second call in the current code return NULL, but
>> the code after your patch behave in a well-defined (and correct) way?
> 
> Yeah, it's not at all clear to me this is solving a real problem. I know
> Andrey mentioned playing around with fault injection in an earlier
> thread, so I'm wondering if there is an artificial fault being injected
> into the second getenv() call. Which does not seem like something that
> should be possible in the real world.
> 
> I definitely agree with the sentiment that as few things as possible
> should happen between calling getenv() and using its result. I've seen
> real bugs there from unexpected invalidation of the static buffer.
> 
> -Peff
Thanks for your comments.
Jeff is right: there were some artificial fault injections imitating valid failures
of different functions (syscalls, libc and so on).
And yes - in the real life there is no problems with current code as there are no other
threads running.
However it's not a good practice to double call getenv() with the same argument:
* Code readability.
* Still no guaranty that the second call will be valid: some linked library may be
compromised or LD_PRELOADed with the aim to create a race with getenv(). I believe
there is no profit doing it here but it's just an explanation.

In my opinion, here it's ok to save the pointer returned from the single getnev() call doing as few actions as possible between getenv() and strtol() calls. I'll change the patch.

-- 
Best regards,
Andrey Okoshkin
Andrey Okoshkin· Oct 25, 2017, 10:49 UTC · re: Eric Sunshine · lore

Re: [PATCH] merge-recursive: check GIT_MERGE_VERBOSITY only once

24.10.2017 19:45, Eric Sunshine wrote:
> I feel uncomfortable about this change, not due to lack of thread
> safety, but due to the distance between the getenv() invocation and
> its point of use. See below for more detail.
Thanks, the usage must be just after the assignment.
-- 
Best regards,
Andrey Okoshkin
Andrey Okoshkin· Oct 25, 2017, 10:49 UTC · re: Stefan Beller · lore

Re: [PATCH] merge-recursive: check GIT_MERGE_VERBOSITY only once

Thanks for your review.
24.10.2017 19:28, Stefan Beller wrote:
> So I think this function is never called from within a threaded environment
> in git.
You are right, it's just a hypothetic case.
 
Show 12 quoted lines
> Despite not being in a threaded environment, I wonder if we want to
> minimize the time between  calling getenv and the use of the result,
> i.e. declare merge_verbosity here, but assign it later, just before the
> condition?
> 
> (The compiler may shuffle stuff around anyway, so this is a
> moot suggestion; It gears mostly towards making the code more
> readable/maintainable when presenting this part of the code
> to the user.)
> 
> With or without this change:
> Reviewed-by: Stefan Beller <sbeller@google.com>

Yes, in current situation it's more for readability. And I'll make the usage of merge_verbosity just after the assignment.

-- 
Best regards,
Andrey Okoshkin

← back to recent threads