Re: [PATCH 7/9] fetch: fetch submodules in parallel

From: Stefan Beller <sbeller@google.com>
Date: Aug 28, 2015, 17:45 UTC
Message-ID: <CAGZ79kYWJ89MuU4iJH=Mvr_jH72d+EM0hCeDhfpQuapYQH0Nkg@mail.gmail.com>
In-Reply-To: <xmqqfv332v84.fsf@gitster.mtv.corp.google.com>

On Fri, Aug 28, 2015 at 10:12 AM, Junio C Hamano <gitster@pobox.com> wrote:

Show 32 quoted lines

> Jonathan Nieder <jrnieder@gmail.com> writes:
>
>> Stefan Beller wrote:
>>> On Thu, Aug 27, 2015 at 6:14 PM, Stefan Beller <sbeller@google.com> wrote:
>>
>>>> This makes use of the new task queue and the syncing feature of
>>>> run-command to fetch a number of submodules at the same time.
>>>>
>>>> The output will look like it would have been run sequential,
>>>> but faster.
>>>
>>> And it breaks the tests t5526-fetch-submodules.sh as the output is done
>>> on stderr only, instead of putting "Fetching submodule <submodule-path>
>>> to stdout. :(
>>>
>>> I guess combining stdout and stderr is not a good strategy after all now.
>>
>> IMHO the "Fetching submodule <submodule-path>" output always should have
>> gone to stderr.  It is not output that scripts would be relying on ---
>> it is just progress output.
>>
>> So a preliminary patch doing that (and updating tests) would make sense
>> to me.
>
> Sounds good.
>
> I personally do not think the "we still do all the output from a
> single process while blocking output from others" buffering
> implemented in this n-th round (by the way, please use [PATCH v$n
> N/M]) is worth doing, though.  It does not make the output machine
> parseable, because the reader does not get any signal in what order
> output of these subprocesses arrive anyway.

> The payload does not
> even have "here is the beginning of output from the process that
> handled the submodule X" to delimit them.

Oh it does, but it is not handled by the buffering code, but the application code, so for git-fetch we would have "Fetching submodule <path> " while for git submodule foreach we would have "Entering <submodule path>"

Show 6 quoted lines

>
> My preference is still (1) leave standard error output all connected
> to the same fd without multiplexing, and (2) line buffer standard
> output so that the output is at least readable as a text, in a
> similar way a log of an irc channel where everybody is talking at
> the same time.

In case of fetch passing the quiet flag becomes mandatory then, because the the per-repo progress is put to stderr, which also deletes previous output, to have the nice counters.