Re: [PATCH 7/9] fetch: fetch submodules in parallel

From: Junio C Hamano <gitster@pobox.com>
Date: Aug 28, 2015, 18:27 UTC
Message-ID: <xmqqpp271d6v.fsf@gitster.mtv.corp.google.com>
In-Reply-To: <20150828182008.GC8165@google.com>

Jonathan Nieder <jrnieder@gmail.com> writes:

Show 21 quoted lines

> Junio C Hamano wrote:
>
>> My preference is still (1) leave standard error output all connected
>> to the same fd without multiplexing, and (2) line buffer standard
>> output so that the output is at least readable as a text, in a
>> similar way a log of an irc channel where everybody is talking at
>> the same time.
>
> There is something nice about the immediacy of seeing output from all
> the subprocesses at the same time in that model.
>
> But for commands that show progress like "git clone", "git checkout",
> and "git fetch", it does not work well at all.  They provide output
> that updates itself by putting a carriage return at the end of each
> chunk of output, like this:
>
>  remote: Finding sources:  11% (18/155)           \r
>  remote: Finding sources:  12% (19/155)           \r
>
> With multiple commands producing such output, they will overwrite each
> other's lines, producing a mixture that is confusing and unuseful.

That example also illustrates why it is not a useful to buffer all of these lines and showing them once.

Show 5 quoted lines

> Ideally what I as a user want to see is something like what "prove"
> writes, showing progress on the multiple tasks that are taking place
> at once:
>
>  ===(     103;1  0/?  8/?  3/?  11/?  6/?  16/?  1/?  1/? )==============

Tell me how that "buffer all and show them once" helps us to get near that ideal.

> That would require more sophisticated inter-process communication than
> seems necessary for the first version of parallel "git submodule
> update".

Exactly. Why waste memory to buffer and stall the entire output from other processes in the interim solution, then?