git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 2/2] fetch: fetch submodules in parallel

From
Junio C Hamano <gitster@pobox.com>
Date
Sep 14, 2015, 17:56 UTC
Message-ID
<xmqqsi6gx4uh.fsf@gitster.mtv.corp.google.com>
In-Reply-To
<20150914171736.GA1548@sigill.intra.peff.net>
Jeff King <peff@peff.net> writes:
Show 16 quoted lines
> I don't think you need exact timing information.  This is no different
> than running the commands themselves, with stdout and stderr writing to
> a pty that your terminal emulator will then read() from. If the program
> produces intermingled stdout/stderr that clogs up the terminal, that is
> its problem.
>
> The only difference is that we're going to save it and later replay it
> all very quickly.  So I think it would be sufficient just to retain the
> original order.
>
>> I will add documentation explaining why the async output case
>> will only deal with one channel. I chose stderr as that's already
>> available and needed in this use case.
>
> I suspect you could just set child->stdout_to_stderr in this case, and
> then you get your ordering for free.

I think we are in agreement; that is exactly what I wanted to say when I said "I offhand do not think the latter [i.e. the callers have to dup them together] is unreasonable". Thanks for stating it more clearly and explicitly.

Show 12 quoted lines
> To handle multiple channels, I think you could just do a linked list of
> buffers rather than a single strbuf. Like:
>
>   struct io_chunk {
> 	int channel;
> 	char *buf;
> 	size_t len;
> 	struct io_chunk *next;
>   };
>
> and just keep appending chunks to the list (and to dump them, just walk
> the list, writing each to the appropriate channel descriptor).

Perhaps, but let's not overdesign things before we have a concrete example codepath that benefits from such a thing. It is hard to detect a misdesign without a real usage pattern.

Previous: Junio C Hamano
Message 13 of 13 in “[RFC PATCHv1 0/2] Parallel git submodule fetching”
  1. Stefan BellerSep 11, 2015
  2. 1/2 Sending "Fetching submodule <foo>" output to stderrStefan Beller, Sep 11, 2015
  3. 2/2 fetch: fetch submodules in parallelStefan Beller, Sep 11, 2015
  4. Junio C HamanoSep 12, 2015
  5. Stefan BellerSep 14, 2015
  6. Jeff KingSep 14, 2015
  7. Stefan BellerSep 14, 2015
  8. Jonathan NiederSep 14, 2015
  9. Jeff KingSep 14, 2015
  10. [PATCHv2] fetch submodules in parallelStefan Beller, Sep 14, 2015
  11. [PATCHv2] fetch: fetch submodules in parallelStefan Beller, Sep 14, 2015
  12. Junio C HamanoSep 14, 2015
  13. Junio C HamanoSep 14, 2015

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.