Re: [PATCH v3 24/34] t/perf/p7519: speed up test using "test-tool touch"

From: Junio C Hamano <gitster@pobox.com>
Date: Jul 13, 2021, 18:07 UTC
Message-ID: <xmqqtuky141l.fsf@gitster.g>
In-Reply-To: <797f4e85-33cc-df7e-31d8-3a4f95a9109a@jeffhostetler.com>

Jeff Hostetler <git@jeffhostetler.com> writes:

Show 13 quoted lines

> FWIW, the xargs is clustering the 10,000 files into ~4 command lines,
> so there is a little bit of Windows process overhead, but not that
> much.
>
> 	seq 1 10000 | xargs wc -l | grep total
>
>> I'd really like to modify test_seq to use seq when it's available and
>> fall back to the looping-in-shell when we need to for various
>> platforms.
>> Maybe it'd even make sense to write a 'test-tool seq' and make
>> test_seq use that just so we can rip out that super lame shell
>> looping.
>>

So what lame in this picture is not shell, or process overhead, but I/O performance.

I've seen some noises about Windows file creation performance raised as an issue when doing initial checkout followed by "git clone", and an idea floated to create a bunch of open file handles for writing in threads when checkout (really the caller that repeatedly calls entry.c:write_entry() by iterating the in-core index) starts, and write out the contents in parallel, as a workaround. When I heard it, I somehow thought it was meant as a not-so-funny joke, but from the sounds of it, the I/O performance may be so horrible to require such a hack to be usable there. Sigh...