Re: Why the Makefile is so eager to re-build & re-link
- From
Johannes Sixt <j6t@kdbg.org>
- Date
- Jun 24, 2021, 21:30 UTC
- Message-ID
- <fb23a23e-13be-14a8-4fbe-5ca2b4bcdb52@kdbg.org>
- In-Reply-To
- <YNSh0CskelTwuZq0@coredump.intra.peff.net>
Am 24.06.21 um 17:16 schrieb Jeff King:
Show 41 quoted lines
> On Thu, Jun 24, 2021 at 03:16:48PM +0200, Ævar Arnfjörð Bjarmason wrote:
>> * {command,config}-list.h (and in-flight, my hook-list.h): Every time
>> you touch a Documentation/git-*.txt we need to re-generate these, and
>> since their mtime changes we re-compile and re-link all the way up to
>> libgit and our other tools.
>>
>> I think the best solution here is to make the generate-*.sh
>> shellscripts faster (just one takes ~300ms of nested shellscripting,
>> just to grep out the first few lines of every git-*.txt, in e.g. Perl
>> or a smarter awk script this would be <5ms).
>
> Yeah, I think Eric mentioned he had looked into doing this in perl, but
> we weren't entirely happy with the dependency. Here's another really odd
> thing I noticed:
>
> $ time sh ./generate-cmdlist.sh command-list.txt >one
> real 0m1.323s
> user 0m1.531s
> sys 0m0.834s
>
> $ time sh -x ./generate-cmdlist.sh command-list.txt >two
> [a bunch of trace output]
> real 0m0.513s
> user 0m0.754s
> sys 0m0.168s
>
> $ cmp one two
> [no output]
>
> Er, what? Running with "-x" makes it almost 3 times faster to generate
> the same output? I'd have said this is an anomaly, but it's repeatable
> (and swapping the order produces the same output, so it's not some weird
> priming thing). And then to top it all off, redirecting the trace is
> slow again:
>
> $ time sh -x ./generate-cmdlist.sh command-list.txt >two 2>/dev/null
> real 0m1.363s
> user 0m1.538s
> sys 0m0.902s
>
> A little mini-mystery that I think I may leave unsolved for now.Strange, really. Reminds me of the case that the `read` built-in must read input byte by byte if stdin is not connected to a (searchable) file.
I have two patches that speed up generate-cmdlist.sh a bit:
generate-cmdlist.sh: replace for loop by printf's auto-repeat feature (https://github.com/j6t/git/commit/b6d05f653bede727bc001f299b57969f62d3bc03) generate-cmdlist.sh: spawn fewer processes (https://github.com/j6t/git/commit/fd8721ee8fae06d7b584fa5166f32bf5521ca304)
that I can submit (give me some time, though) or interested parties could pick up.
-- Hannes