From: D. Ben Knoble Date: Tue, 09 Jun 2026 13:40:25 GMT Subject: Re: [PATCH v2] describe: limit default ref iteration to tags Message-ID: In-Reply-To: <20260609110957.GB1509396@coredump.intra.peff.net> On Tue, Jun 9, 2026 at 7:10 AM Jeff King wrote: > > On Mon, Jun 08, 2026 at 07:32:14PM -0700, Tamir Duberstein wrote: > > > The benchmark checkout had 120,532 refs, of which 330 were tags. With > > `$repo` naming the checkout, `$commit` an exactly tagged commit, and > > `$parent` and `$this` the two binaries, I ran: > > > > hyperfine --warmup 3 --runs 15 \ > > --command-name parent \ > > '$parent -C $repo describe --exact-match $commit' \ > > --command-name 'this commit' \ > > '$this -C $repo describe --exact-match $commit' > > > > The results were: > > > > Benchmark 1: parent > > Time (mean ± σ): 171.7 ms ± 18.5 ms [User: 23.9 ms, System: 133.6 ms] > > Range (min … max): 142.3 ms … 198.3 ms 15 runs > > > > Benchmark 2: this commit > > Time (mean ± σ): 9.9 ms ± 1.1 ms [User: 3.3 ms, System: 4.7 ms] > > Range (min … max): 8.8 ms … 13.1 ms 15 runs > > > > Summary > > this commit ran > > 17.35 ± 2.63 times faster than parent > > > > Both revisions were built with -O3, -mcpu=native, and ThinLTO using > > Apple clang 21.0.0 on macOS 26.5. The machine was a MacBook Pro > > (Mac16,6) with a 16-core Apple M4 Max (12 performance and four > > efficiency cores) and 128 GB RAM. > > This patch looks fine to me, but let me pick a nit for a minute, because > I think there is a broader conversation to be had. > > Given the discussion in earlier rounds and sibling topics, I assume the > commit message here was AI-generated. And it's OK in the sense that it > is describing what happened and I assume is entirely accurate. But as a > human reader, it feels so much more verbose than what I'd expect, as it > is full of semi-irrelevant details. Why set --warmup and --runs? Why > bother with --command-name, which just means you have to show the > commands separately anyway? Is the amount of RAM in the machine > important for this test? Surely it could be if it was absurdly tiny, but > in general, no, I would not expect it to be. [You probably know this] It is common in academic papers to report benchmarks with details about the hardware and how they were run to contextualize the results and help with reproducibility. Of course, Git's commits do not form an academic paper… so I have no real opinion on what to see here. But I've seen a few other mails where having perf test outputs or similar was suggested (maybe that was to be reserved for the cover letter? idk). _If_ we show all the hyperfine details, I think it's reasonable to use --command-name to make distinguishing the versions easy, unless it's obvious from the path/to/git in each benchmark (which I think I've seen from Peff's benchmark reports before?). Someone with better lore skills can probably dig up a few exemplars of how to write about performance in a commit message? -- D. Ben Knoble