Re: [PATCH v3 25/25] p5326: perf tests for MIDX bitmaps

From: Jeff King <peff@peff.net>
Date: Aug 12, 2021, 21:18 UTC
Message-ID: <YRWQPDd2S6Hi/2Ge@coredump.intra.peff.net>
In-Reply-To: <82e8133bf4f6ecf2ca509f6d9e2e0d369d7f19e3.1627420428.git.me@ttaylorr.com>

On Tue, Jul 27, 2021 at 05:20:28PM -0400, Taylor Blau wrote:

Show 10 quoted lines

> These new performance tests demonstrate effectively the same behavior as
> p5310, but use a multi-pack bitmap instead of a single-pack one.
> 
> Notably, p5326 does not create a MIDX bitmap with multiple packs. This
> is so we can measure a direct comparison between it and p5310. Any
> difference between the two is measuring just the overhead of using MIDX
> bitmaps.
> 
> Here are the results of p5310 and p5326 together, measured at the same
> time and on the same machine (using a Xenon W-2255 CPU):

Neat. I think having separate perf regression tests for regular and mix bitmaps will be useful, but being able to compare the pack and mix versions is a cherry on top.

There was one funny number:

>     5310.2: repack to disk                                96.78(93.39+11.33)
>     5326.2: setup multi-pack index                        78.99(75.29+11.58)

In p5310, that step is repacking and writing bitmaps. With the midx, it's repacking, then writing a midx with bitmaps. I'd expect the latter to be strictly slower than the former, but here it's faster.

Running the code locally, I got similar results (with p5310 just a tiny bit faster). So it may have just been noise or some other timing issue.

  As an aside, I think that test is a little bit bogus due to
  GIT_PERF_REPEAT_COUNT; the first trial will generate bitmaps from
  scratch, and then subsequent runs will reuse partial results. It
  probably should "rm -f .git/objects/*.bitmap" within the test. We can
  deal with that separately, though.

-Peff