git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH] bitmaps: don't recurse into trees already in the bitmap

From
Patrick Steinhardt <ps@pks.im>
Date
Jun 18, 2021, 13:35 UTC
Message-ID
<YMyhCTaHmm6oNFpB@xps>
In-Reply-To
<YMyYz6Tavj5l6S8n@coredump.intra.peff.net>
On Fri, Jun 18, 2021 at 08:59:59AM -0400, Jeff King wrote:
Show 26 quoted lines
> On Wed, Jun 16, 2021 at 02:31:04PM +0200, Patrick Steinhardt wrote:
> 
> > > These many-refs scenarios make sense as something that is difficult to
> > > verify using a single fork of an open-source project, but is common in
> > > many closed-source projects that do not use forking to reduce the ref
> > > count per repo.
> > 
> > Agreed. What I typically do to emulate this is to use some version of
> > following command to create refs for "$n" commits.
> > 
> >     git log --all --format="tformat:create refs/commit/%h %H" |
> >         shuf | head -n "$n" | git update-ref --stdin
> > 
> > It's obviously not ideal given that resulting refs are distributed at
> > random. But combined with a sufficiently large repo, it's still helped
> > me at times to reproduce adverse performance at times.
> 
> Yeah, I've done something similar. But I'd caution people into reading
> too much into performance tests from something like that. What I've
> found over the years is that traversal and bitmap performance can be
> somewhat sensitive to tree shape and bitmap placement (which in turn is
> often influenced by ref placement, if you are using delta islands or the
> recently added pack.preferBitmapTips).
> 
> More than once I've spent many head-scratching hours trying to determine
> why some real-world repo performs better or worse than expected. :)

I couldn't agree more. I've also had my fair share of weird performance characteristics in completely unexpected ways. Unfortunately, it has made me become quite cautious about bitmaps given that they've already caused their fair share of perfomance regressions.

But your work here actually encourages me to give it another try soonish and see what kind of repo shapes make them explode this time.

Patrick
Previous: Jeff KingNext: Jeff King
Message 6 of 9 in “bitmaps: don't recurse into trees already in the bitmap”
  1. bitmaps: don't recurse into trees already in the bitmapJeff King, Jun 14, 2021
  2. Jeff KingJun 14, 2021
  3. Derrick StoleeJun 15, 2021
  4. Patrick SteinhardtJun 16, 2021
  5. Jeff KingJun 18, 2021
  6. Patrick SteinhardtJun 18, 2021
  7. Jeff KingJun 18, 2021
  8. Patrick SteinhardtJun 22, 2021
  9. Jeff KingJun 22, 2021

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.