From: Jeff King <peff@peff.net>
Date: Thu, 19 Nov 2015 13:58:08 GMT
Subject: Re: [PATCH v2] add test to demonstrate that shallow recursive clones fail
Message-ID: <20151119135808.GA9353@sigill.intra.peff.net>
In-Reply-To: <CAOE36qj2m4e3hw73-QoLbbpGv4RiyhBt_ou7eN4i4q8pF15rdA@mail.gmail.com>

On Wed, Nov 18, 2015 at 03:40:02PM -0800, Terry Parker wrote:

> > +Terry, who did optimize the JGit implementation for bitmaps,
> > as we also had a "lots of refs" hoarder repo, which underperformed
> > before.
> 
> The performance issue with the "hoarder" repo was that the bitmap
> commit selection algorithm in JGit was selecting too many commits and
> the hoarder repo caused OOMs.
> 
> JGit's selection algorithm walks each branch and makes sure commits
> are selected every 100-200 commits for "recent" history and every
> 5000-5100 commits for more distant history. IIUC cgit sorts all commits
> by date and does a similar selection. Since it doesn't attempt to get
> even coverage on each branch, the selected commits may have
> large gaps for a particular branch.

Thanks for the data point. I agree that cgit seems to have the opposite
problem: selecting fewer commits, which gives worse performance at read
time, but no OOM at write time. :)

I suspect there's some low-hanging fruit in tweaking cgit's
implementation, where we might be able to do a better job of hitting
near the tips of tags without having to drastically increase the number
of bitmapped commits. But I haven't looked closely.

-Peff