Re: [RFC PATCH] index-pack: improve performance on NFS

From: Jeff King <peff@peff.net>
Date: Oct 29, 2018, 15:20 UTC
Message-ID: <20181029152032.GK17668@sigill.intra.peff.net>
In-Reply-To: <xmqqbm7da88t.fsf@gitster-ct.c.googlers.com>

On Mon, Oct 29, 2018 at 09:48:02AM +0900, Junio C Hamano wrote:

Show 16 quoted lines

> > Of course any cache raises questions of cache invalidation, but I think
> > we've already dealt with that for this case. When we use
> > OBJECT_INFO_QUICK, that is a sign that we want to make this kind of
> > accuracy/speed tradeoff (which does a similar caching thing with
> > packfiles).
> >
> > So putting that all together, could we have something like:
> 
> I think this conceptually is a vast improvement relative to
> ".cloning" optimization.  Obviously this does not have the huge
> downside of the other approach that turns the collision detection
> completely off.
> 
> A real question is how much performance gain, relative to ".cloning"
> thing, this approach gives us.  If it gives us 80% or more of the
> gain compared to doing no checking, I'd say we have a clear winner.

My test runs showed it improving index-pack by about 3%, versus 4% for no collision checking at all. But there was easily 1% of noise. And much more importantly, that was on a Linux system on ext4, where stat is fast. I'd be much more curious to hear timing results from people on macOS or Windows, or from Geert's original NFS case.

-Peff