Re: [PATCH] read-cache: call verify_hdr() in a background thread

From: Jeff King <peff@peff.net>
Date: Mar 24, 2017, 15:36 UTC
Message-ID: <20170324153617.smmzju5mygqt5vux@sigill.intra.peff.net>
In-Reply-To: <1490362071-46932-2-git-send-email-git@jeffhostetler.com>

On Fri, Mar 24, 2017 at 01:27:51PM +0000, git@jeffhostetler.com wrote:

Show 13 quoted lines

> From: Jeff Hostetler <jeffhost@microsoft.com>
> 
> Teash do_read_index() in read-cache.c to call verify_hdr()
> in a background thread while the forground thread parses
> the index and builds the_index.
> 
> This is a performance optimization to reduce the overall
> time required to get the index into memory.
> 
> Testing on Windows (using the OpenSSL SHA1 routine) showed
> that parsing the index and computing the SHA1 take almost
> equal time, so this patch effectively reduces the startup
> time by 1/2.

Have you considered just skipping the sha1 check on read (possibly with an option)?

Its purpose is to detect disk bit-rot. Checking it for every single operation may be a bit excessive, especially because it gets rewritten a lot. Nobody really cared until now because they don't have index files that are hundreds of megabytes.

-Peff