From: Jeff Hostetler Date: Fri, 24 Mar 2017 15:46:14 GMT Subject: Re: [PATCH] read-cache: call verify_hdr() in a background thread Message-ID: <1743d048-8d6b-c14e-161b-351e6d79fb95@jeffhostetler.com> In-Reply-To: <20170324153617.smmzju5mygqt5vux@sigill.intra.peff.net> On 3/24/2017 11:36 AM, Jeff King wrote: > On Fri, Mar 24, 2017 at 01:27:51PM +0000, git@jeffhostetler.com wrote: > >> From: Jeff Hostetler >> >> Teash do_read_index() in read-cache.c to call verify_hdr() >> in a background thread while the forground thread parses >> the index and builds the_index. >> >> This is a performance optimization to reduce the overall >> time required to get the index into memory. >> >> Testing on Windows (using the OpenSSL SHA1 routine) showed >> that parsing the index and computing the SHA1 take almost >> equal time, so this patch effectively reduces the startup >> time by 1/2. > > Have you considered just skipping the sha1 check on read (possibly with > an option)? > > Its purpose is to detect disk bit-rot. Checking it for every single > operation may be a bit excessive, especially because it gets rewritten a > lot. Nobody really cared until now because they don't have index files > that are hundreds of megabytes. Yes, we have done that in a version of the client customized for the virtual file system effort and have been using it internally without incident for quite a while. And we can get a patch for that instead (or include it with this one) if there's interest. I tried to limit my proposal here to increasing performance while preserving the existing behavior. Jeff