From: Jeff Hostetler Date: Fri, 19 Feb 2021 19:12:42 GMT Subject: Re: [PATCH] read-cache: make the index write buffer size 128K Message-ID: In-Reply-To: On 2/17/21 9:48 PM, Neeraj K. Singh via GitGitGadget wrote: > From: Neeraj Singh > > Writing an index 8K at a time invokes the OS filesystem and caching code > very frequently, introducing noticeable overhead while writing large > indexes. When experimenting with different write buffer sizes on Windows > writing the Windows OS repo index (260MB), most of the benefit came by > bumping the index write buffer size to 64K. I picked 128K to ensure that > we're past the knee of the curve. > > With this change, the time under do_write_index for an index with 3M > files goes from ~1.02s to ~0.72s. [...] > > -#define WRITE_BUFFER_SIZE 8192 > +#define WRITE_BUFFER_SIZE (128 * 1024) > static unsigned char write_buffer[WRITE_BUFFER_SIZE]; > static unsigned long write_buffer_len; [...] Very nice. I can confirm that this gives nice gains on Windows. (I'm using the Office repo which has a 188MB index file (2.1M files at HEAD). Running "git status" shows a gain of about 200ms. We get a smaller gain on Mac of about 50ms (again, using the Office repo). So, you may add my sign-off or ACK to this. Signed-off-by: Jeff Hostetler FWIW, You might take a look at `t/perf/p0007-write-cache.sh` Update it as follows: ``` diff --git a/t/perf/p0007-write-cache.sh b/t/perf/p0007-write-cache.sh index 09595264f0..337280ff1c 100755 --- a/t/perf/p0007-write-cache.sh +++ b/t/perf/p0007-write-cache.sh @@ -4,7 +4,8 @@ test_description="Tests performance of writing the index" . ./perf-lib.sh -test_perf_default_repo +test_perf_large_repo test_expect_success "setup repo" ' if git rev-parse --verify refs/heads/p0006-ballast^{commit} ``` Then you can run it like this: $ cd t/perf $ GIT_PERF_LARGE_REPO=/path/to/your/enlistment ./p0007-write-cache Then you can run it with the small and then with the large buffer and get times for essentially just the index write in isolation. Hope this helps, Jeff