Re: [PATCH] read-cache.c: fix index memory allocation

From: Nguyen Thai Ngoc Duy <pclouds@gmail.com>
Date: Oct 24, 2011, 23:34 UTC
Message-ID: <20111024233427.GA24956@duynguyen-vnpc.dektech.internal>
In-Reply-To: <4EA5DFB2.3050406@lsrfire.ath.cx>

On Mon, Oct 24, 2011 at 11:59:14PM +0200, René Scharfe wrote:

Show 15 quoted lines

> Am 24.10.2011 09:07, schrieb Junio C Hamano:
> > Thanks.
> > 
> > This approach may be the most appropriate for the maintenance track, but
> > for the purpose of going forward, I wonder if we really want to keep the
> > "estimate and allocate a large pool, and carve out individual pieces".
> > 
> > This bulk-allocate dates back to the days when we didn't have ondisk vs
> > incore representation differences, IIRC, and as the result we deliberately
> > leak cache entries whenever an entry in the index is replaced with a new
> > one. Does the overhead to allocate individually really kill us that much
> > for say a tree with 30k files in it?
> 
> Something like this (applies to master)?  Very basic testing didn't show
> any slowdown of git status in the Linux repo.

"git status" is slow. If your changes causes slowdown, it won't likely stand out while other fast commands may show (read_cache() is used in nearly all commands). So I tested using the following patch.

The result on linux-2.6 shows about 10-20 us slowdown per each read_cache() call (30-40 us on webkit, ~50k files) I think your patch is good enough :-)

-- 8< --

diff --git a/Makefile b/Makefile
index 3139c19..c8b47bc 100644
--- a/Makefile
+++ b/Makefile
@@ -444,6 +444,7 @@ TEST_PROGRAMS_NEED_X += test-string-pool
 TEST_PROGRAMS_NEED_X += test-subprocess
 TEST_PROGRAMS_NEED_X += test-svn-fe
 TEST_PROGRAMS_NEED_X += test-treap
+TEST_PROGRAMS_NEED_X += test-read-cache
 
 TEST_PROGRAMS = $(patsubst %,%$X,$(TEST_PROGRAMS_NEED_X))
 
diff --git a/test-read-cache.c b/test-read-cache.c
index e69de29..1ad80a0 100644
--- a/test-read-cache.c
+++ b/test-read-cache.c
@@ -0,0 +1,14 @@
+#include "cache.h"
+
+int main(int argc, char **argv)
+{
+	struct timeval tv1, tv2;
+	int i;
+
+	gettimeofday(&tv1, NULL);
+	for (i = 0; i < 100; i++)
+		read_cache();
+	gettimeofday(&tv2, NULL);
+	printf("%lu\n", (unsigned long)(tv2.tv_usec - tv1.tv_usec));
+	return 0;
+}
-- 8< --