Re: pack operation is thrashing my server

From: Linus Torvalds <torvalds@linux-foundation.org>
Date: Aug 16, 2008, 00:34 UTC
Message-ID: <alpine.LFD.1.10.0808151729070.3324@nehalem.linux-foundation.org>
In-Reply-To: <alpine.LFD.1.10.0808141544150.3324@nehalem.linux-foundation.org>

On Thu, 14 Aug 2008, Linus Torvalds wrote:

> 
> So yeah, trees are the problem here, and yes, avoiding inflating them 
> would help - but mainly because we do it something like four times per 
> object on average!

Interestingly, it turns out that git also hits a sad performance downside of using zlib.

We always tend to set "stream.avail_out" to the exact size of the expected output. And it turns out that that means that the fast-path case of inffast.c doesn't trigger as often as it could. This (idiotic) patch actually seems to help performance on git rev-list by about 5%.

But maybe it's just me seeing things. But I did this because of the entry assumptions in inflate_fast(), that code only triggers for the case of strm->avail_out >= 258.

Sad, if true.

		Linus

---
 sha1_file.c |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/sha1_file.c b/sha1_file.c
index a57155d..5ca7ce2 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -1500,11 +1500,11 @@ static void *unpack_compressed_entry(struct packed_git *p,
 	z_stream stream;
 	unsigned char *buffer, *in;
 
-	buffer = xmalloc(size + 1);
+	buffer = xmalloc(size + 256 + 1);
 	buffer[size] = 0;
 	memset(&stream, 0, sizeof(stream));
 	stream.next_out = buffer;
-	stream.avail_out = size;
+	stream.avail_out = size + 256;
 
 	inflateInit(&stream);
 	do {