From: david@lang.hm
Date: Fri, 07 Dec 2007 21:24:41 GMT
Subject: Re: RAM consumption when working with the gcc repo
Message-ID: <Pine.LNX.4.64.0712071323260.12607@asgard.lang.hm>
In-Reply-To: <9e4733910712071207p750c14f4h7abc5d637da3a478@mail.gmail.com>

On Fri, 7 Dec 2007, Jon Smirl wrote:

> I noticed two things when doing a repack of the gcc repo. First is
> that the git process is getting to be way too big. Turning off the
> delta caches had minimal impact. Why does the process still grow to
> 4.8GB?
>
> Putting this in perspective, this is a 4.8GB process constructing a
> 330MB file. Something isn't right. Memory leak or inefficient data
> structure?

keep in mind that that 330MB file is _very_ heavily compressed. the simple 
zlib compression is probably getting you 10:1 or 20:1 compression and the 
delta compression is a significant multiplier on top of that.

David Lang

> The second issue is that the repack process slows way down on the last
> 10% of the packing process. I don't believe this was caused by
> swapping since my disk light wasn't on. It takes a long to do the last
> 10% as it did for the first 70%. This seems to be correlated with the
> size of the process getting so large.
>
>