Re: [PATCH] pack-objects: only throw away data during memory pressure

From: Johannes Schindelin <johannes.schindelin@gmx.de>
Date: Feb 11, 2008, 16:08 UTC
Message-ID: <alpine.LSU.1.00.0802111606440.3870@racer.site>
In-Reply-To: <alpine.LFD.1.00.0802111054080.2732@xanadu.home>

Hi,

On Mon, 11 Feb 2008, Nicolas Pitre wrote:

Show 26 quoted lines

> On Mon, 11 Feb 2008, Johannes Schindelin wrote:
> 
> > On Mon, 11 Feb 2008, Martin Koegler wrote:
> > 
> > > What about this not really tested patch for dealing with memory 
> > > pressure in git-pack-objects?
> > > 
> > > It will slow down the repack in the case of memory pressure, but 
> > > missing memory will not affect the results.
> > 
> > It almost helped:
> > 
> > $ /usr/bin/time git repack -a -d -f --window=250 --depth=250
> > Counting objects: 2477715, done.
> > fatal: Out of memory, malloc failed411764)
> > Command exited with non-zero status 1
> > 10050.12user 240.63system 2:53:37elapsed 98%CPU (0avgtext+0avgdata 
> > 0maxresident)k
> > 0inputs+0outputs (29555major+94032945minor)pagefaults 0swaps
> > 
> > So, it ran longer until it ran out of memory.
> 
> What it can do for you is to limit the window memory usage much more 
> without affecting the end result, say to 128MB.  Of course the repack is 
> then going to progress much slower if active purging of the window 
> memory is involved.

Oh, well. I did not think of setting a smaller windowMemory. Will try that next.

> If you still run out of memory at that point then there is not much more 
> to do besides using a new memory allocator that doesn't suffer as much 
> from memory fragmentation, or find the possible memory leak that no one 
> else found so far.

Completely forgot to say: I am running with ememoa (that is the Google allocator, right?).

Ciao, Dscho