Re: [PATCH] Wrap inflateInit to retry allocation after releasing pack memory

From: R. Tyler Ballance <tyler@slide.com>
Date: Jan 8, 2009, 18:15 UTC
Message-ID: <1231438552.8870.645.camel@starfruit>
In-Reply-To: <alpine.LFD.2.00.0901071941210.3283@localhost.localdomain>

On Wed, 2009-01-07 at 19:54 -0800, Linus Torvalds wrote:

Show 19 quoted lines

> 
> On Wed, 7 Jan 2009, Shawn O. Pearce wrote:
> >
> > If we are running low on virtual memory we should release pack
> > windows if zlib's inflateInit fails due to an out of memory error.
> > It may be that we are running under a low ulimit and are getting
> > tight on address space.  Shedding unused windows may get us
> > sufficient working space to continue.
> 
> Let's do this (more complete) wrapping instead, ok?
> 
> This one _just_ wraps things, btw - it doesn't do the "retry on low memory 
> error" part, at least not yet. I think that's an independent issue from 
> the reporting.
> 
> Hmm? 
> 
> Tyler - does this make the corruption errors go away, and be replaced by 
> hard failures with "out of memory" reporting?

Yeah, looks like it:

        tyler@grapefruit:~/source/git/linux-2.6> export
        START=56d18e9932ebf4e8eca42d2ce509450e6c9c1666
        tyler@grapefruit:~/source/git/linux-2.6> git reset --hard
        HEAD is now at 9e42d0c Merge
        git://git.kernel.org/pub/scm/linux/kernel/git/davem/sparc-2.6
        tyler@grapefruit:~/source/git/linux-2.6> git reset --hard $START
        HEAD is now at 56d18e9 Merge branch 'upstream' of
        git://ftp.linux-mips.org/pub/scm/upstream-linus
        tyler@grapefruit:~/source/git/linux-2.6> ulimit -v `echo "350 *
        1024" | bc -l`
        tyler@grapefruit:~/source/git/linux-2.6> limit
        cputime         unlimited
        filesize        unlimited
        datasize        unlimited
        stacksize       8MB
        coredumpsize    0kB
        memoryuse       2561MB
        maxproc         24564
        descriptors     1024
        memorylocked    64kB
        addressspace    350MB
        maxfilelocks    unlimited
        sigpending      24564
        msgqueue        819200
        nice            0
        rt_priority     0
        tyler@grapefruit:~/source/git/linux-2.6> git pull
        Updating 56d18e9..9e42d0c
        fatal: Out of memory? inflateInit failed
        tyler@grapefruit:~/source/git/linux-2.6> which git
        /home/tyler/bin/git
        tyler@grapefruit:~/source/git/linux-2.6>

> 
> This patch is potentially pretty noisy, on purpose. I didn't remove the 
> reporting from places that already do so - some of them have stricter 
> errors than this.

I'm assuming this patch is going to be reworked, if so, I'll back it out of our internal 1.6.1 build and anxiously await The Real Deal(tm)

Cheers

-- 
-R. Tyler Ballance
Slide, Inc.