Re: Public repro case! Re: [PATCH/RFC] Allow writing loose objects that are corrupted in a pack file

From: Linus Torvalds <torvalds@linux-foundation.org>
Date: Jan 8, 2009, 01:08 UTC
Message-ID: <alpine.LFD.2.00.0901071702190.3283@localhost.localdomain>
In-Reply-To: <1231376259.8870.633.camel@starfruit>

On Wed, 7 Jan 2009, R. Tyler Ballance wrote:

Show 11 quoted lines

> > 
> > Can you cnfirm that your "reproducible" case starts working with that 
> > addition to your ~/.gitconfig? If so, the solution is pretty simple: we 
> > should just lower the default pack windowsize.
> 
> This certainly corrected the issue, is there some magic
> packedgitwindowsize that i should be looking at my own repository (our
> internal one) in order to prevent the issue from occurring? 
> 
> Looking into .git/objects/pack, I think the two biggest pack files are
> 3.5G and 177MBG respectively :-!

So there's a few rules to packedgitwindowsize:

 - we need to be able to have at least two windows open at a time, in 
   addition to all the "normal" memory git needs just for objects, of 
   course. And quite frankly, you'd be better off with a few more windows, 
  even if that obviously implies smaller windows.

 - the window size really wants to be a round power-of-two number, and at 
   _least_ it wants to be a nice multiple of the 2*page size.

So if you have a virtual memory limit of 1.5GB, I'd hesitate to make the pack window size less than 512M, and 256M is probably better. That way, I'd expect you to be able to always have at least four windows open (assuming a reasonably generous half a gigabyte for "other stuff").

And quite frankly, there's not a huge downside to making them smaller. At "just" 32MB, you'll still fit plenty of data in one pack window, and while it will cost you a few mmap/unmap's to switch windows around, most operations simply will not likely ever notice. At least not under Linux, where mmap/munmap is pretty cheap.

		Linus