Re: [PATCH] pack-objects: only throw away data during memory pressure

From: Brian Downing <bdowning@lavos.net>
Date: Feb 12, 2008, 17:14 UTC
Message-ID: <20080212171403.GG27535@lavos.net>
In-Reply-To: <alpine.LFD.1.00.0802120910440.2732@xanadu.home>

On Tue, Feb 12, 2008 at 09:26:25AM -0500, Nicolas Pitre wrote:

Show 8 quoted lines

> I think your use case has merits, but the previous behavior had 
> semantics problems.  We always had constant window size with dynamic 
> memory usage, and now we have constant window size with bounded memory 
> usage.
> 
> If what you want is really to have a dynamic window size using a 
> constant memory usage then it needs a different and coherent way to be 
> specified.

Sometimes I want a bounded window size with bounded memory usage; i.e. a maximum of 50 entries OR 256 megs worth. That's for everyday repacking of my troublesome repository; without the window going down to less than 10 or so for the large files, it still takes way too long, but doing the whole thing at 10 makes for very poor packing.

So that gives four options:

1. No memory limit, constant entry depth.  Original Git behavior.
2. The above with an additional memory-based depth limit.  This is 
   what was added with --memory-limit.
3. Constant entry depth with a memory-usage limit.  This is what the
   proposed patch does.
4. Dynamic entry depth, with a memory-based limit.  This is I believe
   what you are proposing above, and what I emulate by setting
   --window=$bignum --window-memory=x.

I'm willing to try and make all of those work. (Though frankly I don't care much about #3; setting the window entry size to something "large enough" seems a simple enough work-around for me, and it prevents what's probably some truly ridiculous behavior if you have a gigantic number of tiny, say, tree objects. Having a cap on window depth stops that case from taking a truly inordinate amount of time.)

However, I can't figure out what sensible command-line and/or config parameters would be for the cases above. Any ideas?

-bcd