Re: WARNING! Object DB conversion (was Re: [PATCH] write-tree performance problems)

From: Linus Torvalds <torvalds@osdl.org>
Date: Apr 20, 2005, 14:29 UTC
Message-ID: <Pine.LNX.4.58.0504200725110.6467@ppc970.osdl.org>
In-Reply-To: <2cfc4032050420050655265d3a@mail.gmail.com>

On Wed, 20 Apr 2005, Jon Seymour wrote:

Show 7 quoted lines

> 
> Am I correct to understand that with this change, all the objects in the 
> database are still being compressed (so no net performance benefit), but by 
> doing the SHA1 calculations before compression you are keeping open the 
> possibility that at some point in the future you may use a different 
> compression technique (including none at all) for some or all of the 
> objects?

Correct. There is zero performance benefit to this right now, and the only reason for doing it is because it will allow other things to happen.

Note that the other things include:
 - change the compression format to make it cheaper
 - _keep_ the same compression format, but notice that we already have an 
   object by looking at the uncompressed one.

I'm actually leaning towards just #2 at this time. I like how things compress, and it sure is simple. The fact that we use the equivalent of "-9" may be expensive, but the thing is, we don't actually write new files that often, and it's "just" CPU time (no seeking on disk or anything like that), which tends to get cheaper over time.

So I suspect that once I optimize the tree writing to notice that "oh, I already have this tree object", and thus build it up but never compressing it, "write-tree" performance will go up _hugely_ even without removing the compressioin. Because most of the time, write-tree actually only needs to create a couple of small new tree objects.

			Linus