Re: If I were redoing git from scratch...

From: Linus Torvalds <torvalds@osdl.org>
Date: Nov 4, 2006, 23:15 UTC
Message-ID: <Pine.LNX.4.64.0611041509370.25218@g5.osdl.org>
In-Reply-To: <Pine.LNX.4.64.0611041436050.25218@g5.osdl.org>

On Sat, 4 Nov 2006, Linus Torvalds wrote:

Show 5 quoted lines

> 
> In addition to that, we need one pointer per hash entry, and in order to 
> keep the hash list size down we need that hash array to be about 25% free, 
> so say 1.5 pointers per object: ~6 bytes or ~12 bytes depending on whether 
> it's a 32- or 64-bit architecture.

Btw, one of the things I considered (but rejected as being _too_ far out for now) during the memory shrinking thing was to make both 32-bit and 64-bit entities use a 32-bit hash table entry.

The way to do that would be to instead of using a pointer, use a 32-bit integer where the low ~10 bits are an index into the allocation buffer (since we batch allocations), and the rest of the bits would be an index into which batch-buffer it is.

Exactly because 8 bytes per hash entry is actually right now a big part of the object memory allocation overhead on 64-bit architectures, and cutting it down to just 4 bytes would help save memory.

I never got around to it, if only because I actually just compile my user-land git stuff as 32-bit, even on my ppc64 system. And partly because I had shrunk the object allocations enough that I just felt pretty happy with it anyway, and the change would have been pretty painful. But on 64-bit architectures, the hash table right now is about a third of the whole memory overhead of the object database, and cutting it down by half would actually be noticeable.