From: Derek Fawcus <dfawcus@cisco.com>
Date: Fri, 15 Apr 2005 19:00:54 GMT
Subject: Re: space compression (again)
Message-ID: <20050415200054.F6735@mrwint.cisco.com>
In-Reply-To: <Pine.LNX.4.61.0504151437100.27637@cag.csail.mit.edu>

On Fri, Apr 15, 2005 at 02:45:55PM -0400, C. Scott Ananian wrote:
> > - we already have wasted space due to the low-level filesystem (as
> >   opposed to "git") usually being block-based, which means that space
> >   utilization for small objects tends to suck. So you really want to
> >   prefer objects that are several kB (compressed), and a small block just
> >   wastes tons of space.
> 
> Not on (say) reiserfs, and not over the network.  I'm proposing (at the 
> moment) easy conversion from chunked to unchunked disk representation,
> so that you can leave things unchunked if (for example) you know you're 
> running ext2 with a large block size.

Or if one does not care about space,  and simply want's speed,  add another
layer of indirection - a flattened container object which has hashses as
normal,  then as it's content simply has the 'chunk list object' and the
'chunk objects' concatenated.

It's then a per user / database as to if the flattened objects,  or the
heirarcal objects are storred locally.

DF