From: Martin Uecker <muecker@gmx.de>
Date: Tue, 19 Apr 2005 12:39:35 GMT
Subject: Re: space compression (again)
Message-ID: <20050419123935.GA8091@macavity>
In-Reply-To: <20050416173702.GA12605@macavity>

On Sat, Apr 16, 2005 at 07:37:02PM +0200, Martin Uecker wrote:
> On Sat, Apr 16, 2005 at 11:11:00AM -0400, C. Scott Ananian wrote:
 
> > The rsync approach does not use fixed chunk boundaries; this is necessary 
> > to ensure good storage reuse for the expected case (ie; inserting a single 
> > line at the start or in the middle of the file, which changes all the 
> > chunk boundaries).
> 
> Yes. The chunk boundaries should be determined deterministically
> from local properties of the data. Use a rolling checksum over
> some small window and split the file it it hits a special value (0).
> This is what the rsyncable patch to zlib does.

This is certainly uninteresting for source code repositories
but for people who manage repositories of rsyncable binary
packages this would save a lot of space, bandwidth and
cpu time (compared to rsync because the scanning phase is
not necessary anymore). 

Martin

-- 
One night, when little Giana from Milano was fast asleep,
she had a strange dream.