git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: space compression (again)

From
MUMartin Uecker <muecker@gmx.de>
Date
Apr 19, 2005, 12:39 UTC
Message-ID
<20050419123935.GA8091@macavity>
In-Reply-To
<20050416173702.GA12605@macavity>
On Sat, Apr 16, 2005 at 07:37:02PM +0200, Martin Uecker wrote:
> On Sat, Apr 16, 2005 at 11:11:00AM -0400, C. Scott Ananian wrote:
Show 9 quoted lines
> > The rsync approach does not use fixed chunk boundaries; this is necessary 
> > to ensure good storage reuse for the expected case (ie; inserting a single 
> > line at the start or in the middle of the file, which changes all the 
> > chunk boundaries).
> 
> Yes. The chunk boundaries should be determined deterministically
> from local properties of the data. Use a rolling checksum over
> some small window and split the file it it hits a special value (0).
> This is what the rsyncable patch to zlib does.

This is certainly uninteresting for source code repositories but for people who manage repositories of rsyncable binary packages this would save a lot of space, bandwidth and cpu time (compared to rsync because the scanning phase is not necessary anymore). 

Martin
-- 
One night, when little Giana from Milano was fast asleep,
she had a strange dream.
Previous: Martin UeckerNext: Derek Fawcus
Message 9 of 10 in “space compression (again)”
  1. C. Scott AnanianApr 15, 2005
  2. Linus TorvaldsApr 15, 2005
  3. C. Scott AnanianApr 15, 2005
  4. Derek FawcusApr 15, 2005
  5. Linus TorvaldsApr 15, 2005
  6. Martin UeckerApr 16, 2005
  7. C. Scott AnanianApr 16, 2005
  8. Martin UeckerApr 16, 2005
  9. Martin UeckerApr 19, 2005
  10. Derek FawcusApr 15, 2005

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.