git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: article: Using a rolling hash to break up binary files

From
Shawn Pearce <spearce@spearce.org>
Date
Aug 1, 2014, 03:26 UTC
Message-ID
<CAJo=hJtQXb_0G0iqSdVWFwvWnCvowiL94S3UeG18BAkp2ih57Q@mail.gmail.com>
In-Reply-To
<D7F507613B1F45379A43C1BCF2F088B0@PhilipOakley>
On Thu, Jul 31, 2014 at 3:31 PM, Philip Oakley <philipoakley@iee.org> wrote:
Show 8 quoted lines
> I thought it worth bring to the list's attention a  recent article on
> CodeProject that may be of interest to those looking at splitting binary
> files into deterministic hunks.
>
> http://www.codeproject.com/Articles/801608/Using-a-rolling-hash-to-break-up-binary-files
>
> It's based on Rabin and Karp's algorithm
> http://en.wikipedia.org/wiki/Rabin%E2%80%93Karp_algorithm.

If I remember right, this is how bup[1] works. Its certainly what we do for delta compressing files.

[1] https://github.com/bup/bup
Previous: Philip Oakley
Message 2 of 2 in “article: Using a rolling hash to break up binary files”
  1. Philip OakleyJul 31, 2014
  2. Shawn PearceAug 1, 2014

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.