# article: Using a rolling hash to break up binary files

2 messages from 2014-07-31 to 2014-08-01. Participants: Philip Oakley, Shawn Pearce.
Thread: https://gitlist.dev/t/37268

## Philip Oakley, 2014-07-31 22:31

Subject: article: Using a rolling hash to break up binary files
Message-ID: <D7F507613B1F45379A43C1BCF2F088B0@PhilipOakley>
URL: https://gitlist.dev/e/D7F507613B1F45379A43C1BCF2F088B0%40PhilipOakley

```
I thought it worth bring to the list's attention a  recent article on 
CodeProject that may be of interest to those looking at splitting binary 
files into deterministic hunks.

http://www.codeproject.com/Articles/801608/Using-a-rolling-hash-to-break-up-binary-files

It's based on Rabin and Karp's algorithm 
http://en.wikipedia.org/wiki/Rabin%E2%80%93Karp_algorithm.

--
Philip 

```

## Shawn Pearce, 2014-08-01 03:26

Subject: Re: article: Using a rolling hash to break up binary files
Message-ID: <CAJo=hJtQXb_0G0iqSdVWFwvWnCvowiL94S3UeG18BAkp2ih57Q@mail.gmail.com>
URL: https://gitlist.dev/e/CAJo%3DhJtQXb_0G0iqSdVWFwvWnCvowiL94S3UeG18BAkp2ih57Q%40mail.gmail.com
In-Reply-To: <D7F507613B1F45379A43C1BCF2F088B0@PhilipOakley>

```
On Thu, Jul 31, 2014 at 3:31 PM, Philip Oakley <philipoakley@iee.org> wrote:
> I thought it worth bring to the list's attention a  recent article on
> CodeProject that may be of interest to those looking at splitting binary
> files into deterministic hunks.
>
> http://www.codeproject.com/Articles/801608/Using-a-rolling-hash-to-break-up-binary-files
>
> It's based on Rabin and Karp's algorithm
> http://en.wikipedia.org/wiki/Rabin%E2%80%93Karp_algorithm.

If I remember right, this is how bup[1] works. Its certainly what we
do for delta compressing files.

[1] https://github.com/bup/bup

```