Re: Settings for minimizing repacking (and keeping 'rsync' happy)

From: Jeff King <peff@peff.net>
Date: Jul 29, 2019, 20:16 UTC
Message-ID: <20190729201610.GG14943@sigill.intra.peff.net>
In-Reply-To: <87tvb55799.fsf@evledraar.gmail.com>

On Mon, Jul 29, 2019 at 02:56:34PM +0200, Ævar Arnfjörð Bjarmason wrote:

Show 11 quoted lines

> The thread I started at
> https://public-inbox.org/git/87bmhiykvw.fsf@evledraar.gmail.com/ should
> also be of interest. I.e. we could have some knobs to create more
> "stable" packs, I know rsync does some in-file hashing, but I don't
> if/how that works if you have 1 file split into N where some chunks in
> the N are in the one file.
> 
> But it's possible to imagine a repacking algorithm that would keep
> producing entirely new packs but arrange for it to be ordered/delta'd in
> such a way that it optimizes for page-by-page similarity to an older
> pack to some degree.

I actually think that's the part that rsync does well. We don't keep page-by-page similarity, but rsync (and other tools like borg) are really good at finding the moved chunks. The problem is just that it doesn't know to compare chunks between two files with unrelated names.

-Peff