Re: [PATCH] add the ability to create and retrieve delta objects

From: Chris Mason <mason@suse.com>
Date: May 3, 2005, 16:35 UTC
Message-ID: <200505031235.31984.mason@suse.com>
In-Reply-To: <Pine.LNX.4.61.0505031153550.32767@cag.csail.mit.edu>

On Tuesday 03 May 2005 11:57, C. Scott Ananian wrote:

Show 9 quoted lines

> On Tue, 3 May 2005, Chris Mason wrote:
> > your delta generator later this week.  Some quick and dirty space numbers
> > to show why we need to pack the files together:
>
> Are you accurately accounting for the cost of the extra hard/soft links
> your scheme requires?  Ie the directories get larger, lookups take
> slightly longer, etc.  Also access to a given file takes longer, and the
> deltas are referring to *other* packed files which *also* take longer to
> decompress and access...

My patch doesn't create any extra directory entries because the file for the packed file is unlinked after all the hard links are made. Even if I kept the packed file directory entry, I'd adding one directory entry and saving an average 6-7 inodes per commit.

>
> How much better does delta-fication do, compared to just packing?

The best case for just packing is to pack the blobs, trees and commits all into one object. Doing all three brought the tree down from 2.5GB to 1.57GB.

The delta patch does pack trees together, but not into the same file as the blobs, and commits are not packed at all. This is just because it is a pain to carry those changes around; it'll be easy to do later.

With the delta patch, the tree is around 900MB, I estimate packing the commits and trees into the blob files would save another 200MB.

Because space savings is so tightly coupled with packing ratios, a script to repack blobs, trees and commits from multiple commits will give much better compression. Right now the patch does not delta trees or commits, but it might make sense to delta the trees via the repacking script.

-chris