Re: Achieving efficient storage of weirdly structured repos

From: Nicolas Pitre <nico@cam.org>
Date: Apr 7, 2008, 00:36 UTC
Message-ID: <alpine.LFD.1.00.0804062028260.2947@xanadu.home>
In-Reply-To: <20080407001833.GA16558@sigill.intra.peff.net>

On Sun, 6 Apr 2008, Jeff King wrote:

Show 12 quoted lines

> On Sun, Apr 06, 2008 at 08:13:10PM -0400, Nicolas Pitre wrote:
> 
> > Well, in your example, the large image part should already be common to 
> > many objects due to deltas if they're really the same: different objects 
> > will only have different EXIF data plus a delta reference to the same 
> > base image object. So in a way the split is already there.  Needs only 
> > that some applications exploit this information at runtime.
> 
> Yes, the resulting packfiles find the deltas and are pretty efficient
> (although it is quite slow to pack).  However, the delta information is
> not used at all for inexact rename detection. Are you proposing to make
> that information available to the rename detector?

In practice I don't know how well that would work since the current heuristic groups deltas and their base according to the name under which those objects are known. So it is possible that some inexact renames end up creating objects that currently never delta against each other even if that would be the right thing to do.

But in some cases, that might be beneficial to look at the delta object themselves when diffing files as the delta might already contain the information telling the upper layer that file A and B are in fact 90% the same and that they differ from offset X to Y only.

Nicolas