Re: Packfile can't be mapped

From: Nicolas Pitre <nico@cam.org>
Date: Aug 28, 2006, 16:43 UTC
Message-ID: <Pine.LNX.4.64.0608281236130.3683@localhost.localdomain>
In-Reply-To: <9e4733910608280844q41ea1a61r33c3eadc3e942604@mail.gmail.com>

On Mon, 28 Aug 2006, Jon Smirl wrote:

Show 17 quoted lines

> On 8/28/06, Nicolas Pitre <nico@cam.org> wrote:
> > > Of course the major problem with pack-local, stream global
> > > dictionaries is it voids the ability to reuse that zlib'd content
> > > from that pack in another pack without wholesale copying the
> > > dictionary as well.  This is an issue for servers which want to
> > > copy out the pack entry without recompressing it but also want the
> > > storage savings from the global dictionaries.
> >
> > Why would copying the dictionary as well be a problem?  How large might
> > it be?  Can it be stored deflated itself?
> 
> The dictionaries are only 4KB. But they have to match up with the
> objects compressed using them. If you bring an object straight down
> out of a dictionary based pack and make it standalone you won't be
> able to read it. You need the dictionary to decode it. What if the
> local and remote packs have been packed using two different
> dictionaries? You can't directly move objects between them.

I guess we'll be able to reuse the same dictionary when objects are all from the same pack.

If not then they could be recompressed which is costly but never as much as delta matching initially was when it was always done.

Anyway let's look at the size saving first. The implied costs could then be evaluated and weighted accordingly.

Nicolas