Re: Packfile can't be mapped

From: Shawn Pearce <spearce@spearce.org>
Date: Aug 28, 2006, 16:48 UTC
Message-ID: <20060828164855.GB22451@spearce.org>
In-Reply-To: <Pine.LNX.4.64.0608281026110.3683@localhost.localdomain>

Nicolas Pitre <nico@cam.org> wrote:

Show 10 quoted lines

> On Mon, 28 Aug 2006, Shawn Pearce wrote:
> > Of course the major problem with pack-local, stream global
> > dictionaries is it voids the ability to reuse that zlib'd content
> > from that pack in another pack without wholesale copying the
> > dictionary as well.  This is an issue for servers which want to
> > copy out the pack entry without recompressing it but also want the
> > storage savings from the global dictionaries.
> 
> Why would copying the dictionary as well be a problem?  How large might 
> it be?  Can it be stored deflated itself?

Largest size is like 200 bytes smaller than the window size. Times 3 as we would want to store 3 dictionaries, though maybe only 2 if the blob dictionary proves to be worse than not having one at all for a given project. Since its just a binary buffer holding bytes which frequently appear in our compressed objects its easily deflatable; especially when you consider its primarily storing US-ASCII text.

I was definately planning on the dictionary being deflated in the pack.

We could alawys use the SHA1 of a dictionary to signal to the client what dictionary was in use. If the dictionary itself was treated like any other SHA1 object then we might be able to transfer the server's current dictionaries to the client along with everything else in the same pack if the client doesn't have the necessary dictionary yet.

-- 
Shawn.