Re: Mozilla .git tree

From: Shawn Pearce <spearce@spearce.org>
Date: Sep 2, 2006, 17:51 UTC
Message-ID: <20060902175125.GC27826@spearce.org>
In-Reply-To: <7v3bba1qg5.fsf@assigned-by-dhcp.cox.net>

Junio C Hamano <junkio@cox.net> wrote:

Show 6 quoted lines

> > I think we probably should have done this when the binary headers
> > were introduced into loose objects.
> 
> No.  That was purely .pack format and did not affect .idx
> format.  Honestly .idx is purely a local matter and not as
> important to keep stable as the .pack format.

I think you missed the point of my comment. I apologize for it not being clear the first time.

The loose objects have format #1 (legacy text format compressed as one big zlib stream) and now format #2 (pack object header followed by data-only zlib stream).

In looking at this dictionary packing code that I'm working on right now I may be introducing format #3 (dictionary based compression rather than zlib compression).

We have no easy way to mark the version number of the loose objects.
I'm sure we can shoehorn in something though.

> > Sure the scheme you outlined allows a 64 bit difference but
> > uncompressed objects already can't be larger than 2**32-1...
> 
> Where do we have that limitation?

sha1_file.c uses "unsigned long" a lot to deal with size of an object, deflated. iirc unsigned long is only 32 bits even in many 64 bit architectures. Or am I wrong?

> I think your "allow zlib to eat the remainder of the current
> window and slide window when it exhausts the current window"
> logic is a very good on and makes it unnecessary to know the
> tail of each object.

I'll send patches to sha1_file.c probably later tonight. My last round of offset changes was the start of that, this next round should finish it. Its mostly a copy-n-paste from fast-import.c so it should go quickly.

-- 
Shawn.

-- 
VGER BF report: U 0.960524