Re: Diff format in packs

From: Jon Smirl <jonsmirl@gmail.com>
Date: Jul 31, 2006, 23:08 UTC
Message-ID: <9e4733910607311608v4938c940pe922257c13a149d2@mail.gmail.com>
In-Reply-To: <20060731223221.GB24888@spearce.org>

On 7/31/06, Shawn Pearce <spearce@spearce.org> wrote:

Show 5 quoted lines

> It should be reasonably simple to extract each revision from a
> single ,v file into its full undeltafied form, compute its SHA1,
> compress it with zlib, and append it into a pack file.  Do that
> for every file and toss the SHA1 values, file names and revision
> numbers off into a table somewhere.

cvs2svn has put everything needed into a nice db4 database that I can generate the trees from once I get the revisions into git. I'll modify cvs2svn to record the SHA1 for the deltas when they are originally scanned. The change set algorithm used by cvs2svn is the only one of the bunch that can process moz cvs without errors or losing check-ins. It takes 3.5hrs to generate the changeset db. There are over 1M deltas to sort and 250K commits.

Show 6 quoted lines

> So I think I'm suggesting you find a way to generate the base objects
> yourself right into a pack file, rather than using the higher level
> GIT executables to do it.  You may be able to reuse some of the
> code in GIT but I know its writer code is organized for writing
> loose objects, not for appending new objects into a new pack file,
> so some surgery would probably be required.

This makes sense, just put the objects into the pack and forget about deltas for now. I can add deltas in the next rewrite or just use git-repack. Since cvs2svn is in Python I need to figure out how to write a pack file from that environment. At least Python has sha1 and zlib support built-in.

What is more important is that I get a working incremental update functioning. I'm hoping that I can rely on cvsps for that. Cutting everyone over at once is unlikely to happen.

-- 
Jon Smirl
jonsmirl@gmail.com