Re: Request for detailed documentation of git pack protocol

From: Nicolas Pitre <nico@cam.org>
Date: May 14, 2009, 18:13 UTC
Message-ID: <alpine.LFD.2.00.0905141353040.6741@xanadu.home>
In-Reply-To: <200905141024.17525.jnareb@gmail.com>

On Thu, 14 May 2009, Jakub Narebski wrote:

Show 5 quoted lines

> I was afraid of this: that the people who know pack protocol good
> enough to be able to write it down are otherwise busy. But we get
> detailed / updated packfile and pack index format descriptions some
> time ago (thanks all that contributed to it!). I hope that the same
> would happen with pack _protocol_ description.

If someone with the wish for such a document volunteers to work on it then I'm sure people with fuller knowledge will review and comment on the result as appropriate.

> I was hoping of document in RFC format; dreaming about having it
> submitted to IETF as (at least) unofficial RFC like Atom Publication
> Protocol (or is it proper RFC these days?), and then accepted like
> HTTP protocol.

I think we'd have to move to a new version of the protocol for that. The current protocol, even if it does the job, is not particularly elegant.

Show 7 quoted lines

> > And lets not even start to mention Dulwich not completing a thin
> > pack before storing it on disk.  Those sorts of on disk things
> > matter to other more popular Git implementations (c git, jgit).
> 
> Ugh! Errr... aren't thin packs send only if other side has the
> capability for it? What is then Dulwich doing announcing such 
> capability when not supporting it correctly...

They probably don't bother because in theory you don't need to complete a thin pack for the system to still work. We require that any pack never contain a delta which base object is in a different pack because that makes for better performances when accessing the pack and when repacking. And not doing so makes pack validation (think verify-pack) impossible without the dependent objects, and that makes incremental repacking much much harder wrt prevention of delta cycles.

Those validation tools from C git (fsck, verify-pack, etc.) should be quite useful for people wishing to implement their own git.

Nicolas