Re: Weird shallow-tree conversion state, and branches of shallow trees

On Sun, 15 Apr 2007, Robin H. Johnson wrote:

Well, in git, you can actyally just take the tree entry for that subdirectory, and it already is cryptographic proof that two subdirectories match.

(It's not signed, but if you actually want to sign it, you can do so, either inside git - by using a tag object that points to that subdirectory - or outside git by just creating a Manifest that contains a list of subdirectories and their tree SHA1's, and signing that).

In fact, in git, there's an explicit command to generate that "Manifest of directories in the top level", and it's called

	git ls-tree HEAD

and it will give you cryptographic hashes of each file/directory in the top level of a repository. So just sign that, ie do

	git ls-tree HEAD > Manifest
	gpg -sa -u "$username" Manifest 

or something like that. And you're done. Add the "-r" flag to get the recursive manifest containing *all* files, rather than just the SHA1's of the directories themselves.

Of course, you could just sign and tag the HEAD itself, which is what the kernel does, since one signature will guarantee everything under it.

We do that for the daily (or rather, nightly) snapshots for the kernel. There's no "Manifest", but look at

	http://www.kernel.org/pub/linux/kernel/v2.6/snapshots/

and you'll see files like

	patch-2.6.21-rc6-git8.bz2       15-Apr-2007 07:01   38K 	 
	patch-2.6.21-rc6-git8.bz2.sign  15-Apr-2007 07:01  248   
	patch-2.6.21-rc6-git8.gz        15-Apr-2007 07:01   42K  
	patch-2.6.21-rc6-git8.gz.sign   15-Apr-2007 07:01  248   
	patch-2.6.21-rc6-git8.id        15-Apr-2007 07:01   41   
	patch-2.6.21-rc6-git8.log       15-Apr-2007 07:01   63K  
	patch-2.6.21-rc6-git8.sign      15-Apr-2007 07:01  248  

where only the patches are signed, but the system *could* have signed the ID file too (the 41-byte "patch-2.6.21-rc6-git8.id" contains the 40-byte HEX representation of the SHA of the HEAD of the snapshot, and a newline).

That 41-byte ID file really is sufficient to describe the whole thing, after all (although you then need to have the git tree in question to actually get the list of files, aka the "Manifest", so if you want that list, you'd have to do the "git ls-tree" thing.

Sure. And every single Manifest file is pointless *inside* git, since git maintains its own cryptographically secure manifest file anyway. But it's trivial to generate them for external use, if you want to.

That sounds like the nightly snapshots the kernel does, except we only do them nightly, and we don't actually validate anythign at all, we just sign things as being from the "master.kernel.org" site (so the signature does mean something, but only that *that* site thinks it is valid).

If you do a single 41-byte thing, you could use git itself to validate the whole tree. But if you want to have people able to validate any random single file in a tar-file without having git installed, you'd have to:

 - have the "full manifest" (aka "git ls-tree -r HEAD")

 - have a trivial script that generates "git ID's" of files, which looks 
   something like this:

	#!/bin/sh
	# generate a "git ID" for one or more files
	while test -n "$1"
	do
		file="$1"
		len=$(stat --format "%s" "$file")
		echo -n " $file (blob $len): "
		# Generate the "git ID" for a blob:
		( echo -e -n "blob $len\0" ; cat "$file") | sha1sum
		shift
	done

and now you can check each file in the Manifest even without having git installed.

			Linus