Re: Git and SHA-1 security (again)

From: brian m. carlson <sandals@crustytoothpaste.net>
Date: Jul 17, 2016, 15:42 UTC
Message-ID: <20160717154234.GC6644@vauxhall.crustytoothpaste.net>
In-Reply-To: <CACsJy8C+2=qv5Vu=tGeDTK_Q+XSAv3qEJw0nrHbEWU7psDf=Cg@mail.gmail.com>

On Sun, Jul 17, 2016 at 05:19:02PM +0200, Duy Nguyen wrote:

Show 19 quoted lines

> On Sun, Jul 17, 2016 at 4:21 PM, brian m. carlson
> <sandals@crustytoothpaste.net> wrote:
> > On Sun, Jul 17, 2016 at 10:01:38AM +0200, Johannes Schindelin wrote:
> >> Out of curiosity: have you considered something like padding the SHA-1s
> >> with, say 0xa1, to the size of the new hash and using that padding to
> >> distinguish between old vs new hash?
> >
> > I'm going to end up having to do something similar because of the issue
> > of submodules.  Submodules may still be SHA-1, while the main repo may
> > be a newer hash.  I was going to zero-pad, however.  I was also, at
> > least at first, going to force a separate .git dir for those, to avoid
> > having to try to store two separate types of objects in the same repo.
> 
> If it's just the external hash representation, can we go with a prefix
> <algo><colon> to identify the hash algorithm? For example
> sha256:1234... is SHA-256 while 1235... by default is SHA-1 (but we
> could switch the default to SHA-256 via config file later SHA-1 is
> dead and nobody wants to type sha256: every time). It catches
> incorrect hash algorithm references.

I'd make it such that the default is that of the repo. If the current repo is generating SHA-256, say, then 473a0f4 refers to the empty blob. If you want to refer to an SHA-1 object, then you write sha-1:e69de29.

On disk, multihash[0] seems like the right way to go. We'd serialize references to the SHA-1 and SHA-256 empty blobs as 1114e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 and 1220473a0f4c3be8a93681a267e3b1e9a7dcda1185436fe141f7749120a303721813 respectively. This makes parsing significantly easier. On disk, we could write them into the object database as 1114e6/9de2… and 122047/3a0f….

We could implement the default hash algorithm as extensions.hash and the on-disk format (which would be a requirement for extensions.hash) as extensions.explicitHash.

As I said, I'm not planning on multiple hash support at first, but it doesn't appear impossible if we go this route. We might still have to rewrite objects, but we can verify signatures over the legacy SHA-1 objects by forcing them into the old-style object format.

[0] https://github.com/jbenet/multihash

-- 
brian m. carlson / brian with sandals: Houston, Texas, US
+1 832 623 2791 | https://www.crustytoothpaste.net/~bmc | My opinion only
OpenPGP: https://keybase.io/bk2204