Re: RFC v3: Another proposed hash function transition plan

From: Jonathan Nieder <jrnieder@gmail.com>
Date: Sep 13, 2017, 16:30 UTC
Message-ID: <20170913163052.GA27425@aiede.mtv.corp.google.com>
In-Reply-To: <alpine.DEB.2.21.1.1709131340030.4132@virtualbox>

Hi Dscho,

Johannes Schindelin wrote:

> So even if the code to generate a bidirectional old <-> new hash mapping
> might be with us forever, it *definitely* should be optional ("optional"
> at least as in "config setting"), allowing developers who only work with
> new-hash repositories to save the time and electrons.

Agreed. This is a good reason not to store the sha1 inside the sha256-encoded objects. I think that is exactly what Brandon was saying in response to Junio --- did you read it differently?

[...]

> ... or Git would simply handle the absence of the generation number header
> gracefully, so that sha1-content == sha3-content...

Part of the sha1-content is references to other objects using their sha1-name, so it is not possible to have sha1-content == sha3-content.

That said, I am also leaning against including generation numbers as part of this design.

There is an argument for including generation numbers. It is much simpler to have generation numbers in *all* commit objects than only in some, since it means the slop-based heuristics for faking generation numbers using commit timestamp can be completely avoided for a repository using such a format. Including generation numbers in all commit objects is a painless thing to do during a format change, since it can happen without harming round-tripping.

Treating generation numbers as derived data (as in Jeff King's preferred design, if I have understood his replies correctly) would also be possible but it does not interact well with shallow clone or narrow clone.

All that said, for simplicity I still lean against including generation numbers as part of a hash function transition. Nothing stops us from having another format change later.

This is a particularly hard decision because I don't have a strong preference. That leads me to err on the side of simplicity.

I will make sure to discuss this issue in my patch to Documentation/technical/, so we don't have to repeat the same conversations again and again.

[...]

Show 13 quoted lines

> Taking a step back, though, it may be a good idea to leave the generation
> number business for later, as much fun as it is to get side tracked and
> focus on relatively trivial stuff instead of the far more difficult and
> complex task to get the transition plan to a new hash ironed out.
>
> For example, I am still in favor of SHA-256 over SHA3-256, after learning
> some background details from in-house cryptographers: it provides
> essentially the same level of security, according to my sources, while
> hardware support seems to be coming to SHA-256 a lot sooner than to
> SHA3-256.
>
> Which hash algorithm to choose is a tough question to answer, and
> discussing generation numbers will sadly not help us answer it any quicker.

This is unrelated to Brandon's message, except for his use of SHA3 as a placeholder for "the next hash function".

My assumption based on previous conversations (and other external conversations like [1]) is that we are going to use SHA2-256 and have a pretty strong consensus for that. Don't worry!

As a side note, I am probably misreading, but I found this set of paragraphs a bit condescending. It sounds to me like you are saying "You are making the wrong choice of hash function and everything else you are describing is irrelevant when compared to that monumental mistake. Please stop working on things I don't consider important". With that reading it is quite demotivating to read.

An alternative reading is that you are saying that the transition plan described in this thread is not ironed out. Can you spell that out more? What particular aspect of the transition plan (which is of course orthogonal to the choice of hash function) are you discontent with?

Thanks and hope that helps, Jonathan

[1] https://www.imperialviolet.org/2017/05/31/skipsha3.html