# Re: [RFC] adding support for md5 2 messages from 2006-08-19 to 2006-08-19. Participants: linux@horizon.com, Petr Baudis. Thread: https://gitlist.dev/t/5328 ## linux@horizon.com, 2006-08-19 03:19 Subject: Re: [RFC] adding support for md5 Message-ID: <20060819031931.486.qmail@science.horizon.com> URL: https://gitlist.dev/e/20060819031931.486.qmail%40science.horizon.com ``` This is a Very Dumb Idea. I normally try to be polite, but this concept is particularly deserving of scorn. The idea that more choice is a good thing is sometimes seductive, but when it comes to standards, that's not a good idea. It's like the famous joke told about the dumb inhabitants of your favorite ethnic region: they're going to try driving on the other side of the road. Starting next month, all the cars will drive on the other side. If that goes well, the month after they'll add trucks and buses. If some disaster arose that required Git to change hash functions, it would be possible to build a conversion utility, although all the signatures in tag objects would break; you'd have to regenerate them, too. But this is an all-or-nothing change. Trying to support *multiple* simultaneous hash functions is a mess. Git depends very fundamentally on identical objects having identical hashes. The in-memory merge depends on it. The network protocol depends on it. Various kludges can be imagined to handle blob objects with multiple names, but tree objects quickly become unworkable. Let's break down the solutions. There are basically four classes, depending on 1) whether objects are stored in the database indexed by both hashes, or just one, and 2) Whether pointers to objects include both hashes, or just one. If you include both hashes everywhere, then you've just built a larger hash function that's the concatenation of SHA-1 and MD5, and while it works sanely, it just makes the object IDs even bigger, and there are obviously no speed benefts. But this is the most reasonable alternative. If pointers include both hashes, but objects are indexed by only one, then to find an object by pointer requires two lookups, and you still need to hash every blob twice when committing get the values to put in the tree objects. So obviosuly no faster than the first option. Okay, so pointers are only one hash. If they were always the same hash, the second hash would be utterly pointless, so we're assuming the database contains a mix. If objects are indexed by both hashes, then you can hash new blobs once to check to see if they're already in the database, but if they're not, you have to hash again with the another algorithm. On the other hand, if you index by only one, then every object being checked to see if it's already in the database needs to be hashed twice so it can be looked up twice. What were the claimed speed gains? But more to the point, any system which stores one arbitrary hash as a pointer makes the the tree object created to describe a directory no longer unique, which results in its hash being fundamentally non-unique, which cascades all the way up to the commit object. So you can get silly things like the need for a merge commit to record the merge of trees that are actually identical. Just a big mess. ``` ## Petr Baudis, 2006-08-19 22:30 Subject: Re: [RFC] adding support for md5 Message-ID: <20060819223027.GA23891@pasky.or.cz> URL: https://gitlist.dev/e/20060819223027.GA23891%40pasky.or.cz In-Reply-To: <20060819031931.486.qmail@science.horizon.com> ``` Pre Scriptum: I do not advocate supporting many hashing functions just for the heck of it, but I don't like the "full sweep" approach _when_ (not "in case") we _need_ to change the hash function. After all, currently we have to operate with the fact that the next one will likely fall victim of attacks in time as well and this is going to be periodical operation. Dear diary, on Sat, Aug 19, 2006 at 05:19:31AM CEST, I got a letter where linux@horizon.com said that... > So you can get silly things like the need for a merge commit to > record the merge of trees that are actually identical. You can create an ordering of hashes based on their strength and then rehash your objects to max() of hash types used in the operation. So, if you're about to merge md5(x) and sha1(y), you first recompute the first id as sha1(x). Of course "recomputing" needs to be a wee bit more complex: you need to first substitute tree id for a common hash as well (and recursively do the same in the trees); slow, yes, but you can cache it. It gets worse when you are about to recompute ancestry, so you need to set up the hash usage transition so that you don't ever have to - such an event is sufficiently rare that each branch keeper can declare a flag day "before now I merged only md5 commits and from now on I will merge only sha1 commits". The tool can help to enforce this. This strategy will not help in 100% of the cases (md5git-based tool takes sha1git-created commit and re-creates it based on a snapshot and the precise same metadata) but I guess it's good enough. It's good that this is just theory so far, though. I'd say let it be for now (at least from the "emergency hash switch" standpoint) and do it when it's needed; you will have to have people upgrade their Git at that point anyway and the code would bitrot just sitting there and adding so-far unnecessary complexity. We have worse problems at our hands right now anyway. -- Petr "Pasky" Baudis Stuff: http://pasky.or.cz/ Snow falling on Perl. White noise covering line noise. Hides all the bugs too. -- J. Putnam ```