git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: Which hash function to use, was Re: RFC: Another proposed hash function transition plan

From
BWBrandon Williams <bmwill@google.com>
Date
Jun 15, 2017, 17:36 UTC
Message-ID
<20170615173616.GA176947@google.com>
In-Reply-To
<alpine.DEB.2.21.1.1706151122180.4200@virtualbox>
On 06/15, Johannes Schindelin wrote:
Show 41 quoted lines
> Hi,
> 
> I thought it better to revive this old thread rather than start a new
> thread, so as to automatically reach everybody who chimed in originally.
> 
> On Mon, 6 Mar 2017, Brandon Williams wrote:
> 
> > On 03/06, brian m. carlson wrote:
> >
> > > On Sat, Mar 04, 2017 at 06:35:38PM -0800, Linus Torvalds wrote:
> > >
> > > > Btw, I do think the particular choice of hash should still be on the
> > > > table. sha-256 may be the obvious first choice, but there are
> > > > definitely a few reasons to consider alternatives, especially if
> > > > it's a complete switch-over like this.
> > > > 
> > > > One is large-file behavior - a parallel (or tree) mode could improve
> > > > on that noticeably. BLAKE2 does have special support for that, for
> > > > example. And SHA-256 does have known attacks compared to SHA-3-256
> > > > or BLAKE2 - whether that is due to age or due to more effort, I
> > > > can't really judge. But if we're switching away from SHA1 due to
> > > > known attacks, it does feel like we should be careful.
> > > 
> > > I agree with Linus on this.  SHA-256 is the slowest option, and it's
> > > the one with the most advanced cryptanalysis.  SHA-3-256 is faster on
> > > 64-bit machines (which, as we've seen on the list, is the overwhelming
> > > majority of machines using Git), and even BLAKE2b-256 is stronger.
> > > 
> > > Doing this all over again in another couple years should also be a
> > > non-goal.
> > 
> > I agree that when we decide to move to a new algorithm that we should
> > select one which we plan on using for as long as possible (much longer
> > than a couple years).  While writing the document we simply used
> > "sha256" because it was more tangible and easier to reference.
> 
> The SHA-1 transition *requires* a knob telling Git that the current
> repository uses a hash function different from SHA-1.
> 
> It would make *a whole of a lot of sense* to make that knob *not* Boolean,
> but to specify *which* hash function is in use.

100% agree on this point. I believe the current plan is to have the hashing function used for a repository be a repository format extension which would be a value (most likely a string like 'sha1', 'sha256', 'black2', etc) stored in a repository's .git/config. This way, upon startup git will die or ignore a repository which uses a hashing function which it does not recognize or does not compiled to handle.

I hope (and expect) that the end produce of this transition is a nice, clean hashing API and interface with sufficient abstractions such that if I wanted to switch to a different hashing function I would just need to implement the interface with the new hashing function and ensure that 'verify_repository_format' allows the new function.

Show 30 quoted lines
> 
> That way, it will be easier to switch another time when it becomes
> necessary.
> 
> And it will also make it easier for interested parties to use a different
> hash function in their infrastructure if they want.
> 
> And it lifts part of that burden that we have to consider *very carefully*
> which function to pick. We still should be more careful than in 2005, when
> Git was born, and when, incidentally, when the first attacks on SHA-1
> became known, of course. We were just lucky for almost 12 years.
> 
> Now, with Dunning-Kruger in mind, I feel that my degree in mathematics
> equips me with *just enough* competence to know just how little *even I*
> know about cryptography.
> 
> The smart thing to do, hence, was to get involved in this discussion and
> act as Lt Tawney Madison between us Git developers and experts in
> cryptography.
> 
> It just so happens that I work at a company with access to excellent
> cryptographers, and as we own the largest Git repository on the planet, we
> have a vested interest in ensuring Git's continued success.
> 
> After a couple of conversations with a couple of experts who I cannot
> thank enough for their time and patience, let alone their knowledge about
> this matter, it would appear that we may not have had a complete enough
> picture yet to even start to make the decision on the hash function to
> use.
> 
-- 
Brandon Williams
Previous: Jeff KingNext: Junio C Hamano
Message 25 of 113 in “RFC: Another proposed hash function transition plan”
  1. Jonathan NiederMar 4, 2017
  2. Linus TorvaldsMar 5, 2017
  3. brian m. carlsonMar 6, 2017
  4. Brandon WilliamsMar 6, 2017
  5. Which hash function to use, was Re: RFC: Another proposed hash function transition planJohannes Schindelin, Jun 15, 2017
  6. Mike HommeyJun 15, 2017
  7. Jeff KingJun 15, 2017
  8. Ævar Arnfjörð BjarmasonJun 15, 2017
  9. Johannes SchindelinJun 15, 2017
  10. Adam LangleyJun 15, 2017
  11. brian m. carlsonJun 15, 2017
  12. Ævar Arnfjörð BjarmasonJun 15, 2017
  13. brian m. carlsonJun 16, 2017
  14. Ævar Arnfjörð BjarmasonJun 16, 2017
  15. Johannes SchindelinJun 16, 2017
  16. Adam LangleyJun 16, 2017
  17. Junio C HamanoJun 16, 2017
  18. Junio C HamanoJun 16, 2017
  19. Jonathan NiederJun 16, 2017
  20. Ævar Arnfjörð BjarmasonJun 16, 2017
  21. Jeff KingJun 16, 2017
  22. Johannes SchindelinJun 19, 2017
  23. Mike HommeyJun 15, 2017
  24. Jeff KingJun 16, 2017
  25. Brandon WilliamsJun 15, 2017
  26. Junio C HamanoJun 15, 2017
  27. Jonathan NiederJun 15, 2017
  28. RFC v3: Another proposed hash function transition planJonathan Nieder, Mar 7, 2017
  29. Shawn PearceMar 9, 2017
  30. Jonathan NiederMar 9, 2017
  31. Jeff KingMar 10, 2017
  32. Jonathan NiederMar 10, 2017
  33. technical doc: add a design doc for hash function transitionJonathan Nieder, Sep 28, 2017
  34. Junio C HamanoSep 29, 2017
  35. Junio C HamanoSep 29, 2017
  36. Jonathan NiederSep 29, 2017
  37. Junio C HamanoOct 2, 2017
  38. Jason CooperOct 2, 2017
  39. Junio C HamanoOct 2, 2017
  40. Jason CooperOct 2, 2017
  41. Junio C HamanoOct 3, 2017
  42. Jason CooperOct 3, 2017
  43. Junio C HamanoOct 4, 2017
  44. Junio C HamanoSep 6, 2017
  45. Junio C HamanoSep 8, 2017
  46. Jeff KingSep 8, 2017
  47. Brandon WilliamsSep 11, 2017
  48. Johannes SchindelinSep 13, 2017
  49. demerphqSep 13, 2017
  50. Jonathan NiederSep 13, 2017
  51. Johannes SchindelinSep 14, 2017
  52. Jonathan NiederSep 14, 2017
  53. Johannes SchindelinSep 14, 2017
  54. Linus TorvaldsSep 13, 2017
  55. Johannes SchindelinSep 14, 2017
  56. Gilles Van AsscheSep 18, 2017
  57. Johannes SchindelinSep 18, 2017
  58. Gilles Van AsscheSep 19, 2017
  59. Johannes SchindelinSep 29, 2017
  60. Joan DaemenSep 29, 2017
  61. Johannes SchindelinSep 29, 2017
  62. Joan DaemenSep 30, 2017
  63. Johannes SchindelinOct 2, 2017
  64. Jonathan NiederSep 18, 2017
  65. Jason CooperSep 26, 2017
  66. Johannes SchindelinSep 26, 2017
  67. technical doc: add a design doc for hash function transitionStefan Beller, Sep 26, 2017
  68. Jonathan NiederSep 26, 2017
  69. Jonathan NiederSep 26, 2017
  70. Jason CooperOct 2, 2017
  71. Brandon WilliamsOct 2, 2017
  72. Jason CooperOct 2, 2017
  73. Linus TorvaldsOct 2, 2017
  74. Jeff KingOct 2, 2017
  75. Jonathan NiederSep 13, 2017
  76. Junio C HamanoSep 13, 2017
  77. Stefan BellerSep 13, 2017
  78. Jonathan NiederSep 13, 2017
  79. Junio C HamanoSep 14, 2017
  80. Johannes SchindelinSep 14, 2017
  81. demerphqSep 14, 2017
  82. Johannes SchindelinSep 14, 2017
  83. Junio C HamanoSep 13, 2017
  84. Jonathan NiederSep 13, 2017
  85. Junio C HamanoSep 14, 2017
  86. Johannes SchindelinSep 14, 2017
  87. Brandon WilliamsSep 14, 2017
  88. Jonathan NiederSep 14, 2017
  89. Philip OakleySep 15, 2017
  90. David LangMar 5, 2017
  91. Jonathan NiederMar 6, 2017
  92. Mike HommeyMar 7, 2017
  93. Jeff KingMar 6, 2017
  94. Junio C HamanoMar 6, 2017
  95. Jonathan TanMar 6, 2017
  96. Linus TorvaldsMar 6, 2017
  97. Brandon WilliamsMar 6, 2017
  98. Junio C HamanoMar 6, 2017
  99. Jeff KingMar 7, 2017
  100. Ian JacksonMar 7, 2017
  101. Linus TorvaldsMar 7, 2017
  102. Ian JacksonMar 8, 2017
  103. Johannes SchindelinMar 8, 2017
  104. Johannes SchindelinMar 8, 2017
  105. Use base32?Jason Hennessey, Mar 20, 2017
  106. Michael SteuerMar 20, 2017
  107. Jacob KellerMar 20, 2017
  108. Michael SteuerMar 21, 2017
  109. The Keccak TeamMar 13, 2017
  110. Jonathan NiederMar 13, 2017
  111. ankostisMar 13, 2017
  112. Johannes SchindelinMar 17, 2017
  113. Jeff KingMar 6, 2017

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.