git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: SHA1 collisions found

From
Jeff King <peff@peff.net>
Date
Mar 3, 2017, 11:13 UTC
Message-ID
<20170303111347.6uzuhvmpdwr27qjw@sigill.intra.peff.net>
In-Reply-To
<20170302214610.GA86054@google.com>
On Thu, Mar 02, 2017 at 01:46:10PM -0800, Brandon Williams wrote:
Show 5 quoted lines
> There were a few of us discussing this sort of approach internally.  We
> also figured that, given some performance hit, you could maintain your
> repo in sha256 and do some translation to sha1 if you need to push or
> fetch to a server which has the the repo in a sha1 format.  This way you
> can convert your repo independently of the rest of the world.

Yeah, you definitely _can_ convert between the two. It's just expensive to do so on the fly. We'd potentially want to be able to during the transition period just to help people get all the work converted over. But I had assumed that the conversion would be a mix of:

  1. Unpublished work (or work which is otherwise OK to be rewritten)
     could be converted to the new hash.
  2. Old history could be grafted with a parent pointer that mentions
     the tip of the old history by its new hash, but the pointed-to
     parent contains sha1s.
> As for storing the translation table, you should really only need to
> maintain the table until old clients are phased out and all of the repos
> of a project have experienced flag day and have been converted to
> sha256.

I think you've read more into my "conversion" than I intended. The old history won't get rewritten. It will just be grafted onto the bottom of the commit history you've got, and the new trees will all be written with the new hash.

So you still have those old objects hanging around that refer to things by their sha1 (not to mention bug trackers, commit messages, etc, which all use commit ids). And you want to be able to quickly resolve those references.

What _does_ get rewritten is what's in your ref files, your pack .idx, etc. Those are all sha256 (or whatever), and work as sha1's do now. Looking up a sha1 reference from an old object just goes through the extra level of indirection.

-Peff
Previous: Brandon WilliamsNext: Ian Jackson
Message 70 of 136 in “SHA1 collisions found”
  1. Joey HessFeb 23, 2017
  2. Junio C HamanoFeb 23, 2017
  3. Junio C HamanoFeb 23, 2017
  4. David LangFeb 23, 2017
  5. Jakub NarębskiFeb 23, 2017
  6. Jeff KingFeb 23, 2017
  7. Joey HessFeb 23, 2017
  8. Linus TorvaldsFeb 23, 2017
  9. Joey HessFeb 23, 2017
  10. Linus TorvaldsFeb 23, 2017
  11. Jeff KingFeb 23, 2017
  12. Linus TorvaldsFeb 23, 2017
  13. Jeff KingFeb 23, 2017
  14. Linus TorvaldsFeb 23, 2017
  15. Jeff KingFeb 23, 2017
  16. Øyvind A. HolmFeb 23, 2017
  17. Joey HessFeb 23, 2017
  18. Joey HessFeb 23, 2017
  19. Morten WelinderFeb 23, 2017
  20. Geert UytterhoevenFeb 24, 2017
  21. Jeff KingFeb 23, 2017
  22. David LangFeb 23, 2017
  23. David LangFeb 23, 2017
  24. David LangFeb 23, 2017
  25. Linus TorvaldsFeb 23, 2017
  26. Linus TorvaldsFeb 23, 2017
  27. Joey HessFeb 23, 2017
  28. Linus TorvaldsFeb 23, 2017
  29. Junio C HamanoFeb 23, 2017
  30. Duy NguyenFeb 24, 2017
  31. brian m. carlsonFeb 25, 2017
  32. René ScharfeFeb 27, 2017
  33. brian m. carlsonFeb 28, 2017
  34. Ian JacksonFeb 24, 2017
  35. ankostisFeb 24, 2017
  36. Junio C HamanoFeb 24, 2017
  37. David LangFeb 24, 2017
  38. Junio C HamanoFeb 24, 2017
  39. Stefan BellerFeb 24, 2017
  40. Junio C HamanoFeb 24, 2017
  41. Junio C HamanoFeb 24, 2017
  42. ankostisFeb 24, 2017
  43. Junio C HamanoFeb 24, 2017
  44. ankostisFeb 25, 2017
  45. Jason CooperFeb 26, 2017
  46. brian m. carlsonFeb 26, 2017
  47. Linus TorvaldsFeb 26, 2017
  48. Ævar Arnfjörð BjarmasonFeb 26, 2017
  49. Jeff KingFeb 26, 2017
  50. Transition plan for git to move to a new hash functionIan Jackson, Feb 27, 2017
  51. Markus TrippelsdorfFeb 27, 2017
  52. Ian JacksonFeb 27, 2017
  53. Tony FinchFeb 27, 2017
  54. brian m. carlsonFeb 28, 2017
  55. Ian JacksonMar 2, 2017
  56. brian m. carlsonMar 4, 2017
  57. Ian JacksonMar 5, 2017
  58. brian m. carlsonMar 5, 2017
  59. Philip OakleyFeb 24, 2017
  60. Jeff KingFeb 24, 2017
  61. Linus TorvaldsFeb 25, 2017
  62. Linus TorvaldsFeb 25, 2017
  63. Jeff KingFeb 25, 2017
  64. Junio C HamanoFeb 26, 2017
  65. Junio C HamanoFeb 25, 2017
  66. Jason CooperFeb 26, 2017
  67. Jeff KingFeb 26, 2017
  68. brian m. carlsonFeb 26, 2017
  69. Brandon WilliamsMar 2, 2017
  70. Jeff KingMar 3, 2017
  71. Ian JacksonMar 3, 2017
  72. Jeff KingMar 3, 2017
  73. Linus TorvaldsMar 2, 2017
  74. Junio C HamanoMar 2, 2017
  75. Linus TorvaldsMar 2, 2017
  76. Joey HessMar 2, 2017
  77. Linus TorvaldsMar 2, 2017
  78. Mike HommeyMar 3, 2017
  79. Linus TorvaldsMar 3, 2017
  80. Jeff KingMar 3, 2017
  81. Stefan BellerMar 3, 2017
  82. David LangFeb 25, 2017
  83. Stefan BellerFeb 25, 2017
  84. Jeff KingFeb 25, 2017
  85. David LangFeb 25, 2017
  86. Jeff KingFeb 25, 2017
  87. David LangFeb 25, 2017
  88. Jacob KellerFeb 25, 2017
  89. Jacob KellerFeb 25, 2017
  90. grarpampFeb 25, 2017
  91. Ian JacksonFeb 24, 2017
  92. Ian JacksonFeb 25, 2017
  93. brian m. carlsonFeb 25, 2017
  94. Jeff KingFeb 25, 2017
  95. Mike HommeyFeb 25, 2017
  96. brian m. carlsonFeb 26, 2017
  97. Jason CooperFeb 24, 2017
  98. ankostisFeb 25, 2017
  99. Jakub NarębskiFeb 24, 2017
  100. Santiago TorresFeb 24, 2017
  101. Jakub NarębskiFeb 24, 2017
  102. Øyvind A. HolmFeb 24, 2017
  103. Jeff KingFeb 24, 2017
  104. Jakub NarębskiFeb 24, 2017
  105. Lars SchneiderFeb 25, 2017
  106. Jeff KingFeb 26, 2017
  107. Junio C HamanoFeb 26, 2017
  108. Thomas BraunFeb 26, 2017
  109. Jeff KingFeb 26, 2017
  110. Geert UytterhoevenFeb 27, 2017
  111. Jeff KingFeb 27, 2017
  112. Morten WelinderFeb 27, 2017
  113. Jeff KingFeb 23, 2017
  114. Linus TorvaldsFeb 23, 2017
  115. Jeff KingFeb 23, 2017
  116. 1/3 add collision-detecting sha1 implementationJeff King, Feb 23, 2017
  117. Stefan BellerFeb 23, 2017
  118. Jeff KingFeb 24, 2017
  119. Linus TorvaldsFeb 24, 2017
  120. Jeff KingFeb 24, 2017
  121. 2/3 sha1dc: adjust header includes for gitJeff King, Feb 23, 2017
  122. 3/3 Makefile: add USE_SHA1DC knobJeff King, Feb 23, 2017
  123. HW42Feb 24, 2017
  124. Jeff KingFeb 24, 2017
  125. Linus TorvaldsFeb 23, 2017
  126. Junio C HamanoFeb 28, 2017
  127. Junio C HamanoFeb 28, 2017
  128. Jeff KingFeb 28, 2017
  129. Dan ShumowMar 1, 2017
  130. Linus TorvaldsFeb 28, 2017
  131. Shawn PearceFeb 28, 2017
  132. Linus TorvaldsFeb 28, 2017
  133. Dan ShumowFeb 28, 2017
  134. Marc StevensFeb 28, 2017
  135. Linus TorvaldsFeb 28, 2017
  136. Jeff KingMar 1, 2017

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.