git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 0/4] hash.h: support choosing a separate SHA-1 for non-cryptographic uses

From
Taylor Blau <me@ttaylorr.com>
Date
Sep 5, 2024, 16:23 UTC
Message-ID
<ZtnbFXL7W5DvW8UN@nand.local>
In-Reply-To
<xmqq34me5crn.fsf@gitster.g>
On Thu, Sep 05, 2024 at 08:41:16AM -0700, Junio C Hamano wrote:
Show 20 quoted lines
> Jeff King <peff@peff.net> writes:
>
> > Probably the solution is:
> >
> >   - renaming packfiles into place should use finalize_object_file() to
> >     avoid collisions.  That happens for tmp-objdir migration already,
> >     but we should do it more directly in index-pack (and maybe even
> >     pack-objects). And possibly we should implement an actual
> >     byte-for-byte comparison if we think we saw a collision, rather than
> >     just assuming that the write was effectively a noopi (see the FIXME
> >     in that function). That becomes more important if the checksum gets
> >     more likely to collide accidentally (we essentially ignore the
> >     possibility that sha1 would ever do so).
>
> Yes.  I somehow mistakenly thought that Taylor analized the code
> path when brian raised the "we rename over, overwriting existing
> files" and I included fixing it as one of the steps necessary to
> safely switch the tail sum to weaker and faster hash, but after
> reading the thread again, it seems that I was hallucinating X-<.
> This needs to be corrected.

Just to make sure I understand you here... this is talking about a prerequisite step for switching index-pack to use a faster hash, right?

If so, I agree, but would note that this series does not yet switch index-pack to use the non-collision detecting SHA-1 implementation when available, so that would not be a prerequisite for merging this series.

Thanks, Taylor

Previous: Junio C HamanoNext: Junio C Hamano
Message 30 of 99 in “hash.h: support choosing a separate SHA-1 for non-cryptographic uses”
  1. 0/4 hash.h: support choosing a separate SHA-1 for non-cryptographic usesTaylor Blau, Sep 1, 2024
  2. 1/4 sha1: do not redefine `platform_SHA_CTX` and friendsTaylor Blau, Sep 1, 2024
  3. Patrick SteinhardtSep 2, 2024
  4. Taylor BlauSep 3, 2024
  5. 2/4 hash.h: scaffolding for _fast hashing variantsTaylor Blau, Sep 1, 2024
  6. Patrick SteinhardtSep 2, 2024
  7. Junio C HamanoSep 3, 2024
  8. Taylor BlauSep 3, 2024
  9. Junio C HamanoSep 3, 2024
  10. Taylor BlauSep 3, 2024
  11. Patrick SteinhardtSep 4, 2024
  12. Junio C HamanoSep 4, 2024
  13. Taylor BlauSep 3, 2024
  14. 3/4 Makefile: allow specifying a SHA-1 for non-cryptographic usesTaylor Blau, Sep 1, 2024
  15. Patrick SteinhardtSep 2, 2024
  16. Taylor BlauSep 3, 2024
  17. 4/4 csum-file.c: use fast SHA-1 implementation when availableTaylor Blau, Sep 1, 2024
  18. Patrick SteinhardtSep 2, 2024
  19. brian m. carlsonSep 3, 2024
  20. Taylor BlauSep 3, 2024
  21. Junio C HamanoSep 2, 2024
  22. Taylor BlauSep 3, 2024
  23. Junio C HamanoSep 3, 2024
  24. brian m. carlsonSep 2, 2024
  25. Taylor BlauSep 3, 2024
  26. Junio C HamanoSep 3, 2024
  27. brian m. carlsonSep 4, 2024
  28. Jeff KingSep 5, 2024
  29. Junio C HamanoSep 5, 2024
  30. Taylor BlauSep 5, 2024
  31. Junio C HamanoSep 5, 2024
  32. Taylor BlauSep 5, 2024
  33. Taylor BlauSep 5, 2024
  34. Taylor BlauSep 5, 2024
  35. Jeff KingSep 5, 2024
  36. Junio C HamanoSep 5, 2024
  37. 0/4 hash.h: support choosing a separate SHA-1 for non-cryptographic usesTaylor Blau, Sep 5, 2024
  38. 1/4 sha1: do not redefine `platform_SHA_CTX` and friendsTaylor Blau, Sep 5, 2024
  39. 2/4 hash.h: scaffolding for _fast hashing variantsTaylor Blau, Sep 5, 2024
  40. 3/4 Makefile: allow specifying a SHA-1 for non-cryptographic usesTaylor Blau, Sep 5, 2024
  41. 4/4 csum-file.c: use fast SHA-1 implementation when availableTaylor Blau, Sep 5, 2024
  42. 0/9 hash.h: support choosing a separate SHA-1 for non-cryptographic usesTaylor Blau, Sep 6, 2024
  43. 1/9 finalize_object_file(): check for name collision before renamingTaylor Blau, Sep 6, 2024
  44. 2/9 finalize_object_file(): refactor unlink_or_warn() placementTaylor Blau, Sep 6, 2024
  45. 3/9 finalize_object_file(): implement collision checkTaylor Blau, Sep 6, 2024
  46. Junio C HamanoSep 6, 2024
  47. Chris TorekSep 6, 2024
  48. Jeff KingSep 10, 2024
  49. Junio C HamanoSep 10, 2024
  50. Patrick SteinhardtSep 16, 2024
  51. Taylor BlauSep 16, 2024
  52. Taylor BlauSep 16, 2024
  53. Junio C HamanoSep 17, 2024
  54. 4/9 pack-objects: use finalize_object_file() to rename pack/idx/etcTaylor Blau, Sep 6, 2024
  55. 5/9 i5500-git-daemon.sh: use compile-able version of Git without OpenSSLTaylor Blau, Sep 6, 2024
  56. Jeff KingSep 11, 2024
  57. Jeff KingSep 11, 2024
  58. Junio C HamanoSep 12, 2024
  59. Junio C HamanoSep 11, 2024
  60. Jeff KingSep 11, 2024
  61. 6/9 sha1: do not redefine `platform_SHA_CTX` and friendsTaylor Blau, Sep 6, 2024
  62. 7/9 hash.h: scaffolding for _fast hashing variantsTaylor Blau, Sep 6, 2024
  63. 8/9 Makefile: allow specifying a SHA-1 for non-cryptographic usesTaylor Blau, Sep 6, 2024
  64. 9/9 csum-file.c: use fast SHA-1 implementation when availableTaylor Blau, Sep 6, 2024
  65. Junio C HamanoSep 6, 2024
  66. 0/8 hash.h: support choosing a separate SHA-1 for non-cryptographic usesTaylor Blau, Sep 24, 2024
  67. 1/8 finalize_object_file(): check for name collision before renamingTaylor Blau, Sep 24, 2024
  68. Junio C HamanoSep 25, 2024
  69. 2/8 finalize_object_file(): refactor unlink_or_warn() placementTaylor Blau, Sep 24, 2024
  70. 3/8 finalize_object_file(): implement collision checkTaylor Blau, Sep 24, 2024
  71. Jeff KingSep 24, 2024
  72. Taylor BlauSep 24, 2024
  73. Jeff KingSep 24, 2024
  74. Taylor BlauSep 25, 2024
  75. Junio C HamanoSep 24, 2024
  76. Taylor BlauSep 24, 2024
  77. 4/8 pack-objects: use finalize_object_file() to rename pack/idx/etcTaylor Blau, Sep 24, 2024
  78. Junio C HamanoSep 24, 2024
  79. 5/8 sha1: do not redefine `platform_SHA_CTX` and friendsTaylor Blau, Sep 24, 2024
  80. 6/8 hash.h: scaffolding for _unsafe hashing variantsTaylor Blau, Sep 24, 2024
  81. 7/8 Makefile: allow specifying a SHA-1 for non-cryptographic usesTaylor Blau, Sep 24, 2024
  82. 8/8 csum-file.c: use unsafe SHA-1 implementation when availableTaylor Blau, Sep 24, 2024
  83. Jeff KingSep 24, 2024
  84. Elijah NewrenSep 25, 2024
  85. Junio C HamanoSep 25, 2024
  86. Taylor BlauSep 25, 2024
  87. Taylor BlauSep 25, 2024
  88. 0/8 hash.h: support choosing a separate SHA-1 for non-cryptographic usesTaylor Blau, Sep 26, 2024
  89. 1/8 finalize_object_file(): check for name collision before renamingTaylor Blau, Sep 26, 2024
  90. 2/8 finalize_object_file(): refactor unlink_or_warn() placementTaylor Blau, Sep 26, 2024
  91. 3/8 finalize_object_file(): implement collision checkTaylor Blau, Sep 26, 2024
  92. 4/8 pack-objects: use finalize_object_file() to rename pack/idx/etcTaylor Blau, Sep 26, 2024
  93. 5/8 sha1: do not redefine `platform_SHA_CTX` and friendsTaylor Blau, Sep 26, 2024
  94. 6/8 hash.h: scaffolding for _unsafe hashing variantsTaylor Blau, Sep 26, 2024
  95. 7/8 Makefile: allow specifying a SHA-1 for non-cryptographic usesTaylor Blau, Sep 26, 2024
  96. 8/8 csum-file.c: use unsafe SHA-1 implementation when availableTaylor Blau, Sep 26, 2024
  97. Elijah NewrenSep 26, 2024
  98. Junio C HamanoSep 27, 2024
  99. Jeff KingSep 27, 2024

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.