git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v4 2/4] core.fsync: introduce granular fsync control

From
Junio C Hamano <gitster@pobox.com>
Date
Mar 9, 2022, 20:03 UTC
Message-ID
<xmqqpmmuki68.fsf@gitster.g>
In-Reply-To
<YiiuqK/tCnQOXrSV@ncase>
Patrick Steinhardt <ps@pks.im> writes:
Show 12 quoted lines
>> If the user cares about fsynching loose object files in the right
>> way, we shouldn't leave loose ref files not following the safe
>> safety level, regardless of how this new core.fsync knobs would look
>> like.
>> 
>> I think we three are in agreement on that.
>
> Is there anything I can specifically do to help out with this topic? We
> have again hit data loss in production because we don't sync loose refs
> to disk before renaming them into place, so I'd really love to sort out
> this issue somehow so that I can revive my patch series which fixes the
> known repository corruption [1].

How about doing a series to unconditionally sync loose ref creation and modification?

Alternatively, we could link it to the existing configuration to control synching of object files.

I do not think core.fsyncObjectFiles having "object" in its name is a good reason not to think those who set it to true only care about the loose object files and nothing else. It is more sensible to consider that those who set it to true cares about the repository integrity more than those who set it to false, I would think.

But that (i.e. doing it conditionally and choose which knob to use) is one extra thing that needs justification, so starting from unconditional fsync_or_die() may be the best way to ease it in.

Previous: Ævar Arnfjörð BjarmasonNext: Patrick Steinhardt
Message 51 of 122 in “A design for future-proofing fsync() configuration”
  1. 0/2 A design for future-proofing fsync() configurationNeeraj K. Singh via GitGitGadget, Dec 4, 2021
  2. 1/2 fsync: add writeout-only mode for fsyncing repo dataNeeraj Singh via GitGitGadget, Dec 4, 2021
  3. Neeraj SinghDec 6, 2021
  4. 2/2 core.fsync: introduce granular fsync controlNeeraj Singh via GitGitGadget, Dec 4, 2021
  5. 0/3 A design for future-proofing fsync() configurationNeeraj K. Singh via GitGitGadget, Dec 7, 2021
  6. 1/3 core.fsyncmethod: add writeout-only modeNeeraj Singh via GitGitGadget, Dec 7, 2021
  7. Patrick SteinhardtDec 7, 2021
  8. Ævar Arnfjörð BjarmasonDec 7, 2021
  9. Neeraj SinghDec 7, 2021
  10. Ævar Arnfjörð BjarmasonDec 7, 2021
  11. Neeraj SinghDec 7, 2021
  12. 2/3 core.fsync: introduce granular fsync controlNeeraj Singh via GitGitGadget, Dec 7, 2021
  13. Patrick SteinhardtDec 7, 2021
  14. Neeraj SinghDec 7, 2021
  15. Ævar Arnfjörð BjarmasonDec 7, 2021
  16. Neeraj SinghDec 7, 2021
  17. Ævar Arnfjörð BjarmasonDec 8, 2021
  18. Neeraj SinghDec 9, 2021
  19. Junio C HamanoDec 9, 2021
  20. Ævar Arnfjörð BjarmasonDec 9, 2021
  21. Neeraj SinghDec 9, 2021
  22. Neeraj SinghJan 18, 2022
  23. Ævar Arnfjörð BjarmasonJan 19, 2022
  24. Ævar Arnfjörð BjarmasonJan 19, 2022
  25. Neeraj SinghJan 28, 2022
  26. 3/3 core.fsync: new option to harden the indexNeeraj Singh via GitGitGadget, Dec 7, 2021
  27. Patrick SteinhardtDec 7, 2021
  28. Neeraj SinghDec 8, 2021
  29. 0/4 A design for future-proofing fsync() configurationNeeraj K. Singh via GitGitGadget, Dec 9, 2021
  30. 1/4 core.fsyncmethod: add writeout-only modeNeeraj Singh via GitGitGadget, Dec 9, 2021
  31. 2/4 core.fsync: introduce granular fsync controlNeeraj Singh via GitGitGadget, Dec 9, 2021
  32. 3/4 core.fsync: new option to harden the indexNeeraj Singh via GitGitGadget, Dec 9, 2021
  33. 4/4 core.fsync: add a `derived-metadata` aggregate optionNeeraj Singh via GitGitGadget, Dec 9, 2021
  34. Neeraj SinghJan 8, 2022
  35. rsbecker@nexbridge.comJan 9, 2022
  36. Neeraj SinghJan 10, 2022
  37. 0/4 A design for future-proofing fsync() configurationNeeraj K. Singh via GitGitGadget, Feb 1, 2022
  38. 1/4 core.fsyncmethod: add writeout-only modeNeeraj Singh via GitGitGadget, Feb 1, 2022
  39. 2/4 core.fsync: introduce granular fsync controlNeeraj Singh via GitGitGadget, Feb 1, 2022
  40. Junio C HamanoFeb 2, 2022
  41. Junio C HamanoFeb 2, 2022
  42. Neeraj SinghFeb 11, 2022
  43. Junio C HamanoFeb 11, 2022
  44. Neeraj SinghFeb 11, 2022
  45. Junio C HamanoFeb 11, 2022
  46. rsbecker@nexbridge.comFeb 12, 2022
  47. Patrick SteinhardtFeb 14, 2022
  48. Junio C HamanoFeb 14, 2022
  49. Patrick SteinhardtMar 9, 2022
  50. Ævar Arnfjörð BjarmasonMar 9, 2022
  51. Junio C HamanoMar 9, 2022
  52. Patrick SteinhardtMar 10, 2022
  53. Junio C HamanoMar 10, 2022
  54. Neeraj SinghMar 9, 2022
  55. Neeraj SinghFeb 11, 2022
  56. 3/4 core.fsync: new option to harden the indexNeeraj Singh via GitGitGadget, Feb 1, 2022
  57. 4/4 core.fsync: add a `derived-metadata` aggregate optionNeeraj Singh via GitGitGadget, Feb 1, 2022
  58. 0/5 A design for future-proofing fsync() configurationNeeraj K. Singh via GitGitGadget, Mar 9, 2022
  59. 1/5 wrapper: move inclusion of CSPRNG headers the wrapper.c fileNeeraj Singh via GitGitGadget, Mar 9, 2022
  60. Junio C HamanoMar 9, 2022
  61. Neeraj SinghMar 10, 2022
  62. brian m. carlsonMar 10, 2022
  63. Neeraj SinghMar 10, 2022
  64. 2/5 core.fsyncmethod: add writeout-only modeNeeraj Singh via GitGitGadget, Mar 9, 2022
  65. Junio C HamanoMar 9, 2022
  66. 3/5 core.fsync: introduce granular fsync controlNeeraj Singh via GitGitGadget, Mar 9, 2022
  67. Junio C HamanoMar 10, 2022
  68. Neeraj SinghMar 10, 2022
  69. Junio C HamanoMar 10, 2022
  70. Neeraj SinghMar 10, 2022
  71. Junio C HamanoMar 10, 2022
  72. Junio C HamanoMar 10, 2022
  73. Neeraj SinghMar 10, 2022
  74. Junio C HamanoMar 10, 2022
  75. Johannes SchindelinMar 10, 2022
  76. Junio C HamanoMar 10, 2022
  77. 4/5 core.fsync: new option to harden the indexNeeraj Singh via GitGitGadget, Mar 9, 2022
  78. 5/5 core.fsync: documentation and user-friendly aggregate optionsNeeraj Singh via GitGitGadget, Mar 9, 2022
  79. Future-proofed syncing of refsPatrick Steinhardt, Mar 10, 2022
  80. 6/8 core.fsync: add `fsync_component()` wrapper which doesn't diePatrick Steinhardt, Mar 10, 2022
  81. Junio C HamanoMar 10, 2022
  82. Neeraj SinghMar 10, 2022
  83. 7/8 core.fsync: new option to harden loose referencesPatrick Steinhardt, Mar 10, 2022
  84. Junio C HamanoMar 10, 2022
  85. Neeraj SinghMar 10, 2022
  86. Neeraj SinghMar 10, 2022
  87. Junio C HamanoMar 11, 2022
  88. Patrick SteinhardtMar 11, 2022
  89. Ævar Arnfjörð BjarmasonMar 11, 2022
  90. 8/8 core.fsync: new option to harden packed referencesPatrick Steinhardt, Mar 10, 2022
  91. Junio C HamanoMar 10, 2022
  92. Patrick SteinhardtMar 11, 2022
  93. 0/6 A design for future-proofing fsync() configurationNeeraj K. Singh via GitGitGadget, Mar 10, 2022
  94. 1/6 wrapper: make inclusion of Windows csprng header tightly scopedNeeraj Singh via GitGitGadget, Mar 10, 2022
  95. 3/6 core.fsync: introduce granular fsync control infrastructureNeeraj Singh via GitGitGadget, Mar 10, 2022
  96. 4/6 core.fsync: add configuration parsingNeeraj Singh via GitGitGadget, Mar 10, 2022
  97. Jiang XinMar 28, 2022
  98. Neeraj SinghMar 28, 2022
  99. 5/6 core.fsync: new option to harden the indexNeeraj Singh via GitGitGadget, Mar 10, 2022
  100. 2/6 core.fsyncmethod: add writeout-only modeNeeraj Singh via GitGitGadget, Mar 10, 2022
  101. 6/6 core.fsync: documentation and user-friendly aggregate optionsNeeraj Singh via GitGitGadget, Mar 10, 2022
  102. core.fsync: documentation and user-friendly aggregate optionsNeeraj Singh, Mar 15, 2022
  103. Junio C HamanoMar 15, 2022
  104. Neeraj SinghMar 15, 2022
  105. do we have too much fsync() configuration in 'next'? (was: [PATCH v7] core.fsync: documentation and user-friendly aggregate options)Ævar Arnfjörð Bjarmason, Mar 23, 2022
  106. Neeraj SinghMar 25, 2022
  107. Ævar Arnfjörð BjarmasonMar 26, 2022
  108. Junio C HamanoMar 26, 2022
  109. Neeraj SinghMar 26, 2022
  110. Ævar Arnfjörð BjarmasonMar 26, 2022
  111. Neeraj SinghMar 27, 2022
  112. Ævar Arnfjörð BjarmasonMar 27, 2022
  113. Patrick SteinhardtMar 28, 2022
  114. Ævar Arnfjörð BjarmasonMar 28, 2022
  115. Neeraj SinghMar 28, 2022
  116. Neeraj SinghMar 30, 2022
  117. Junio C HamanoMar 10, 2022
  118. Neeraj SinghMar 11, 2022
  119. Neeraj SinghMar 11, 2022
  120. Junio C HamanoMar 13, 2022
  121. core.fsync: new option to harden referencesPatrick Steinhardt, Mar 11, 2022
  122. SZEDER GáborMar 25, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.