git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v5 5/5] core.fsync: documentation and user-friendly aggregate options

From
NGNeeraj Singh via GitGitGadget <gitgitgadget@gmail.com>
Date
Mar 9, 2022, 23:03 UTC
Message-ID
<2d71346b10e8beb3c44bdf8e06694e4dafc657d5.1646866998.git.gitgitgadget@gmail.com>
In-Reply-To
<pull.1093.v5.git.1646866998.gitgitgadget@gmail.com>
From: Neeraj Singh <neerajsi@microsoft.com>

This commit adds aggregate options for the core.fsync setting that are more user-friendly. These options are specified in terms of 'levels of safety', indicating which Git operations are considered to be sync points for durability.

The new documentation is also included here in its entirety for ease of review.

Signed-off-by: Neeraj Singh <neerajsi@microsoft.com>
---
 Documentation/config/core.txt | 36 +++++++++++++++++++++++++++++++++++
 cache.h                       | 23 +++++++++++++++++++---
 config.c                      |  6 ++++++
 3 files changed, 62 insertions(+), 3 deletions(-)
diff --git a/Documentation/config/core.txt b/Documentation/config/core.txt
index 74399072843..973805e8a98 100644
--- a/Documentation/config/core.txt
+++ b/Documentation/config/core.txt
@@ -547,6 +547,42 @@ core.whitespace::
   is relevant for `indent-with-non-tab` and when Git fixes `tab-in-indent`
   errors. The default tab width is 8. Allowed values are 1 to 63.
 
+core.fsync::
+	A comma-separated list of parts of the repository which should be
+	hardened via the core.fsyncMethod when created or modified. You can
+	disable hardening of any component by prefixing it with a '-'. Later
+	items take precedence over earlier ones in the comma-separated list.
+	For example, `core.fsync=all,-pack-metadata` means "harden everything
+	except pack metadata." Items that are not hardened may be lost in the
+	event of an unclean system shutdown. Unless you have special
+	requirements, it is recommended that you leave this option as default
+	or pick one of `committed`, `added`, or `all`.
++
+* `none` disables fsync completely. This value must be specified alone.
+* `loose-object` hardens objects added to the repo in loose-object form.
+* `pack` hardens objects added to the repo in packfile form.
+* `pack-metadata` hardens packfile bitmaps and indexes.
+* `commit-graph` hardens the commit graph file.
+* `index` hardens the index when it is modified.
+* `objects` is an aggregate option that is equivalent to
+  `loose-object,pack`.
+* `derived-metadata` is an aggregate option that is equivalent to
+  `pack-metadata,commit-graph`.
+* `default` is an aggregate option that is equivalent to
+  `objects,derived-metadata,-loose-object`. This mode is enabled by default.
+  It has good performance, but risks losing recent work if the system shuts
+  down uncleanly, since commits, trees, and blobs in loose-object form may be
+  lost.
+* `committed` is an aggregate option that is currently equivalent to
+  `objects`. This mode sacrifices some performance to ensure that all work
+  that is committed to the repository with `git commit` or similar commands
+  is preserved.
+* `added` is an aggregate option that is currently equivalent to
+  `committed,index`. This mode sacrifices additional performance to
+  ensure that the results of commands like `git add` and similar operations
+  are preserved.
+* `all` is an aggregate option that syncs all individual components above.
+
 core.fsyncMethod::
 	A value indicating the strategy Git will use to harden repository data
 	using fsync and related primitives.
diff --git a/cache.h b/cache.h
index 3fefe8f6f60..833f0236e68 100644
--- a/cache.h
+++ b/cache.h
@@ -1007,9 +1007,26 @@ enum fsync_component {
 	FSYNC_COMPONENT_INDEX			= 1 << 4,
 };
 
-#define FSYNC_COMPONENTS_DEFAULT (FSYNC_COMPONENT_PACK | \
-				  FSYNC_COMPONENT_PACK_METADATA | \
-				  FSYNC_COMPONENT_COMMIT_GRAPH)
+#define FSYNC_COMPONENTS_OBJECTS (FSYNC_COMPONENT_LOOSE_OBJECT | \
+				  FSYNC_COMPONENT_PACK)
+
+#define FSYNC_COMPONENTS_DERIVED_METADATA (FSYNC_COMPONENT_PACK_METADATA | \
+					   FSYNC_COMPONENT_COMMIT_GRAPH)
+
+#define FSYNC_COMPONENTS_DEFAULT (FSYNC_COMPONENTS_OBJECTS | \
+				  FSYNC_COMPONENTS_DERIVED_METADATA | \
+				  ~FSYNC_COMPONENT_LOOSE_OBJECT)
+
+#define FSYNC_COMPONENTS_COMMITTED (FSYNC_COMPONENTS_OBJECTS)
+
+#define FSYNC_COMPONENTS_ADDED (FSYNC_COMPONENTS_COMMITTED | \
+				FSYNC_COMPONENT_INDEX)
+
+#define FSYNC_COMPONENTS_ALL (FSYNC_COMPONENT_LOOSE_OBJECT | \
+			      FSYNC_COMPONENT_PACK | \
+			      FSYNC_COMPONENT_PACK_METADATA | \
+			      FSYNC_COMPONENT_COMMIT_GRAPH | \
+			      FSYNC_COMPONENT_INDEX)
 
 /*
  * A bitmask indicating which components of the repo should be fsynced.
diff --git a/config.c b/config.c
index bb44f6f506d..3976ec74fd4 100644
--- a/config.c
+++ b/config.c
@@ -1332,6 +1332,12 @@ static const struct fsync_component_entry {
 	{ "pack-metadata", FSYNC_COMPONENT_PACK_METADATA },
 	{ "commit-graph", FSYNC_COMPONENT_COMMIT_GRAPH },
 	{ "index", FSYNC_COMPONENT_INDEX },
+	{ "objects", FSYNC_COMPONENTS_OBJECTS },
+	{ "derived-metadata", FSYNC_COMPONENTS_DERIVED_METADATA },
+	{ "default", FSYNC_COMPONENTS_DEFAULT },
+	{ "committed", FSYNC_COMPONENTS_COMMITTED },
+	{ "added", FSYNC_COMPONENTS_ADDED },
+	{ "all", FSYNC_COMPONENTS_ALL },
 };
 
 static enum fsync_component parse_fsync_components(const char *var, const char *string)
-- 
gitgitgadget
Previous: Neeraj Singh via GitGitGadgetNext: Patrick Steinhardt
Message 78 of 122 in “A design for future-proofing fsync() configuration”
  1. 0/2 A design for future-proofing fsync() configurationNeeraj K. Singh via GitGitGadget, Dec 4, 2021
  2. 1/2 fsync: add writeout-only mode for fsyncing repo dataNeeraj Singh via GitGitGadget, Dec 4, 2021
  3. Neeraj SinghDec 6, 2021
  4. 2/2 core.fsync: introduce granular fsync controlNeeraj Singh via GitGitGadget, Dec 4, 2021
  5. 0/3 A design for future-proofing fsync() configurationNeeraj K. Singh via GitGitGadget, Dec 7, 2021
  6. 1/3 core.fsyncmethod: add writeout-only modeNeeraj Singh via GitGitGadget, Dec 7, 2021
  7. Patrick SteinhardtDec 7, 2021
  8. Ævar Arnfjörð BjarmasonDec 7, 2021
  9. Neeraj SinghDec 7, 2021
  10. Ævar Arnfjörð BjarmasonDec 7, 2021
  11. Neeraj SinghDec 7, 2021
  12. 2/3 core.fsync: introduce granular fsync controlNeeraj Singh via GitGitGadget, Dec 7, 2021
  13. Patrick SteinhardtDec 7, 2021
  14. Neeraj SinghDec 7, 2021
  15. Ævar Arnfjörð BjarmasonDec 7, 2021
  16. Neeraj SinghDec 7, 2021
  17. Ævar Arnfjörð BjarmasonDec 8, 2021
  18. Neeraj SinghDec 9, 2021
  19. Junio C HamanoDec 9, 2021
  20. Ævar Arnfjörð BjarmasonDec 9, 2021
  21. Neeraj SinghDec 9, 2021
  22. Neeraj SinghJan 18, 2022
  23. Ævar Arnfjörð BjarmasonJan 19, 2022
  24. Ævar Arnfjörð BjarmasonJan 19, 2022
  25. Neeraj SinghJan 28, 2022
  26. 3/3 core.fsync: new option to harden the indexNeeraj Singh via GitGitGadget, Dec 7, 2021
  27. Patrick SteinhardtDec 7, 2021
  28. Neeraj SinghDec 8, 2021
  29. 0/4 A design for future-proofing fsync() configurationNeeraj K. Singh via GitGitGadget, Dec 9, 2021
  30. 1/4 core.fsyncmethod: add writeout-only modeNeeraj Singh via GitGitGadget, Dec 9, 2021
  31. 2/4 core.fsync: introduce granular fsync controlNeeraj Singh via GitGitGadget, Dec 9, 2021
  32. 3/4 core.fsync: new option to harden the indexNeeraj Singh via GitGitGadget, Dec 9, 2021
  33. 4/4 core.fsync: add a `derived-metadata` aggregate optionNeeraj Singh via GitGitGadget, Dec 9, 2021
  34. Neeraj SinghJan 8, 2022
  35. rsbecker@nexbridge.comJan 9, 2022
  36. Neeraj SinghJan 10, 2022
  37. 0/4 A design for future-proofing fsync() configurationNeeraj K. Singh via GitGitGadget, Feb 1, 2022
  38. 1/4 core.fsyncmethod: add writeout-only modeNeeraj Singh via GitGitGadget, Feb 1, 2022
  39. 2/4 core.fsync: introduce granular fsync controlNeeraj Singh via GitGitGadget, Feb 1, 2022
  40. Junio C HamanoFeb 2, 2022
  41. Junio C HamanoFeb 2, 2022
  42. Neeraj SinghFeb 11, 2022
  43. Junio C HamanoFeb 11, 2022
  44. Neeraj SinghFeb 11, 2022
  45. Junio C HamanoFeb 11, 2022
  46. rsbecker@nexbridge.comFeb 12, 2022
  47. Patrick SteinhardtFeb 14, 2022
  48. Junio C HamanoFeb 14, 2022
  49. Patrick SteinhardtMar 9, 2022
  50. Ævar Arnfjörð BjarmasonMar 9, 2022
  51. Junio C HamanoMar 9, 2022
  52. Patrick SteinhardtMar 10, 2022
  53. Junio C HamanoMar 10, 2022
  54. Neeraj SinghMar 9, 2022
  55. Neeraj SinghFeb 11, 2022
  56. 3/4 core.fsync: new option to harden the indexNeeraj Singh via GitGitGadget, Feb 1, 2022
  57. 4/4 core.fsync: add a `derived-metadata` aggregate optionNeeraj Singh via GitGitGadget, Feb 1, 2022
  58. 0/5 A design for future-proofing fsync() configurationNeeraj K. Singh via GitGitGadget, Mar 9, 2022
  59. 1/5 wrapper: move inclusion of CSPRNG headers the wrapper.c fileNeeraj Singh via GitGitGadget, Mar 9, 2022
  60. Junio C HamanoMar 9, 2022
  61. Neeraj SinghMar 10, 2022
  62. brian m. carlsonMar 10, 2022
  63. Neeraj SinghMar 10, 2022
  64. 2/5 core.fsyncmethod: add writeout-only modeNeeraj Singh via GitGitGadget, Mar 9, 2022
  65. Junio C HamanoMar 9, 2022
  66. 3/5 core.fsync: introduce granular fsync controlNeeraj Singh via GitGitGadget, Mar 9, 2022
  67. Junio C HamanoMar 10, 2022
  68. Neeraj SinghMar 10, 2022
  69. Junio C HamanoMar 10, 2022
  70. Neeraj SinghMar 10, 2022
  71. Junio C HamanoMar 10, 2022
  72. Junio C HamanoMar 10, 2022
  73. Neeraj SinghMar 10, 2022
  74. Junio C HamanoMar 10, 2022
  75. Johannes SchindelinMar 10, 2022
  76. Junio C HamanoMar 10, 2022
  77. 4/5 core.fsync: new option to harden the indexNeeraj Singh via GitGitGadget, Mar 9, 2022
  78. 5/5 core.fsync: documentation and user-friendly aggregate optionsNeeraj Singh via GitGitGadget, Mar 9, 2022
  79. Future-proofed syncing of refsPatrick Steinhardt, Mar 10, 2022
  80. 6/8 core.fsync: add `fsync_component()` wrapper which doesn't diePatrick Steinhardt, Mar 10, 2022
  81. Junio C HamanoMar 10, 2022
  82. Neeraj SinghMar 10, 2022
  83. 7/8 core.fsync: new option to harden loose referencesPatrick Steinhardt, Mar 10, 2022
  84. Junio C HamanoMar 10, 2022
  85. Neeraj SinghMar 10, 2022
  86. Neeraj SinghMar 10, 2022
  87. Junio C HamanoMar 11, 2022
  88. Patrick SteinhardtMar 11, 2022
  89. Ævar Arnfjörð BjarmasonMar 11, 2022
  90. 8/8 core.fsync: new option to harden packed referencesPatrick Steinhardt, Mar 10, 2022
  91. Junio C HamanoMar 10, 2022
  92. Patrick SteinhardtMar 11, 2022
  93. 0/6 A design for future-proofing fsync() configurationNeeraj K. Singh via GitGitGadget, Mar 10, 2022
  94. 1/6 wrapper: make inclusion of Windows csprng header tightly scopedNeeraj Singh via GitGitGadget, Mar 10, 2022
  95. 3/6 core.fsync: introduce granular fsync control infrastructureNeeraj Singh via GitGitGadget, Mar 10, 2022
  96. 4/6 core.fsync: add configuration parsingNeeraj Singh via GitGitGadget, Mar 10, 2022
  97. Jiang XinMar 28, 2022
  98. Neeraj SinghMar 28, 2022
  99. 5/6 core.fsync: new option to harden the indexNeeraj Singh via GitGitGadget, Mar 10, 2022
  100. 2/6 core.fsyncmethod: add writeout-only modeNeeraj Singh via GitGitGadget, Mar 10, 2022
  101. 6/6 core.fsync: documentation and user-friendly aggregate optionsNeeraj Singh via GitGitGadget, Mar 10, 2022
  102. core.fsync: documentation and user-friendly aggregate optionsNeeraj Singh, Mar 15, 2022
  103. Junio C HamanoMar 15, 2022
  104. Neeraj SinghMar 15, 2022
  105. do we have too much fsync() configuration in 'next'? (was: [PATCH v7] core.fsync: documentation and user-friendly aggregate options)Ævar Arnfjörð Bjarmason, Mar 23, 2022
  106. Neeraj SinghMar 25, 2022
  107. Ævar Arnfjörð BjarmasonMar 26, 2022
  108. Junio C HamanoMar 26, 2022
  109. Neeraj SinghMar 26, 2022
  110. Ævar Arnfjörð BjarmasonMar 26, 2022
  111. Neeraj SinghMar 27, 2022
  112. Ævar Arnfjörð BjarmasonMar 27, 2022
  113. Patrick SteinhardtMar 28, 2022
  114. Ævar Arnfjörð BjarmasonMar 28, 2022
  115. Neeraj SinghMar 28, 2022
  116. Neeraj SinghMar 30, 2022
  117. Junio C HamanoMar 10, 2022
  118. Neeraj SinghMar 11, 2022
  119. Neeraj SinghMar 11, 2022
  120. Junio C HamanoMar 13, 2022
  121. core.fsync: new option to harden referencesPatrick Steinhardt, Mar 11, 2022
  122. SZEDER GáborMar 25, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.