git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v7 21/22] gc: automatically write commit-graph files

From
Derrick Stolee <stolee@gmail.com>
Date
Jun 27, 2018, 13:24 UTC
Message-ID
<20180627132447.142473-22-dstolee@microsoft.com>
In-Reply-To
<20180627132447.142473-1-dstolee@microsoft.com>

The commit-graph file is a very helpful feature for speeding up git operations. In order to make it more useful, make it possible to write the commit-graph file during standard garbage collection operations.

Add a 'gc.commitGraph' config setting that triggers writing a commit-graph file after any non-trivial 'git gc' command. Defaults to false while the commit-graph feature matures. We specifically do not want to have this on by default until the commit-graph feature is fully integrated with history-modifying features like shallow clones.

Helped-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
---
 Documentation/config.txt |  9 ++++++---
 Documentation/git-gc.txt |  4 ++++
 builtin/gc.c             |  6 ++++++
 t/t5318-commit-graph.sh  | 14 ++++++++++++++
 4 files changed, 30 insertions(+), 3 deletions(-)
diff --git a/Documentation/config.txt b/Documentation/config.txt
index 1cc18a828c..978deecfee 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -904,9 +904,12 @@ core.notesRef::
 This setting defaults to "refs/notes/commits", and it can be overridden by
 the `GIT_NOTES_REF` environment variable.  See linkgit:git-notes[1].
 
-core.commitGraph::
-	Enable git commit graph feature. Allows reading from the
-	commit-graph file.
+gc.commitGraph::
+	If true, then gc will rewrite the commit-graph file when
+	linkgit:git-gc[1] is run. When using linkgit:git-gc[1]
+	'--auto' the commit-graph will be updated if housekeeping is
+	required. Default is false. See linkgit:git-commit-graph[1]
+	for details.
 
 core.sparseCheckout::
 	Enable "sparse checkout" feature. See section "Sparse checkout" in
diff --git a/Documentation/git-gc.txt b/Documentation/git-gc.txt
index 24b2dd44fe..f5bc98ccb3 100644
--- a/Documentation/git-gc.txt
+++ b/Documentation/git-gc.txt
@@ -136,6 +136,10 @@ The optional configuration variable `gc.packRefs` determines if
 it within all non-bare repos or it can be set to a boolean value.
 This defaults to true.
 
+The optional configuration variable `gc.commitGraph` determines if
+'git gc' should run 'git commit-graph write'. This can be set to a
+boolean value. This defaults to false.
+
 The optional configuration variable `gc.aggressiveWindow` controls how
 much time is spent optimizing the delta compression of the objects in
 the repository when the --aggressive option is specified.  The larger
diff --git a/builtin/gc.c b/builtin/gc.c
index ccfb1ceaeb..b7dd206f41 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -20,6 +20,7 @@
 #include "sigchain.h"
 #include "argv-array.h"
 #include "commit.h"
+#include "commit-graph.h"
 #include "packfile.h"
 #include "object-store.h"
 #include "pack.h"
@@ -40,6 +41,7 @@ static int aggressive_depth = 50;
 static int aggressive_window = 250;
 static int gc_auto_threshold = 6700;
 static int gc_auto_pack_limit = 50;
+static int gc_write_commit_graph = 0;
 static int detach_auto = 1;
 static timestamp_t gc_log_expire_time;
 static const char *gc_log_expire = "1.day.ago";
@@ -129,6 +131,7 @@ static void gc_config(void)
 	git_config_get_int("gc.aggressivedepth", &aggressive_depth);
 	git_config_get_int("gc.auto", &gc_auto_threshold);
 	git_config_get_int("gc.autopacklimit", &gc_auto_pack_limit);
+	git_config_get_bool("gc.writecommitgraph", &gc_write_commit_graph);
 	git_config_get_bool("gc.autodetach", &detach_auto);
 	git_config_get_expiry("gc.pruneexpire", &prune_expire);
 	git_config_get_expiry("gc.worktreepruneexpire", &prune_worktrees_expire);
@@ -641,6 +644,9 @@ int cmd_gc(int argc, const char **argv, const char *prefix)
 	if (pack_garbage.nr > 0)
 		clean_pack_garbage();
 
+	if (gc_write_commit_graph)
+		write_commit_graph_reachable(get_object_directory(), 0);
+
 	if (auto_gc && too_many_loose_objects())
 		warning(_("There are too many unreachable loose objects; "
 			"run 'git prune' to remove them."));
diff --git a/t/t5318-commit-graph.sh b/t/t5318-commit-graph.sh
index 2208c266b5..5947de3d24 100755
--- a/t/t5318-commit-graph.sh
+++ b/t/t5318-commit-graph.sh
@@ -245,6 +245,20 @@ test_expect_success 'perform fast-forward merge in full repo' '
 	test_cmp expect output
 '
 
+test_expect_success 'check that gc computes commit-graph' '
+	cd "$TRASH_DIRECTORY/full" &&
+	git commit --allow-empty -m "blank" &&
+	git commit-graph write --reachable &&
+	cp $objdir/info/commit-graph commit-graph-before-gc &&
+	git reset --hard HEAD~1 &&
+	git config gc.writeCommitGraph true &&
+	git gc &&
+	cp $objdir/info/commit-graph commit-graph-after-gc &&
+	! test_cmp commit-graph-before-gc commit-graph-after-gc &&
+	git commit-graph write --reachable &&
+	test_cmp commit-graph-after-gc $objdir/info/commit-graph
+'
+
 # the verify tests below expect the commit-graph to contain
 # exactly the commits reachable from the commits/8 branch.
 # If the file changes the set of commits in the list, then the

-- 
2.18.0.24.g1b579a2ee9
Previous: Derrick StoleeNext: Junio C Hamano
Message 111 of 125 in “Integrate commit-graph into 'fsck' and 'gc'”
  1. 00/21 Integrate commit-graph into 'fsck' and 'gc'Derrick Stolee, Jun 4, 2018
  2. 01/21 commit-graph: UNLEAK before die()Derrick Stolee, Jun 4, 2018
  3. 02/21 commit-graph: fix GRAPH_MIN_SIZEDerrick Stolee, Jun 4, 2018
  4. 03/21 commit-graph: parse commit from chosen graphDerrick Stolee, Jun 4, 2018
  5. 05/21 commit-graph: load a root tree from specific graphDerrick Stolee, Jun 4, 2018
  6. 07/21 commit-graph: verify catches corrupt signatureDerrick Stolee, Jun 4, 2018
  7. 06/21 commit-graph: add 'verify' subcommandDerrick Stolee, Jun 4, 2018
  8. 08/21 commit-graph: verify required chunks are presentDerrick Stolee, Jun 4, 2018
  9. 09/21 commit-graph: verify corrupt OID fanout and lookupDerrick Stolee, Jun 4, 2018
  10. 11/21 commit-graph: verify root tree OIDsDerrick Stolee, Jun 4, 2018
  11. 12/21 commit-graph: verify parent listDerrick Stolee, Jun 4, 2018
  12. 10/21 commit-graph: verify objects existDerrick Stolee, Jun 4, 2018
  13. 18/21 commit-graph: use string-list API for inputDerrick Stolee, Jun 4, 2018
  14. 20/21 gc: automatically write commit-graph filesDerrick Stolee, Jun 4, 2018
  15. Ævar Arnfjörð BjarmasonJun 6, 2018
  16. 19/21 commit-graph: add '--reachable' optionDerrick Stolee, Jun 4, 2018
  17. 21/21 commit-graph: update design documentDerrick Stolee, Jun 4, 2018
  18. 13/21 commit-graph: verify generation numberDerrick Stolee, Jun 4, 2018
  19. 17/21 fsck: verify commit-graphDerrick Stolee, Jun 4, 2018
  20. Ævar Arnfjörð BjarmasonJun 6, 2018
  21. Derrick StoleeJun 6, 2018
  22. 16/21 commit-graph: verify contents match checksumDerrick Stolee, Jun 4, 2018
  23. 14/21 commit-graph: verify commit dateDerrick Stolee, Jun 4, 2018
  24. 15/21 commit-graph: test for corrupted octopus edgeDerrick Stolee, Jun 4, 2018
  25. 04/21 commit: force commit to parse from object databaseDerrick Stolee, Jun 4, 2018
  26. Derrick StoleeJun 4, 2018
  27. Ævar Arnfjörð BjarmasonJun 5, 2018
  28. Derrick StoleeJun 5, 2018
  29. 00/21 Integrate commit-graph into 'fsck' and 'gc'Derrick Stolee, Jun 6, 2018
  30. 05/21 commit-graph: load a root tree from specific graphDerrick Stolee, Jun 6, 2018
  31. 17/21 fsck: verify commit-graphDerrick Stolee, Jun 6, 2018
  32. 15/21 commit-graph: test for corrupted octopus edgeDerrick Stolee, Jun 6, 2018
  33. 21/21 commit-graph: update design documentDerrick Stolee, Jun 6, 2018
  34. 13/21 commit-graph: verify generation numberDerrick Stolee, Jun 6, 2018
  35. 20/21 gc: automatically write commit-graph filesDerrick Stolee, Jun 6, 2018
  36. 10/21 commit-graph: verify objects existDerrick Stolee, Jun 6, 2018
  37. 19/21 commit-graph: add '--reachable' optionDerrick Stolee, Jun 6, 2018
  38. 03/21 commit-graph: parse commit from chosen graphDerrick Stolee, Jun 6, 2018
  39. 18/21 commit-graph: use string-list API for inputDerrick Stolee, Jun 6, 2018
  40. Ævar Arnfjörð BjarmasonJun 6, 2018
  41. Derrick StoleeJun 6, 2018
  42. Ævar Arnfjörð BjarmasonJun 6, 2018
  43. Derrick StoleeJun 6, 2018
  44. Derrick StoleeJun 8, 2018
  45. 06/21 commit-graph: add 'verify' subcommandDerrick Stolee, Jun 6, 2018
  46. 16/21 commit-graph: verify contents match checksumDerrick Stolee, Jun 6, 2018
  47. 09/21 commit-graph: verify corrupt OID fanout and lookupDerrick Stolee, Jun 6, 2018
  48. 04/21 commit: force commit to parse from object databaseDerrick Stolee, Jun 6, 2018
  49. 14/21 commit-graph: verify commit dateDerrick Stolee, Jun 6, 2018
  50. 07/21 commit-graph: verify catches corrupt signatureDerrick Stolee, Jun 6, 2018
  51. 08/21 commit-graph: verify required chunks are presentDerrick Stolee, Jun 6, 2018
  52. Ævar Arnfjörð BjarmasonJun 6, 2018
  53. 12/21 commit-graph: verify parent listDerrick Stolee, Jun 6, 2018
  54. 11/21 commit-graph: verify root tree OIDsDerrick Stolee, Jun 6, 2018
  55. 01/21 commit-graph: UNLEAK before die()Derrick Stolee, Jun 6, 2018
  56. 02/21 commit-graph: fix GRAPH_MIN_SIZEDerrick Stolee, Jun 6, 2018
  57. 00/21 Integrate commit-graph into 'fsck' and 'gc'Derrick Stolee, Jun 8, 2018
  58. 02/21 commit-graph: fix GRAPH_MIN_SIZEDerrick Stolee, Jun 8, 2018
  59. 03/21 commit-graph: parse commit from chosen graphDerrick Stolee, Jun 8, 2018
  60. 05/21 commit-graph: load a root tree from specific graphDerrick Stolee, Jun 8, 2018
  61. 06/21 commit-graph: add 'verify' subcommandDerrick Stolee, Jun 8, 2018
  62. 09/21 commit-graph: verify corrupt OID fanout and lookupDerrick Stolee, Jun 8, 2018
  63. 12/21 commit-graph: verify parent listDerrick Stolee, Jun 8, 2018
  64. Junio C HamanoJun 12, 2018
  65. 10/21 commit-graph: verify objects existDerrick Stolee, Jun 8, 2018
  66. 17/21 fsck: verify commit-graphDerrick Stolee, Jun 8, 2018
  67. 21/21 commit-graph: update design documentDerrick Stolee, Jun 8, 2018
  68. 16/21 commit-graph: verify contents match checksumDerrick Stolee, Jun 8, 2018
  69. 18/21 commit-graph: use string-list API for inputDerrick Stolee, Jun 8, 2018
  70. 19/21 commit-graph: add '--reachable' optionDerrick Stolee, Jun 8, 2018
  71. 15/21 commit-graph: test for corrupted octopus edgeDerrick Stolee, Jun 8, 2018
  72. 20/21 gc: automatically write commit-graph filesDerrick Stolee, Jun 8, 2018
  73. SZEDER GáborJun 8, 2018
  74. Derrick StoleeJun 25, 2018
  75. 11/21 commit-graph: verify root tree OIDsDerrick Stolee, Jun 8, 2018
  76. 14/21 commit-graph: verify commit dateDerrick Stolee, Jun 8, 2018
  77. 13/21 commit-graph: verify generation numberDerrick Stolee, Jun 8, 2018
  78. 08/21 commit-graph: verify required chunks are presentDerrick Stolee, Jun 8, 2018
  79. 07/21 commit-graph: verify catches corrupt signatureDerrick Stolee, Jun 8, 2018
  80. 04/21 commit: force commit to parse from object databaseDerrick Stolee, Jun 8, 2018
  81. 01/21 commit-graph: UNLEAK before die()Derrick Stolee, Jun 8, 2018
  82. Jakub NarębskiJun 8, 2018
  83. Derrick StoleeJun 8, 2018
  84. 00/22 Integrate commit-graph into 'fsck' and 'gc'Derrick Stolee, Jun 27, 2018
  85. 01/22 t5318-commit-graph.sh: use core.commitGraphDerrick Stolee, Jun 27, 2018
  86. Junio C HamanoJun 27, 2018
  87. 02/22 commit-graph: UNLEAK before die()Derrick Stolee, Jun 27, 2018
  88. 03/22 commit-graph: fix GRAPH_MIN_SIZEDerrick Stolee, Jun 27, 2018
  89. 05/22 commit: force commit to parse from object databaseDerrick Stolee, Jun 27, 2018
  90. 04/22 commit-graph: parse commit from chosen graphDerrick Stolee, Jun 27, 2018
  91. 06/22 commit-graph: load a root tree from specific graphDerrick Stolee, Jun 27, 2018
  92. SZEDER GáborJul 11, 2018
  93. coccinelle: update commit.cocciDerrick Stolee, Jul 13, 2018
  94. Eric SunshineJul 13, 2018
  95. 08/22 commit-graph: verify catches corrupt signatureDerrick Stolee, Jun 27, 2018
  96. 07/22 commit-graph: add 'verify' subcommandDerrick Stolee, Jun 27, 2018
  97. Jonathan TanJun 27, 2018
  98. Derrick StoleeJun 29, 2018
  99. 12/22 commit-graph: verify root tree OIDsDerrick Stolee, Jun 27, 2018
  100. 13/22 commit-graph: verify parent listDerrick Stolee, Jun 27, 2018
  101. 14/22 commit-graph: verify generation numberDerrick Stolee, Jun 27, 2018
  102. 16/22 commit-graph: test for corrupted octopus edgeDerrick Stolee, Jun 27, 2018
  103. 18/22 fsck: verify commit-graphDerrick Stolee, Jun 27, 2018
  104. 17/22 commit-graph: verify contents match checksumDerrick Stolee, Jun 27, 2018
  105. Junio C HamanoJun 27, 2018
  106. 11/22 commit-graph: verify objects existDerrick Stolee, Jun 27, 2018
  107. 10/22 commit-graph: verify corrupt OID fanout and lookupDerrick Stolee, Jun 27, 2018
  108. 22/22 commit-graph: update design documentDerrick Stolee, Jun 27, 2018
  109. Junio C HamanoJun 27, 2018
  110. 15/22 commit-graph: verify commit dateDerrick Stolee, Jun 27, 2018
  111. 21/22 gc: automatically write commit-graph filesDerrick Stolee, Jun 27, 2018
  112. Junio C HamanoJun 27, 2018
  113. Derrick StoleeJun 27, 2018
  114. SZEDER GáborAug 3, 2018
  115. t5318: use 'test_cmp_bin' to compare commit-graph filesSZEDER Gábor, Aug 12, 2018
  116. Derrick StoleeAug 13, 2018
  117. SZEDER GáborAug 13, 2018
  118. t5318: use 'test_cmp_bin' to compare commit-graph filesSZEDER Gábor, Aug 13, 2018
  119. 20/22 commit-graph: add '--reachable' optionDerrick Stolee, Jun 27, 2018
  120. Junio C HamanoJun 27, 2018
  121. Christian CouderSep 11, 2018
  122. Derrick StoleeSep 11, 2018
  123. Christian CouderSep 11, 2018
  124. 09/22 commit-graph: verify required chunks are presentDerrick Stolee, Jun 27, 2018
  125. 19/22 commit-graph: use string-list API for inputDerrick Stolee, Jun 27, 2018

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.