git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v4 4/7] gc: add gc.bigPackThreshold config

From
Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Date
Mar 24, 2018, 07:42 UTC
Message-ID
<20180324074308.18934-8-pclouds@gmail.com>
In-Reply-To
<20180324074308.18934-1-pclouds@gmail.com>

The --keep-largest-pack option is not very convenient to use because you need to tell gc to do this explicitly (and probably on just a few large repos).

Add a config key that enables this mode when packs larger than a limit are found. Note that there's a slight behavior difference compared to --keep-largest-pack: all packs larger than the threshold are kept, not just the largest one.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 Documentation/config.txt |  8 ++++++++
 Documentation/git-gc.txt |  6 ++++--
 builtin/gc.c             | 26 ++++++++++++++++++++------
 3 files changed, 32 insertions(+), 8 deletions(-)
diff --git a/Documentation/config.txt b/Documentation/config.txt
index ce9102cea8..d63db3f12c 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -1558,6 +1558,14 @@ gc.autoDetach::
 	Make `git gc --auto` return immediately and run in background
 	if the system supports it. Default is true.
 
+gc.bigPackThreshold::
+	If non-zero, all packs larger than this limit are kept when
+	`git gc` is run. This is very similar to `--keep-base-pack`
+	except that all packs that meet the threshold are kept, not
+	just the base pack. Defaults to zero. Common unit suffixes of
+	'k', 'm', or 'g' are supported.
+
+
 gc.logExpiry::
 	If the file gc.log exists, then `git gc --auto` won't run
 	unless that file is more than 'gc.logExpiry' old.  Default is
diff --git a/Documentation/git-gc.txt b/Documentation/git-gc.txt
index bf81b8de30..0adf381b52 100644
--- a/Documentation/git-gc.txt
+++ b/Documentation/git-gc.txt
@@ -55,7 +55,8 @@ all loose objects are combined into a single pack using
 disables automatic packing of loose objects.
 +
 If the number of packs exceeds the value of `gc.autoPackLimit`,
-then existing packs (except those marked with a `.keep` file)
+then existing packs (except those marked with a `.keep` file
+or over `gc.bigPackThreshold` limit)
 are consolidated into a single pack by using the `-A` option of
 'git repack'. Setting `gc.autoPackLimit` to 0 disables
 automatic consolidation of packs.
@@ -80,7 +81,8 @@ automatic consolidation of packs.
 
 --keep-largest-pack::
 	All packs except the largest pack and those marked with a
-	`.keep` files are consolidated into a single pack.
+	`.keep` files are consolidated into a single pack. When this
+	option is used, `gc.bigPackThreshold` is ignored.
 
 Configuration
 -------------
diff --git a/builtin/gc.c b/builtin/gc.c
index 9a09cf53b0..53a0500898 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -39,6 +39,7 @@ static timestamp_t gc_log_expire_time;
 static const char *gc_log_expire = "1.day.ago";
 static const char *prune_expire = "2.weeks.ago";
 static const char *prune_worktrees_expire = "3.months.ago";
+static unsigned long big_pack_threshold;
 
 static struct argv_array pack_refs_cmd = ARGV_ARRAY_INIT;
 static struct argv_array reflog = ARGV_ARRAY_INIT;
@@ -126,6 +127,8 @@ static void gc_config(void)
 	git_config_get_expiry("gc.worktreepruneexpire", &prune_worktrees_expire);
 	git_config_get_expiry("gc.logexpiry", &gc_log_expire);
 
+	git_config_get_ulong("gc.bigpackthreshold", &big_pack_threshold);
+
 	git_config(git_default_config, NULL);
 }
 
@@ -164,7 +167,7 @@ static int too_many_loose_objects(void)
 	return needed;
 }
 
-static void find_base_packs(struct string_list *packs)
+static void find_base_packs(struct string_list *packs, unsigned long limit)
 {
 	struct packed_git *p, *base = NULL;
 
@@ -173,7 +176,10 @@ static void find_base_packs(struct string_list *packs)
 	for (p = packed_git; p; p = p->next) {
 		if (!p->pack_local)
 			continue;
-		if (!base || base->pack_size < p->pack_size) {
+		if (limit) {
+			if (p->pack_size >= limit)
+				string_list_append(packs, p->pack_name);
+		} else if (!base || base->pack_size < p->pack_size) {
 			base = p;
 		}
 	}
@@ -245,9 +251,15 @@ static int need_to_gc(void)
 	 * we run "repack -A -d -l".  Otherwise we tell the caller
 	 * there is no need.
 	 */
-	if (too_many_packs())
-		add_repack_all_option(NULL);
-	else if (too_many_loose_objects())
+	if (too_many_packs()) {
+		struct string_list keep_pack = STRING_LIST_INIT_NODUP;
+
+		if (big_pack_threshold)
+			find_base_packs(&keep_pack, big_pack_threshold);
+
+		add_repack_all_option(&keep_pack);
+		string_list_clear(&keep_pack, 0);
+	} else if (too_many_loose_objects())
 		add_repack_incremental_option();
 	else
 		return 0;
@@ -462,7 +474,9 @@ int cmd_gc(int argc, const char **argv, const char *prefix)
 
 		if (keep_base_pack != -1) {
 			if (keep_base_pack)
-				find_base_packs(&keep_pack);
+				find_base_packs(&keep_pack, 0);
+		} else if (big_pack_threshold) {
+			find_base_packs(&keep_pack, big_pack_threshold);
 		}
 
 		add_repack_all_option(&keep_pack);
-- 
2.17.0.rc0.348.gd5a49e0b6f
Previous: Nguyễn Thái Ngọc DuyNext: Nguyễn Thái Ngọc Duy
Message 51 of 110 in “Combine t/helper binaries into a single one”
  1. 00/36 Combine t/helper binaries into a single oneNguyễn Thái Ngọc Duy, Mar 17, 2018
  2. 01/36 t/helper: add an empty test-tool programNguyễn Thái Ngọc Duy, Mar 17, 2018
  3. 03/36 t/helper: merge test-sha1 into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  4. 10/36 t/helper: merge test-dump-cache-tree into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  5. 13/36 t/helper: merge test-genrandom into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  6. 11/36 t/helper: merge test-dump-split-index into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  7. 19/36 t/helper: merge test-online-cpus into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  8. 22/36 t/helepr: merge test-read-cache into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  9. Eric SunshineMar 18, 2018
  10. 26/36 t/helper: merge test-run-command into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  11. 28/36 t/helper: merge test-sha1-array into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  12. 31/36 t/helper: merge test-string-list into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  13. 33/36 t/helper: merge test-subprocess into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  14. 35/36 t/helper: merge test-wildmatch into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  15. 34/36 t/helper: merge test-urlmatch-.. into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  16. Eric SunshineMar 18, 2018
  17. 36/36 t/helper: merge test-write-cache into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  18. 32/36 t/helper: merge test-submodule-config into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  19. 30/36 t/helper: merge test-strcmp-offset into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  20. 27/36 t/helper: merge test-scrap-cache-tree into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  21. 29/36 t/helper: merge test-sigchain into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  22. 18/36 t/helper: merge test-mktemp into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  23. 25/36 t/helper: merge test-revision-walking into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  24. 21/36 t/helper: merge test-prio-queue into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  25. 24/36 t/helper: merge test-regex into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  26. 20/36 t/helper: merge test-path-utils into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  27. 23/36 t/helper: merge test-ref-store into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  28. 12/36 t/helper: merge test-example-decorate into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  29. 14/36 t/helper: merge test-hashmap into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  30. 17/36 t/helper: merge (unused) test-mergesort into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  31. 15/36 t/helper: merge test-index-version into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  32. 16/36 t/helper: merge (unused) test-match-trees into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  33. 08/36 t/helper: merge (unused) test-delta into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  34. 05/36 t/helper: merge test-config into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  35. 02/36 t/helper: merge test-chmtime into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  36. 09/36 t/helper: merge test-drop-caches into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  37. 04/36 t/helper: merge test-lazy-init-name-hash into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  38. Eric SunshineMar 18, 2018
  39. Duy NguyenMar 18, 2018
  40. Eric SunshineMar 18, 2018
  41. Jeff HostetlerMar 19, 2018
  42. Duy NguyenMar 19, 2018
  43. 07/36 t/helper: merge test-date into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  44. 06/36 t/helper: merge test-ctype into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  45. 00/36 Combine t/helper binaries into a single oneNguyễn Thái Ngọc Duy, Mar 24, 2018
  46. 1/7 t7700: have closing quote of a test at the beginning of lineNguyễn Thái Ngọc Duy, Mar 24, 2018
  47. 01/36 t/helper: add an empty test-tool programNguyễn Thái Ngọc Duy, Mar 24, 2018
  48. 2/7 repack: add --keep-pack optionNguyễn Thái Ngọc Duy, Mar 24, 2018
  49. 03/36 t/helper: merge test-sha1 into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  50. 05/36 t/helper: merge test-config into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  51. 4/7 gc: add gc.bigPackThreshold configNguyễn Thái Ngọc Duy, Mar 24, 2018
  52. 06/36 t/helper: merge test-ctype into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  53. 04/36 t/helper: merge test-lazy-init-name-hash into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  54. 07/36 t/helper: merge test-date into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  55. 5/7 gc: handle a corner case in gc.bigPackThresholdNguyễn Thái Ngọc Duy, Mar 24, 2018
  56. 6/7 gc --auto: exclude base pack if not enough mem to "repack -ad"Nguyễn Thái Ngọc Duy, Mar 24, 2018
  57. 7/7 pack-objects: show some progress when counting kept objectsNguyễn Thái Ngọc Duy, Mar 24, 2018
  58. 3/7 gc: add --keep-largest-pack optionNguyễn Thái Ngọc Duy, Mar 24, 2018
  59. Ævar Arnfjörð BjarmasonMar 24, 2018
  60. Duy NguyenMar 24, 2018
  61. Junio C HamanoMar 26, 2018
  62. 02/36 t/helper: merge test-chmtime into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  63. 00/36 Combine t/helper binaries into a single oneNguyễn Thái Ngọc Duy, Mar 24, 2018
  64. 01/36 t/helper: add an empty test-tool programNguyễn Thái Ngọc Duy, Mar 24, 2018
  65. Johannes SchindelinMar 26, 2018
  66. Duy NguyenMar 26, 2018
  67. Johannes SchindelinMar 26, 2018
  68. SZEDER GáborMar 26, 2018
  69. Johannes SchindelinMar 27, 2018
  70. 02/36 t/helper: merge test-chmtime into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  71. 06/36 t/helper: merge test-ctype into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  72. 08/36 t/helper: merge (unused) test-delta into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  73. 07/36 t/helper: merge test-date into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  74. 09/36 t/helper: merge test-drop-caches into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  75. 13/36 t/helper: merge test-genrandom into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  76. 14/36 t/helper: merge test-hashmap into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  77. 16/36 t/helper: merge (unused) test-match-trees into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  78. 12/36 t/helper: merge test-example-decorate into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  79. 26/36 t/helper: merge test-run-command into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  80. 27/36 t/helper: merge test-scrap-cache-tree into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  81. 29/36 t/helper: merge test-sigchain into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  82. 28/36 t/helper: merge test-sha1-array into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  83. 30/36 t/helper: merge test-strcmp-offset into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  84. 33/36 t/helper: merge test-subprocess into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  85. 35/36 t/helper: merge test-wildmatch into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  86. 34/36 t/helper: merge test-urlmatch-normalization into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  87. 36/36 t/helper: merge test-write-cache into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  88. 32/36 t/helper: merge test-submodule-config into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  89. 31/36 t/helper: merge test-string-list into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  90. 15/36 t/helper: merge test-index-version into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  91. 25/36 t/helper: merge test-revision-walking into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  92. 18/36 t/helper: merge test-mktemp into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  93. 24/36 t/helper: merge test-regex into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  94. 23/36 t/helper: merge test-ref-store into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  95. 17/36 t/helper: merge (unused) test-mergesort into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  96. 22/36 t/helper: merge test-read-cache into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  97. 19/36 t/helper: merge test-online-cpus into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  98. 10/36 t/helper: merge test-dump-cache-tree into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  99. 11/36 t/helper: merge test-dump-split-index into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  100. 20/36 t/helper: merge test-path-utils into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  101. 21/36 t/helper: merge test-prio-queue into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  102. 03/36 t/helper: merge test-sha1 into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  103. 05/36 t/helper: merge test-config into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  104. 04/36 t/helper: merge test-lazy-init-name-hash into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  105. Ævar Arnfjörð BjarmasonMar 24, 2018
  106. Duy NguyenMar 24, 2018
  107. Junio C HamanoMar 26, 2018
  108. Johannes SchindelinMar 27, 2018
  109. Junio C HamanoMar 27, 2018
  110. Johannes SchindelinMar 27, 2018

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.