git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v4 5/7] gc: handle a corner case in gc.bigPackThreshold

From
Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Date
Mar 24, 2018, 07:42 UTC
Message-ID
<20180324074308.18934-10-pclouds@gmail.com>
In-Reply-To
<20180324074308.18934-1-pclouds@gmail.com>

This config allows us to keep <N> packs back if their size is larger than a limit. But if this N >= gc.autoPackLimit, we may have a problem. We are supposed to reduce the number of packs after a threshold because it affects performance.

We could tell the user that they have incompatible gc.bigPackThreshold and gc.autoPackLimit, but it's kinda hard when 'git gc --auto' runs in background. Instead let's fall back to the next best stategy: try to reduce the number of packs anyway, but keep the base pack out. This reduces the number of packs to two and hopefully won't take up too much resources to repack (the assumption still is the base pack takes most resources to handle).

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 Documentation/config.txt | 6 +++++-
 builtin/gc.c             | 8 +++++++-
 2 files changed, 12 insertions(+), 2 deletions(-)
diff --git a/Documentation/config.txt b/Documentation/config.txt
index d63db3f12c..cf862d3edf 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -1564,7 +1564,11 @@ gc.bigPackThreshold::
 	except that all packs that meet the threshold are kept, not
 	just the base pack. Defaults to zero. Common unit suffixes of
 	'k', 'm', or 'g' are supported.
-
++
+Note that if the number of kept packs is more than gc.autoPackLimit,
+this configuration variable is ignored, all packs except the base pack
+will be repacked. After this the number of packs should go below
+gc.autoPackLimit and gc.bigPackThreshold should be respected again.
 
 gc.logExpiry::
 	If the file gc.log exists, then `git gc --auto` won't run
diff --git a/builtin/gc.c b/builtin/gc.c
index 53a0500898..74d3aaa270 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -254,8 +254,14 @@ static int need_to_gc(void)
 	if (too_many_packs()) {
 		struct string_list keep_pack = STRING_LIST_INIT_NODUP;
 
-		if (big_pack_threshold)
+		if (big_pack_threshold) {
 			find_base_packs(&keep_pack, big_pack_threshold);
+			if (keep_pack.nr >= gc_auto_pack_limit) {
+				big_pack_threshold = 0;
+				string_list_clear(&keep_pack, 0);
+				find_base_packs(&keep_pack, 0);
+			}
+		}
 
 		add_repack_all_option(&keep_pack);
 		string_list_clear(&keep_pack, 0);
-- 
2.17.0.rc0.348.gd5a49e0b6f
Previous: Nguyễn Thái Ngọc DuyNext: Nguyễn Thái Ngọc Duy
Message 55 of 110 in “Combine t/helper binaries into a single one”
  1. 00/36 Combine t/helper binaries into a single oneNguyễn Thái Ngọc Duy, Mar 17, 2018
  2. 01/36 t/helper: add an empty test-tool programNguyễn Thái Ngọc Duy, Mar 17, 2018
  3. 03/36 t/helper: merge test-sha1 into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  4. 10/36 t/helper: merge test-dump-cache-tree into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  5. 13/36 t/helper: merge test-genrandom into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  6. 11/36 t/helper: merge test-dump-split-index into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  7. 19/36 t/helper: merge test-online-cpus into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  8. 22/36 t/helepr: merge test-read-cache into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  9. Eric SunshineMar 18, 2018
  10. 26/36 t/helper: merge test-run-command into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  11. 28/36 t/helper: merge test-sha1-array into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  12. 31/36 t/helper: merge test-string-list into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  13. 33/36 t/helper: merge test-subprocess into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  14. 35/36 t/helper: merge test-wildmatch into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  15. 34/36 t/helper: merge test-urlmatch-.. into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  16. Eric SunshineMar 18, 2018
  17. 36/36 t/helper: merge test-write-cache into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  18. 32/36 t/helper: merge test-submodule-config into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  19. 30/36 t/helper: merge test-strcmp-offset into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  20. 27/36 t/helper: merge test-scrap-cache-tree into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  21. 29/36 t/helper: merge test-sigchain into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  22. 18/36 t/helper: merge test-mktemp into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  23. 25/36 t/helper: merge test-revision-walking into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  24. 21/36 t/helper: merge test-prio-queue into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  25. 24/36 t/helper: merge test-regex into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  26. 20/36 t/helper: merge test-path-utils into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  27. 23/36 t/helper: merge test-ref-store into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  28. 12/36 t/helper: merge test-example-decorate into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  29. 14/36 t/helper: merge test-hashmap into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  30. 17/36 t/helper: merge (unused) test-mergesort into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  31. 15/36 t/helper: merge test-index-version into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  32. 16/36 t/helper: merge (unused) test-match-trees into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  33. 08/36 t/helper: merge (unused) test-delta into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  34. 05/36 t/helper: merge test-config into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  35. 02/36 t/helper: merge test-chmtime into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  36. 09/36 t/helper: merge test-drop-caches into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  37. 04/36 t/helper: merge test-lazy-init-name-hash into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  38. Eric SunshineMar 18, 2018
  39. Duy NguyenMar 18, 2018
  40. Eric SunshineMar 18, 2018
  41. Jeff HostetlerMar 19, 2018
  42. Duy NguyenMar 19, 2018
  43. 07/36 t/helper: merge test-date into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  44. 06/36 t/helper: merge test-ctype into test-toolNguyễn Thái Ngọc Duy, Mar 17, 2018
  45. 00/36 Combine t/helper binaries into a single oneNguyễn Thái Ngọc Duy, Mar 24, 2018
  46. 1/7 t7700: have closing quote of a test at the beginning of lineNguyễn Thái Ngọc Duy, Mar 24, 2018
  47. 01/36 t/helper: add an empty test-tool programNguyễn Thái Ngọc Duy, Mar 24, 2018
  48. 2/7 repack: add --keep-pack optionNguyễn Thái Ngọc Duy, Mar 24, 2018
  49. 03/36 t/helper: merge test-sha1 into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  50. 05/36 t/helper: merge test-config into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  51. 4/7 gc: add gc.bigPackThreshold configNguyễn Thái Ngọc Duy, Mar 24, 2018
  52. 06/36 t/helper: merge test-ctype into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  53. 04/36 t/helper: merge test-lazy-init-name-hash into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  54. 07/36 t/helper: merge test-date into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  55. 5/7 gc: handle a corner case in gc.bigPackThresholdNguyễn Thái Ngọc Duy, Mar 24, 2018
  56. 6/7 gc --auto: exclude base pack if not enough mem to "repack -ad"Nguyễn Thái Ngọc Duy, Mar 24, 2018
  57. 7/7 pack-objects: show some progress when counting kept objectsNguyễn Thái Ngọc Duy, Mar 24, 2018
  58. 3/7 gc: add --keep-largest-pack optionNguyễn Thái Ngọc Duy, Mar 24, 2018
  59. Ævar Arnfjörð BjarmasonMar 24, 2018
  60. Duy NguyenMar 24, 2018
  61. Junio C HamanoMar 26, 2018
  62. 02/36 t/helper: merge test-chmtime into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  63. 00/36 Combine t/helper binaries into a single oneNguyễn Thái Ngọc Duy, Mar 24, 2018
  64. 01/36 t/helper: add an empty test-tool programNguyễn Thái Ngọc Duy, Mar 24, 2018
  65. Johannes SchindelinMar 26, 2018
  66. Duy NguyenMar 26, 2018
  67. Johannes SchindelinMar 26, 2018
  68. SZEDER GáborMar 26, 2018
  69. Johannes SchindelinMar 27, 2018
  70. 02/36 t/helper: merge test-chmtime into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  71. 06/36 t/helper: merge test-ctype into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  72. 08/36 t/helper: merge (unused) test-delta into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  73. 07/36 t/helper: merge test-date into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  74. 09/36 t/helper: merge test-drop-caches into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  75. 13/36 t/helper: merge test-genrandom into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  76. 14/36 t/helper: merge test-hashmap into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  77. 16/36 t/helper: merge (unused) test-match-trees into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  78. 12/36 t/helper: merge test-example-decorate into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  79. 26/36 t/helper: merge test-run-command into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  80. 27/36 t/helper: merge test-scrap-cache-tree into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  81. 29/36 t/helper: merge test-sigchain into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  82. 28/36 t/helper: merge test-sha1-array into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  83. 30/36 t/helper: merge test-strcmp-offset into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  84. 33/36 t/helper: merge test-subprocess into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  85. 35/36 t/helper: merge test-wildmatch into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  86. 34/36 t/helper: merge test-urlmatch-normalization into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  87. 36/36 t/helper: merge test-write-cache into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  88. 32/36 t/helper: merge test-submodule-config into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  89. 31/36 t/helper: merge test-string-list into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  90. 15/36 t/helper: merge test-index-version into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  91. 25/36 t/helper: merge test-revision-walking into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  92. 18/36 t/helper: merge test-mktemp into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  93. 24/36 t/helper: merge test-regex into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  94. 23/36 t/helper: merge test-ref-store into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  95. 17/36 t/helper: merge (unused) test-mergesort into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  96. 22/36 t/helper: merge test-read-cache into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  97. 19/36 t/helper: merge test-online-cpus into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  98. 10/36 t/helper: merge test-dump-cache-tree into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  99. 11/36 t/helper: merge test-dump-split-index into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  100. 20/36 t/helper: merge test-path-utils into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  101. 21/36 t/helper: merge test-prio-queue into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  102. 03/36 t/helper: merge test-sha1 into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  103. 05/36 t/helper: merge test-config into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  104. 04/36 t/helper: merge test-lazy-init-name-hash into test-toolNguyễn Thái Ngọc Duy, Mar 24, 2018
  105. Ævar Arnfjörð BjarmasonMar 24, 2018
  106. Duy NguyenMar 24, 2018
  107. Junio C HamanoMar 26, 2018
  108. Johannes SchindelinMar 27, 2018
  109. Junio C HamanoMar 27, 2018
  110. Johannes SchindelinMar 27, 2018

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.