git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v8 7/9] gc: add `gc.repackFilter` config option

From
Christian Couder <christian.couder@gmail.com>
Date
Oct 2, 2023, 16:55 UTC
Message-ID
<20231002165504.1325153-8-christian.couder@gmail.com>
In-Reply-To
<20231002165504.1325153-1-christian.couder@gmail.com>

A previous commit has implemented `git repack --filter=<filter-spec>` to allow users to filter out some objects from the main pack and move them into a new different pack.

Users might want to perform such a cleanup regularly at the same time as they perform other repacks and cleanups, so as part of `git gc`.

Let's allow them to configure a <filter-spec> for that purpose using a new gc.repackFilter config option.

Now when `git gc` will perform a repack with a <filter-spec> configured through this option and not empty, the repack process will be passed a corresponding `--filter=<filter-spec>` argument.

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
 Documentation/config/gc.txt |  5 +++++
 builtin/gc.c                |  6 ++++++
 t/t6500-gc.sh               | 13 +++++++++++++
 3 files changed, 24 insertions(+)
diff --git a/Documentation/config/gc.txt b/Documentation/config/gc.txt
index ca47eb2008..2153bde7ac 100644
--- a/Documentation/config/gc.txt
+++ b/Documentation/config/gc.txt
@@ -145,6 +145,11 @@ Multiple hooks are supported, but all must exit successfully, else the
 operation (either generating a cruft pack or unpacking unreachable
 objects) will be halted.
 
+gc.repackFilter::
+	When repacking, use the specified filter to move certain
+	objects into a separate packfile.  See the
+	`--filter=<filter-spec>` option of linkgit:git-repack[1].
+
 gc.rerereResolved::
 	Records of conflicted merge you resolved earlier are
 	kept for this many days when 'git rerere gc' is run.
diff --git a/builtin/gc.c b/builtin/gc.c
index 00192ae5d3..98148e98fe 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -61,6 +61,7 @@ static timestamp_t gc_log_expire_time;
 static const char *gc_log_expire = "1.day.ago";
 static const char *prune_expire = "2.weeks.ago";
 static const char *prune_worktrees_expire = "3.months.ago";
+static char *repack_filter;
 static unsigned long big_pack_threshold;
 static unsigned long max_delta_cache_size = DEFAULT_DELTA_CACHE_SIZE;
 
@@ -170,6 +171,8 @@ static void gc_config(void)
 	git_config_get_ulong("gc.bigpackthreshold", &big_pack_threshold);
 	git_config_get_ulong("pack.deltacachesize", &max_delta_cache_size);
 
+	git_config_get_string("gc.repackfilter", &repack_filter);
+
 	git_config(git_default_config, NULL);
 }
 
@@ -355,6 +358,9 @@ static void add_repack_all_option(struct string_list *keep_pack)
 
 	if (keep_pack)
 		for_each_string_list(keep_pack, keep_one_pack, NULL);
+
+	if (repack_filter && *repack_filter)
+		strvec_pushf(&repack, "--filter=%s", repack_filter);
 }
 
 static void add_repack_incremental_option(void)
diff --git a/t/t6500-gc.sh b/t/t6500-gc.sh
index 69509d0c11..232e403b66 100755
--- a/t/t6500-gc.sh
+++ b/t/t6500-gc.sh
@@ -202,6 +202,19 @@ test_expect_success 'one of gc.reflogExpire{Unreachable,}=never does not skip "e
 	grep -E "^trace: (built-in|exec|run_command): git reflog expire --" trace.out
 '
 
+test_expect_success 'gc.repackFilter launches repack with a filter' '
+	test_when_finished "rm -rf bare.git" &&
+	git clone --no-local --bare . bare.git &&
+
+	git -C bare.git -c gc.cruftPacks=false gc &&
+	test_stdout_line_count = 1 ls bare.git/objects/pack/*.pack &&
+
+	GIT_TRACE=$(pwd)/trace.out git -C bare.git -c gc.repackFilter=blob:none \
+		-c repack.writeBitmaps=false -c gc.cruftPacks=false gc &&
+	test_stdout_line_count = 2 ls bare.git/objects/pack/*.pack &&
+	grep -E "^trace: (built-in|exec|run_command): git repack .* --filter=blob:none ?.*" trace.out
+'
+
 prepare_cruft_history () {
 	test_commit base &&
 
-- 
2.42.0.305.g5bfd918c90
Previous: Christian CouderNext: Christian Couder
Message 157 of 161 in “Repack objects into separate packfiles based on a filter”
  1. 0/9 Repack objects into separate packfiles based on a filterChristian Couder, Jun 14, 2023
  2. 1/9 pack-objects: allow `--filter` without `--stdout`Christian Couder, Jun 14, 2023
  3. Taylor BlauJun 21, 2023
  4. Christian CouderJul 5, 2023
  5. 2/9 pack-objects: add `--print-filtered` to print omitted objectsChristian Couder, Jun 14, 2023
  6. Junio C HamanoJun 15, 2023
  7. Taylor BlauJun 21, 2023
  8. Christian CouderJun 21, 2023
  9. Taylor BlauJun 21, 2023
  10. 3/9 t/helper: add 'find-pack' test-toolChristian Couder, Jun 14, 2023
  11. Junio C HamanoJun 15, 2023
  12. Christian CouderJun 21, 2023
  13. Taylor BlauJun 21, 2023
  14. 5/9 repack: refactor finishing pack-objects commandChristian Couder, Jun 14, 2023
  15. Junio C HamanoJun 16, 2023
  16. Taylor BlauJun 21, 2023
  17. Christian CouderJun 21, 2023
  18. Taylor BlauJun 21, 2023
  19. 4/9 repack: refactor piping an oid to a commandChristian Couder, Jun 14, 2023
  20. Junio C HamanoJun 15, 2023
  21. Taylor BlauJun 21, 2023
  22. Christian CouderJun 21, 2023
  23. 6/9 repack: add `--filter=<filter-spec>` optionChristian Couder, Jun 14, 2023
  24. Junio C HamanoJun 16, 2023
  25. Taylor BlauJun 21, 2023
  26. Christian CouderJun 21, 2023
  27. Taylor BlauJun 22, 2023
  28. Christian CouderJun 21, 2023
  29. Junio C HamanoJun 21, 2023
  30. Christian CouderJun 22, 2023
  31. Junio C HamanoJun 22, 2023
  32. Taylor BlauJun 21, 2023
  33. Christian CouderJul 5, 2023
  34. 8/9 repack: implement `--filter-to` for storing filtered out objectsChristian Couder, Jun 14, 2023
  35. Junio C HamanoJun 16, 2023
  36. Taylor BlauJun 21, 2023
  37. Christian CouderJun 21, 2023
  38. Taylor BlauJun 21, 2023
  39. Junio C HamanoJun 21, 2023
  40. Christian CouderJul 5, 2023
  41. 7/9 gc: add `gc.repackFilter` config optionChristian Couder, Jun 14, 2023
  42. 9/9 gc: add `gc.repackFilterTo` config optionChristian Couder, Jun 14, 2023
  43. Junio C HamanoJun 16, 2023
  44. Junio C HamanoJun 14, 2023
  45. Junio C HamanoJun 16, 2023
  46. 0/8 Repack objects into separate packfiles based on a filterChristian Couder, Jul 5, 2023
  47. 1/8 pack-objects: allow `--filter` without `--stdout`Christian Couder, Jul 5, 2023
  48. 2/8 t/helper: add 'find-pack' test-toolChristian Couder, Jul 5, 2023
  49. 3/8 repack: refactor finishing pack-objects commandChristian Couder, Jul 5, 2023
  50. 4/8 repack: refactor finding pack prefixChristian Couder, Jul 5, 2023
  51. 5/8 repack: add `--filter=<filter-spec>` optionChristian Couder, Jul 5, 2023
  52. Junio C HamanoJul 5, 2023
  53. Christian CouderJul 24, 2023
  54. Junio C HamanoJul 24, 2023
  55. Christian CouderJul 25, 2023
  56. Junio C HamanoJul 25, 2023
  57. Junio C HamanoJul 25, 2023
  58. Christian CouderAug 8, 2023
  59. Taylor BlauAug 9, 2023
  60. Junio C HamanoAug 9, 2023
  61. Junio C HamanoAug 9, 2023
  62. Jeff KingAug 10, 2023
  63. Junio C HamanoJul 5, 2023
  64. Christian CouderJul 24, 2023
  65. 6/8 gc: add `gc.repackFilter` config optionChristian Couder, Jul 5, 2023
  66. 8/8 gc: add `gc.repackFilterTo` config optionChristian Couder, Jul 5, 2023
  67. 7/8 repack: implement `--filter-to` for storing filtered out objectsChristian Couder, Jul 5, 2023
  68. Junio C HamanoJul 5, 2023
  69. Christian CouderJul 24, 2023
  70. Junio C HamanoJul 24, 2023
  71. Robert CoupJul 25, 2023
  72. Junio C HamanoJul 25, 2023
  73. Christian CouderJul 25, 2023
  74. 0/8 Repack objects into separate packfiles based on a filterChristian Couder, Jul 24, 2023
  75. 1/8 pack-objects: allow `--filter` without `--stdout`Christian Couder, Jul 24, 2023
  76. Taylor BlauJul 25, 2023
  77. Junio C HamanoJul 25, 2023
  78. 3/8 repack: refactor finishing pack-objects commandChristian Couder, Jul 24, 2023
  79. Taylor BlauJul 25, 2023
  80. 4/8 repack: refactor finding pack prefixChristian Couder, Jul 24, 2023
  81. Taylor BlauJul 25, 2023
  82. Christian CouderAug 8, 2023
  83. 2/8 t/helper: add 'find-pack' test-toolChristian Couder, Jul 24, 2023
  84. Taylor BlauJul 25, 2023
  85. Christian CouderAug 8, 2023
  86. 5/8 repack: add `--filter=<filter-spec>` optionChristian Couder, Jul 24, 2023
  87. Taylor BlauJul 25, 2023
  88. Christian CouderAug 8, 2023
  89. Taylor BlauAug 9, 2023
  90. 6/8 gc: add `gc.repackFilter` config optionChristian Couder, Jul 24, 2023
  91. Taylor BlauJul 25, 2023
  92. Christian CouderAug 8, 2023
  93. Taylor BlauAug 9, 2023
  94. 8/8 gc: add `gc.repackFilterTo` config optionChristian Couder, Jul 24, 2023
  95. 7/8 repack: implement `--filter-to` for storing filtered out objectsChristian Couder, Jul 24, 2023
  96. Taylor BlauJul 25, 2023
  97. 0/8 Repack objects into separate packfiles based on a filterChristian Couder, Aug 8, 2023
  98. 7/8 repack: implement `--filter-to` for storing filtered out objectsChristian Couder, Aug 8, 2023
  99. 5/8 repack: add `--filter=<filter-spec>` optionChristian Couder, Aug 8, 2023
  100. Taylor BlauAug 9, 2023
  101. 4/8 repack: refactor finding pack prefixChristian Couder, Aug 8, 2023
  102. Taylor BlauAug 9, 2023
  103. 3/8 repack: refactor finishing pack-objects commandChristian Couder, Aug 8, 2023
  104. 6/8 gc: add `gc.repackFilter` config optionChristian Couder, Aug 8, 2023
  105. 8/8 gc: add `gc.repackFilterTo` config optionChristian Couder, Aug 8, 2023
  106. 1/8 pack-objects: allow `--filter` without `--stdout`Christian Couder, Aug 8, 2023
  107. 2/8 t/helper: add 'find-pack' test-toolChristian Couder, Aug 8, 2023
  108. Taylor BlauAug 9, 2023
  109. Taylor BlauAug 9, 2023
  110. Junio C HamanoAug 9, 2023
  111. Christian CouderAug 12, 2023
  112. 0/8 Repack objects into separate packfiles based on a filterChristian Couder, Aug 12, 2023
  113. 1/8 pack-objects: allow `--filter` without `--stdout`Christian Couder, Aug 12, 2023
  114. 2/8 t/helper: add 'find-pack' test-toolChristian Couder, Aug 12, 2023
  115. 3/8 repack: refactor finishing pack-objects commandChristian Couder, Aug 12, 2023
  116. 4/8 repack: refactor finding pack prefixChristian Couder, Aug 12, 2023
  117. 5/8 repack: add `--filter=<filter-spec>` optionChristian Couder, Aug 12, 2023
  118. 6/8 gc: add `gc.repackFilter` config optionChristian Couder, Aug 12, 2023
  119. 7/8 repack: implement `--filter-to` for storing filtered out objectsChristian Couder, Aug 12, 2023
  120. 8/8 gc: add `gc.repackFilterTo` config optionChristian Couder, Aug 12, 2023
  121. Junio C HamanoAug 15, 2023
  122. Taylor BlauAug 15, 2023
  123. Junio C HamanoAug 15, 2023
  124. Taylor BlauAug 15, 2023
  125. Junio C HamanoAug 15, 2023
  126. Taylor BlauAug 16, 2023
  127. Junio C HamanoAug 16, 2023
  128. Christian CouderSep 11, 2023
  129. 0/9 Repack objects into separate packfiles based on a filterChristian Couder, Sep 11, 2023
  130. 4/9 repack: refactor finding pack prefixChristian Couder, Sep 11, 2023
  131. 5/9 pack-bitmap-write: rebuild using new bitmap when remappingChristian Couder, Sep 11, 2023
  132. 2/9 t/helper: add 'find-pack' test-toolChristian Couder, Sep 11, 2023
  133. 6/9 repack: add `--filter=<filter-spec>` optionChristian Couder, Sep 11, 2023
  134. 1/9 pack-objects: allow `--filter` without `--stdout`Christian Couder, Sep 11, 2023
  135. 8/9 repack: implement `--filter-to` for storing filtered out objectsChristian Couder, Sep 11, 2023
  136. 7/9 gc: add `gc.repackFilter` config optionChristian Couder, Sep 11, 2023
  137. 3/9 repack: refactor finishing pack-objects commandChristian Couder, Sep 11, 2023
  138. 9/9 gc: add `gc.repackFilterTo` config optionChristian Couder, Sep 11, 2023
  139. 0/9 Repack objects into separate packfiles based on a filterChristian Couder, Sep 25, 2023
  140. 1/9 pack-objects: allow `--filter` without `--stdout`Christian Couder, Sep 25, 2023
  141. 2/9 t/helper: add 'find-pack' test-toolChristian Couder, Sep 25, 2023
  142. 3/9 repack: refactor finishing pack-objects commandChristian Couder, Sep 25, 2023
  143. 4/9 repack: refactor finding pack prefixChristian Couder, Sep 25, 2023
  144. 5/9 pack-bitmap-write: rebuild using new bitmap when remappingChristian Couder, Sep 25, 2023
  145. 7/9 gc: add `gc.repackFilter` config optionChristian Couder, Sep 25, 2023
  146. 6/9 repack: add `--filter=<filter-spec>` optionChristian Couder, Sep 25, 2023
  147. 8/9 repack: implement `--filter-to` for storing filtered out objectsChristian Couder, Sep 25, 2023
  148. 9/9 gc: add `gc.repackFilterTo` config optionChristian Couder, Sep 25, 2023
  149. Junio C HamanoSep 25, 2023
  150. Taylor BlauSep 25, 2023
  151. 0/9 Repack objects into separate packfiles based on a filterChristian Couder, Oct 2, 2023
  152. 1/9 pack-objects: allow `--filter` without `--stdout`Christian Couder, Oct 2, 2023
  153. 2/9 t/helper: add 'find-pack' test-toolChristian Couder, Oct 2, 2023
  154. 3/9 repack: refactor finishing pack-objects commandChristian Couder, Oct 2, 2023
  155. 4/9 repack: refactor finding pack prefixChristian Couder, Oct 2, 2023
  156. 5/9 pack-bitmap-write: rebuild using new bitmap when remappingChristian Couder, Oct 2, 2023
  157. 7/9 gc: add `gc.repackFilter` config optionChristian Couder, Oct 2, 2023
  158. 9/9 gc: add `gc.repackFilterTo` config optionChristian Couder, Oct 2, 2023
  159. 8/9 repack: implement `--filter-to` for storing filtered out objectsChristian Couder, Oct 2, 2023
  160. 6/9 repack: add `--filter=<filter-spec>` optionChristian Couder, Oct 2, 2023
  161. Taylor BlauOct 2, 2023

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.