git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v4 3/3] gc: add gc.repackFilter config option

From
Christian Couder <christian.couder@gmail.com>
Date
Dec 21, 2022, 04:04 UTC
Message-ID
<20221221040446.2860985-4-christian.couder@gmail.com>
In-Reply-To
<20221221040446.2860985-1-christian.couder@gmail.com>

A previous commit has implemented `git repack --filter=<filter-spec>` to allow users to remove objects that are available on a promisor remote but that they don't want locally.

Users might want to perform such a cleanup regularly at the same time as they perform other repacks and cleanups, so as part of `git gc`.

Let's allow them to configure a <filter-spec> for that purpose using a new gc.repackFilter config option.

Now when `git gc` will perform a repack with a <filter-spec> configured through this option and not empty, the repack process will be passed a corresponding `--filter=<filter-spec>` argument.

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
 Documentation/config/gc.txt |  9 +++++++++
 builtin/gc.c                |  6 ++++++
 t/t6500-gc.sh               | 19 +++++++++++++++++++
 3 files changed, 34 insertions(+)
diff --git a/Documentation/config/gc.txt b/Documentation/config/gc.txt
index 38fea076a2..9359136f14 100644
--- a/Documentation/config/gc.txt
+++ b/Documentation/config/gc.txt
@@ -130,6 +130,15 @@ or rebase occurring.  Since these changes are not part of the current
 project most users will want to expire them sooner, which is why the
 default is more aggressive than `gc.reflogExpire`.
 
+gc.repackFilter::
+	When repacking, use the specified filter to omit certain
+	objects from the resulting packfile. WARNING: this could
+	easily corrupt the current repo and lose data if ANY of the
+	omitted objects hasn't been already pushed to a remote. Be
+	very careful about objects that might have been created
+	locally! See the `--filter=<filter-spec>` option of
+	linkgit:git-repack[1].
+
 gc.rerereResolved::
 	Records of conflicted merge you resolved earlier are
 	kept for this many days when 'git rerere gc' is run.
diff --git a/builtin/gc.c b/builtin/gc.c
index 02455fdcd7..bf28619723 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -52,6 +52,7 @@ static timestamp_t gc_log_expire_time;
 static const char *gc_log_expire = "1.day.ago";
 static const char *prune_expire = "2.weeks.ago";
 static const char *prune_worktrees_expire = "3.months.ago";
+static char *repack_filter;
 static unsigned long big_pack_threshold;
 static unsigned long max_delta_cache_size = DEFAULT_DELTA_CACHE_SIZE;
 
@@ -161,6 +162,8 @@ static void gc_config(void)
 	git_config_get_ulong("gc.bigpackthreshold", &big_pack_threshold);
 	git_config_get_ulong("pack.deltacachesize", &max_delta_cache_size);
 
+	git_config_get_string("gc.repackfilter", &repack_filter);
+
 	git_config(git_default_config, NULL);
 }
 
@@ -346,6 +349,9 @@ static void add_repack_all_option(struct string_list *keep_pack)
 
 	if (keep_pack)
 		for_each_string_list(keep_pack, keep_one_pack, NULL);
+
+	if (repack_filter && *repack_filter)
+		strvec_pushf(&repack, "--filter=%s", repack_filter);
 }
 
 static void add_repack_incremental_option(void)
diff --git a/t/t6500-gc.sh b/t/t6500-gc.sh
index d9acb63951..b1492b521a 100755
--- a/t/t6500-gc.sh
+++ b/t/t6500-gc.sh
@@ -56,6 +56,7 @@ test_expect_success 'gc -h with invalid configuration' '
 '
 
 test_expect_success 'gc is not aborted due to a stale symref' '
+	test_when_finished "rm -rf remote client" &&
 	git init remote &&
 	(
 		cd remote &&
@@ -202,6 +203,24 @@ test_expect_success 'one of gc.reflogExpire{Unreachable,}=never does not skip "e
 	grep -E "^trace: (built-in|exec|run_command): git reflog expire --" trace.out
 '
 
+test_expect_success 'gc.repackFilter launches repack with a filter' '
+	test_when_finished "rm -rf server client" &&
+	test_create_repo server &&
+	git -C server config uploadpack.allowFilter true &&
+	git -C server config uploadpack.allowAnySHA1InWant true &&
+	test_commit -C server 1 &&
+	git clone --bare --no-local server client &&
+	git -C client config remote.origin.promisor true &&
+	git -C client rev-list --objects --all --missing=print >objects &&
+	test $(grep -c "^?" objects) = 0 &&
+
+	GIT_TRACE=$(pwd)/trace.out git -C client -c gc.repackFilter=blob:none -c repack.writeBitmaps=false -c gc.pruneExpire=now gc &&
+
+	grep -E "^trace: (built-in|exec|run_command): git repack .* --filter=blob:none ?.*" trace.out &&
+	git -C client rev-list --objects --all --missing=print >objects &&
+	test $(grep -c "^?" objects) = 1
+'
+
 prepare_cruft_history () {
 	test_commit base &&
 
-- 
2.39.0.59.g395bcb85bc.dirty
Previous: Junio C HamanoNext: Patrick Steinhardt
Message 28 of 46 in “Implement filtering repacks”
  1. 0/3 Implement filtering repacksChristian Couder, Oct 12, 2022
  2. 1/3 pack-objects: allow --filter without --stdoutChristian Couder, Oct 12, 2022
  3. 2/3 repack: add --filter=<filter-spec> optionChristian Couder, Oct 12, 2022
  4. 3/3 repack: introduce --force to force filteringChristian Couder, Oct 12, 2022
  5. Junio C HamanoOct 14, 2022
  6. Christian CouderOct 20, 2022
  7. Taylor BlauOct 28, 2022
  8. Junio C HamanoOct 28, 2022
  9. Christian CouderNov 7, 2022
  10. Christian CouderNov 7, 2022
  11. 0/2 Implement filtering repacksChristian Couder, Oct 25, 2022
  12. 1/2 pack-objects: allow --filter without --stdoutChristian Couder, Oct 25, 2022
  13. 2/2 repack: add --filter=<filter-spec> optionChristian Couder, Oct 25, 2022
  14. Taylor BlauOct 28, 2022
  15. Christian CouderNov 7, 2022
  16. 0/2 Implement filtering repacksChristian Couder, Nov 22, 2022
  17. 1/2 pack-objects: allow --filter without --stdoutChristian Couder, Nov 22, 2022
  18. 2/2 repack: add --filter=<filter-spec> optionChristian Couder, Nov 22, 2022
  19. Junio C HamanoNov 23, 2022
  20. Christian CouderDec 21, 2022
  21. Junio C HamanoNov 23, 2022
  22. 0/3 Implement filtering repacksChristian Couder, Dec 21, 2022
  23. 1/3 pack-objects: allow --filter without --stdoutChristian Couder, Dec 21, 2022
  24. Patrick SteinhardtJan 4, 2023
  25. 2/3 repack: add --filter=<filter-spec> optionChristian Couder, Dec 21, 2022
  26. Patrick SteinhardtJan 4, 2023
  27. Junio C HamanoJan 5, 2023
  28. 3/3 gc: add gc.repackFilter config optionChristian Couder, Dec 21, 2022
  29. Patrick SteinhardtJan 4, 2023
  30. 0/3 upload-pack: support a missing-actionChristian Couder, May 15, 2024
  31. 1/3 rev-list: refactor --missing=<missing-action>Christian Couder, May 15, 2024
  32. Junio C HamanoMay 15, 2024
  33. 2/3 pack-objects: use the missing action APIChristian Couder, May 15, 2024
  34. Junio C HamanoMay 15, 2024
  35. Christian CouderMay 24, 2024
  36. 3/3 upload-pack: allow configuring a missing-actionChristian Couder, May 15, 2024
  37. Junio C HamanoMay 15, 2024
  38. Christian CouderMay 24, 2024
  39. Junio C HamanoMay 24, 2024
  40. Christian CouderMay 28, 2024
  41. Junio C HamanoMay 28, 2024
  42. Christian CouderMay 31, 2024
  43. Junio C HamanoJun 1, 2024
  44. Christian CouderJun 3, 2024
  45. Junio C HamanoJun 3, 2024
  46. Christian CouderMay 15, 2024

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.