git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v2 09/10] builtin/gc.c: make `gc.cruftPacks` enabled by default

From
Taylor Blau <me@ttaylorr.com>
Date
Apr 18, 2023, 20:40 UTC
Message-ID
<b6784ddfe2906f7c04b3050bd9ba63a884ddb047.1681850424.git.me@ttaylorr.com>
In-Reply-To
<cover.1681850424.git.me@ttaylorr.com>

Back in 5b92477f89 (builtin/gc.c: conditionally avoid pruning objects via loose, 2022-05-20), `git gc` learned the `--cruft` option and `gc.cruftPacks` configuration to opt-in to writing cruft packs when collecting or pruning unreachable objects.

Cruft packs were introduced with the merge in a50036da1a (Merge branch 'tb/cruft-packs', 2022-06-03). They address the problem of "loose object explosions", where Git will write out many individual loose objects when there is a large number of unreachable objects that have not yet aged past `--prune=<date>`.

Instead of keeping track of those unreachable yet recent objects via their loose object file's mtime, cruft packs collect all unreachable objects into a single pack with a corresponding `*.mtimes` file that acts as a table to store the mtimes of all unreachable objects. This prevents the need to store unreachable objects as loose as they age out of the repository, and avoids the problem of loose object explosions.

Beyond avoiding loose object explosions, cruft packs also act as a more efficient mechanism to store unreachable objects as they age out of a repository. This is because pairs of similar unreachable objects serve as delta bases for one another.

In 5b92477f89, the feature was introduced as experimental. Since then, GitHub has been running these patches in every repository generating hundreds of millions of cruft packs along the way. The feature is battle-tested, and avoids many pathological cases such as above. Users who either run `git gc` manually, or via `git maintenance` can benefit from having cruft packs.

As such, enable cruft pack generation to take place by default (by making `gc.cruftPacks` have the default of "true" rather than "false).

Signed-off-by: Taylor Blau <me@ttaylorr.com>
---
 Documentation/config/feature.txt |  3 ---
 Documentation/config/gc.txt      |  2 +-
 Documentation/git-gc.txt         |  5 +++--
 Documentation/gitformat-pack.txt |  4 ++--
 builtin/gc.c                     |  6 +-----
 t/t6500-gc.sh                    | 12 ++++--------
 6 files changed, 11 insertions(+), 21 deletions(-)
diff --git a/Documentation/config/feature.txt b/Documentation/config/feature.txt
index e52bc6b858..17b4d39f89 100644
--- a/Documentation/config/feature.txt
+++ b/Documentation/config/feature.txt
@@ -14,9 +14,6 @@ feature.experimental::
 +
 * `fetch.negotiationAlgorithm=skipping` may improve fetch negotiation times by
 skipping more commits at a time, reducing the number of round trips.
-+
-* `gc.cruftPacks=true` reduces disk space used by unreachable objects during
-garbage collection, preventing loose object explosions.
 
 feature.manyFiles::
 	Enable config options that optimize for repos with many files in the
diff --git a/Documentation/config/gc.txt b/Documentation/config/gc.txt
index 8d5353e9e0..7f95c866e1 100644
--- a/Documentation/config/gc.txt
+++ b/Documentation/config/gc.txt
@@ -84,7 +84,7 @@ gc.packRefs::
 gc.cruftPacks::
 	Store unreachable objects in a cruft pack (see
 	linkgit:git-repack[1]) instead of as loose objects. The default
-	is `false`.
+	is `true`.
 
 gc.pruneExpire::
 	When 'git gc' is run, it will call 'prune --expire 2.weeks.ago'
diff --git a/Documentation/git-gc.txt b/Documentation/git-gc.txt
index fef382a70f..90806fd26a 100644
--- a/Documentation/git-gc.txt
+++ b/Documentation/git-gc.txt
@@ -54,9 +54,10 @@ other housekeeping tasks (e.g. rerere, working trees, reflog...) will
 be performed as well.
 
 
---cruft::
+--[no-]cruft::
 	When expiring unreachable objects, pack them separately into a
-	cruft pack instead of storing them as loose objects.
+	cruft pack instead of storing them as loose objects. `--cruft`
+	is on by default.
 
 --prune=<date>::
 	Prune loose objects older than date (default is 2 weeks ago,
diff --git a/Documentation/gitformat-pack.txt b/Documentation/gitformat-pack.txt
index e06af02f21..0c1be2dbe8 100644
--- a/Documentation/gitformat-pack.txt
+++ b/Documentation/gitformat-pack.txt
@@ -611,8 +611,8 @@ result of repeatedly resetting the objects' mtimes to the present time.
 
 If you are GC-ing repositories in a mixed version environment, consider omitting
 the `--cruft` option when using linkgit:git-repack[1] and linkgit:git-gc[1], and
-leaving the `gc.cruftPacks` configuration unset until all writers understand
-cruft packs.
+setting the `gc.cruftPacks` configuration to "false" until all writers
+understand cruft packs.
 
 === Alternatives
 
diff --git a/builtin/gc.c b/builtin/gc.c
index 53ef137e1d..ece01e966f 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -48,7 +48,7 @@ static const char * const builtin_gc_usage[] = {
 
 static int pack_refs = 1;
 static int prune_reflogs = 1;
-static int cruft_packs = -1;
+static int cruft_packs = 1;
 static int aggressive_depth = 50;
 static int aggressive_window = 250;
 static int gc_auto_threshold = 6700;
@@ -608,10 +608,6 @@ int cmd_gc(int argc, const char **argv, const char *prefix)
 	if (prune_expire && parse_expiry_date(prune_expire, &dummy))
 		die(_("failed to parse prune expiry value %s"), prune_expire);
 
-	prepare_repo_settings(the_repository);
-	if (cruft_packs < 0)
-		cruft_packs = the_repository->settings.gc_cruft_packs;
-
 	if (aggressive) {
 		strvec_push(&repack, "-f");
 		if (aggressive_depth > 0)
diff --git a/t/t6500-gc.sh b/t/t6500-gc.sh
index 3ba2ae5140..69509d0c11 100755
--- a/t/t6500-gc.sh
+++ b/t/t6500-gc.sh
@@ -216,11 +216,9 @@ assert_no_cruft_packs () {
 }
 
 for argv in \
-	"gc --cruft" \
+	"gc" \
 	"-c gc.cruftPacks=true gc" \
-	"-c gc.cruftPacks=false gc --cruft" \
-	"-c feature.experimental=true gc" \
-	"-c gc.cruftPacks=true -c feature.experimental=false gc"
+	"-c gc.cruftPacks=false gc --cruft"
 do
 	test_expect_success "git $argv generates a cruft pack" '
 		test_when_finished "rm -fr repo" &&
@@ -244,11 +242,9 @@ do
 done
 
 for argv in \
-	"gc" \
+	"gc --no-cruft" \
 	"-c gc.cruftPacks=false gc" \
-	"-c gc.cruftPacks=true gc --no-cruft" \
-	"-c feature.expiremental=true -c gc.cruftPacks=false gc" \
-	"-c feature.experimental=false gc"
+	"-c gc.cruftPacks=true gc --no-cruft"
 do
 	test_expect_success "git $argv does not generate a cruft pack" '
 		test_when_finished "rm -fr repo" &&

-- 
2.40.0.362.gc67ee7c2ff
Previous: Taylor BlauNext: Junio C Hamano
Message 42 of 48 in “gc: enable cruft packs by default”
  1. 00/10 gc: enable cruft packs by defaultTaylor Blau, Apr 17, 2023
  2. 01/10 pack-write.c: plug a leak in stage_tmp_packfiles()Taylor Blau, Apr 17, 2023
  3. Jeff KingApr 18, 2023
  4. Taylor BlauApr 18, 2023
  5. 02/10 builtin/repack.c: fix incorrect reference to '-C'Taylor Blau, Apr 17, 2023
  6. 03/10 builtin/gc.c: ignore cruft packs with `--keep-largest-pack`Taylor Blau, Apr 17, 2023
  7. Junio C HamanoApr 17, 2023
  8. Taylor BlauApr 17, 2023
  9. Jeff KingApr 18, 2023
  10. Derrick StoleeApr 18, 2023
  11. 04/10 t/t5304-prune.sh: prepare for `gc --cruft` by defaultTaylor Blau, Apr 17, 2023
  12. 05/10 t/t9300-fast-import.sh: prepare for `gc --cruft` by defaultTaylor Blau, Apr 17, 2023
  13. Jeff KingApr 18, 2023
  14. Taylor BlauApr 18, 2023
  15. 06/10 t/t6500-gc.sh: refactor cruft pack testsTaylor Blau, Apr 17, 2023
  16. 07/10 t/t6500-gc.sh: add additional test casesTaylor Blau, Apr 17, 2023
  17. Jeff KingApr 18, 2023
  18. Taylor BlauApr 18, 2023
  19. 08/10 t/t6501-freshen-objects.sh: prepare for `gc --cruft` by defaultTaylor Blau, Apr 17, 2023
  20. Jeff KingApr 18, 2023
  21. Taylor BlauApr 18, 2023
  22. Jeff KingApr 22, 2023
  23. 09/10 builtin/gc.c: make `gc.cruftPacks` enabled by defaultTaylor Blau, Apr 17, 2023
  24. Jeff KingApr 18, 2023
  25. Taylor BlauApr 18, 2023
  26. 10/10 repository.h: drop unused `gc_cruft_packs`Taylor Blau, Apr 17, 2023
  27. Jeff KingApr 18, 2023
  28. Jeff KingApr 18, 2023
  29. Taylor BlauApr 18, 2023
  30. 00/10 gc: enable cruft packs by defaultTaylor Blau, Apr 18, 2023
  31. 01/10 pack-write.c: plug a leak in stage_tmp_packfiles()Taylor Blau, Apr 18, 2023
  32. Junio C HamanoApr 19, 2023
  33. Taylor BlauApr 20, 2023
  34. Junio C HamanoApr 20, 2023
  35. 02/10 builtin/repack.c: fix incorrect reference to '-C'Taylor Blau, Apr 18, 2023
  36. 03/10 builtin/gc.c: ignore cruft packs with `--keep-largest-pack`Taylor Blau, Apr 18, 2023
  37. 04/10 t/t5304-prune.sh: prepare for `gc --cruft` by defaultTaylor Blau, Apr 18, 2023
  38. 05/10 t/t6501-freshen-objects.sh: prepare for `gc --cruft` by defaultTaylor Blau, Apr 18, 2023
  39. 06/10 t/t6500-gc.sh: refactor cruft pack testsTaylor Blau, Apr 18, 2023
  40. 07/10 t/t6500-gc.sh: add additional test casesTaylor Blau, Apr 18, 2023
  41. 08/10 t/t9300-fast-import.sh: prepare for `gc --cruft` by defaultTaylor Blau, Apr 18, 2023
  42. 09/10 builtin/gc.c: make `gc.cruftPacks` enabled by defaultTaylor Blau, Apr 18, 2023
  43. Junio C HamanoApr 19, 2023
  44. Taylor BlauApr 20, 2023
  45. Junio C HamanoApr 20, 2023
  46. Taylor BlauApr 20, 2023
  47. 10/10 repository.h: drop unused `gc_cruft_packs`Taylor Blau, Apr 18, 2023
  48. Junio C HamanoApr 19, 2023

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.