git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 3/8] builtin/maintenance: introduce "geometric-repack" task

From
Justin Tobler <jltobler@gmail.com>
Date
Oct 16, 2025, 20:51 UTC
Message-ID
<uos7cczvzlrwjgcyhzyfirck62qjnb4zcoy6ga2o3pbnba7cfj@ag2pnonze5tu>
In-Reply-To
<20251016-pks-maintenance-geometric-strategy-v1-3-18943d474203@pks.im>
On 25/10/16 09:26AM, Patrick Steinhardt wrote:
Show 19 quoted lines
> Introduce a new "geometric-repack" task. This task uses our geometric
> repack infrastructure as provided by git-repack(1) itself, which is a
> strategy that especially hosting providers tend to use to amortize the
> costs of repacking objects.
> 
> There is one issue though with geometric repacks, namely that they
> unconditionally pack all loose objects, regardless of whether or not
> they are reachable. This is done because it means that we can completely
> skip the reachability step, which significantly speeds up the operation.
> But it has the big downside that we are unable to expire objects over
> time.
> 
> To address this issue we thus use a split strategy in this new task:
> whenever a geometric repack would merge together all packs, we instead
> do an all-into-one repack. By default, these all-into-one repacks have
> cruft packs enabled, so unreachable objects would now be written into
> their own pack. Consequently, they won't be soaked up during geometric
> repacking anymore and can be expired with the next full repack, assuming
> that their expiry date has surpassed.

So normal geometric repacks don't ever check for unreachable objects, even if all the packs are being merged together. With this new strategy though, when a geometric repack would normally merge together all packs, we instead to an all-into-one repack which does check for unreachable objects.

Does checking for unreachable objects in this case slow down the repack significantly?

> Signed-off-by: Patrick Steinhardt <ps@pks.im>
> ---
[snip]
Show 30 quoted lines
> @@ -1566,6 +1568,101 @@ static int maintenance_task_incremental_repack(struct maintenance_run_opts *opts
>  	return 0;
>  }
>  
> +static int maintenance_task_geometric_repack(struct maintenance_run_opts *opts,
> +					     struct gc_config *cfg)
> +{
> +	struct pack_geometry geometry = {
> +		.split_factor = 2,
> +	};
> +	struct pack_objects_args po_args = {
> +		.local = 1,
> +	};
> +	struct existing_packs existing_packs = EXISTING_PACKS_INIT;
> +	struct string_list kept_packs = STRING_LIST_INIT_DUP;
> +	struct child_process child = CHILD_PROCESS_INIT;
> +	int ret;
> +
> +	existing_packs.repo = the_repository;
> +	existing_packs_collect(&existing_packs, &kept_packs);
> +	pack_geometry_init(&geometry, &existing_packs, &po_args);
> +	pack_geometry_split(&geometry);
> +
> +	child.git_cmd = 1;
> +
> +	strvec_pushl(&child.args, "repack", "-d", "-l", NULL);
> +	if (geometry.split < geometry.pack_nr)
> +		strvec_push(&child.args, "--geometric=2");
> +	else
> +		add_repack_all_option(cfg, NULL, &child.args);

Here we do the full repack when the all packs are to be merged. Makes sense.

[snip]
Show 9 quoted lines
> @@ -1608,6 +1705,11 @@ static const struct maintenance_task tasks[] = {
>  		.background = maintenance_task_incremental_repack,
>  		.auto_condition = incremental_repack_auto_condition,
>  	},
> +	[TASK_GEOMETRIC_REPACK] = {
> +		.name = "geometric-repack",
> +		.background = maintenance_task_geometric_repack,
> +		.auto_condition = geometric_repack_auto_condition,
> +	},
Here we configure the new maintenance task. Nice :)
The rest of this patch looks good.
-Justin
Previous: Patrick SteinhardtNext: Patrick Steinhardt
Message 8 of 69 in “builtin/maintenance: introduce "geometric" strategy”
  1. 0/8 builtin/maintenance: introduce "geometric" strategyPatrick Steinhardt, Oct 16, 2025
  2. 1/8 builtin/gc: remove global `repack` variablePatrick Steinhardt, Oct 16, 2025
  3. Justin ToblerOct 16, 2025
  4. Taylor BlauOct 17, 2025
  5. 2/8 builtin/gc: make `too_many_loose_objects()` reusable without GC configPatrick Steinhardt, Oct 16, 2025
  6. Junio C HamanoOct 16, 2025
  7. 3/8 builtin/maintenance: introduce "geometric-repack" taskPatrick Steinhardt, Oct 16, 2025
  8. Justin ToblerOct 16, 2025
  9. Patrick SteinhardtOct 17, 2025
  10. Taylor BlauOct 17, 2025
  11. Patrick SteinhardtOct 21, 2025
  12. Taylor BlauOct 23, 2025
  13. Patrick SteinhardtOct 24, 2025
  14. 4/8 builtin/maintenance: don't silently ignore invalid strategyPatrick Steinhardt, Oct 16, 2025
  15. 5/8 builtin/maintenance: run maintenance tasks depending on typePatrick Steinhardt, Oct 16, 2025
  16. 6/8 builtin/maintenance: extend "maintenance.strategy" to manual maintenancePatrick Steinhardt, Oct 16, 2025
  17. 7/8 builtin/maintenance: make "gc" strategy accessiblePatrick Steinhardt, Oct 16, 2025
  18. 8/8 builtin/maintenance: introduce "geometric" strategyPatrick Steinhardt, Oct 16, 2025
  19. 0/9 builtin/maintenance: introduce "geometric" strategyPatrick Steinhardt, Oct 21, 2025
  20. 1/9 builtin/gc: remove global `repack` variablePatrick Steinhardt, Oct 21, 2025
  21. 2/9 builtin/gc: make `too_many_loose_objects()` reusable without GC configPatrick Steinhardt, Oct 21, 2025
  22. 3/9 builtin/maintenance: introduce "geometric-repack" taskPatrick Steinhardt, Oct 21, 2025
  23. Taylor BlauOct 23, 2025
  24. Patrick SteinhardtOct 24, 2025
  25. 4/9 builtin/maintenance: make the geometric factor configurablePatrick Steinhardt, Oct 21, 2025
  26. Taylor BlauOct 23, 2025
  27. Patrick SteinhardtOct 24, 2025
  28. Taylor BlauOct 24, 2025
  29. 5/9 builtin/maintenance: don't silently ignore invalid strategyPatrick Steinhardt, Oct 21, 2025
  30. Taylor BlauOct 23, 2025
  31. 6/9 builtin/maintenance: run maintenance tasks depending on typePatrick Steinhardt, Oct 21, 2025
  32. Taylor BlauOct 23, 2025
  33. 7/9 builtin/maintenance: extend "maintenance.strategy" to manual maintenancePatrick Steinhardt, Oct 21, 2025
  34. 8/9 builtin/maintenance: make "gc" strategy accessiblePatrick Steinhardt, Oct 21, 2025
  35. 9/9 builtin/maintenance: introduce "geometric" strategyPatrick Steinhardt, Oct 21, 2025
  36. Taylor BlauOct 23, 2025
  37. Patrick SteinhardtOct 24, 2025
  38. Junio C HamanoOct 23, 2025
  39. Taylor BlauOct 23, 2025
  40. 00/10 builtin/maintenance: introduce "geometric" strategyPatrick Steinhardt, Oct 24, 2025
  41. 01/10 builtin/gc: remove global `repack` variablePatrick Steinhardt, Oct 24, 2025
  42. 02/10 builtin/gc: make `too_many_loose_objects()` reusable without GC configPatrick Steinhardt, Oct 24, 2025
  43. 03/10 builtin/maintenance: introduce "geometric-repack" taskPatrick Steinhardt, Oct 24, 2025
  44. Jeff KingOct 25, 2025
  45. Patrick SteinhardtOct 27, 2025
  46. Jeff KingOct 27, 2025
  47. 04/10 builtin/maintenance: make the geometric factor configurablePatrick Steinhardt, Oct 24, 2025
  48. 05/10 builtin/maintenance: don't silently ignore invalid strategyPatrick Steinhardt, Oct 24, 2025
  49. 06/10 builtin/maintenance: improve readability of strategiesPatrick Steinhardt, Oct 24, 2025
  50. 07/10 builtin/maintenance: run maintenance tasks depending on typePatrick Steinhardt, Oct 24, 2025
  51. 08/10 builtin/maintenance: extend "maintenance.strategy" to manual maintenancePatrick Steinhardt, Oct 24, 2025
  52. 09/10 builtin/maintenance: make "gc" strategy accessiblePatrick Steinhardt, Oct 24, 2025
  53. 10/10 builtin/maintenance: introduce "geometric" strategyPatrick Steinhardt, Oct 24, 2025
  54. Taylor BlauOct 24, 2025
  55. Junio C HamanoOct 24, 2025
  56. 00/10 builtin/maintenance: introduce "geometric" strategyPatrick Steinhardt, Oct 27, 2025
  57. 01/10 builtin/gc: remove global `repack` variablePatrick Steinhardt, Oct 27, 2025
  58. 02/10 builtin/gc: make `too_many_loose_objects()` reusable without GC configPatrick Steinhardt, Oct 27, 2025
  59. 03/10 builtin/maintenance: introduce "geometric-repack" taskPatrick Steinhardt, Oct 27, 2025
  60. 04/10 builtin/maintenance: make the geometric factor configurablePatrick Steinhardt, Oct 27, 2025
  61. 05/10 builtin/maintenance: don't silently ignore invalid strategyPatrick Steinhardt, Oct 27, 2025
  62. 06/10 builtin/maintenance: improve readability of strategiesPatrick Steinhardt, Oct 27, 2025
  63. 07/10 builtin/maintenance: run maintenance tasks depending on typePatrick Steinhardt, Oct 27, 2025
  64. 08/10 builtin/maintenance: extend "maintenance.strategy" to manual maintenancePatrick Steinhardt, Oct 27, 2025
  65. 09/10 builtin/maintenance: make "gc" strategy accessiblePatrick Steinhardt, Oct 27, 2025
  66. 10/10 builtin/maintenance: introduce "geometric" strategyPatrick Steinhardt, Oct 27, 2025
  67. Junio C HamanoOct 27, 2025
  68. Patrick SteinhardtOct 27, 2025
  69. Junio C HamanoOct 27, 2025

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.