git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v3 02/10] repack: fix trying to use preferred pack in alternates

From
Patrick Steinhardt <ps@pks.im>
Date
Apr 13, 2023, 11:16 UTC
Message-ID
<c5db1bc587701df35f2cb32270ab734ffdec2e32.1681384405.git.ps@pks.im>
In-Reply-To
<cover.1681384405.git.ps@pks.im>

When doing a geometric repack with multi-pack-indices, then we ask git-multi-pack-index(1) to use the largest packfile as the preferred pack. It can happen though that the largest packfile is not part of the main object database, but instead part of an alternate object database. The result is that git-multi-pack-index(1) will not be able to find the preferred pack and print a warning. It then falls back to use the first packfile that the multi-pack-index shall reference.

Fix this bug by only considering packfiles as preferred pack that are local. This is the right thing to do given that a multi-pack-index should never reference packfiles borrowed from an alternate.

While at it, rename the function `get_largest_active_packfile()` to `get_preferred_pack()` to better document its intent.

Helped-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 builtin/repack.c            | 26 +++++++++++++++++++++-----
 t/t7703-repack-geometric.sh | 32 ++++++++++++++++++++++++++++++++
 2 files changed, 53 insertions(+), 5 deletions(-)
diff --git a/builtin/repack.c b/builtin/repack.c
index 87f73c8923..687419776d 100644
--- a/builtin/repack.c
+++ b/builtin/repack.c
@@ -446,8 +446,10 @@ static void split_pack_geometry(struct pack_geometry *geometry, int factor)
 	geometry->split = split;
 }
 
-static struct packed_git *get_largest_active_pack(struct pack_geometry *geometry)
+static struct packed_git *get_preferred_pack(struct pack_geometry *geometry)
 {
+	uint32_t i;
+
 	if (!geometry) {
 		/*
 		 * No geometry means either an all-into-one repack (in which
@@ -462,7 +464,21 @@ static struct packed_git *get_largest_active_pack(struct pack_geometry *geometry
 	}
 	if (geometry->split == geometry->pack_nr)
 		return NULL;
-	return geometry->pack[geometry->pack_nr - 1];
+
+	/*
+	 * The preferred pack is the largest pack above the split line. In
+	 * other words, it is the largest pack that does not get rolled up in
+	 * the geometric repack.
+	 */
+	for (i = geometry->pack_nr; i > geometry->split; i--)
+		/*
+		 * A pack that is not local would never be included in a
+		 * multi-pack index. We thus skip over any non-local packs.
+		 */
+		if (geometry->pack[i - 1]->pack_local)
+			return geometry->pack[i - 1];
+
+	return NULL;
 }
 
 static void clear_pack_geometry(struct pack_geometry *geometry)
@@ -589,7 +605,7 @@ static int write_midx_included_packs(struct string_list *include,
 {
 	struct child_process cmd = CHILD_PROCESS_INIT;
 	struct string_list_item *item;
-	struct packed_git *largest = get_largest_active_pack(geometry);
+	struct packed_git *preferred = get_preferred_pack(geometry);
 	FILE *in;
 	int ret;
 
@@ -610,9 +626,9 @@ static int write_midx_included_packs(struct string_list *include,
 	if (write_bitmaps)
 		strvec_push(&cmd.args, "--bitmap");
 
-	if (largest)
+	if (preferred)
 		strvec_pushf(&cmd.args, "--preferred-pack=%s",
-			     pack_basename(largest));
+			     pack_basename(preferred));
 
 	if (refs_snapshot)
 		strvec_pushf(&cmd.args, "--refs-snapshot=%s", refs_snapshot);
diff --git a/t/t7703-repack-geometric.sh b/t/t7703-repack-geometric.sh
index 8821fbd2dd..d6db362541 100755
--- a/t/t7703-repack-geometric.sh
+++ b/t/t7703-repack-geometric.sh
@@ -281,4 +281,36 @@ test_expect_success '--geometric with pack.packSizeLimit' '
 	)
 '
 
+test_expect_success '--geometric --write-midx with packfiles in main and alternate ODB' '
+	test_when_finished "rm -fr shared member" &&
+
+	# Create a shared repository that will serve as the alternate object
+	# database for the member linked to it. It has got some objects on its
+	# own that are packed into a single packfile.
+	git init shared &&
+	test_commit -C shared common-object &&
+	git -C shared repack -ad &&
+
+	# We create member so that its alternates file points to the shared
+	# repository. We then create a commit in it so that git-repack(1) has
+	# something to repack.
+	# of the shared object database.
+	git clone --shared shared member &&
+	test_commit -C member unique-object &&
+	git -C member repack --geometric=2 --write-midx 2>err &&
+	test_must_be_empty err &&
+
+	# We should see that a new packfile was generated.
+	find shared/.git/objects/pack -type f -name "*.pack" >packs &&
+	test_line_count = 1 packs &&
+
+	# We should also see a multi-pack-index. This multi-pack-index should
+	# never refer to any packfiles in the alternate object database.
+	test -f member/.git/objects/pack/multi-pack-index &&
+	test-tool read-midx member/.git/objects >packs.midx &&
+	grep "^pack-.*\.idx$" packs.midx | sort >actual &&
+	basename member/.git/objects/pack/pack-*.idx >expect &&
+	test_cmp expect actual
+'
+
 test_done
-- 
2.40.0
Previous: Derrick StoleeNext: Patrick Steinhardt
Message 40 of 67 in “repack: fix geometric repacking with gitalternates”
  1. repack: fix geometric repacking with gitalternatesPatrick Steinhardt, Apr 4, 2023
  2. Taylor BlauApr 4, 2023
  3. Taylor BlauApr 4, 2023
  4. Patrick SteinhardtApr 5, 2023
  5. Derrick StoleeApr 10, 2023
  6. Taylor BlauApr 10, 2023
  7. Patrick SteinhardtApr 11, 2023
  8. Taylor BlauApr 11, 2023
  9. Patrick SteinhardtApr 12, 2023
  10. Patrick SteinhardtApr 11, 2023
  11. Patrick SteinhardtApr 11, 2023
  12. Taylor BlauApr 11, 2023
  13. Taylor BlauApr 10, 2023
  14. 0/8 repack: fix geometric repacking with gitalternatesPatrick Steinhardt, Apr 12, 2023
  15. 1/8 midx: fix segfault with no packs and invalid preferred packPatrick Steinhardt, Apr 12, 2023
  16. Taylor BlauApr 12, 2023
  17. Patrick SteinhardtApr 13, 2023
  18. 2/8 repack: fix trying to use preferred pack in alternatesPatrick Steinhardt, Apr 12, 2023
  19. Taylor BlauApr 12, 2023
  20. Patrick SteinhardtApr 13, 2023
  21. 3/8 repack: fix generating multi-pack-index with only non-local packsPatrick Steinhardt, Apr 12, 2023
  22. Taylor BlauApr 12, 2023
  23. 4/8 pack-objects: fix error when packing same pack twicePatrick Steinhardt, Apr 12, 2023
  24. Taylor BlauApr 12, 2023
  25. 5/8 pack-objects: fix error when same packfile is included and excludedPatrick Steinhardt, Apr 12, 2023
  26. Taylor BlauApr 12, 2023
  27. 6/8 pack-objects: extend test coverage of `--stdin-packs` with alternatesPatrick Steinhardt, Apr 12, 2023
  28. 7/8 repack: honor `-l` when calculating pack geometryPatrick Steinhardt, Apr 12, 2023
  29. Junio C HamanoApr 12, 2023
  30. Junio C HamanoApr 13, 2023
  31. Patrick SteinhardtApr 13, 2023
  32. 8/8 repack: disable writing bitmaps when doing a local geometric repackPatrick Steinhardt, Apr 12, 2023
  33. Taylor BlauApr 12, 2023
  34. Patrick SteinhardtApr 13, 2023
  35. Patrick SteinhardtApr 13, 2023
  36. Taylor BlauApr 12, 2023
  37. 00/10 repack: fix geometric repacking with gitalternatesPatrick Steinhardt, Apr 13, 2023
  38. 01/10 midx: fix segfault with no packs and invalid preferred packPatrick Steinhardt, Apr 13, 2023
  39. Derrick StoleeApr 13, 2023
  40. 02/10 repack: fix trying to use preferred pack in alternatesPatrick Steinhardt, Apr 13, 2023
  41. 03/10 repack: fix generating multi-pack-index with only non-local packsPatrick Steinhardt, Apr 13, 2023
  42. 04/10 pack-objects: split out `--stdin-packs` tests into separate filePatrick Steinhardt, Apr 13, 2023
  43. 05/10 pack-objects: fix error when packing same pack twicePatrick Steinhardt, Apr 13, 2023
  44. 06/10 pack-objects: fix error when same packfile is included and excludedPatrick Steinhardt, Apr 13, 2023
  45. 07/10 pack-objects: extend test coverage of `--stdin-packs` with alternatesPatrick Steinhardt, Apr 13, 2023
  46. 08/10 t/helper: allow chmtime to print verbosely without modifying mtimePatrick Steinhardt, Apr 13, 2023
  47. 09/10 repack: honor `-l` when calculating pack geometryPatrick Steinhardt, Apr 13, 2023
  48. Derrick StoleeApr 13, 2023
  49. Patrick SteinhardtApr 13, 2023
  50. Junio C HamanoApr 13, 2023
  51. 10/10 repack: disable writing bitmaps when doing a local repackPatrick Steinhardt, Apr 13, 2023
  52. Derrick StoleeApr 13, 2023
  53. Junio C HamanoApr 14, 2023
  54. Patrick SteinhardtApr 14, 2023
  55. 00/10 repack: fix geometric repacking with gitalternatesPatrick Steinhardt, Apr 14, 2023
  56. 01/10 midx: fix segfault with no packs and invalid preferred packPatrick Steinhardt, Apr 14, 2023
  57. 03/10 repack: fix generating multi-pack-index with only non-local packsPatrick Steinhardt, Apr 14, 2023
  58. 02/10 repack: fix trying to use preferred pack in alternatesPatrick Steinhardt, Apr 14, 2023
  59. 04/10 pack-objects: split out `--stdin-packs` tests into separate filePatrick Steinhardt, Apr 14, 2023
  60. 05/10 pack-objects: fix error when packing same pack twicePatrick Steinhardt, Apr 14, 2023
  61. 06/10 pack-objects: fix error when same packfile is included and excludedPatrick Steinhardt, Apr 14, 2023
  62. 07/10 pack-objects: extend test coverage of `--stdin-packs` with alternatesPatrick Steinhardt, Apr 14, 2023
  63. 08/10 t/helper: allow chmtime to print verbosely without modifying mtimePatrick Steinhardt, Apr 14, 2023
  64. 09/10 repack: honor `-l` when calculating pack geometryPatrick Steinhardt, Apr 14, 2023
  65. 10/10 repack: disable writing bitmaps when doing a local repackPatrick Steinhardt, Apr 14, 2023
  66. Derrick StoleeApr 14, 2023
  67. Junio C HamanoApr 14, 2023

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.