git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH] repack: fix geometric repacking with gitalternates

From
Patrick Steinhardt <ps@pks.im>
Date
Apr 4, 2023, 11:08 UTC
Message-ID
<a07ed50feeec4bfc3e9736bf493b9876896bcdd2.1680606445.git.ps@pks.im>

Performing geometric repacking with repositories that have alternate object directories set up is causing errors in different scenarios.

- Repacking fails when the repository linked to the object
  directory does not have any objects on its own.
  ```
  $ git init shared
  $ git -C shared commit --allow-empty --message something
  $ git clone --shared shared member
  $ git -C member repack --geometric=2 --write-midx
  Nothing new to pack.
  warning: unknown preferred pack: 'pack-3e1a94a8dc9bb4defb0d98ce2ebf325312d76362.pack'
  error: multi-pack-index died of signal 7
  ```
- Repacking fails when the repository linked to the object
  directory has the exact same packfile as the linked-to object
  directory.
  ```
  $ git init shared
  $ git -C shared commit --allow-empty --message something
  $ git -C shared repack -Ad
  $ cp -r shared member
  $ realpath shared/.git/objects >member/.git/objects/info/alternates
  $ git -C member repack --geometric 2 --write-midx
  fatal: could not find pack 'pack-d404037a861afe456e07a1aefb3655150f1299f0.pack'
  ```

Both issues have the same underlying root cause, which is that geometric repacks don't honor whether packfiles are local or not. As a result, they will try to include packs part of the alternate object directory and then at a later point fail to locate them as they do not exist in the object directory of the repository we're about to repack.

Skip over packfiles that aren't local. This will cause geometric repacks to never include packfiles of its alternates.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 builtin/repack.c            |  6 ++++
 t/t7703-repack-geometric.sh | 59 +++++++++++++++++++++++++++++++++++++
 2 files changed, 65 insertions(+)
diff --git a/builtin/repack.c b/builtin/repack.c
index 87f73c8923..c6d12fa4bd 100644
--- a/builtin/repack.c
+++ b/builtin/repack.c
@@ -333,6 +333,12 @@ static void init_pack_geometry(struct pack_geometry **geometry_p,
 	geometry = *geometry_p;
 
 	for (p = get_all_packs(the_repository); p; p = p->next) {
+		/*
+		 * We don't want to repack packfiles which are not part of the
+		 * primary object database.
+		 */
+		if (!p->pack_local)
+			continue;
 		if (!pack_kept_objects) {
 			/*
 			 * Any pack that has its pack_keep bit set will appear
diff --git a/t/t7703-repack-geometric.sh b/t/t7703-repack-geometric.sh
index 8821fbd2dd..9f8bc663e4 100755
--- a/t/t7703-repack-geometric.sh
+++ b/t/t7703-repack-geometric.sh
@@ -281,4 +281,63 @@ test_expect_success '--geometric with pack.packSizeLimit' '
 	)
 '
 
+packed_objects() {
+	git verify-pack -v "$@" >tmp-object-list &&
+	sed -n -e "s/^\([0-9a-f][0-9a-f]*\).*\(commit\|tree\|blob\|tag\).*/\1/p" <tmp-object-list &&
+	rm -f tmp-object-list
+}
+
+test_expect_success '--geometric with no local objects creates no pack' '
+	git init shared &&
+	test_when_finished "rm -fr shared" &&
+	test_commit -C shared "shared" &&
+	git -C shared repack -Ad &&
+
+	# Set up the member repository so that it does not have
+	# any objects on its own.
+	git clone --shared shared member &&
+	test_when_finished "rm -fr member" &&
+
+	git -C member repack --geometric 2 --write-midx &&
+	find member/.git/objects/pack -type f >actual &&
+	test_must_be_empty actual
+'
+
+test_expect_success '--geometric does not include shared packfiles' '
+	git init shared &&
+	test_when_finished "rm -fr shared" &&
+	test_commit -C shared "shared" &&
+	git -C shared repack -Ad &&
+
+	git clone --shared shared member &&
+	test_when_finished "rm -fr member" &&
+	git -C member commit --allow-empty --message "not-shared" &&
+	git -C member repack --geometric 2 --write-midx &&
+
+	# We expect the created packfile to only contain the new commit.
+	packed_objects member/.git/objects/pack/pack-*.idx >actual &&
+	git -C member rev-parse HEAD >expect &&
+	test_cmp expect actual
+'
+
+test_expect_success '--geometric with same packfile in shared repository' '
+	git init shared &&
+	test_when_finished "rm -fr shared" &&
+	test_commit -C shared "shared" &&
+	git -C shared repack -Ad &&
+
+	# Prepare the member repository so that it got the exact same packfile
+	# as the shared repository and set up gitalternates.
+	cp -r shared member &&
+	test_when_finished "rm -fr member" &&
+	test-tool path-utils real_path shared/.git/objects >member/.git/objects/info/alternates &&
+	find shared/.git/objects -type f >expect &&
+
+	# After repacking, contents of the member repository should not have
+	# changed.
+	git -C member repack --geometric 2 --write-midx 2>error &&
+	find shared/.git/objects -type f >actual &&
+	test_cmp expect actual
+'
+
 test_done
-- 
2.40.0
Next: Taylor Blau
Message 1 of 67 in “repack: fix geometric repacking with gitalternates”
  1. repack: fix geometric repacking with gitalternatesPatrick Steinhardt, Apr 4, 2023
  2. Taylor BlauApr 4, 2023
  3. Taylor BlauApr 4, 2023
  4. Patrick SteinhardtApr 5, 2023
  5. Derrick StoleeApr 10, 2023
  6. Taylor BlauApr 10, 2023
  7. Patrick SteinhardtApr 11, 2023
  8. Taylor BlauApr 11, 2023
  9. Patrick SteinhardtApr 12, 2023
  10. Patrick SteinhardtApr 11, 2023
  11. Patrick SteinhardtApr 11, 2023
  12. Taylor BlauApr 11, 2023
  13. Taylor BlauApr 10, 2023
  14. 0/8 repack: fix geometric repacking with gitalternatesPatrick Steinhardt, Apr 12, 2023
  15. 1/8 midx: fix segfault with no packs and invalid preferred packPatrick Steinhardt, Apr 12, 2023
  16. Taylor BlauApr 12, 2023
  17. Patrick SteinhardtApr 13, 2023
  18. 2/8 repack: fix trying to use preferred pack in alternatesPatrick Steinhardt, Apr 12, 2023
  19. Taylor BlauApr 12, 2023
  20. Patrick SteinhardtApr 13, 2023
  21. 3/8 repack: fix generating multi-pack-index with only non-local packsPatrick Steinhardt, Apr 12, 2023
  22. Taylor BlauApr 12, 2023
  23. 4/8 pack-objects: fix error when packing same pack twicePatrick Steinhardt, Apr 12, 2023
  24. Taylor BlauApr 12, 2023
  25. 5/8 pack-objects: fix error when same packfile is included and excludedPatrick Steinhardt, Apr 12, 2023
  26. Taylor BlauApr 12, 2023
  27. 6/8 pack-objects: extend test coverage of `--stdin-packs` with alternatesPatrick Steinhardt, Apr 12, 2023
  28. 7/8 repack: honor `-l` when calculating pack geometryPatrick Steinhardt, Apr 12, 2023
  29. Junio C HamanoApr 12, 2023
  30. Junio C HamanoApr 13, 2023
  31. Patrick SteinhardtApr 13, 2023
  32. 8/8 repack: disable writing bitmaps when doing a local geometric repackPatrick Steinhardt, Apr 12, 2023
  33. Taylor BlauApr 12, 2023
  34. Patrick SteinhardtApr 13, 2023
  35. Patrick SteinhardtApr 13, 2023
  36. Taylor BlauApr 12, 2023
  37. 00/10 repack: fix geometric repacking with gitalternatesPatrick Steinhardt, Apr 13, 2023
  38. 01/10 midx: fix segfault with no packs and invalid preferred packPatrick Steinhardt, Apr 13, 2023
  39. Derrick StoleeApr 13, 2023
  40. 02/10 repack: fix trying to use preferred pack in alternatesPatrick Steinhardt, Apr 13, 2023
  41. 03/10 repack: fix generating multi-pack-index with only non-local packsPatrick Steinhardt, Apr 13, 2023
  42. 04/10 pack-objects: split out `--stdin-packs` tests into separate filePatrick Steinhardt, Apr 13, 2023
  43. 05/10 pack-objects: fix error when packing same pack twicePatrick Steinhardt, Apr 13, 2023
  44. 06/10 pack-objects: fix error when same packfile is included and excludedPatrick Steinhardt, Apr 13, 2023
  45. 07/10 pack-objects: extend test coverage of `--stdin-packs` with alternatesPatrick Steinhardt, Apr 13, 2023
  46. 08/10 t/helper: allow chmtime to print verbosely without modifying mtimePatrick Steinhardt, Apr 13, 2023
  47. 09/10 repack: honor `-l` when calculating pack geometryPatrick Steinhardt, Apr 13, 2023
  48. Derrick StoleeApr 13, 2023
  49. Patrick SteinhardtApr 13, 2023
  50. Junio C HamanoApr 13, 2023
  51. 10/10 repack: disable writing bitmaps when doing a local repackPatrick Steinhardt, Apr 13, 2023
  52. Derrick StoleeApr 13, 2023
  53. Junio C HamanoApr 14, 2023
  54. Patrick SteinhardtApr 14, 2023
  55. 00/10 repack: fix geometric repacking with gitalternatesPatrick Steinhardt, Apr 14, 2023
  56. 01/10 midx: fix segfault with no packs and invalid preferred packPatrick Steinhardt, Apr 14, 2023
  57. 03/10 repack: fix generating multi-pack-index with only non-local packsPatrick Steinhardt, Apr 14, 2023
  58. 02/10 repack: fix trying to use preferred pack in alternatesPatrick Steinhardt, Apr 14, 2023
  59. 04/10 pack-objects: split out `--stdin-packs` tests into separate filePatrick Steinhardt, Apr 14, 2023
  60. 05/10 pack-objects: fix error when packing same pack twicePatrick Steinhardt, Apr 14, 2023
  61. 06/10 pack-objects: fix error when same packfile is included and excludedPatrick Steinhardt, Apr 14, 2023
  62. 07/10 pack-objects: extend test coverage of `--stdin-packs` with alternatesPatrick Steinhardt, Apr 14, 2023
  63. 08/10 t/helper: allow chmtime to print verbosely without modifying mtimePatrick Steinhardt, Apr 14, 2023
  64. 09/10 repack: honor `-l` when calculating pack geometryPatrick Steinhardt, Apr 14, 2023
  65. 10/10 repack: disable writing bitmaps when doing a local repackPatrick Steinhardt, Apr 14, 2023
  66. Derrick StoleeApr 14, 2023
  67. Junio C HamanoApr 14, 2023

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.