git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v2 2/8] builtin/multi-pack-index.c: support `--stdin-packs` mode

From
Taylor Blau <me@ttaylorr.com>
Date
Sep 15, 2021, 18:24 UTC
Message-ID
<59556e554565120929549239f1aee5a76d80ac8d.1631730270.git.me@ttaylorr.com>
In-Reply-To
<cover.1631730270.git.me@ttaylorr.com>

To power a new `--write-midx` mode, `git repack` will want to write a multi-pack index containing a certain set of packs in the repository.

This new option will be used by `git repack` to write a MIDX which contains only the packs which will survive after the repack (that is, it will exclude any packs which are about to be deleted).

This patch effectively exposes the function implemented in the previous commit via the `git multi-pack-index` builtin. An alternative approach would have been to call that function from the `git repack` builtin directly, but this introduces awkward problems around closing and reopening the object store, so the MIDX will be written out-of-process.

Signed-off-by: Taylor Blau <me@ttaylorr.com>
---
 Documentation/git-multi-pack-index.txt |  4 ++++
 builtin/multi-pack-index.c             | 27 ++++++++++++++++++++++++++
 t/t5319-multi-pack-index.sh            | 15 ++++++++++++++
 3 files changed, 46 insertions(+)
diff --git a/Documentation/git-multi-pack-index.txt b/Documentation/git-multi-pack-index.txt
index a9df3dbd32..009c989ef8 100644
--- a/Documentation/git-multi-pack-index.txt
+++ b/Documentation/git-multi-pack-index.txt
@@ -45,6 +45,10 @@ write::
 
 	--[no-]bitmap::
 		Control whether or not a multi-pack bitmap is written.
+
+	--stdin-packs::
+		Write a multi-pack index containing only the set of
+		line-delimited pack index basenames provided over stdin.
 --
 
 verify::
diff --git a/builtin/multi-pack-index.c b/builtin/multi-pack-index.c
index 73c0113b48..047647b5f2 100644
--- a/builtin/multi-pack-index.c
+++ b/builtin/multi-pack-index.c
@@ -47,6 +47,7 @@ static struct opts_multi_pack_index {
 	const char *preferred_pack;
 	unsigned long batch_size;
 	unsigned flags;
+	int stdin_packs;
 } opts;
 
 static struct option common_opts[] = {
@@ -61,6 +62,16 @@ static struct option *add_common_options(struct option *prev)
 	return parse_options_concat(common_opts, prev);
 }
 
+static void read_packs_from_stdin(struct string_list *to)
+{
+	struct strbuf buf = STRBUF_INIT;
+	while (strbuf_getline(&buf, stdin) != EOF)
+		string_list_append(to, buf.buf);
+	string_list_sort(to);
+
+	strbuf_release(&buf);
+}
+
 static int cmd_multi_pack_index_write(int argc, const char **argv)
 {
 	struct option *options;
@@ -70,6 +81,8 @@ static int cmd_multi_pack_index_write(int argc, const char **argv)
 			   N_("pack for reuse when computing a multi-pack bitmap")),
 		OPT_BIT(0, "bitmap", &opts.flags, N_("write multi-pack bitmap"),
 			MIDX_WRITE_BITMAP | MIDX_WRITE_REV_INDEX),
+		OPT_BOOL(0, "stdin-packs", &opts.stdin_packs,
+			 N_("write multi-pack index containing only given indexes")),
 		OPT_END(),
 	};
 
@@ -86,6 +99,20 @@ static int cmd_multi_pack_index_write(int argc, const char **argv)
 
 	FREE_AND_NULL(options);
 
+	if (opts.stdin_packs) {
+		struct string_list packs = STRING_LIST_INIT_DUP;
+		int ret;
+
+		read_packs_from_stdin(&packs);
+
+		ret = write_midx_file_only(opts.object_dir, &packs,
+					   opts.preferred_pack, opts.flags);
+
+		string_list_clear(&packs, 0);
+
+		return ret;
+
+	}
 	return write_midx_file(opts.object_dir, opts.preferred_pack,
 			       opts.flags);
 }
diff --git a/t/t5319-multi-pack-index.sh b/t/t5319-multi-pack-index.sh
index bb04f0f23b..385f0a3efd 100755
--- a/t/t5319-multi-pack-index.sh
+++ b/t/t5319-multi-pack-index.sh
@@ -168,6 +168,21 @@ test_expect_success 'write midx with two packs' '
 
 compare_results_with_midx "two packs"
 
+test_expect_success 'write midx with --stdin-packs' '
+	rm -fr $objdir/pack/multi-pack-index &&
+
+	idx="$(find $objdir/pack -name "test-2-*.idx")" &&
+	basename "$idx" >in &&
+
+	git multi-pack-index write --stdin-packs <in &&
+
+	test-tool read-midx $objdir | grep "\.idx$" >packs &&
+
+	test_cmp packs in
+'
+
+compare_results_with_midx "mixed mode (one pack + extra)"
+
 test_expect_success 'write progress off for redirected stderr' '
 	git multi-pack-index --object-dir=$objdir write 2>err &&
 	test_line_count = 0 err
-- 
2.33.0.96.g73915697e6
Previous: Taylor BlauNext: Josh Steadmon
Message 39 of 76 in “repack: introduce `--write-midx`”
  1. 0/8 repack: introduce `--write-midx`Taylor Blau, Sep 11, 2021
  2. 1/8 midx: expose 'write_midx_file_only()' publiclyTaylor Blau, Sep 11, 2021
  3. Junio C HamanoSep 11, 2021
  4. Taylor BlauSep 11, 2021
  5. Ævar Arnfjörð BjarmasonSep 11, 2021
  6. Taylor BlauSep 11, 2021
  7. 2/8 builtin/multi-pack-index.c: support --stdin-packs modeTaylor Blau, Sep 11, 2021
  8. Ævar Arnfjörð BjarmasonSep 11, 2021
  9. Taylor BlauSep 11, 2021
  10. Taylor BlauSep 11, 2021
  11. Eric SunshineSep 12, 2021
  12. Taylor BlauSep 12, 2021
  13. Ævar Arnfjörð BjarmasonSep 12, 2021
  14. Junio C HamanoSep 12, 2021
  15. Ævar Arnfjörð BjarmasonSep 12, 2021
  16. Jeff KingSep 14, 2021
  17. Taylor BlauSep 14, 2021
  18. Eric SunshineSep 15, 2021
  19. 4/8 builtin/repack.c: keep track of existing packs unconditionallyTaylor Blau, Sep 11, 2021
  20. 3/8 midx: preliminary support for `--refs-snapshot`Taylor Blau, Sep 11, 2021
  21. 5/8 builtin/repack.c: extract showing progress to a variableTaylor Blau, Sep 11, 2021
  22. 6/8 builtin/repack.c: support writing a MIDX while repackingTaylor Blau, Sep 11, 2021
  23. 7/8 builtin/repack.c: make largest pack preferredTaylor Blau, Sep 11, 2021
  24. Ævar Arnfjörð BjarmasonSep 11, 2021
  25. Taylor BlauSep 11, 2021
  26. 8/8 builtin/repack.c: pass `--refs-snapshot` when writing bitmapsTaylor Blau, Sep 11, 2021
  27. Ævar Arnfjörð BjarmasonSep 11, 2021
  28. Ævar Arnfjörð BjarmasonSep 11, 2021
  29. Taylor BlauSep 11, 2021
  30. Jeff KingSep 14, 2021
  31. Taylor BlauSep 14, 2021
  32. Ævar Arnfjörð BjarmasonSep 14, 2021
  33. Taylor BlauSep 15, 2021
  34. Taylor BlauSep 11, 2021
  35. 0/8 repack: introduce `--write-midx`Taylor Blau, Sep 15, 2021
  36. 1/8 midx: expose `write_midx_file_only()` publiclyTaylor Blau, Sep 15, 2021
  37. Jonathan TanSep 22, 2021
  38. Taylor BlauSep 23, 2021
  39. 2/8 builtin/multi-pack-index.c: support `--stdin-packs` modeTaylor Blau, Sep 15, 2021
  40. Josh SteadmonSep 22, 2021
  41. Taylor BlauSep 23, 2021
  42. Jonathan TanSep 22, 2021
  43. Taylor BlauSep 23, 2021
  44. 3/8 midx: preliminary support for `--refs-snapshot`Taylor Blau, Sep 15, 2021
  45. Josh SteadmonSep 22, 2021
  46. Taylor BlauSep 23, 2021
  47. Jonathan TanSep 22, 2021
  48. Taylor BlauSep 23, 2021
  49. 4/8 builtin/repack.c: keep track of existing packs unconditionallyTaylor Blau, Sep 15, 2021
  50. Jonathan TanSep 22, 2021
  51. Taylor BlauSep 23, 2021
  52. 5/8 builtin/repack.c: extract showing progress to a variableTaylor Blau, Sep 15, 2021
  53. 6/8 builtin/repack.c: support writing a MIDX while repackingTaylor Blau, Sep 15, 2021
  54. Jonathan TanSep 22, 2021
  55. Taylor BlauSep 23, 2021
  56. 7/8 builtin/repack.c: make largest pack preferredTaylor Blau, Sep 15, 2021
  57. 8/8 builtin/repack.c: pass `--refs-snapshot` when writing bitmapsTaylor Blau, Sep 15, 2021
  58. Jonathan TanSep 24, 2021
  59. Taylor BlauOct 1, 2021
  60. Junio C HamanoSep 15, 2021
  61. Junio C HamanoSep 15, 2021
  62. Taylor BlauSep 15, 2021
  63. Junio C HamanoSep 16, 2021
  64. 0/9 repack: introduce `--write-midx`Taylor Blau, Sep 29, 2021
  65. 1/9 midx: expose `write_midx_file_only()` publiclyTaylor Blau, Sep 29, 2021
  66. 2/9 builtin/multi-pack-index.c: support `--stdin-packs` modeTaylor Blau, Sep 29, 2021
  67. 3/9 midx: preliminary support for `--refs-snapshot`Taylor Blau, Sep 29, 2021
  68. 4/9 builtin/repack.c: keep track of existing packs unconditionallyTaylor Blau, Sep 29, 2021
  69. 5/9 builtin/repack.c: rename variables that deal with non-kept packsTaylor Blau, Sep 29, 2021
  70. 6/9 builtin/repack.c: extract showing progress to a variableTaylor Blau, Sep 29, 2021
  71. 7/9 builtin/repack.c: support writing a MIDX while repackingTaylor Blau, Sep 29, 2021
  72. 8/9 builtin/repack.c: make largest pack preferredTaylor Blau, Sep 29, 2021
  73. 9/9 builtin/repack.c: pass `--refs-snapshot` when writing bitmapsTaylor Blau, Sep 29, 2021
  74. Junio C HamanoSep 29, 2021
  75. Jonathan TanOct 1, 2021
  76. Taylor BlauOct 1, 2021

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.