[PATCH v4 00/16] repack: incremental MIDX/bitmap-based repacking
- From
Taylor Blau <me@ttaylorr.com>
- Date
- May 19, 2026, 15:57 UTC
- Message-ID
- <cover.1779206239.git.me@ttaylorr.com>
- In-Reply-To
- <cover.1774820449.git.me@ttaylorr.com>
Here is another small reroll of my series to implement the last remaining component of the incremental MIDX/bitmap-based repacking strategy [1].
The changes since v3 are fairly small:
- `repack_prepare_midx_command()` now calls its third parameter "subcommand" instead of "verb".
- `clear_incremental_midx_files()` now avoids a redundant conditional around `r->objects` and reuses a local `struct odb_source *` when closing any open incremental MIDX handles.
As usual, a range-diff is included below for convenience. Thanks in advance for reviewing!
[1]: https://lore.kernel.org/git/cover.1777507303.git.me@ttaylorr.com/
Taylor Blau (16): midx-write: handle noop writes when converting incremental chains midx: use `strset` for retained MIDX files midx: build `keep_hashes` array in order midx: use `strvec` for `keep_hashes` midx: introduce `--no-write-chain-file` for incremental MIDX writes midx: support custom `--base` for incremental MIDX writes repack: track the ODB source via existing_packs midx: expose `midx_layer_contains_pack()` repack-midx: factor out `repack_prepare_midx_command()` repack-midx: extract `repack_fill_midx_stdin_packs()` repack-geometry: prepare for incremental MIDX repacking builtin/repack.c: convert `--write-midx` to an `OPT_CALLBACK` packfile: ensure `close_pack_revindex()` frees in-memory revindex repack: implement incremental MIDX repacking repack: introduce `--write-midx=incremental` repack: allow `--write-midx=incremental` without `--geometric`
Documentation/config/repack.adoc | 18 + Documentation/git-multi-pack-index.adoc | 32 +- Documentation/git-repack.adoc | 44 +- builtin/multi-pack-index.c | 48 +- builtin/repack.c | 102 +++- midx-write.c | 206 ++++--- midx.c | 102 ++-- midx.h | 11 +- packfile.c | 2 + repack-geometry.c | 48 +- repack-midx.c | 710 +++++++++++++++++++++++- repack.c | 58 +- repack.h | 26 +- t/meson.build | 1 + t/t5334-incremental-multi-pack-index.sh | 63 +++ t/t5335-compact-multi-pack-index.sh | 113 ++++ t/t7705-repack-incremental-midx.sh | 525 ++++++++++++++++++ 17 files changed, 1909 insertions(+), 200 deletions(-) create mode 100755 t/t7705-repack-incremental-midx.sh
Range-diff against v3:
1: d6c27317c25 = 1: ead11e610c8 midx-write: handle noop writes when converting incremental chains
2: 629c8d23116 = 2: ece55bf2957 midx: use `strset` for retained MIDX files
3: e303bf6a4ac = 3: 5609d1941e6 midx: build `keep_hashes` array in order
4: 42d76c70060 = 4: 13b7c808860 midx: use `strvec` for `keep_hashes`
5: 2c80aa34fac = 5: cac3fd54bf0 midx: introduce `--no-write-chain-file` for incremental MIDX writes
6: 2a05f4b86f3 = 6: 1bbb387d6b6 midx: support custom `--base` for incremental MIDX writes
7: 92aba3d366f = 7: 4a93adb3ad3 repack: track the ODB source via existing_packs
8: d3ac65c1f11 = 8: 8d1b8b1d301 midx: expose `midx_layer_contains_pack()`
9: 1bd2f194c6f ! 9: 42111e5f75d repack-midx: factor out `repack_prepare_midx_command()`
@@ Commit message
subcommands), so extract the common portions of the command setup into a
reusable `repack_prepare_midx_command()` helper.
- The extracted helper sets `git_cmd`, pushes the `multi-pack-index`
- subcommand and verb, and handles `--progress`/`--no-progress` and
- `--bitmap` flags. The remaining arguments that are specific to the
- `write` subcommand (such as `--stdin-packs`) are left to the caller.
+ The extracted helper sets `git_cmd`, pushes `multi-pack-index` and a
+ subcommand, and handles `--progress`/`--no-progress` and `--bitmap`
+ flags. The remaining arguments that are specific to the `write`
+ subcommand (such as `--stdin-packs`) are left to the caller.
No functional changes are included in this patch.
@@ repack-midx.c: static void remove_redundant_bitmaps(struct string_list *include,
+static void repack_prepare_midx_command(struct child_process *cmd,
+ struct repack_write_midx_opts *opts,
-+ const char *verb)
++ const char *subcommand)
+{
+ cmd->git_cmd = 1;
+
-+ strvec_pushl(&cmd->args, "multi-pack-index", verb, NULL);
++ strvec_pushl(&cmd->args, "multi-pack-index", subcommand, NULL);
+
+ if (opts->show_progress)
+ strvec_push(&cmd->args, "--progress");
10: 2a87a1e4561 = 10: ed76e6efd1c repack-midx: extract `repack_fill_midx_stdin_packs()`
11: 3d32b9c88da = 11: 9665f1b3a64 repack-geometry: prepare for incremental MIDX repacking
12: 1f7a5479bb8 = 12: e0db62b9f10 builtin/repack.c: convert `--write-midx` to an `OPT_CALLBACK`
13: b155f25d53c = 13: c8c846b1ac1 packfile: ensure `close_pack_revindex()` frees in-memory revindex
14: ef012314930 = 14: 78f1a98c1ea repack: implement incremental MIDX repacking
15: 04cfecd5136 ! 15: 95bf8b21fff repack: introduce `--write-midx=incremental`
@@ midx.c: void clear_midx_file(struct repository *r)
+void clear_incremental_midx_files(struct repository *r,
+ const struct strvec *keep_hashes)
+{
++ struct odb_source *source = r->objects->sources;
+ struct strbuf chain = STRBUF_INIT;
+
-+ get_midx_chain_filename(r->objects->sources, &chain);
++ get_midx_chain_filename(source, &chain);
+
-+ if (r->objects) {
-+ struct odb_source *source = r->objects->sources;
-+ for (source = r->objects->sources; source; source = source->next) {
-+ struct odb_source_files *files = odb_source_files_downcast(source);
-+ if (files->packed->midx)
-+ close_midx(files->packed->midx);
-+ files->packed->midx = NULL;
-+ }
++ for (; source; source = source->next) {
++ struct odb_source_files *files = odb_source_files_downcast(source);
++ if (files->packed->midx)
++ close_midx(files->packed->midx);
++ files->packed->midx = NULL;
+ }
+
+ if (!keep_hashes && remove_path(chain.buf))
16: 1c05dfce579 = 16: 8bd0ec98dc3 repack: allow `--write-midx=incremental` without `--geometric`base-commit: 7bcaabddcf68bd0702697da5904c3b68c52f94cf
-- 2.54.0.175.g8bd0ec98dc3