Re: MIDX woes, was Re: [ANNOUNCE] Git v2.54.0-rc2
- From
Jeff King <peff@peff.net>
- Date
- Apr 16, 2026, 05:17 UTC
- Message-ID
- <20260416051732.GA48541@coredump.intra.peff.net>
- In-Reply-To
- <xmqq5x5s540j.fsf@gitster.g>
On Wed, Apr 15, 2026 at 02:04:44PM -0700, Junio C Hamano wrote:
Show 33 quoted lines
> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes: > > >> * Further work on incremental repacking using MIDX/bitmap > > > > I just noticed that a fetch with v2.54.0-rc2 into an existing repository > > rendered it unusable for Git v2.53.0: > > > > fatal: multi-pack-index version 2 not recognized > > > > Is it possible that v2.54.0-rc2 forcefully uses a MIDX version that has > > _just_ been introduced? > > > > If so, that might have been a premature bump of the default MIDX version, > > if even the preceding Git version fails to handle that MIDX version. It is > > guaranteed to cause substantial problems in setups where e.g. libgit2 or > > JGit is expected to be used interchangeably with Git. It also causes > > problems when users have to downgrade Git, or use several Git versions > > side by side (e.g. using GitHub Desktop, which bundles its own version of > > Git). > > Is b2ec8e90 (midx: do not require packs to be sorted in > lexicographic order, 2026-02-24), which says > > This change produces MIDXs which may not be correctly read with external > tools or older versions of Git. Though older versions of Git know how to > gracefully degrade and ignore any MIDX(s) they consider corrupt, > external tools may not be as robust. To avoid unintentionally breaking > any such tools, guard this change behind a version bump in the MIDX's > on-disk format. > > relevant? The version bump seems to be doing more harm to "older > versions of Git" that "know how to gracefully degrade" by not > allowing them to degrade.
Eek. The midx _should_ be an optional component, so returning an error during load would then just look in the regular pack .idx files. But there are a few die() calls in the loading function. :(
Something like the patch below on top of v2.53.0 gets the desired effect (we print "version 2 not recognized to stderr" and then everything works anyway). But of course we can't go back in time now to fix it (and earlier versions).
I wonder how libgit2 and JGit handle this:
- Looking in the source, JGit will throw an exception, which is
presumably caught and handled OK (at least seems to using the "jgit"
CLI I have handy). So that's good. - libgit2 seems to "return midx_error()" if the signature or version
isn't matched. I don't have an easy tool built against it, but
looking at the code I think it would quietly fall back to using the
actual packs.So it really is just our old versions that are the problem.
I think removing the .midx file (and optionally regenerating with the old version) would be the appropriate workaround, but I wonder how hard it would be to go back to generating v1 midx files by default. I know v2 is a building block for more advanced features, but for those who are not using those features yet it is a strict regression.
-Peff
---
diff --git a/midx.c b/midx.c index a75ea99a0d..79ce6a1e3b 100644 --- a/midx.c +++ b/midx.c @@ -134,22 +134,26 @@ static struct multi_pack_index *load_multi_pack_index_one(struct odb_source *sou CALLOC_ARRAY(m, 1); m->data = midx_map; m->data_len = midx_size; m->source = source; m->signature = get_be32(m->data); - if (m->signature != MIDX_SIGNATURE) - die(_("multi-pack-index signature 0x%08x does not match signature 0x%08x"), + if (m->signature != MIDX_SIGNATURE) { + error(_("multi-pack-index signature 0x%08x does not match signature 0x%08x"), m->signature, MIDX_SIGNATURE); + goto cleanup_fail; + } m->version = m->data[MIDX_BYTE_FILE_VERSION]; - if (m->version != MIDX_VERSION) - die(_("multi-pack-index version %d not recognized"), + if (m->version != MIDX_VERSION) { + error(_("multi-pack-index version %d not recognized"), m->version); + goto cleanup_fail; + } hash_version = m->data[MIDX_BYTE_HASH_VERSION]; if (hash_version != oid_version(r->hash_algo)) { error(_("multi-pack-index hash version %u does not match version %u"), hash_version, oid_version(r->hash_algo)); goto cleanup_fail; }