From: Jeff King Date: Thu, 16 Apr 2026 05:17:32 GMT Subject: Re: MIDX woes, was Re: [ANNOUNCE] Git v2.54.0-rc2 Message-ID: <20260416051732.GA48541@coredump.intra.peff.net> In-Reply-To: On Wed, Apr 15, 2026 at 02:04:44PM -0700, Junio C Hamano wrote: > Johannes Schindelin writes: > > >> * Further work on incremental repacking using MIDX/bitmap > > > > I just noticed that a fetch with v2.54.0-rc2 into an existing repository > > rendered it unusable for Git v2.53.0: > > > > fatal: multi-pack-index version 2 not recognized > > > > Is it possible that v2.54.0-rc2 forcefully uses a MIDX version that has > > _just_ been introduced? > > > > If so, that might have been a premature bump of the default MIDX version, > > if even the preceding Git version fails to handle that MIDX version. It is > > guaranteed to cause substantial problems in setups where e.g. libgit2 or > > JGit is expected to be used interchangeably with Git. It also causes > > problems when users have to downgrade Git, or use several Git versions > > side by side (e.g. using GitHub Desktop, which bundles its own version of > > Git). > > Is b2ec8e90 (midx: do not require packs to be sorted in > lexicographic order, 2026-02-24), which says > > This change produces MIDXs which may not be correctly read with external > tools or older versions of Git. Though older versions of Git know how to > gracefully degrade and ignore any MIDX(s) they consider corrupt, > external tools may not be as robust. To avoid unintentionally breaking > any such tools, guard this change behind a version bump in the MIDX's > on-disk format. > > relevant? The version bump seems to be doing more harm to "older > versions of Git" that "know how to gracefully degrade" by not > allowing them to degrade. Eek. The midx _should_ be an optional component, so returning an error during load would then just look in the regular pack .idx files. But there are a few die() calls in the loading function. :( Something like the patch below on top of v2.53.0 gets the desired effect (we print "version 2 not recognized to stderr" and then everything works anyway). But of course we can't go back in time now to fix it (and earlier versions). I wonder how libgit2 and JGit handle this: - Looking in the source, JGit will throw an exception, which is presumably caught and handled OK (at least seems to using the "jgit" CLI I have handy). So that's good. - libgit2 seems to "return midx_error()" if the signature or version isn't matched. I don't have an easy tool built against it, but looking at the code I think it would quietly fall back to using the actual packs. So it really is just our old versions that are the problem. I think removing the .midx file (and optionally regenerating with the old version) would be the appropriate workaround, but I wonder how hard it would be to go back to generating v1 midx files by default. I know v2 is a building block for more advanced features, but for those who are not using those features yet it is a strict regression. -Peff --- diff --git a/midx.c b/midx.c index a75ea99a0d..79ce6a1e3b 100644 --- a/midx.c +++ b/midx.c @@ -134,22 +134,26 @@ static struct multi_pack_index *load_multi_pack_index_one(struct odb_source *sou CALLOC_ARRAY(m, 1); m->data = midx_map; m->data_len = midx_size; m->source = source; m->signature = get_be32(m->data); - if (m->signature != MIDX_SIGNATURE) - die(_("multi-pack-index signature 0x%08x does not match signature 0x%08x"), + if (m->signature != MIDX_SIGNATURE) { + error(_("multi-pack-index signature 0x%08x does not match signature 0x%08x"), m->signature, MIDX_SIGNATURE); + goto cleanup_fail; + } m->version = m->data[MIDX_BYTE_FILE_VERSION]; - if (m->version != MIDX_VERSION) - die(_("multi-pack-index version %d not recognized"), + if (m->version != MIDX_VERSION) { + error(_("multi-pack-index version %d not recognized"), m->version); + goto cleanup_fail; + } hash_version = m->data[MIDX_BYTE_HASH_VERSION]; if (hash_version != oid_version(r->hash_algo)) { error(_("multi-pack-index hash version %u does not match version %u"), hash_version, oid_version(r->hash_algo)); goto cleanup_fail; }