git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: MIDX woes, was Re: [ANNOUNCE] Git v2.54.0-rc2

From
Jeff King <peff@peff.net>
Date
Apr 16, 2026, 05:17 UTC
Message-ID
<20260416051732.GA48541@coredump.intra.peff.net>
In-Reply-To
<xmqq5x5s540j.fsf@gitster.g>
On Wed, Apr 15, 2026 at 02:04:44PM -0700, Junio C Hamano wrote:
Show 33 quoted lines
> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
> >>  * Further work on incremental repacking using MIDX/bitmap
> >
> > I just noticed that a fetch with v2.54.0-rc2 into an existing repository
> > rendered it unusable for Git v2.53.0:
> >
> >   fatal: multi-pack-index version 2 not recognized
> >
> > Is it possible that v2.54.0-rc2 forcefully uses a MIDX version that has
> > _just_ been introduced?
> >
> > If so, that might have been a premature bump of the default MIDX version,
> > if even the preceding Git version fails to handle that MIDX version. It is
> > guaranteed to cause substantial problems in setups where e.g. libgit2 or
> > JGit is expected to be used interchangeably with Git. It also causes
> > problems when users have to downgrade Git, or use several Git versions
> > side by side (e.g. using GitHub Desktop, which bundles its own version of
> > Git).
> 
> Is b2ec8e90 (midx: do not require packs to be sorted in
> lexicographic order, 2026-02-24), which says
> 
>     This change produces MIDXs which may not be correctly read with external
>     tools or older versions of Git. Though older versions of Git know how to
>     gracefully degrade and ignore any MIDX(s) they consider corrupt,
>     external tools may not be as robust. To avoid unintentionally breaking
>     any such tools, guard this change behind a version bump in the MIDX's
>     on-disk format.
>     
> relevant?  The version bump seems to be doing more harm to "older
> versions of Git" that "know how to gracefully degrade" by not
> allowing them to degrade.

Eek. The midx _should_ be an optional component, so returning an error during load would then just look in the regular pack .idx files. But there are a few die() calls in the loading function. :(

Something like the patch below on top of v2.53.0 gets the desired effect (we print "version 2 not recognized to stderr" and then everything works anyway). But of course we can't go back in time now to fix it (and earlier versions).

I wonder how libgit2 and JGit handle this:
  - Looking in the source, JGit will throw an exception, which is
    presumably caught and handled OK (at least seems to using the "jgit"
    CLI I have handy). So that's good.
  - libgit2 seems to "return midx_error()" if the signature or version
    isn't matched. I don't have an easy tool built against it, but
    looking at the code I think it would quietly fall back to using the
    actual packs.
So it really is just our old versions that are the problem.

I think removing the .midx file (and optionally regenerating with the old version) would be the appropriate workaround, but I wonder how hard it would be to go back to generating v1 midx files by default. I know v2 is a building block for more advanced features, but for those who are not using those features yet it is a strict regression.

-Peff
---
diff --git a/midx.c b/midx.c
index a75ea99a0d..79ce6a1e3b 100644
--- a/midx.c
+++ b/midx.c
@@ -134,22 +134,26 @@ static struct multi_pack_index *load_multi_pack_index_one(struct odb_source *sou
 
 	CALLOC_ARRAY(m, 1);
 	m->data = midx_map;
 	m->data_len = midx_size;
 	m->source = source;
 
 	m->signature = get_be32(m->data);
-	if (m->signature != MIDX_SIGNATURE)
-		die(_("multi-pack-index signature 0x%08x does not match signature 0x%08x"),
+	if (m->signature != MIDX_SIGNATURE) {
+		error(_("multi-pack-index signature 0x%08x does not match signature 0x%08x"),
 		      m->signature, MIDX_SIGNATURE);
+		goto cleanup_fail;
+	}
 
 	m->version = m->data[MIDX_BYTE_FILE_VERSION];
-	if (m->version != MIDX_VERSION)
-		die(_("multi-pack-index version %d not recognized"),
+	if (m->version != MIDX_VERSION) {
+		error(_("multi-pack-index version %d not recognized"),
 		      m->version);
+		goto cleanup_fail;
+	}
 
 	hash_version = m->data[MIDX_BYTE_HASH_VERSION];
 	if (hash_version != oid_version(r->hash_algo)) {
 		error(_("multi-pack-index hash version %u does not match version %u"),
 		      hash_version, oid_version(r->hash_algo));
 		goto cleanup_fail;
 	}
Previous: Junio C HamanoNext: Jeff King
Message 4 of 30 in “[ANNOUNCE] Git v2.54.0-rc2”
  1. Junio C HamanoApr 14, 2026
  2. MIDX woes, was Re: [ANNOUNCE] Git v2.54.0-rc2Johannes Schindelin, Apr 15, 2026
  3. Junio C HamanoApr 15, 2026
  4. Jeff KingApr 16, 2026
  5. Jeff KingApr 16, 2026
  6. Derrick StoleeApr 16, 2026
  7. Junio C HamanoApr 16, 2026
  8. Taylor BlauApr 16, 2026
  9. Derrick StoleeApr 19, 2026
  10. Junio C HamanoApr 20, 2026
  11. Taylor BlauApr 16, 2026
  12. Jeff KingApr 16, 2026
  13. Junio C HamanoApr 16, 2026
  14. Junio C HamanoApr 16, 2026
  15. Jeff KingApr 16, 2026
  16. Junio C HamanoApr 16, 2026
  17. Jeff KingApr 16, 2026
  18. Junio C HamanoApr 17, 2026
  19. MIDX: revert the default version to v1Junio C Hamano, Apr 16, 2026
  20. Junio C HamanoApr 16, 2026
  21. Junio C HamanoApr 16, 2026
  22. Taylor BlauApr 16, 2026
  23. Jeff KingApr 16, 2026
  24. Junio C HamanoApr 16, 2026
  25. Jeff KingApr 16, 2026
  26. Jeff KingApr 16, 2026
  27. Taylor BlauApr 16, 2026
  28. Jeff KingApr 16, 2026
  29. Junio C HamanoApr 17, 2026
  30. Elijah NewrenApr 17, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.