Re: [PATCH 09/17] midx: do not require packs to be sorted in lexicographic order

From: Taylor Blau <me@ttaylorr.com>
Date: Dec 9, 2025, 02:11 UTC
Message-ID: <aTeFVDGo69ljiQP9@nand.local>
In-Reply-To: <aTeEZX4036A9YecX@nand.local>

On Mon, Dec 08, 2025 at 09:07:33PM -0500, Taylor Blau wrote:

Show 18 quoted lines

> On Mon, Dec 08, 2025 at 07:26:53PM +0100, Patrick Steinhardt wrote:
> > On Sat, Dec 06, 2025 at 03:31:25PM -0500, Taylor Blau wrote:
> > > Note that this produces MIDXs which may be incompatible with earlier
> > > versions of Git that have stricter requirements on the layout of packs
> > > within a MIDX. This patch does *not* modify the version number of the
> > > MIDX format, since existing versions of Git already know to gracefully
> > > ignore a MIDX with packs that appear out-of-order.
> >
> > Interesting. Did you verify how other implementations of Git behave if
> > we start to relax this requirement? It seems like a somewhat dangerous
> > assumption to me that this will just continue to work.
>
> That's a great point. It looks like current libgit2 assumes[1] that the
> list is sorted and complains loudly if it is not. Presumably other
> implementations behave similarly.
>
> I think that is a compelling enough argument to swing us towards
> bumping the version number to avoid compatibility issues.

I had another thought about how we might work around this without forcing a compatibility issue, but it's a non-starter. I wanted to share it on the list for posterity regardless.

I was going to add that we could instead consider adding a new chunk to the MIDX format that lists the pack names in the order that they should appear in the pseudo-pack order. Absent of that chunk, the pseudo-pack order would be defined by the lexicographic order of pack names. If the chunk exists, it would supersede that ordering.

But that just kicks the can down the road, since implementations like libgit2 would think that they could read a *.midx file, but then they'd produce all sorts of errors when trying to read its corresponding *.bitmap file by permuting its bits out-of-order.

(I'm not sure off-hand whether or not libgit2 supports reading MIDX bitmaps to begin with. Regardless, we should not introduce the possibility for such a breakage in clients that *do* support reading MIDX bitmaps, whether or not libgit2 is such a client.)

Thanks, Taylor