Re: Efficiently storing SHA-1 ↔ SHA-256 mappings in compatibility mode

From: Eric Wong <e@80x24.org>
Date: Aug 29, 2025, 19:51 UTC
Message-ID: <20250829195109.M344703@dcvr>
In-Reply-To: <aLDNj5GPYA9nR3xR@fruit.crustytoothpaste.net>

"brian m. carlson" <sandals@crustytoothpaste.net> wrote:

> SQLite is not an option because it performs poorly with Java and we want
> our formats to work with other implementations, like JGit.  That's why
> we created reftable instead of using SQLite.

Interesting. I would've thought it'd be a solved problem by now given the popularity of both SQLite and Java. I actually expected choosing a more common/standard file format would make life easier for hackers of alternative git implementations.

Searching for `pure java sqlite' reveals some projects, but I'm not a Java user at all so can't comment on the quality of implementations: https://html.duckduckgo.com/html/?q=pure+java+sqlite

Fwiw, the US Library of Congress has SQLite as a recommended data format for several years, now: https://www.loc.gov/preservation/digital/formats/fdd/fdd000461.shtml Nowadays I'm more concerned about the continued relevance or even existence of the LoC than SQLite.

Show 5 quoted lines

> Also, in general, I'm not interested in being tied to a single
> implementation.  If the developers of SQLite decide to dramatically
> change the license of all their code like Oracle did with Berkeley DB,
> we're going to have a problem.  Yes, we can use the older versions, but
> we'd still need people to maintain the library and update it.

Whereas we'd be implementing and maintaining yet another one-off data format from day one instead of waiting for an eventuality which may never come. SQLite 3 has far surpassed the stability and use of Berkeley DB; there's not a lot of similar formats that might replace it (e.g. GDBM, LMDB). So I'd expect there'd be no shortage of hackers able to maintain a usable fork if push comes to shove.

Fwiw, I consider the proliferation of data formats and protocols the biggest threat to digital freedom and security. Even when the implementations are open source it feels like a huge drain to have to constantly deal reviewing/porting code to deal with more formats.