Re: [PATCH 1/9] docs: update pack index v3 format
- From
brian m. carlson <sandals@crustytoothpaste.net>
- Date
- Sep 20, 2025, 15:23 UTC
- Message-ID
- <aM7G22LJ47nx8Wqk@fruit.crustytoothpaste.net>
- In-Reply-To
- <xmqq7bxu14fw.fsf@gitster.g>
On 2025-09-19 at 22:08:03, Junio C Hamano wrote:
Show 29 quoted lines
> "brian m. carlson" <sandals@crustytoothpaste.net> writes: > > > Our current pack index v3 format uses 4-byte integers to find the > > trailer of the file. This effectively means that the file cannot be > > much larger than 2^32. While this might at first seem to be okay, we > > expect that each object will have at least 64 bytes worth of data, which > > means that no more than about 67 million objects can be stored. > > > > Again, this might seem fine, but unfortunately, we know of many users > > who attempt to create repos with extremely large numbers of commits to > > get a "high score," and we've already seen repositories with at least 55 > > million commits. In the interests of gracefully handling repositories > > even for these well-intentioned but ultimately misguided users, let's > > change these lengths to 8 bytes. > > Very sensible. > > I do also agree that 32-byte is the natural size for the trailing > hash, but I found that the two paragraphs below was far more than > necessary. As they argue, we use a truncated hash anywhere in our > file formats, so I would have understood if the explanation were > > "20" in "A copy of the 20-byte SHA-256 checksum" is an obvious > typo, as SHA-256 is longer than that. Fix it to "32". > > instead of these two paragraphs. > > Or did we mean to use a truncated hash back when this transition > design was proposed originally?
I think we intended to use a 20-byte value originally because we felt we didn't need the full 32 bytes for an index or pack checksum. However, as I mentioned, we use the 32-byte checksum for SHA-256 already, so all it does is add complexity to try to mandate a 20-byte value.
-- brian m. carlson (they/them) Toronto, Ontario, CA