Re: reftable [v6]: new ref storage format

From: Junio C Hamano <gitster@pobox.com>
Date: Aug 8, 2017, 23:34 UTC
Message-ID: <xmqqy3qt8wi0.fsf@gitster.mtv.corp.google.com>
In-Reply-To: <CAJo=hJsyoFeCQbeJ=2XCRcE1U0zYaRr8VvzXHwkPwisdfUm71Q@mail.gmail.com>

Shawn Pearce <spearce@spearce.org> writes:

Show 6 quoted lines

> Given that the index can now also be multi-level, I don't expect to
> see a 2G index. A 2G index forces the reader to load the entire 2G to
> take advantage of the restart table. It may be more efficient for such
> a reader to have had the writer make a mutli-level index, instead of a
> single monster index block. And so perhaps the writer shouldn't make a
> 2G index block that she is forced to buffer. :)

Ah, OK, then it is sensible to have all table blocks to have the same format, and restart at the beginning to help readers would be a fine choice. For the same "let's make them as consistent" sake, I am tempted to suggest that we lift "the index block can be 2G" and have it also be within uint_24(), perhaps? Otherwise the readers would have to read (or mmap) the whole 2G.