Re: reftable [v3]: new ref storage format

From: Shawn Pearce <spearce@spearce.org>
Date: Jul 29, 2017, 02:31 UTC
Message-ID: <CAJo=hJt5A6wL1i4d8WAivL-+LymZd_M6XjUBJ+vVoYfvuVm23w@mail.gmail.com>
In-Reply-To: <CAMy9T_F9oH926ShUGpFqJz4bMrJofvbsfb0_oME2NhsYvObq9Q@mail.gmail.com>

On Fri, Jul 28, 2017 at 7:18 PM, Michael Haggerty <mhagger@alum.mit.edu> wrote:

Show 12 quoted lines

> On Fri, Jul 28, 2017 at 3:12 PM, Shawn Pearce <spearce@spearce.org> wrote:
>> I'm with you this far, and like the {min,max}_update_index in the
>> header. I'm concerned about update_index in 32 bits. At some point you
>> need to reset the counter, or the repository is broken. 4b updates is
>> enough for anyone? I'd feel better about this being a 64 bit field.
>
> Yes, I was a little bit nervous about 32 bits, too. But that's a *lot*
> of updates: one per second for 136 years. If that limit were ever
> reached, there could be a compaction step, where any update indices
> that don't have associated reflog entries are "compacted out" of the
> numerical sequence and the remaining indices are renumbered
> contiguously.

I considered that, but its a bit of a pain for the writer to renumber the remaining records.

> But it's ok with me to make it 64 bits. Usually those extra bytes
> would be appear as FFFFFFFF and so should prefix- and zlib-compress
> well.

That was my thought. Within a single reference these will prefix compress right out, and zlib will fix any sins within the log block at restart points.