Re: reftable [v2]: new ref storage format

From: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date: Jul 19, 2017, 14:02 UTC
Message-ID: <87k234tti7.fsf@gmail.com>
In-Reply-To: <CAJo=hJuP9GdudFsA_ToFQwx-zESaDHRDXHLxmvAXSX5CKmh7JQ@mail.gmail.com>

On Tue, Jul 18 2017, Shawn Pearce jotted:

Show 16 quoted lines

> On Mon, Jul 17, 2017 at 12:51 PM, Junio C Hamano <gitster@pobox.com> wrote:
>> Shawn Pearce <spearce@spearce.org> writes:
>>> where `time_sec` is the update time in seconds since the epoch.  The
>>> `reverse_int32` function inverses the value so lexographical ordering
>>> the network byte order time sorts more recent records first:
>>>
>>>     reverse_int(int32 t) {
>>>       return 0xffffffff - t;
>>>     }
>>
>> Is 2038 an issue, or by that time we'd all be retired together with
>> this file format and it won't be our problem?
>
> Based on discussion with Michael Haggerty, this is now an 8 byte field
> storing microseconds since the epoch. We should be good through year
> 9999.

I think this should be s/microseconds/nanoseconds/, not because there's some great need to get better resolution than nanoseconds, but because:

 a) We already have WIP code (bp/fsmonitor) that's storing 64 bit
    nanoseconds since the epoch, albeit for the index, not for refs.

 b) There are several filesystems that have nanosecond resolution now,
    and it's likely more will start using that.

Thus:

 x) If you use such a filesystem you'll lose time resolution with this
    ref backend v.s. storing them on disk, which isn't itself a big
    deal, but more importantly you lose 1=1 time mapping as you
    transition and convert between the two.

 y) Our own code will need to juggle second resolution epochs
    (traditional FSs, any 32bit epoch format), microseconds (this
    proposal), and nanoseconds (new FSs, bp/fsmonitor) internally in
    various places.

    Let's not make this harder than it needs to be and just settle on
    two epoch resolution formats if we can help it, and so far it looks
    like we can.

The downside is that instead of lasting through the year 9999 the 64 bit nanosecond resolution is only good up until the year 2554, which I think is an acceptable trade-off given the above.