Re: reftable [v6]: new ref storage format

From: Stefan Beller <sbeller@google.com>
Date: Aug 7, 2017, 18:27 UTC
Message-ID: <CAGZ79kZO517Bg+O2z3dr+oHqO_av1__FU3h9bP=rhXJUSDnRjA@mail.gmail.com>
In-Reply-To: <CAJo=hJtg0PAVHT1phbArdra8+4LfnEEuaj3fBid==BXkZghi8g@mail.gmail.com>

On Sun, Aug 6, 2017 at 6:47 PM, Shawn Pearce <spearce@spearce.org> wrote:

Show 11 quoted lines

> 6th iteration of the reftable storage format.
>
> You can read a rendered version of this here:
> https://googlers.googlesource.com/sop/jgit/+/reftable/Documentation/technical/reftable.md
>
> Changes from v5:
> - extensions.refStorage = reftable is used to select this format.
>
> - Log records can be explicitly deleted (for refs/stash).
> - Log records may use Michael Haggerty's chained idea to compress before zlib.
>   This saved ~5.8% on one of my example repositories.

Some observations:

Also the bits in the records changed in v5 or v6:
  0x0..0x3 is valid for a ref,
  obj records have a ccnt
  0x0, 0x1, 0x4..0x7 are used in the logs

We have the following block indicators:
  'r'  ref block
  'o' object block
  'g' log block

  high bit for any index.

Without prior knowledge an index doesn't indicate if it indexes refs, objects or logs. To find out, one must follow an arbitrary entry which points to either an index again or at a block marked with 'r', 'o' or 'g'.

Okay with me.

> The index may be organized into a multi-level index, where ...
> which may in turn point to either index blocks (3rd level) or ref blocks (leaf level).

So we allow 3 levels at most?

The file format structure marks the indexes '?', should that be rather '*' to indicate there can be more than one index block?