git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2 5/5] reftable: document reading and writing indices

From
Patrick Steinhardt <ps@pks.im>
Date
Feb 6, 2024, 07:04 UTC
Message-ID
<ZcHZ4HM8f1eGr710@tanuki>
In-Reply-To
<oslbbspnu4spohamhenhxyqv23fct7ltuqkdl67liw774opxwj@jjprhh7zy34o>
On Mon, Feb 05, 2024 at 07:43:07PM -0600, jltobler wrote:
Show 45 quoted lines
> On 24/02/01 08:52AM, Patrick Steinhardt wrote:
> > The way the index gets written and read is not trivial at all and
> > requires the reader to piece together a bunch of parts to figure out how
> > it works. Add some documentation to hopefully make this easier to
> > understand for the next reader.
> > 
> > Signed-off-by: Patrick Steinhardt <ps@pks.im>
> > ---
> >  reftable/reader.c | 27 +++++++++++++++++++++++++++
> >  reftable/writer.c | 23 +++++++++++++++++++++++
> >  2 files changed, 50 insertions(+)
> > 
> > diff --git a/reftable/reader.c b/reftable/reader.c
> > index 278f727a3d..6011d0aa04 100644
> > --- a/reftable/reader.c
> > +++ b/reftable/reader.c
> > @@ -508,11 +508,38 @@ static int reader_seek_indexed(struct reftable_reader *r,
> >  	if (err < 0)
> >  		goto done;
> >  
> > +	/*
> > +	 * The index may consist of multiple levels, where each level may have
> > +	 * multiple index blocks. We start by doing a linear search in the
> > +	 * highest layer that identifies the relevant index block as well as
> > +	 * the record inside that block that corresponds to our wanted key.
> > +	 */
> >  	err = reader_seek_linear(&index_iter, &want_index);
> >  	if (err < 0)
> >  		goto done;
> >  
> > +	/*
> > +	 * Traverse down the levels until we find a non-index entry.
> > +	 */
> >  	while (1) {
> > +		/*
> > +		 * In case we seek a record that does not exist the index iter
> > +		 * will tell us that the iterator is over. This works because
> > +		 * the last index entry of the current level will contain the
> > +		 * last key it knows about. So in case our seeked key is larger
> > +		 * than the last indexed key we know that it won't exist.
> 
> The last block in the highest-level index section should end with the
> record key of greatest value. Doesn't that mean the initial linear seek
> should be sufficient to stop the iterator from continuing if the wanted
> record key is of a greater value?

Yes. But we only notice it here when calling `table_iter_next()`. The call to `reader_seek_linear()` will not return an end-of-iterator indication.

Is there any way you think this comment can be improved to clarify this?
Patrick
Previous: jltobler
Message 21 of 21 in “reftable: fix writing multi-level indices”
  1. 0/5 reftable: fix writing multi-level indicesPatrick Steinhardt, Jan 26, 2024
  2. 1/5 reftable/reader: be more careful about errors in indexed seeksPatrick Steinhardt, Jan 26, 2024
  3. 2/5 reftable/writer: use correct type to iterate through index entriesPatrick Steinhardt, Jan 26, 2024
  4. 3/5 reftable/writer: simplify writing index recordsPatrick Steinhardt, Jan 26, 2024
  5. Toon ClaesJan 31, 2024
  6. Patrick SteinhardtFeb 1, 2024
  7. Kristoffer HaugsbakkJan 31, 2024
  8. Patrick SteinhardtFeb 1, 2024
  9. 4/5 reftable/writer: fix writing multi-level indicesPatrick Steinhardt, Jan 26, 2024
  10. 5/5 reftable: document reading and writing indicesPatrick Steinhardt, Jan 26, 2024
  11. Junio C HamanoJan 26, 2024
  12. 0/5 reftable: fix writing multi-level indicesPatrick Steinhardt, Feb 1, 2024
  13. 1/5 reftable/reader: be more careful about errors in indexed seeksPatrick Steinhardt, Feb 1, 2024
  14. 2/5 reftable/writer: use correct type to iterate through index entriesPatrick Steinhardt, Feb 1, 2024
  15. 3/5 reftable/writer: simplify writing index recordsPatrick Steinhardt, Feb 1, 2024
  16. 4/5 reftable/writer: fix writing multi-level indicesPatrick Steinhardt, Feb 1, 2024
  17. jltoblerFeb 5, 2024
  18. Patrick SteinhardtFeb 6, 2024
  19. 5/5 reftable: document reading and writing indicesPatrick Steinhardt, Feb 1, 2024
  20. jltoblerFeb 6, 2024
  21. Patrick SteinhardtFeb 6, 2024

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.