git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH] reftable/record: bound an object record's offset count by its length

From
Mm@suozzo.co <m@suozzo.co>
Date
Oct 19, 2026, 17:14 UTC
Message-ID
<41edbeff828fcb4a2f3bc903802b14e3aa66f46b@suozzo.co>
In-Reply-To
<atXK0ska_v2rFn8W@pks.im>
October 19, 2026 at 3:46 AM, "Patrick Steinhardt" <ps@pks.im mailto:ps@pks.im?to=%22Patrick%20Steinhardt%22%20%3Cps%40pks.im%3E > wrote:
Show 26 quoted lines
> 
> On Wed, Oct 14, 2026 at 02:58:03PM +0000, Matthew Suozzo via GitGitGadget wrote:
> 
> > 
> > From: Matthew Suozzo <m@suozzo.co>
> >  
> >  An object record's count of offsets is a 64-bit varint from the table,
> >  and reftable_obj_record_decode() allocates the offsets array from it
> >  through REFTABLE_ALLOC_ARRAY(), which takes a size_t. Where size_t is
> >  32 bits, a count of 2^32 + 1 allocates room for one offset, and the
> >  loop, which runs to the full count, writes an offset from the table
> >  for each varint left in the block past the end of the array. Where
> >  size_t is 64 bits, a large count only makes a large allocation that
> >  the decode then fails to fill.
> >  
> >  Nothing in Git reaches this decoder from a repository: a block of a
> >  type other than the one wanted is refused before its records are
> >  decoded, and the object index, the only reader of object blocks, is
> >  used only by the unit tests. The decoder is still wrong.
> >  
> >  Each offset takes at least a byte, so refuse a count larger than what
> >  is left of the record.
> > 
> Sorry, but this whole commit message reads very AI-generated to me, as
> it is both overly verbose and hard to parse. I would welcome it if you
> could maybe touch up the message for normal human beings.
yeah it was AI generated and i copy-edited it. it didn't feel too excessive since i wanted to be precise about the issue's mechanism, scope (32-bit only), severity (not really reachable), and describe the fix.
any pointers on how you'd reword? remove the exposition?
Show 17 quoted lines
> 
> > 
> > diff --git a/Makefile b/Makefile
> >  index a96be506b5..66b7270fc0 100644
> >  --- a/Makefile
> >  +++ b/Makefile
> >  @@ -1551,6 +1551,7 @@ CLAR_TEST_SUITES += u-reftable-block
> >  CLAR_TEST_SUITES += u-reftable-merged
> >  CLAR_TEST_SUITES += u-reftable-pq
> >  CLAR_TEST_SUITES += u-reftable-readwrite
> >  +CLAR_TEST_SUITES += u-reftable-record
> >  CLAR_TEST_SUITES += u-reftable-stack
> >  CLAR_TEST_SUITES += u-reftable-table
> >  CLAR_TEST_SUITES += u-reftable-tree
> > 
> Makes sense, even though this should probably have been in its own
> patch.
happy to move it to a separate patch.
Show 29 quoted lines
> 
> > 
> > diff --git a/reftable/record.c b/reftable/record.c
> >  index 1fce441930..96eceddca7 100644
> >  --- a/reftable/record.c
> >  +++ b/reftable/record.c
> >  @@ -589,6 +589,10 @@ static int reftable_obj_record_decode(void *rec, struct reftable_buf key,
> >  if (count == 0)
> >  return start.len - in.len;
> >  
> >  + /* each offset takes at least a byte */
> >  + if (count > in.len)
> >  + return REFTABLE_FORMAT_ERROR;
> > 
> This check doesn't really seem to match what you're describing. What you
> describe is an integer overflow, but you only check for a very specific
> edge ecase here. I would have expected for example that...
> 
> > 
> > REFTABLE_ALLOC_ARRAY(r->offsets, count);
> > 
> ... this line already detects the overflow and then refuses the
> operation. Because `REFTABLE_ALLOC_ARRAY()` itself already tries to
> check for overflows.
> 
> That being said, if you take a look at `reftable_alloc_size()` you can
> see that the input is both of type `size_t`. And _that_ is indeed broken
> on platforms where `size_t` is 32 bits, as we'd potentially wrap. But
> that fact is not even mentioned as part of the commit message at all.
isn't that "REFTABLE_ALLOC_ARRAY(), which takes a size_t" in the original message?
> 
> So overall I'm not convinced that this fix makes sense, we should rather
> fix the source of the problem, which is `reftable_alloc_size()`.
that felt like a pretty disruptive change i was hesitant to put up as my first patch but i'd be happy to make the change.
Show 5 quoted lines
> 
> Thanks!
> 
> Patrick
>
Previous: Patrick SteinhardtNext: Patrick Steinhardt
Message 3 of 10 in “reftable/record: bound an object record's offset count by its length”
  1. reftable/record: bound an object record's offset count by its lengthMatthew Suozzo via GitGitGadget, Oct 14, 2026
  2. Patrick SteinhardtOct 19, 2026
  3. m@suozzo.coOct 19, 2026
  4. Patrick SteinhardtOct 20, 2026
  5. Junio C HamanoOct 20, 2026
  6. Patrick SteinhardtOct 20, 2026
  7. Aniket GuptaOct 20, 2026
  8. 0/2 reftable: widen allocation size arguments to uintmax_tMatthew Suozzo via GitGitGadget, Oct 21, 2026
  9. 1/2 Makefile: build the reftable record unit testsMatthew Suozzo via GitGitGadget, Oct 21, 2026
  10. 2/2 reftable/basics: widen allocation size arguments to uintmax_tMatthew Suozzo via GitGitGadget, Oct 21, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.