git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 4/4] Fix delta offset overflow

From
Junio C Hamano <gitster@pobox.com>
Date
Aug 11, 2017, 18:50 UTC
Message-ID
<xmqqtw1eym56.fsf@gitster.mtv.corp.google.com>
In-Reply-To
<20170811065732.GA15128@mail.zuhause>
Martin Koegler <martin.koegler@chello.at> writes:
Show 9 quoted lines
> On Thu, Aug 10, 2017 at 01:49:24PM -0700, Junio C Hamano wrote:
>> The lower 4-byte of moff (before incrementing it with msize) were
>> already encoded to the output stream before this hunk.  Shouldn't
>> we be checking if moff would fit in uint32_t _before_ that happens?
>
> moff is otherwise only decremented or assigned with an offset generated by
> create_delta_index. These offsets are limited by 4GB.
>
> Any larger offets would be a programming bug - so qualify for just a "assert".

OK, in that case, I agree that a check before encoding moff into (upto) 4 output bytes is unnecessary. Sorry, I didn't read the function that populates index->hash[] before responding, and I admit that I haven't read it for a while.

Show 7 quoted lines
>> Cutting it off at here by resetting msize to 0 might help the next
>> iteration (I didn't check, but is the effect of it is to corrupt the
>> "val" rolling checksum and make it unlikely that the hash
>> computation would not find a correct match?) but it somehow feels
>> like closing the barn door after the horse has already bolted...
>
> The current code produces incorrect deltas - its not just a checksum issue.

Again, I mis-read what role msize was playing in the original (or in your update). I'd need to re-read that part of the code to make sure I get how your change will fix the issue.

Thanks.
Show 9 quoted lines
> By the way: 
>
> Somebody interested in JGIT should also look at these two bugs:
>
> https://github.com/eclipse/jgit/blob/005e5feb4ecd08c4e4d141a38b9e7942accb3212/org.eclipse.jgit/src/org/eclipse/jgit/internal/storage/pack/DeltaEncoder.java
> copy would also encode beyond 4GB - producing truncated delta offset.
>
> https://github.com/eclipse/jgit/blob/005e5feb4ecd08c4e4d141a38b9e7942accb3212/org.eclipse.jgit/src/org/eclipse/jgit/internal/storage/pack/BinaryDelta.java
> apply uses int for decoding length values.
I'll cc: an obvious suspect; thanks for the note.
Previous: Martin KoeglerNext: Junio C Hamano
Message 5 of 6 in “Convert zlib.c to size_t”
  1. 3/4 Convert zlib.c to size_tMartin Koegler, Aug 10, 2017
  2. 4/4 Fix delta offset overflowMartin Koegler, Aug 10, 2017
  3. Junio C HamanoAug 10, 2017
  4. Martin KoeglerAug 11, 2017
  5. Junio C HamanoAug 11, 2017
  6. Junio C HamanoAug 10, 2017

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.