git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: invalid tree and commit object

From
René Scharfe <l.s.r@web.de>
Date
May 9, 2020, 19:24 UTC
Message-ID
<938f0818-7e57-b883-009f-01db88ef8f65@web.de>
In-Reply-To
<xmqqpnbduiec.fsf@gitster.c.googlers.com>
Am 09.05.20 um 19:28 schrieb Junio C Hamano:
Show 37 quoted lines
> René Scharfe <l.s.r@web.de> writes:
>
>> So I got curious if such trees might be in popular repos, wrote the patch
>> below and checked around a bit, but couldn't find any.
>>
>> Is there a smarter way to check for duplicates?  One that doesn't need
>> allocations?  Perhaps by having a version of tree_entry_extract() that
>> seeks backwards somehow?
>
> I've never looked into seeking backwards in a tree object, but in
> unpack-trees, I had to deal with this exact problem that a blob
> 'hello' sorts before 'hello.c' which in turn sorts after a tree
> 'hello' because of the "implicit slash after the name for an entry
> for a tree object" rule by introducing the "cache_bottom" hack in
> the traversal logic to limit how far we must scan back.
>
> We may be able to limit the list of "seen recently" names in a
> similar way.
>
> If the tree we are scanning has 'hello' (blob), 'hello.c' and
> 'hellp', the bottom pointer initially would be at 'hello' (blob),
> then stay there when we see 'hello.c' (because the next entry might
> be 'hello' (tree) that would crash with 'hello'), and when we see
> the entry 'hellp', we know that the entry at the bottom pointer
> 'hello' (blob) cannot crash with any entry that comes later in the
> tree object we are scanning, so we can advance the bottom pointer
> forward.  To decide if we can advance the bottom pointer beyond
> 'hello.c' (blob), we see if 'hello.c' (tree) can appear after the
> current entry we are looking at (i.e. 'hellp'), and we know it
> cannot without violating the sort order.  So the bottom would move
> to point at 'hellp' we just saw.
>
> If we had 'hello' (tree) instead of 'hellp', when we look at it
> after looking at 'hello' (blob) and 'hello.c', we scan from the
> bottom pointer up to the previous entry, which is still pointing at
> 'hello' (blob), and notice the crash.
>

Hmm, this could lead to quadratic behavior in the worst case, can't it? Imagine you have:

  a
  a.b
  a.b.c
  ...
  a.b.c/
  a.b/
  a/

If you have a single bottom pointer, it needs to stay at "a" the whole time, so that you can detect the d/f conflict with "a/" at the end. And for all entries in between you need to scan from "a" on and compare each entry -- quadratic. The blobs "a.b" and "a.b.c" don't need to actually be present, but we need to scan all entries to determine "a.b/" and "a.b.c/" are not conflicting anyway. Right?

We could, however, reduce the names we add to the string_list to those that are possible candidates for conflict -- blobs followed by an entry whose name starts with the blob name followed by a dot and trees that follow an entry whose name matches in the same way.

René
Previous: Junio C HamanoNext: Junio C Hamano
Message 6 of 23 in “invalid tree and commit object”
  1. Brandon WilliamsMay 9, 2020
  2. René ScharfeMay 9, 2020
  3. Johannes SchindelinMay 9, 2020
  4. René ScharfeMay 9, 2020
  5. Junio C HamanoMay 9, 2020
  6. René ScharfeMay 9, 2020
  7. Junio C HamanoMay 9, 2020
  8. René ScharfeMay 10, 2020
  9. René ScharfeMay 10, 2020
  10. Junio C HamanoMay 11, 2020
  11. Brandon WilliamsMay 13, 2020
  12. René ScharfeMay 21, 2020
  13. 1/4 fsck: fix a typo in a commentRené Scharfe, May 21, 2020
  14. Denton LiuMay 21, 2020
  15. René ScharfeMay 21, 2020
  16. 2/4 t1450: increase test coverage of in-tree d/f detectionRené Scharfe, May 21, 2020
  17. Denton LiuMay 21, 2020
  18. René ScharfeMay 21, 2020
  19. Junio C HamanoMay 21, 2020
  20. 3/4 t1450: demonstrate undetected in-tree d/f conflictRené Scharfe, May 21, 2020
  21. 4/4 fsck: detect more in-tree d/f conflictsRené Scharfe, May 21, 2020
  22. Junio C HamanoMay 10, 2020
  23. René ScharfeMay 21, 2020

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.