threads / patch / 54660

patchgc: fix cast in compare_tasks_by_selection()

Subject: [PATCH] gc: fix cast in compare_tasks_by_selection()

## tl;dr

2 messages between Nov 17, 2020 and Nov 17, 2020. Diffs are folded; open one to read it.

replies: 1people: 2as markdown or json

René Scharfe· Nov 17, 2020, 21:59 UTC · lore

compare_tasks_by_selection() is used with QSORT and gets passed pointers to the elements of "static struct maintenance_task tasks[]". It casts the *addresses* of these passed pointers to element pointers, though, and thus effectively compares some unrelated values from the stack. Fix the casts to actually compare array elements.

Detected by USan (make SANITIZE=undefined test).

Signed-off-by: René Scharfe <l.s.r@web.de>
---
 builtin/gc.c | 6 ++----
 1 file changed, 2 insertions(+), 4 deletions(-)

Show changes to builtin/gc.c +2 −5

diff --git a/builtin/gc.c b/builtin/gc.c
index 5cd2a43f9f..986b760b30 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -1213,10 +1213,8 @@ static struct maintenance_task tasks[] = {

 static int compare_tasks_by_selection(const void *a_, const void *b_)
 {
-	const struct maintenance_task *a, *b;
-
-	a = (const struct maintenance_task *)&a_;
-	b = (const struct maintenance_task *)&b_;
+	const struct maintenance_task *a = a_;
+	const struct maintenance_task *b = b_;

 	return b->selected_order - a->selected_order;
 }
--
2.29.2

Jeff King· Nov 17, 2020, 22:30 UTC · re: René Scharfe · lore

Re: [PATCH] gc: fix cast in compare_tasks_by_selection()

On Tue, Nov 17, 2020 at 10:59:49PM +0100, René Scharfe wrote:

Show 7 quoted lines

> compare_tasks_by_selection() is used with QSORT and gets passed pointers
> to the elements of "static struct maintenance_task tasks[]".  It casts
> the *addresses* of these passed pointers to element pointers, though,
> and thus effectively compares some unrelated values from the stack.  Fix
> the casts to actually compare array elements.
> 
> Detected by USan (make SANITIZE=undefined test).

I checked the caller here, and indeed, it's passing an array-of-struct so your patch is doing the right thing (not that I doubted it, but that's what review is for).

This qsort void-pointer convention seems to create a lot of confusion (not just the lack of type-safety, but the fact that it's getting a pointer to the element). I felt like we had a discussion about this a while ago, and indeed, I found:

  https://lore.kernel.org/git/7b95417a-c8fb-4f1e-cb09-c36804a3a4d0@web.de/

The whole sub-thread is worth reading, but the macro you arrived at in:

  https://lore.kernel.org/git/c141fb44-904f-e8b6-119f-7d2d6bcfd81a@web.de/

seems pretty reasonable.

-Peff