[PATCH v3 04/22] reftable/basics: handle allocation failures in `reftable_calloc()`

From: Patrick Steinhardt <ps@pks.im>
Date: Sep 30, 2024, 08:08 UTC
Message-ID: <f6ad92ffd01c442dacd3ac6aa448891028636636.1727680272.git.ps@pks.im>
In-Reply-To: <cover.1727680272.git.ps@pks.im>

Handle allocation failures in `reftable_calloc()`.

While at it, remove our use of `st_mult()` that would cause us to die on an overflow. From the caller's point of view there is not much of a difference between arguments that are too large to be multiplied and a request that is too big to handle by the allocator: in both cases the allocation cannot be fulfilled. And in neither of these cases do we want the reftable library to die.

While we could use `unsigned_mult_overflows()` to handle the overflow gracefully, we instead open-code it to further our goal of converting the reftable codebase to become a standalone library that can be reused by external projects.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 reftable/basics.c | 13 ++++++++++---
 1 file changed, 10 insertions(+), 3 deletions(-)

diff --git a/reftable/basics.c b/reftable/basics.c
index 4adc98cf5d..3350bbffa2 100644
--- a/reftable/basics.c
+++ b/reftable/basics.c
@@ -37,9 +37,16 @@ void reftable_free(void *p)
 
 void *reftable_calloc(size_t nelem, size_t elsize)
 {
-	size_t sz = st_mult(nelem, elsize);
-	void *p = reftable_malloc(sz);
-	memset(p, 0, sz);
+	void *p;
+
+	if (nelem && elsize > SIZE_MAX / nelem)
+		return NULL;
+
+	p = reftable_malloc(nelem * elsize);
+	if (!p)
+		return NULL;
+
+	memset(p, 0, nelem * elsize);
 	return p;
 }

-- 
2.46.2.852.g229c0bf0e5.dirty