threads / patch / 48450

patch, 2 partsobject.c: free replace map in raw_object_store_clear

Subject: [PATCH 1/2] object.c: free replace map in raw_object_store_clear

## tl;dr

8 messages between May 9, 2018 and May 10, 2018. Diffs are folded; open one to read it.

replies: 7people: 4as markdown or json

Stefan Beller· May 9, 2018, 23:40 UTC · lore

The replace map for objects was missed to free in the object store in the conversion of 174774cd519 (Merge branch 'sb/object-store-replace', 2018-05-08)

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 object.c | 1 +
 1 file changed, 1 insertion(+)
Show changes to object.c +1 −0
diff --git a/object.c b/object.c
index 9d5b10d5a20..ff28f90c5ef 100644
--- a/object.c
+++ b/object.c
@@ -497,6 +497,7 @@ void raw_object_store_clear(struct raw_object_store *o)
 {
 	FREE_AND_NULL(o->objectdir);
 	FREE_AND_NULL(o->alternate_db);
+	FREE_AND_NULL(o->replace_map);
 
 	free_alt_odbs(o);
 	o->alt_odb_tail = NULL;
-- 
2.17.0.255.g8bfb7c0704
Stefan Beller· May 9, 2018, 23:40 UTC · re: Stefan Beller · lore

[PATCH 2/2] replace-object.c: remove the_repository from prepare_replace_object

This was missed in 5982da9d2ce (replace-object: allow prepare_replace_object to handle arbitrary repositories, 2018-04-11)

Technically the code works correctly as the replace_map is the same size in different repositories, however it is hard to read. So convert the code to the familiar pattern of dereferencing the pointer that we assign in the sizeof itself.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 replace_object.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Show changes to replace_object.c +1 −1
diff --git a/replace_object.c b/replace_object.c
index 246b98cd4f1..801b5c16789 100644
--- a/replace_object.c
+++ b/replace_object.c
@@ -37,7 +37,7 @@ static void prepare_replace_object(struct repository *r)
 		return;
 
 	r->objects->replace_map =
-		xmalloc(sizeof(*the_repository->objects->replace_map));
+		xmalloc(sizeof(*r->objects->replace_map));
 	oidmap_init(r->objects->replace_map, 0);
 
 	for_each_replace_ref(r, register_replace_ref, NULL);
-- 
2.17.0.255.g8bfb7c0704
Junio C Hamano· May 10, 2018, 10:23 UTC · re: Stefan Beller · lore

Re: [PATCH 2/2] replace-object.c: remove the_repository from prepare_replace_object

Stefan Beller <sbeller@google.com> writes:
Show 7 quoted lines
> This was missed in 5982da9d2ce (replace-object: allow
> prepare_replace_object to handle arbitrary repositories, 2018-04-11)
>
> Technically the code works correctly as the replace_map is the same
> size in different repositories, however it is hard to read. So convert
> the code to the familiar pattern of dereferencing the pointer that we
> assign in the sizeof itself.
;-)
We say
	ptr = xmalloc(sizeof(*ptr))
is better because 
	ptr = xmalloc(sizeof(typeof(*ptr)))

is easy to go stale unless we actually use typeof and instead say a concrete type like "struct oidmap".

This one was doing
	ptr = xmalloc(sizeof(*another_ptr))

and it was OK because ptr and another_ptr happened to be of the same type. I wonder if we are making it safer, or making it more obscure to seasoned C programmers, if we introduced a pair of helper macros, perhaps like these:

	#define ALLOCATE(ptr) (ptr) = xmalloc(sizeof(*(ptr)))
	#define CALLOCATE(ptr,cnt) (ptr) = xcalloc((cnt), sizeof(*(ptr)))
The change looks obviously good.  Will queue.
Thanks.
Jeff King· May 10, 2018, 11:56 UTC · re: Junio C Hamano · lore

Re: [PATCH 2/2] replace-object.c: remove the_repository from prepare_replace_object

On Thu, May 10, 2018 at 07:23:13PM +0900, Junio C Hamano wrote:
Show 11 quoted lines
> This one was doing
> 
> 	ptr = xmalloc(sizeof(*another_ptr))
> 
> and it was OK because ptr and another_ptr happened to be of the same
> type.  I wonder if we are making it safer, or making it more obscure
> to seasoned C programmers, if we introduced a pair of helper macros,
> perhaps like these:
> 
> 	#define ALLOCATE(ptr) (ptr) = xmalloc(sizeof(*(ptr)))
> 	#define CALLOCATE(ptr,cnt) (ptr) = xcalloc((cnt), sizeof(*(ptr)))

I've often wondered that, too. It's the natural endgame of the ALLOC_ARRAY() road we've been going down.

-Peff
Derrick Stolee· May 10, 2018, 14:53 UTC · re: Jeff King · lore

Re: [PATCH 2/2] replace-object.c: remove the_repository from prepare_replace_object

On 5/10/2018 7:56 AM, Jeff King wrote:
Show 15 quoted lines
> On Thu, May 10, 2018 at 07:23:13PM +0900, Junio C Hamano wrote:
>
>> This one was doing
>>
>> 	ptr = xmalloc(sizeof(*another_ptr))
>>
>> and it was OK because ptr and another_ptr happened to be of the same
>> type.  I wonder if we are making it safer, or making it more obscure
>> to seasoned C programmers, if we introduced a pair of helper macros,
>> perhaps like these:
>>
>> 	#define ALLOCATE(ptr) (ptr) = xmalloc(sizeof(*(ptr)))
>> 	#define CALLOCATE(ptr,cnt) (ptr) = xcalloc((cnt), sizeof(*(ptr)))
> I've often wondered that, too. It's the natural endgame of the
> ALLOC_ARRAY() road we've been going down.
I'll chime in that I like this idea.

Because I'm trying to learn more about Coccinelle, I added the diff below and ran 'make coccicheck'. It resulted in a 1000-line patch on 'next'. I'll refrain from sending that patch series, but it's nice to know a "simple" transformation is possible.

Using `git grep` I see 230 instances of 'xmalloc' and 261 instances of 'xcalloc'. After the Coccinelle transformation, these are down to 194 and 190, respectively, because the rest allocate in the same line as the definition. It's worth thinking about the macro pattern for those cases.

Show changes to 2 files +16 −0

contrib/coccinelle/alloc.cocci, git-compat-util.h

diff --git a/contrib/coccinelle/alloc.cocci b/contrib/coccinelle/alloc.cocci
new file mode 100644
index 0000000000..95f426c4dc
--- /dev/null
+++ b/contrib/coccinelle/alloc.cocci
@@ -0,0 +1,13 @@
+@@
+expression p;
+@@
+- p = xmalloc(sizeof(*p));
++ ALLOCATE(p);
+
+@@
+expression c;
+expression p;
+@@
+- p = xcalloc(c, sizeof(*p));
++ CALLOCATE(p,c);
+
diff --git a/git-compat-util.h b/git-compat-util.h
index f9e4c5f9bc..5398f217d7 100644
--- a/git-compat-util.h
+++ b/git-compat-util.h
@@ -843,6 +843,9 @@ extern FILE *fopen_or_warn(const char *path, const 
char *mode);
   */
  #define FREE_AND_NULL(p) do { free(p); (p) = NULL; } while (0)

+#define ALLOCATE(ptr) (ptr) = xmalloc(sizeof(*(ptr)))
+#define CALLOCATE(ptr,cnt) (ptr) = xcalloc((cnt), sizeof(*(ptr)))
+
  #define ALLOC_ARRAY(x, alloc) (x) = xmalloc(st_mult(sizeof(*(x)), 
(alloc)))
  #define REALLOC_ARRAY(x, alloc) (x) = xrealloc((x), 
st_mult(sizeof(*(x)), (alloc)))
Stefan Beller· May 10, 2018, 18:32 UTC · re: Derrick Stolee · lore

Re: [PATCH 2/2] replace-object.c: remove the_repository from prepare_replace_object

> Using `git grep` I see 230 instances of 'xmalloc' and 261 instances of
> 'xcalloc'. After the Coccinelle transformation, these are down to 194 and
> 190, respectively, because the rest allocate in the same line as the
> definition. It's worth thinking about the macro pattern for those cases.
Thanks for reporting the coccinelle experiment!

As we follow a strict declare before code, and we do not know if further declarations make use of this already, e.g. given

    struct foo *f = xmalloc(sizeof(*f));
    struct bar b = &f->baz;

we cannot split up the line declaring and assigning f, but the macro has to recreate the assignment upon declaration, for that we'd need to have something like

    ALLOCATE_TYPE(type, name);
which over complicates things IMHO.

Maybe it is worth identifying the pattern where 'f' is not used in further declarations, such that we can make patches as

-    struct foo *f = xmalloc(sizeof(*f));
+   struct foo *f;
    struct baz b = &unrelated;
+
+ ALLOCATE(f);
+

Thanks, Stefan

Junio C Hamano· May 10, 2018, 10:16 UTC · re: Stefan Beller · lore

Re: [PATCH 1/2] object.c: free replace map in raw_object_store_clear

Stefan Beller <sbeller@google.com> writes:
> The replace map for objects was missed to free in the object store in
> the conversion of 174774cd519 (Merge branch 'sb/object-store-replace',
> 2018-05-08)

I need a bit of clarification wrt the above. The above makes it sound like the merge needed a semantic conflict resolution (e.g. one side turned replace_map into a pointer while the other side added a place where the containing structure is freed and now the merge result needs to free the pointer in the new place that frees the containing structure, but the merge forgot to do so). Is that what is going on?

Or is this just a simple "the topic that ends at 174774cd519^2 had this leak that needs to be fixed by this patch; instead of rerolling this is an incremental, because the topic has already been merged to 'master' and it is too late now"?

Looking at this patch in the context of the side branch (instead of in the merged result) already makes sense to me, so I am guessing it is the latter (i.e. not a botched merge that missed semantic conflicts), in which case the proposed log message is a bit too alarming and points readers in a wrong direction. Shouldn't it point at, say, c1274495 ("replace-object: eliminate replace objects prepared flag", 2018-04-11) that turned the oidmap instance into a pointer in raw_object_store?

Thanks.
Show 18 quoted lines
>
> Signed-off-by: Stefan Beller <sbeller@google.com>
> ---
>  object.c | 1 +
>  1 file changed, 1 insertion(+)
>
> diff --git a/object.c b/object.c
> index 9d5b10d5a20..ff28f90c5ef 100644
> --- a/object.c
> +++ b/object.c
> @@ -497,6 +497,7 @@ void raw_object_store_clear(struct raw_object_store *o)
>  {
>  	FREE_AND_NULL(o->objectdir);
>  	FREE_AND_NULL(o->alternate_db);
> +	FREE_AND_NULL(o->replace_map);
>  
>  	free_alt_odbs(o);
>  	o->alt_odb_tail = NULL;
Stefan Beller· May 10, 2018, 17:23 UTC · re: Junio C Hamano · lore

Re: [PATCH 1/2] object.c: free replace map in raw_object_store_clear

Hi Junio,
On Thu, May 10, 2018 at 3:16 AM, Junio C Hamano <gitster@pobox.com> wrote:
Show 11 quoted lines
> Stefan Beller <sbeller@google.com> writes:
>
>> The replace map for objects was missed to free in the object store in
>> the conversion of 174774cd519 (Merge branch 'sb/object-store-replace',
>> 2018-05-08)
>
>
> Or is this just a simple "the topic that ends at 174774cd519^2 had
> this leak that needs to be fixed by this patch; instead of rerolling
> this is an incremental, because the topic has already been merged to
> 'master' and it is too late now"?

This is the case. I wondered if I want to point to the exact commit (which I have trouble pointing to as the whole topic has no memory leak fixes, Closest would be d88f9fdf8b2 (replace-object: move replace_map to object store, 2018-04-11)) or rather just point at the series. I did not think of the confusion that might arise there.

Show 8 quoted lines
> Looking at this patch in the context of the side branch (instead of
> in the merged result) already makes sense to me, so I am guessing it
> is the latter (i.e. not a botched merge that missed semantic
> conflicts), in which case the proposed log message is a bit too
> alarming and points readers in a wrong direction.  Shouldn't it
> point at, say, c1274495 ("replace-object: eliminate replace objects
> prepared flag", 2018-04-11) that turned the oidmap instance into a
> pointer in raw_object_store?

Ah, that is the best place to point at. Makes sense. I'll reroll this,

Thanks, Stefan

← back to recent threads