threads / patch / 45616

patch, 2 partsFix nonnull errors reported by UBSAN with GCC 7.

Subject: [PATCH 1/2] Fix nonnull errors reported by UBSAN with GCC 7.

## tl;dr

15 messages between Apr 6, 2017 and Apr 17, 2017. Diffs are folded; open one to read it.

replies: 14people: 6as markdown or json

Martin Liška· Apr 6, 2017, 08:02 UTC · lore
Hello.
Following patch fixes issues that can be seen with -fsanitize=undefined on GCC 7.
Patch was tested with make test.

Thanks, Martin

From e6d2d5ee5614acdbe67b79aeb0fdc9b53cf3a828 Mon Sep 17 00:00:00 2001
From: marxin <mliska@suse.cz>
Date: Wed, 5 Apr 2017 14:31:32 +0200
Subject: [PATCH 1/2] Fix nonnull errors reported by UBSAN with GCC 7.

Memory functions like memmove and memcpy should not be called with an argument equal to NULL.

Signed-off-by: Martin Liska <mliska@suse.cz>
---
 apply.c            | 7 ++++---
 builtin/ls-files.c | 5 +++--
 2 files changed, 7 insertions(+), 5 deletions(-)
Show changes to 2 files +7 −5

apply.c, builtin/ls-files.c

diff --git a/apply.c b/apply.c
index e6dbab26a..78a83f66b 100644
--- a/apply.c
+++ b/apply.c
@@ -2802,9 +2802,10 @@ static void update_image(struct apply_state *state,
 			img->line + applied_pos + preimage_limit,
 			(img->nr - (applied_pos + preimage_limit)) *
 			sizeof(*img->line));
-	memcpy(img->line + applied_pos,
-	       postimage->line,
-	       postimage->nr * sizeof(*img->line));
+	if (postimage->nr)
+		memcpy(img->line + applied_pos,
+			postimage->line,
+			postimage->nr * sizeof(*img->line));
 	if (!state->allow_overlap)
 		for (i = 0; i < postimage->nr; i++)
 			img->line[applied_pos + i].flag |= LINE_PATCHED;
diff --git a/builtin/ls-files.c b/builtin/ls-files.c
index d449e46db..01d24314d 100644
--- a/builtin/ls-files.c
+++ b/builtin/ls-files.c
@@ -391,8 +391,9 @@ static void prune_cache(const char *prefix, size_t prefixlen)
 		}
 		last = next;
 	}
-	memmove(active_cache, active_cache + pos,
-		(last - pos) * sizeof(struct cache_entry *));
+	if (last - pos > 0)
+		memmove(active_cache, active_cache + pos,
+			(last - pos) * sizeof(struct cache_entry *));
 	active_nr = last - pos;
 }
 

-- 
2.12.2
Jeff King· Apr 6, 2017, 08:34 UTC · re: Martin Liška · lore

Re: [PATCH 1/2] Fix nonnull errors reported by UBSAN with GCC 7.

On Thu, Apr 06, 2017 at 10:02:22AM +0200, Martin Liška wrote:
> Subject: [PATCH 1/2] Fix nonnull errors reported by UBSAN with GCC 7.
> 
> Memory functions like memmove and memcpy should not be called
> with an argument equal to NULL.

Yeah, makes sense. Your fixes are obviously correct. In other cases we've added wrappers like sane_qsort() that do the size check automatically. I'm not sure if we'd want to do the same here.

Either way, it probably makes sense to take this as a quick fix and worry about refactoring as a possible patch on top.

Thanks.
-Peff
Martin Liška· Apr 6, 2017, 09:52 UTC · re: Jeff King · lore

Re: [PATCH v2 1/2] Fix nonnull errors reported by UBSAN with GCC 7.

On 04/06/2017 10:34 AM, Jeff King wrote:
Show 18 quoted lines
> On Thu, Apr 06, 2017 at 10:02:22AM +0200, Martin Liška wrote:
> 
>> Subject: [PATCH 1/2] Fix nonnull errors reported by UBSAN with GCC 7.
>>
>> Memory functions like memmove and memcpy should not be called
>> with an argument equal to NULL.
> 
> Yeah, makes sense. Your fixes are obviously correct. In other cases
> we've added wrappers like sane_qsort() that do the size check
> automatically. I'm not sure if we'd want to do the same here.
> 
> Either way, it probably makes sense to take this as a quick fix and
> worry about refactoring as a possible patch on top.
> 
> Thanks.
> 
> -Peff
> 
Hello.

I'm sending (v2), where I updated commit message and wrapped 2 problematic places to newly introduced macros that do the check. Follow-up patch can change usages of memcpy and memove.

Martin
From 876cfa4f4b2e74d43b9fd93d1056b88ab2ace0cd Mon Sep 17 00:00:00 2001
From: marxin <mliska@suse.cz>
Date: Wed, 5 Apr 2017 14:31:32 +0200
Subject: [PATCH 1/2] Fix nonnull errors reported by UBSAN with GCC 7.

Memory functions like memmove and memcpy should not be called with an argument equal to NULL.

Signed-off-by: Martin Liska <mliska@suse.cz>
---
 apply.c            |  6 +++---
 builtin/ls-files.c |  2 +-
 git-compat-util.h  | 20 ++++++++++++++++++++
 3 files changed, 24 insertions(+), 4 deletions(-)
Show changes to 3 files +24 −4

apply.c, builtin/ls-files.c, git-compat-util.h

diff --git a/apply.c b/apply.c
index e6dbab26a..eacca29fa 100644
--- a/apply.c
+++ b/apply.c
@@ -2802,9 +2802,9 @@ static void update_image(struct apply_state *state,
 			img->line + applied_pos + preimage_limit,
 			(img->nr - (applied_pos + preimage_limit)) *
 			sizeof(*img->line));
-	memcpy(img->line + applied_pos,
-	       postimage->line,
-	       postimage->nr * sizeof(*img->line));
+	MEMCPY(img->line + applied_pos,
+		postimage->line,
+		postimage->nr * sizeof(*img->line));
 	if (!state->allow_overlap)
 		for (i = 0; i < postimage->nr; i++)
 			img->line[applied_pos + i].flag |= LINE_PATCHED;
diff --git a/builtin/ls-files.c b/builtin/ls-files.c
index d449e46db..7caeeb6a6 100644
--- a/builtin/ls-files.c
+++ b/builtin/ls-files.c
@@ -391,7 +391,7 @@ static void prune_cache(const char *prefix, size_t prefixlen)
 		}
 		last = next;
 	}
-	memmove(active_cache, active_cache + pos,
+	MEMMOVE(active_cache, active_cache + pos,
 		(last - pos) * sizeof(struct cache_entry *));
 	active_nr = last - pos;
 }
diff --git a/git-compat-util.h b/git-compat-util.h
index 8a4a3f85e..e5323f6c7 100644
--- a/git-compat-util.h
+++ b/git-compat-util.h
@@ -1002,6 +1002,26 @@ int git_qsort_s(void *base, size_t nmemb, size_t size,
 		die("BUG: qsort_s() failed");			\
 } while (0)
 
+static inline void *sane_memcpy(void *dest, const void *src, size_t n)
+{
+	if (n > 0)
+		return memcpy(dest, src, n);
+	else
+		return dest;
+}
+
+#define MEMCPY(dest, src, n) sane_memcpy(dest, src, n)
+
+static inline void *sane_memmove(void *dest, const void *src, size_t n)
+{
+	if (n > 0)
+		return memmove(dest, src, n);
+	else
+		return dest;
+}
+
+#define MEMMOVE(dest, src, n) sane_memmove(dest, src, n)
+
 #ifndef REG_STARTEND
 #error "Git requires REG_STARTEND support. Compile with NO_REGEX=NeedsStartEnd"
 #endif

-- 
2.12.2
René Scharfe· Apr 6, 2017, 12:26 UTC · re: Martin Liška · lore

Re: [PATCH v2 1/2] Fix nonnull errors reported by UBSAN with GCC 7.

Am 06.04.2017 um 11:52 schrieb Martin Liška:
> I'm sending (v2), where I updated commit message and wrapped 2 problematic
> places to newly introduced macros that do the check. Follow-up patch can
> change usages of memcpy and memove.
Show 14 quoted lines
> diff --git a/apply.c b/apply.c
> index e6dbab26a..eacca29fa 100644
> --- a/apply.c
> +++ b/apply.c
> @@ -2802,9 +2802,9 @@ static void update_image(struct apply_state *state,
>  			img->line + applied_pos + preimage_limit,
>  			(img->nr - (applied_pos + preimage_limit)) *
>  			sizeof(*img->line));
> -	memcpy(img->line + applied_pos,
> -	       postimage->line,
> -	       postimage->nr * sizeof(*img->line));
> +	MEMCPY(img->line + applied_pos,
> +		postimage->line,
> +		postimage->nr * sizeof(*img->line));
Using the existing macro COPY_ARRAY would yield a nicer result:
	COPY_ARRAY(img->line + applied_pos, postimage->line, postimage->nr);
Show 16 quoted lines
>  	if (!state->allow_overlap)
>  		for (i = 0; i < postimage->nr; i++)
>  			img->line[applied_pos + i].flag |= LINE_PATCHED;
> diff --git a/builtin/ls-files.c b/builtin/ls-files.c
> index d449e46db..7caeeb6a6 100644
> --- a/builtin/ls-files.c
> +++ b/builtin/ls-files.c
> @@ -391,7 +391,7 @@ static void prune_cache(const char *prefix, size_t prefixlen)
>  		}
>  		last = next;
>  	}
> -	memmove(active_cache, active_cache + pos,
> +	MEMMOVE(active_cache, active_cache + pos,
>  		(last - pos) * sizeof(struct cache_entry *));
>  	active_nr = last - pos;
>  }
Perhaps we should add MOVE_ARRAY to complement COPY_ARRAY.
René
Martin Liška· Apr 6, 2017, 15:42 UTC · re: René Scharfe · lore

Re: [PATCH v3 1/2] Fix nonnull errors reported by UBSAN with GCC 7.

On 04/06/2017 02:26 PM, René Scharfe wrote:
Show 21 quoted lines
> Am 06.04.2017 um 11:52 schrieb Martin Liška:
>> I'm sending (v2), where I updated commit message and wrapped 2 problematic
>> places to newly introduced macros that do the check. Follow-up patch can
>> change usages of memcpy and memove.
> 
>> diff --git a/apply.c b/apply.c
>> index e6dbab26a..eacca29fa 100644
>> --- a/apply.c
>> +++ b/apply.c
>> @@ -2802,9 +2802,9 @@ static void update_image(struct apply_state *state,
>>  			img->line + applied_pos + preimage_limit,
>>  			(img->nr - (applied_pos + preimage_limit)) *
>>  			sizeof(*img->line));
>> -	memcpy(img->line + applied_pos,
>> -	       postimage->line,
>> -	       postimage->nr * sizeof(*img->line));
>> +	MEMCPY(img->line + applied_pos,
>> +		postimage->line,
>> +		postimage->nr * sizeof(*img->line));
> 
> Using the existing macro COPY_ARRAY would yield a nicer result:
Yes, it's nicer. I'm sending tested version 3.
Martin
Show 24 quoted lines
> 
> 	COPY_ARRAY(img->line + applied_pos, postimage->line, postimage->nr);
> 
>>  	if (!state->allow_overlap)
>>  		for (i = 0; i < postimage->nr; i++)
>>  			img->line[applied_pos + i].flag |= LINE_PATCHED;
>> diff --git a/builtin/ls-files.c b/builtin/ls-files.c
>> index d449e46db..7caeeb6a6 100644
>> --- a/builtin/ls-files.c
>> +++ b/builtin/ls-files.c
>> @@ -391,7 +391,7 @@ static void prune_cache(const char *prefix, size_t prefixlen)
>>  		}
>>  		last = next;
>>  	}
>> -	memmove(active_cache, active_cache + pos,
>> +	MEMMOVE(active_cache, active_cache + pos,
>>  		(last - pos) * sizeof(struct cache_entry *));
>>  	active_nr = last - pos;
>>  }
> 
> Perhaps we should add MOVE_ARRAY to complement COPY_ARRAY.
> 
> René
> 
From 4784ff90b2c4cd0d78a25c8d8ed77f299686348c Mon Sep 17 00:00:00 2001
From: marxin <mliska@suse.cz>
Date: Wed, 5 Apr 2017 14:31:32 +0200
Subject: [PATCH 1/2] Fix nonnull errors reported by UBSAN with GCC 7.

Memory functions like memmove and memcpy should only be called with positive sizes. That is achieved by newly introduced macro MEMMOVE and usage of ARRAY_COPY.

Signed-off-by: Martin Liska <mliska@suse.cz>
---
 apply.c            |  4 +---
 builtin/ls-files.c |  2 +-
 git-compat-util.h  | 10 ++++++++++
 3 files changed, 12 insertions(+), 4 deletions(-)
Show changes to 3 files +12 −4

apply.c, builtin/ls-files.c, git-compat-util.h

diff --git a/apply.c b/apply.c
index e6dbab26a..121f3f414 100644
--- a/apply.c
+++ b/apply.c
@@ -2802,9 +2802,7 @@ static void update_image(struct apply_state *state,
 			img->line + applied_pos + preimage_limit,
 			(img->nr - (applied_pos + preimage_limit)) *
 			sizeof(*img->line));
-	memcpy(img->line + applied_pos,
-	       postimage->line,
-	       postimage->nr * sizeof(*img->line));
+	COPY_ARRAY(img->line + applied_pos, postimage->line, postimage->nr);
 	if (!state->allow_overlap)
 		for (i = 0; i < postimage->nr; i++)
 			img->line[applied_pos + i].flag |= LINE_PATCHED;
diff --git a/builtin/ls-files.c b/builtin/ls-files.c
index d449e46db..7caeeb6a6 100644
--- a/builtin/ls-files.c
+++ b/builtin/ls-files.c
@@ -391,7 +391,7 @@ static void prune_cache(const char *prefix, size_t prefixlen)
 		}
 		last = next;
 	}
-	memmove(active_cache, active_cache + pos,
+	MEMMOVE(active_cache, active_cache + pos,
 		(last - pos) * sizeof(struct cache_entry *));
 	active_nr = last - pos;
 }
diff --git a/git-compat-util.h b/git-compat-util.h
index 8a4a3f85e..b75e21cff 100644
--- a/git-compat-util.h
+++ b/git-compat-util.h
@@ -1002,6 +1002,16 @@ int git_qsort_s(void *base, size_t nmemb, size_t size,
 		die("BUG: qsort_s() failed");			\
 } while (0)
 
+static inline void *sane_memmove(void *dest, const void *src, size_t n)
+{
+	if (n > 0)
+		return memmove(dest, src, n);
+	else
+		return dest;
+}
+
+#define MEMMOVE(dest, src, n) sane_memmove(dest, src, n)
+
 #ifndef REG_STARTEND
 #error "Git requires REG_STARTEND support. Compile with NO_REGEX=NeedsStartEnd"
 #endif

-- 
2.12.2
Johannes Sixt· Apr 6, 2017, 16:33 UTC · re: Martin Liška · lore

Re: [PATCH v3 1/2] Fix nonnull errors reported by UBSAN with GCC 7.

Am 06.04.2017 um 17:42 schrieb Martin Liška:
Show 7 quoted lines
> +static inline void *sane_memmove(void *dest, const void *src, size_t n)
> +{
> +	if (n > 0)
> +		return memmove(dest, src, n);
> +	else
> +		return dest;
> +}
Huh? memmove with n == 0 is well-defined. This wrapper is pointless.
-- Hannes
René Scharfe· Apr 6, 2017, 17:31 UTC · re: Johannes Sixt · lore

Re: [PATCH v3 1/2] Fix nonnull errors reported by UBSAN with GCC 7.

Am 06.04.2017 um 18:33 schrieb Johannes Sixt:
Show 10 quoted lines
> Am 06.04.2017 um 17:42 schrieb Martin Liška:
>> +static inline void *sane_memmove(void *dest, const void *src, size_t n)
>> +{
>> +    if (n > 0)
>> +        return memmove(dest, src, n);
>> +    else
>> +        return dest;
>> +}
>
> Huh? memmove with n == 0 is well-defined. This wrapper is pointless.
memmove(3) with NULL pointers is undefined.  From string.h on Debian:
   extern void *memmove (void *__dest, const void *__src, size_t __n)
        __THROW __nonnull ((1, 2));

Sometimes we use a NULL pointer and a size of zero to represent arrays with no members. That convention is incompatible with memmove(3), but the wrapper above would support it. Checking the size instead of the pointer is preferable because a positive length with NULL pointers should still result in a segfault instead of a silent no-op.

(I'd still prefer a MOVE_ARRAY macro which also infers the element size).

René
Johannes Sixt· Apr 6, 2017, 20:49 UTC · re: René Scharfe · lore

Re: [PATCH v3 1/2] Fix nonnull errors reported by UBSAN with GCC 7.

Am 06.04.2017 um 19:31 schrieb René Scharfe:
Show 13 quoted lines
> Am 06.04.2017 um 18:33 schrieb Johannes Sixt:
>> Am 06.04.2017 um 17:42 schrieb Martin Liška:
>>> +static inline void *sane_memmove(void *dest, const void *src, size_t n)
>>> +{
>>> +    if (n > 0)
>>> +        return memmove(dest, src, n);
>>> +    else
>>> +        return dest;
>>> +}
>>
>> Huh? memmove with n == 0 is well-defined. This wrapper is pointless.
>
> memmove(3) with NULL pointers is undefined.

Then don't hide this helper behind a macro with a suspiciously similar name. Using the name sane_mmemove at the call site is preferable. memmove_or_null or something similar perhaps even more so.

-- Hannes
Martin Liška· Apr 7, 2017, 14:23 UTC · re: Johannes Sixt · lore

Re: [PATCH v3 1/2] Fix nonnull errors reported by UBSAN with GCC 7.

On 04/06/2017 10:49 PM, Johannes Sixt wrote:
Show 19 quoted lines
> Am 06.04.2017 um 19:31 schrieb René Scharfe:
>> Am 06.04.2017 um 18:33 schrieb Johannes Sixt:
>>> Am 06.04.2017 um 17:42 schrieb Martin Liška:
>>>> +static inline void *sane_memmove(void *dest, const void *src, size_t n)
>>>> +{
>>>> +    if (n > 0)
>>>> +        return memmove(dest, src, n);
>>>> +    else
>>>> +        return dest;
>>>> +}
>>>
>>> Huh? memmove with n == 0 is well-defined. This wrapper is pointless.
>>
>> memmove(3) with NULL pointers is undefined.
> 
> Then don't hide this helper behind a macro with a suspiciously similar name. Using the name sane_mmemove at the call site is preferable. memmove_or_null or something similar perhaps even more so.
> 
> -- Hannes
> 
Good. There's tested v4.
Martin
From 0bdf4d717d3d368dd9676d15d20f8592c4d22fde Mon Sep 17 00:00:00 2001
From: marxin <mliska@suse.cz>
Date: Wed, 5 Apr 2017 14:31:32 +0200
Subject: [PATCH 1/2] Fix nonnull errors reported by UBSAN with GCC 7.

Replace call to memmove with newly introduced function memmove_or_null and call to memcpy with COPY_ARRAY macro.

Signed-off-by: Martin Liska <mliska@suse.cz>
---
 apply.c            | 4 +---
 builtin/ls-files.c | 2 +-
 git-compat-util.h  | 8 ++++++++
 3 files changed, 10 insertions(+), 4 deletions(-)
Show changes to 3 files +10 −4

apply.c, builtin/ls-files.c, git-compat-util.h

diff --git a/apply.c b/apply.c
index e6dbab26a..121f3f414 100644
--- a/apply.c
+++ b/apply.c
@@ -2802,9 +2802,7 @@ static void update_image(struct apply_state *state,
 			img->line + applied_pos + preimage_limit,
 			(img->nr - (applied_pos + preimage_limit)) *
 			sizeof(*img->line));
-	memcpy(img->line + applied_pos,
-	       postimage->line,
-	       postimage->nr * sizeof(*img->line));
+	COPY_ARRAY(img->line + applied_pos, postimage->line, postimage->nr);
 	if (!state->allow_overlap)
 		for (i = 0; i < postimage->nr; i++)
 			img->line[applied_pos + i].flag |= LINE_PATCHED;
diff --git a/builtin/ls-files.c b/builtin/ls-files.c
index d449e46db..0a6cc1e8a 100644
--- a/builtin/ls-files.c
+++ b/builtin/ls-files.c
@@ -391,7 +391,7 @@ static void prune_cache(const char *prefix, size_t prefixlen)
 		}
 		last = next;
 	}
-	memmove(active_cache, active_cache + pos,
+	memmove_or_null(active_cache, active_cache + pos,
 		(last - pos) * sizeof(struct cache_entry *));
 	active_nr = last - pos;
 }
diff --git a/git-compat-util.h b/git-compat-util.h
index 8a4a3f85e..81f6e56ac 100644
--- a/git-compat-util.h
+++ b/git-compat-util.h
@@ -1002,6 +1002,14 @@ int git_qsort_s(void *base, size_t nmemb, size_t size,
 		die("BUG: qsort_s() failed");			\
 } while (0)
 
+static inline void *memmove_or_null(void *dest, const void *src, size_t n)
+{
+	if (n > 0)
+		return memmove(dest, src, n);
+	else
+		return dest;
+}
+
 #ifndef REG_STARTEND
 #error "Git requires REG_STARTEND support. Compile with NO_REGEX=NeedsStartEnd"
 #endif

-- 
2.12.2
René Scharfe· Apr 7, 2017, 15:25 UTC · re: Martin Liška · lore

Re: [PATCH v3 1/2] Fix nonnull errors reported by UBSAN with GCC 7.

Am 07.04.2017 um 16:23 schrieb Martin Liška:
Show 22 quoted lines
> On 04/06/2017 10:49 PM, Johannes Sixt wrote:
>> Am 06.04.2017 um 19:31 schrieb René Scharfe:
>>> Am 06.04.2017 um 18:33 schrieb Johannes Sixt:
>>>> Am 06.04.2017 um 17:42 schrieb Martin Liška:
>>>>> +static inline void *sane_memmove(void *dest, const void *src, size_t n)
>>>>> +{
>>>>> +    if (n > 0)
>>>>> +        return memmove(dest, src, n);
>>>>> +    else
>>>>> +        return dest;
>>>>> +}
>>>>
>>>> Huh? memmove with n == 0 is well-defined. This wrapper is pointless.
>>>
>>> memmove(3) with NULL pointers is undefined.
>>
>> Then don't hide this helper behind a macro with a suspiciously similar name. Using the name sane_mmemove at the call site is preferable. memmove_or_null or something similar perhaps even more so.
>>
>> -- Hannes
>>
> 
> Good. There's tested v4.
Thank you!  I'd *still* prefer MOVE_ARRAY, though.  Patches to follow.
René
René Scharfe· Apr 7, 2017, 15:25 UTC · re: Martin Liška · lore

[PATCH 1/2] add MOVE_ARRAY

Add MOVE_ARRAY to complement COPY_ARRAY, which was added in 60566cbb. It provides the same convenience, safety and support for NULL pointers as representations of empty arrays, just as a wrapper for memmove(3) instead of memcpy(3).

Inspired-by: Martin Liska <mliska@suse.cz>
Signed-off-by: Rene Scharfe <l.s.r@web.de>
---
 git-compat-util.h | 8 ++++++++
 1 file changed, 8 insertions(+)
Show changes to git-compat-util.h +8 −0
diff --git a/git-compat-util.h b/git-compat-util.h
index 8a4a3f85e7..3266a71fb4 100644
--- a/git-compat-util.h
+++ b/git-compat-util.h
@@ -812,6 +812,14 @@ static inline void copy_array(void *dst, const void *src, size_t n, size_t size)
 		memcpy(dst, src, st_mult(size, n));
 }
 
+#define MOVE_ARRAY(dst, src, n) move_array((dst), (src), (n), sizeof(*(dst)) + \
+	BUILD_ASSERT_OR_ZERO(sizeof(*(dst)) == sizeof(*(src))))
+static inline void move_array(void *dst, const void *src, size_t n, size_t size)
+{
+	if (n)
+		memmove(dst, src, st_mult(size, n));
+}
+
 /*
  * These functions help you allocate structs with flex arrays, and copy
  * the data directly into the array. For example, if you had:

-- 
2.12.2
René Scharfe· Apr 7, 2017, 15:25 UTC · re: Martin Liška · lore

[PATCH 2/2] use MOVE_ARRAY

Add a semantic patch for converting certain calls of memmove(3) to MOVE_ARRAY() and apply that transformation to the code base. The result is shorter and safer code.

Signed-off-by: Rene Scharfe <l.s.r@web.de>
---
 apply.c                        |  7 +++----
 builtin/ls-files.c             |  3 +--
 builtin/merge.c                |  2 +-
 builtin/pack-objects.c         |  5 ++---
 cache-tree.c                   |  5 ++---
 commit.c                       | 11 ++++-------
 contrib/coccinelle/array.cocci | 37 +++++++++++++++++++++++++++++++++++++
 diffcore-rename.c              |  8 ++++----
 dir.c                          |  4 ++--
 notes-merge.c                  |  3 +--
 parse-options.c                |  2 +-
 read-cache.c                   |  5 ++---
 reflog-walk.c                  |  7 +++----
 refs/files-backend.c           |  5 ++---
 replace_object.c               |  6 ++----
 rerere.c                       |  4 ++--
 string-list.c                  |  5 ++---
 17 files changed, 71 insertions(+), 48 deletions(-)
Show changes to 17 files +71 −48

apply.c, builtin/ls-files.c, builtin/merge.c, builtin/pack-objects.c, cache-tree.c, commit.c, contrib/coccinelle/array.cocci, diffcore-rename.c, dir.c, notes-merge.c, parse-options.c, read-cache.c, reflog-walk.c, refs/files-backend.c, replace_object.c, rerere.c, string-list.c

diff --git a/apply.c b/apply.c
index e6dbab26ad..4e4d13cacc 100644
--- a/apply.c
+++ b/apply.c
@@ -2798,10 +2798,9 @@ static void update_image(struct apply_state *state,
 		img->line_allocated = img->line;
 	}
 	if (preimage_limit != postimage->nr)
-		memmove(img->line + applied_pos + postimage->nr,
-			img->line + applied_pos + preimage_limit,
-			(img->nr - (applied_pos + preimage_limit)) *
-			sizeof(*img->line));
+		MOVE_ARRAY(img->line + applied_pos + postimage->nr,
+			   img->line + applied_pos + preimage_limit,
+			   img->nr - (applied_pos + preimage_limit));
 	memcpy(img->line + applied_pos,
 	       postimage->line,
 	       postimage->nr * sizeof(*img->line));
diff --git a/builtin/ls-files.c b/builtin/ls-files.c
index d449e46db5..1627c26466 100644
--- a/builtin/ls-files.c
+++ b/builtin/ls-files.c
@@ -391,8 +391,7 @@ static void prune_cache(const char *prefix, size_t prefixlen)
 		}
 		last = next;
 	}
-	memmove(active_cache, active_cache + pos,
-		(last - pos) * sizeof(struct cache_entry *));
+	MOVE_ARRAY(active_cache, active_cache + pos, last - pos);
 	active_nr = last - pos;
 }
 
diff --git a/builtin/merge.c b/builtin/merge.c
index 95572b1810..0c0d2e4a94 100644
--- a/builtin/merge.c
+++ b/builtin/merge.c
@@ -536,7 +536,7 @@ static void parse_branch_merge_options(char *bmo)
 		die(_("Bad branch.%s.mergeoptions string: %s"), branch,
 		    split_cmdline_strerror(argc));
 	REALLOC_ARRAY(argv, argc + 2);
-	memmove(argv + 1, argv, sizeof(*argv) * (argc + 1));
+	MOVE_ARRAY(argv + 1, argv, argc + 1);
 	argc++;
 	argv[0] = "branch.*.mergeoptions";
 	parse_options(argc, argv, NULL, builtin_merge_options,
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index 84af7c2324..5fb5d7c546 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -1292,9 +1292,8 @@ static int check_pbase_path(unsigned hash)
 		   done_pbase_paths_alloc);
 	done_pbase_paths_num++;
 	if (pos < done_pbase_paths_num)
-		memmove(done_pbase_paths + pos + 1,
-			done_pbase_paths + pos,
-			(done_pbase_paths_num - pos - 1) * sizeof(unsigned));
+		MOVE_ARRAY(done_pbase_paths + pos + 1, done_pbase_paths + pos,
+			   done_pbase_paths_num - pos - 1);
 	done_pbase_paths[pos] = hash;
 	return 0;
 }
diff --git a/cache-tree.c b/cache-tree.c
index 345ea35963..071164a2ec 100644
--- a/cache-tree.c
+++ b/cache-tree.c
@@ -131,9 +131,8 @@ static int do_invalidate_path(struct cache_tree *it, const char *path)
 			 * move 4 and 5 up one place (2 entries)
 			 * 2 = 6 - 3 - 1 = subtree_nr - pos - 1
 			 */
-			memmove(it->down+pos, it->down+pos+1,
-				sizeof(struct cache_tree_sub *) *
-				(it->subtree_nr - pos - 1));
+			MOVE_ARRAY(it->down + pos, it->down + pos + 1,
+				   it->subtree_nr - pos - 1);
 			it->subtree_nr--;
 		}
 		return 1;
diff --git a/commit.c b/commit.c
index 73c78c2b80..62a71fee92 100644
--- a/commit.c
+++ b/commit.c
@@ -125,10 +125,8 @@ int register_commit_graft(struct commit_graft *graft, int ignore_dups)
 	ALLOC_GROW(commit_graft, commit_graft_nr + 1, commit_graft_alloc);
 	commit_graft_nr++;
 	if (pos < commit_graft_nr)
-		memmove(commit_graft + pos + 1,
-			commit_graft + pos,
-			(commit_graft_nr - pos - 1) *
-			sizeof(*commit_graft));
+		MOVE_ARRAY(commit_graft + pos + 1, commit_graft + pos,
+			   commit_graft_nr - pos - 1);
 	commit_graft[pos] = graft;
 	return 0;
 }
@@ -222,9 +220,8 @@ int unregister_shallow(const unsigned char *sha1)
 	if (pos < 0)
 		return -1;
 	if (pos + 1 < commit_graft_nr)
-		memmove(commit_graft + pos, commit_graft + pos + 1,
-				sizeof(struct commit_graft *)
-				* (commit_graft_nr - pos - 1));
+		MOVE_ARRAY(commit_graft + pos, commit_graft + pos + 1,
+			   commit_graft_nr - pos - 1);
 	commit_graft_nr--;
 	return 0;
 }
diff --git a/contrib/coccinelle/array.cocci b/contrib/coccinelle/array.cocci
index 4ba98b7eaf..a6e5ee927f 100644
--- a/contrib/coccinelle/array.cocci
+++ b/contrib/coccinelle/array.cocci
@@ -27,6 +27,43 @@ expression n;
 
 @@
 type T;
+T *dst;
+T *src;
+expression n;
+@@
+(
+- memmove(
++ MOVE_ARRAY(
+(
+  dst
+|
+  dst + ...
+|
+  &dst[...]
+)
+- , src, (n) * sizeof(*dst)
++ , src, n
+  );
+|
+- memmove(dst,
++ MOVE_ARRAY(dst,
+(
+  src
+|
+  src + ...
+|
+  &src[...]
+)
+- , (n) * sizeof(*src)
++ , n
+  );
+|
+- memmove(dst, src, (n) * sizeof(T));
++ MOVE_ARRAY(dst, src, n);
+)
+
+@@
+type T;
 T *ptr;
 expression n;
 @@
diff --git a/diffcore-rename.c b/diffcore-rename.c
index f7444c86bd..449d5f065f 100644
--- a/diffcore-rename.c
+++ b/diffcore-rename.c
@@ -57,8 +57,8 @@ static int add_rename_dst(struct diff_filespec *two)
 	ALLOC_GROW(rename_dst, rename_dst_nr + 1, rename_dst_alloc);
 	rename_dst_nr++;
 	if (first < rename_dst_nr)
-		memmove(rename_dst + first + 1, rename_dst + first,
-			(rename_dst_nr - first - 1) * sizeof(*rename_dst));
+		MOVE_ARRAY(rename_dst + first + 1, rename_dst + first,
+			   rename_dst_nr - first - 1);
 	rename_dst[first].two = alloc_filespec(two->path);
 	fill_filespec(rename_dst[first].two, two->oid.hash, two->oid_valid,
 		      two->mode);
@@ -98,8 +98,8 @@ static struct diff_rename_src *register_rename_src(struct diff_filepair *p)
 	ALLOC_GROW(rename_src, rename_src_nr + 1, rename_src_alloc);
 	rename_src_nr++;
 	if (first < rename_src_nr)
-		memmove(rename_src + first + 1, rename_src + first,
-			(rename_src_nr - first - 1) * sizeof(*rename_src));
+		MOVE_ARRAY(rename_src + first + 1, rename_src + first,
+			   rename_src_nr - first - 1);
 	rename_src[first].p = p;
 	rename_src[first].score = score;
 	return &(rename_src[first]);
diff --git a/dir.c b/dir.c
index f451bfa48c..f5f5658b1d 100644
--- a/dir.c
+++ b/dir.c
@@ -691,8 +691,8 @@ static struct untracked_cache_dir *lookup_untracked(struct untracked_cache *uc,
 	FLEX_ALLOC_MEM(d, name, name, len);
 
 	ALLOC_GROW(dir->dirs, dir->dirs_nr + 1, dir->dirs_alloc);
-	memmove(dir->dirs + first + 1, dir->dirs + first,
-		(dir->dirs_nr - first) * sizeof(*dir->dirs));
+	MOVE_ARRAY(dir->dirs + first + 1, dir->dirs + first,
+		   dir->dirs_nr - first);
 	dir->dirs_nr++;
 	dir->dirs[first] = d;
 	return d;
diff --git a/notes-merge.c b/notes-merge.c
index 5998605acc..36d5d2f87f 100644
--- a/notes-merge.c
+++ b/notes-merge.c
@@ -99,8 +99,7 @@ static struct notes_merge_pair *find_notes_merge_pair_pos(
 	else {
 		*occupied = 0;
 		if (insert_new && i < len) {
-			memmove(list + i + 1, list + i,
-				(len - i) * sizeof(struct notes_merge_pair));
+			MOVE_ARRAY(list + i + 1, list + i, len - i);
 			memset(list + i, 0, sizeof(struct notes_merge_pair));
 		}
 	}
diff --git a/parse-options.c b/parse-options.c
index a23a1e67f0..b8f768b0d8 100644
--- a/parse-options.c
+++ b/parse-options.c
@@ -524,7 +524,7 @@ int parse_options_step(struct parse_opt_ctx_t *ctx,
 
 int parse_options_end(struct parse_opt_ctx_t *ctx)
 {
-	memmove(ctx->out + ctx->cpidx, ctx->argv, ctx->argc * sizeof(*ctx->out));
+	MOVE_ARRAY(ctx->out + ctx->cpidx, ctx->argv, ctx->argc);
 	ctx->out[ctx->cpidx + ctx->argc] = NULL;
 	return ctx->cpidx + ctx->argc;
 }
diff --git a/read-cache.c b/read-cache.c
index e447751823..1bb5e97979 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -514,9 +514,8 @@ int remove_index_entry_at(struct index_state *istate, int pos)
 	istate->cache_nr--;
 	if (pos >= istate->cache_nr)
 		return 0;
-	memmove(istate->cache + pos,
-		istate->cache + pos + 1,
-		(istate->cache_nr - pos) * sizeof(struct cache_entry *));
+	MOVE_ARRAY(istate->cache + pos, istate->cache + pos + 1,
+		   istate->cache_nr - pos);
 	return 1;
 }
 
diff --git a/reflog-walk.c b/reflog-walk.c
index 99679f5825..25c198b9fd 100644
--- a/reflog-walk.c
+++ b/reflog-walk.c
@@ -95,10 +95,9 @@ static struct commit_info *get_commit_info(struct commit *commit,
 			struct commit_info *result = &lifo->items[i];
 			if (pop) {
 				if (i + 1 < lifo->nr)
-					memmove(lifo->items + i,
-						lifo->items + i + 1,
-						(lifo->nr - i) *
-						sizeof(struct commit_info));
+					MOVE_ARRAY(lifo->items + i,
+						   lifo->items + i + 1,
+						   lifo->nr - i);
 				lifo->nr--;
 			}
 			return result;
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 50188e92f9..a851326445 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -437,9 +437,8 @@ static int remove_entry(struct ref_dir *dir, const char *refname)
 		return -1;
 	entry = dir->entries[entry_index];
 
-	memmove(&dir->entries[entry_index],
-		&dir->entries[entry_index + 1],
-		(dir->nr - entry_index - 1) * sizeof(*dir->entries)
+	MOVE_ARRAY(&dir->entries[entry_index], &dir->entries[entry_index + 1],
+		dir->nr - entry_index - 1
 		);
 	dir->nr--;
 	if (dir->sorted > entry_index)
diff --git a/replace_object.c b/replace_object.c
index f0b39f06d5..3e49965d05 100644
--- a/replace_object.c
+++ b/replace_object.c
@@ -44,10 +44,8 @@ static int register_replace_object(struct replace_object *replace,
 	ALLOC_GROW(replace_object, replace_object_nr + 1, replace_object_alloc);
 	replace_object_nr++;
 	if (pos < replace_object_nr)
-		memmove(replace_object + pos + 1,
-			replace_object + pos,
-			(replace_object_nr - pos - 1) *
-			sizeof(*replace_object));
+		MOVE_ARRAY(replace_object + pos + 1, replace_object + pos,
+			   replace_object_nr - pos - 1);
 	replace_object[pos] = replace;
 	return 0;
 }
diff --git a/rerere.c b/rerere.c
index 3bd55caf3b..7410445ba2 100644
--- a/rerere.c
+++ b/rerere.c
@@ -159,8 +159,8 @@ static struct rerere_dir *find_rerere_dir(const char *hex)
 		ALLOC_GROW(rerere_dir, rerere_dir_nr + 1, rerere_dir_alloc);
 		/* ... and add it in. */
 		rerere_dir_nr++;
-		memmove(rerere_dir + pos + 1, rerere_dir + pos,
-			(rerere_dir_nr - pos - 1) * sizeof(*rerere_dir));
+		MOVE_ARRAY(rerere_dir + pos + 1, rerere_dir + pos,
+			   rerere_dir_nr - pos - 1);
 		rerere_dir[pos] = rr_dir;
 		scan_rerere_dir(rr_dir);
 	}
diff --git a/string-list.c b/string-list.c
index 45016ad86d..df64d4c446 100644
--- a/string-list.c
+++ b/string-list.c
@@ -46,9 +46,8 @@ static int add_entry(int insert_at, struct string_list *list, const char *string
 		REALLOC_ARRAY(list->items, list->alloc);
 	}
 	if (index < list->nr)
-		memmove(list->items + index + 1, list->items + index,
-				(list->nr - index)
-				* sizeof(struct string_list_item));
+		MOVE_ARRAY(list->items + index + 1, list->items + index,
+			   list->nr - index);
 	list->items[index].string = list->strdup_strings ?
 		xstrdup(string) : (char *)string;
 	list->items[index].util = NULL;

-- 
2.12.2
Junio C Hamano· Apr 17, 2017, 01:49 UTC · re: Martin Liška · lore

Re: [PATCH v3 1/2] Fix nonnull errors reported by UBSAN with GCC 7.

Martin Liška <mliska@suse.cz> writes:
Show 7 quoted lines
> From 0bdf4d717d3d368dd9676d15d20f8592c4d22fde Mon Sep 17 00:00:00 2001
> From: marxin <mliska@suse.cz>
> Date: Wed, 5 Apr 2017 14:31:32 +0200
> Subject: [PATCH 1/2] Fix nonnull errors reported by UBSAN with GCC 7.
>
> Replace call to memmove with newly introduced function memmove_or_null
> and call to memcpy with COPY_ARRAY macro.

I didn't closely follow the discussion, but with these three lines (which will be the primary thing future readers of this change in "git log -p" output will rely on), it is unclear why this change was made. For that matter, it is not clear what "nonnull errors" are, either.

Show 19 quoted lines
> Signed-off-by: Martin Liska <mliska@suse.cz>
> ---
>  apply.c            | 4 +---
>  builtin/ls-files.c | 2 +-
>  git-compat-util.h  | 8 ++++++++
>  3 files changed, 10 insertions(+), 4 deletions(-)
>
> diff --git a/apply.c b/apply.c
> index e6dbab26a..121f3f414 100644
> --- a/apply.c
> +++ b/apply.c
> @@ -2802,9 +2802,7 @@ static void update_image(struct apply_state *state,
>  			img->line + applied_pos + preimage_limit,
>  			(img->nr - (applied_pos + preimage_limit)) *
>  			sizeof(*img->line));
> -	memcpy(img->line + applied_pos,
> -	       postimage->line,
> -	       postimage->nr * sizeof(*img->line));
> +	COPY_ARRAY(img->line + applied_pos, postimage->line, postimage->nr);

I am suspecting that postimage->nr can be 0 and newer compliers can give warning "hey what's the point of copying 0 bytes?" which can be squelched by moving to COPY_ARRAY()? If that is the case I like the change (but as I said, it is unclear if that is what is going on here).

Show 11 quoted lines
> diff --git a/builtin/ls-files.c b/builtin/ls-files.c
> index d449e46db..0a6cc1e8a 100644
> --- a/builtin/ls-files.c
> +++ b/builtin/ls-files.c
> @@ -391,7 +391,7 @@ static void prune_cache(const char *prefix, size_t prefixlen)
>  		}
>  		last = next;
>  	}
> -	memmove(active_cache, active_cache + pos,
> +	memmove_or_null(active_cache, active_cache + pos,
>  		(last - pos) * sizeof(struct cache_entry *));

Does this change come with the same or a similar motivation as the above (i.e. pos could be the same as last)?

"Something or NULL" is a name we use for a function that returns something (under normal circumstances) or returns NULL. This wrapper is not about returning NULL at all, as far as I can see, and is misnamed. If it is about "avoid moving 0 bytes", similar to how COPY_ARRAY() is used in the previous hunk, perhaps MOVE_ARRAY() is a better name?

Thanks.
Show 21 quoted lines
>  	active_nr = last - pos;
>  }
> diff --git a/git-compat-util.h b/git-compat-util.h
> index 8a4a3f85e..81f6e56ac 100644
> --- a/git-compat-util.h
> +++ b/git-compat-util.h
> @@ -1002,6 +1002,14 @@ int git_qsort_s(void *base, size_t nmemb, size_t size,
>  		die("BUG: qsort_s() failed");			\
>  } while (0)
>  
> +static inline void *memmove_or_null(void *dest, const void *src, size_t n)
> +{
> +	if (n > 0)
> +		return memmove(dest, src, n);
> +	else
> +		return dest;
> +}
> +
>  #ifndef REG_STARTEND
>  #error "Git requires REG_STARTEND support. Compile with NO_REGEX=NeedsStartEnd"
>  #endif
Johannes Sixt· Apr 17, 2017, 07:59 UTC · re: Junio C Hamano · lore

Re: [PATCH v3 1/2] Fix nonnull errors reported by UBSAN with GCC 7.

Am 17.04.2017 um 03:49 schrieb Junio C Hamano:
Show 6 quoted lines
> "Something or NULL" is a name we use for a function that returns
> something (under normal circumstances) or returns NULL.  This
> wrapper is not about returning NULL at all, as far as I can see, and
> is misnamed.  If it is about "avoid moving 0 bytes", similar to how
> COPY_ARRAY() is used in the previous hunk, perhaps MOVE_ARRAY() is a
> better name?

It is not about "avoid moving 0 bytes", but "if we move 0 bytes, then we allow NULL pointers". Plain memmove/memcpy do not allow the pointers to be NULL even if the count is 0. It just so happens that the implementation of memmove_or_null that permits the relaxed condition looks like "avoid moving 0 bytes".

The name was my suggestion, but I agree that it is not the best name. [Sentence about two most difficult things in software engineering omitted for brevity.]

-- Hannes
Johannes Schindelin· Apr 6, 2017, 08:57 UTC · re: Martin Liška · lore

Re: [PATCH 1/2] Fix nonnull errors reported by UBSAN with GCC 7.

Hi Martin,
On Thu, 6 Apr 2017, Martin Liška wrote:
> Following patch fixes issues that can be seen with -fsanitize=undefined
> on GCC 7.
The commit message says:
	Memory functions like memmove and memcpy should not be called
	with an argument equal to NULL.

But the patch is not about the pointer arguments, it is about the size argument. So I would suggest to say

	Memory functions like memmove and memcpy should only be called
	with positive sizes.

Ciao, Johannes

← back to recent threads