threads / patch / 7889

patch, 5 partsRe: [PATCH 1/5] Make xstrndup common

Subject: Re: [PATCH 1/5] Make xstrndup common

## tl;dr

9 messages between Apr 28, 2007 and Apr 30, 2007. Diffs are folded; open one to read it.

replies: 8people: 6as markdown or json

Josh Triplett· Apr 28, 2007, 18:53 UTC · lore
Daniel Barkalow wrote:
> It was implemented in commit.c; move it with the other x memory functions.
[...]
Show 8 quoted lines
> +static inline char *xstrndup(const char *str, int len)
> +{
> +	char *ret = xmalloc(len + 1);
> +	memcpy(ret, str, len);
> +	ret[len] = '\0';
> +	return ret;
> +}
> +

I don't know if it matters, but this definition of xstrndup, like the version in commit.c, doesn't match the definition of strndup. strndup duplicates a string, copying up to n characters or the length of the string. This xstrndup always copies n characters, reading past the end of the string if it doesn't have at least n characters.

- Josh Triplett
Junio C Hamano· Apr 28, 2007, 22:47 UTC · re: Josh Triplett · lore
Josh Triplett <josh@freedesktop.org> writes:
Show 17 quoted lines
> Daniel Barkalow wrote:
>> It was implemented in commit.c; move it with the other x memory functions.
> [...]
>> +static inline char *xstrndup(const char *str, int len)
>> +{
>> +	char *ret = xmalloc(len + 1);
>> +	memcpy(ret, str, len);
>> +	ret[len] = '\0';
>> +	return ret;
>> +}
>> +
>
> I don't know if it matters, but this definition of xstrndup, like the version
> in commit.c, doesn't match the definition of strndup.  strndup duplicates a
> string, copying up to n characters or the length of the string.  This xstrndup
> always copies n characters, reading past the end of the string if it doesn't
> have at least n characters.

Very well caught indeed, thanks. In the original context in commit.c, this function is always given length computed by inspecting the source text by the caller, so the code was correct, but if we are making it available as a general utility routine, we probably cannot depend on that assumption anymore.

Daniel Barkalow· Apr 29, 2007, 18:19 UTC · re: Josh Triplett · lore
On Sat, 28 Apr 2007, Josh Triplett wrote:
Show 17 quoted lines
> Daniel Barkalow wrote:
> > It was implemented in commit.c; move it with the other x memory functions.
> [...]
> > +static inline char *xstrndup(const char *str, int len)
> > +{
> > +	char *ret = xmalloc(len + 1);
> > +	memcpy(ret, str, len);
> > +	ret[len] = '\0';
> > +	return ret;
> > +}
> > +
> 
> I don't know if it matters, but this definition of xstrndup, like the version
> in commit.c, doesn't match the definition of strndup.  strndup duplicates a
> string, copying up to n characters or the length of the string.  This xstrndup
> always copies n characters, reading past the end of the string if it doesn't
> have at least n characters.

Good catch. Replacing the memcpy with strncpy solves this, right? (Potentially allocating a bit of extra memory if someone is actually using it on too short a string for some reason, of course).

	-Daniel
*This .sig left intentionally blank*
Josh Triplett· Apr 29, 2007, 20:29 UTC · re: Daniel Barkalow · lore
Daniel Barkalow wrote:
Show 21 quoted lines
> On Sat, 28 Apr 2007, Josh Triplett wrote:
>> Daniel Barkalow wrote:
>>> It was implemented in commit.c; move it with the other x memory functions.
>> [...]
>>> +static inline char *xstrndup(const char *str, int len)
>>> +{
>>> +	char *ret = xmalloc(len + 1);
>>> +	memcpy(ret, str, len);
>>> +	ret[len] = '\0';
>>> +	return ret;
>>> +}
>>> +
>> I don't know if it matters, but this definition of xstrndup, like the version
>> in commit.c, doesn't match the definition of strndup.  strndup duplicates a
>> string, copying up to n characters or the length of the string.  This xstrndup
>> always copies n characters, reading past the end of the string if it doesn't
>> have at least n characters.
> 
> Good catch. Replacing the memcpy with strncpy solves this, right? 
> (Potentially allocating a bit of extra memory if someone is actually using 
> it on too short a string for some reason, of course).

That would work, but it seems bad to allocate excess memory. How about just using strlen and setting len to that if shorter, before doing the xmalloc and memcpy? Yes, that makes two passes over the string, but I don't see any way around that.

I just checked the glibc source for strndup, and it does exactly the same thing, except that it uses the glibc-specific function strnlen rather than using strlen and figuring out the smaller of the two lengths. That probably increases efficiency if we have a string longer than, but we can't portably use strnlen, so we'd have to check for it; doesn't seem worth the trouble.

- Josh Triplett
Junio C Hamano· Apr 29, 2007, 20:39 UTC · re: Josh Triplett · lore
Josh Triplett <josh@freedesktop.org> writes:
Show 10 quoted lines
> Daniel Barkalow wrote:
> ...
>> Good catch. Replacing the memcpy with strncpy solves this, right? 
>> (Potentially allocating a bit of extra memory if someone is actually using 
>> it on too short a string for some reason, of course).
>
> That would work, but it seems bad to allocate excess memory.  How about just
> using strlen and setting len to that if shorter, before doing the xmalloc and
> memcpy?  Yes, that makes two passes over the string, but I don't see any way
> around that.

Hand-rolling strnlen() would be needed anyway, because there is no guarantee that the incoming string is NUL terminated. In the worst case the string may point at a region of memory filled with non-NUL to the end, which coincides with a page boundary, and the next page may be an unmapped one; your strlen() would sigbus.

Adam Roben· Apr 29, 2007, 20:40 UTC · re: Josh Triplett · lore
On Apr 29, 2007, at 1:29 PM, Josh Triplett wrote:
Show 36 quoted lines
> Daniel Barkalow wrote:
>> On Sat, 28 Apr 2007, Josh Triplett wrote:
>>> Daniel Barkalow wrote:
>>>> It was implemented in commit.c; move it with the other x memory  
>>>> functions.
>>> [...]
>>>> +static inline char *xstrndup(const char *str, int len)
>>>> +{
>>>> +	char *ret = xmalloc(len + 1);
>>>> +	memcpy(ret, str, len);
>>>> +	ret[len] = '\0';
>>>> +	return ret;
>>>> +}
>>>> +
>>> I don't know if it matters, but this definition of xstrndup, like  
>>> the version
>>> in commit.c, doesn't match the definition of strndup.  strndup  
>>> duplicates a
>>> string, copying up to n characters or the length of the string.   
>>> This xstrndup
>>> always copies n characters, reading past the end of the string if  
>>> it doesn't
>>> have at least n characters.
>>
>> Good catch. Replacing the memcpy with strncpy solves this, right?
>> (Potentially allocating a bit of extra memory if someone is  
>> actually using
>> it on too short a string for some reason, of course).
>
> That would work, but it seems bad to allocate excess memory.  How  
> about just
> using strlen and setting len to that if shorter, before doing the  
> xmalloc and
> memcpy?  Yes, that makes two passes over the string, but I don't  
> see any way
> around that.
    An easy way around that is to do the string copy yourself,  
walking the string until you either find '\0' or reach len copied  
characters.
-Adam
Johannes Schindelin· Apr 30, 2007, 13:12 UTC · re: Josh Triplett · lore
Hi,
On Sun, 29 Apr 2007, Josh Triplett wrote:
Show 27 quoted lines
> Daniel Barkalow wrote:
> > On Sat, 28 Apr 2007, Josh Triplett wrote:
> >> Daniel Barkalow wrote:
> >>> It was implemented in commit.c; move it with the other x memory functions.
> >> [...]
> >>> +static inline char *xstrndup(const char *str, int len)
> >>> +{
> >>> +	char *ret = xmalloc(len + 1);
> >>> +	memcpy(ret, str, len);
> >>> +	ret[len] = '\0';
> >>> +	return ret;
> >>> +}
> >>> +
> >> I don't know if it matters, but this definition of xstrndup, like the 
> >> version in commit.c, doesn't match the definition of strndup.  
> >> strndup duplicates a string, copying up to n characters or the length 
> >> of the string.  This xstrndup always copies n characters, reading 
> >> past the end of the string if it doesn't have at least n characters.
> > 
> > Good catch. Replacing the memcpy with strncpy solves this, right? 
> > (Potentially allocating a bit of extra memory if someone is actually 
> > using it on too short a string for some reason, of course).
> 
> That would work, but it seems bad to allocate excess memory.  How about 
> just using strlen and setting len to that if shorter, before doing the 
> xmalloc and memcpy?  Yes, that makes two passes over the string, but I 
> don't see any way around that.
Unless I am missing something, I think this should work:
static inline char *xstrndup(const char *str, int len)
{
	char *result = strndup(str, len);
	if (result == NULL)
		die ("xstrndup(): out of memory");
	return result;
}
Hmm?

Ciao, Dscho

P.S.: If you feel real paranoid about it, you might insert
	if (result == NULL) {
		release_pack_memory(len, -1);
		result = strndup(str, len);
	}
before the if (...), but I think that's overkill.
Jeff King· Apr 30, 2007, 20:50 UTC · re: Johannes Schindelin · lore
On Mon, Apr 30, 2007 at 03:12:50PM +0200, Johannes Schindelin wrote:
Show 11 quoted lines
> Unless I am missing something, I think this should work:
> 
> static inline char *xstrndup(const char *str, int len)
> {
> 	char *result = strndup(str, len);
> 	if (result == NULL)
> 		die ("xstrndup(): out of memory");
> 	return result;
> }
> 
> Hmm?

I can't speak for the original authors, but I imagine part of the impetus was that strndup is a GNU-ism.

-Peff
Johannes Schindelin· Apr 30, 2007, 22:21 UTC · re: Jeff King · lore
Hi,
On Mon, 30 Apr 2007, Jeff King wrote:
Show 16 quoted lines
> On Mon, Apr 30, 2007 at 03:12:50PM +0200, Johannes Schindelin wrote:
> 
> > Unless I am missing something, I think this should work:
> > 
> > static inline char *xstrndup(const char *str, int len)
> > {
> > 	char *result = strndup(str, len);
> > 	if (result == NULL)
> > 		die ("xstrndup(): out of memory");
> > 	return result;
> > }
> > 
> > Hmm?
> 
> I can't speak for the original authors, but I imagine part of the
> impetus was that strndup is a GNU-ism.
D'oh! I never thought this was a GNU-ism.

Ciao, Dscho

← back to recent threads