threads / patch / 45517

patchstrbuf: support long paths w/o read rights in strbuf_getcwd() on FreeBSD

Subject: [PATCH] strbuf: support long paths w/o read rights in strbuf_getcwd() on FreeBSD

## tl;dr

10 messages between Mar 26, 2017 and Mar 30, 2017. Diffs are folded; open one to read it.

replies: 9people: 6as markdown or json

René Scharfe· Mar 26, 2017, 13:43 UTC · lore

FreeBSD implements getcwd(3) as a syscall, but falls back to a version based on readdir(3) if it fails for some reason. The latter requires permissions to read and execute path components, while the former does not. That means that if our buffer is too small and we're missing rights we could get EACCES, but we may succeed with a bigger buffer.

Keep retrying if getcwd(3) indicates lack of permissions until our buffer can fit PATH_MAX bytes, as that's the maximum supported by the syscall on FreeBSD anyway. This way we do what we can to be able to benefit from the syscall, but we also won't loop forever if there is a real permission issue.

This fixes a regression introduced with 7333ed17 (setup: convert setup_git_directory_gently_1 et al. to strbuf, 2014-07-28) for paths longer than 127 bytes with components that miss read or execute permissions (e.g. 0711 on /home for privacy reasons); we used a fixed PATH_MAX-sized buffer before.

Reported-by: Zenobiusz Kunegunda <zenobiusz.kunegunda@interia.pl>
Signed-off-by: Rene Scharfe <l.s.r@web.de>
---
 strbuf.c        | 11 +++++++++++
 t/t0001-init.sh | 14 ++++++++++++++
 2 files changed, 25 insertions(+)
Show changes to 2 files +25 −0

strbuf.c, t/t0001-init.sh

diff --git a/strbuf.c b/strbuf.c
index ace58e7367..00457940cf 100644
--- a/strbuf.c
+++ b/strbuf.c
@@ -449,6 +449,17 @@ int strbuf_getcwd(struct strbuf *sb)
 			strbuf_setlen(sb, strlen(sb->buf));
 			return 0;
 		}
+
+		/*
+		 * If getcwd(3) is implemented as a syscall that falls
+		 * back to a regular lookup using readdir(3) etc. then
+		 * we may be able to avoid EACCES by providing enough
+		 * space to the syscall as it's not necessarily bound
+		 * to the same restrictions as the fallback.
+		 */
+		if (errno == EACCES && guessed_len < PATH_MAX)
+			continue;
+
 		if (errno != ERANGE)
 			break;
 	}
diff --git a/t/t0001-init.sh b/t/t0001-init.sh
index e424de5363..5f81fbe07c 100755
--- a/t/t0001-init.sh
+++ b/t/t0001-init.sh
@@ -315,6 +315,20 @@ test_expect_success 'init with separate gitdir' '
 	test_path_is_dir realgitdir/refs
 '
 
+test_expect_success 'init in long base path' '
+	# exceed initial buffer size of strbuf_getcwd()
+	component=123456789abcdef &&
+	test_when_finished "chmod 0700 $component; rm -rf $component" &&
+	p31=$component/$component &&
+	p127=$p31/$p31/$p31/$p31 &&
+	mkdir -p $p127 &&
+	chmod 0111 $component &&
+	(
+		cd $p127 &&
+		git init newdir
+	)
+'
+
 test_expect_success 're-init on .git file' '
 	( cd newdir && git init )
 '
-- 
2.12.2
Junio C Hamano· Mar 27, 2017, 00:40 UTC · re: René Scharfe · lore

Re: [PATCH] strbuf: support long paths w/o read rights in strbuf_getcwd() on FreeBSD

René Scharfe <l.s.r@web.de> writes:
Show 5 quoted lines
> FreeBSD implements getcwd(3) as a syscall, but falls back to a version
> based on readdir(3) if it fails for some reason.  The latter requires
> permissions to read and execute path components, while the former does
> not.  That means that if our buffer is too small and we're missing
> rights we could get EACCES, but we may succeed with a bigger buffer.

WOW. Just WOW. Looking at the debugging exchange from the sideline, I didn't expect this end result.

Show 15 quoted lines
> Keep retrying if getcwd(3) indicates lack of permissions until our
> buffer can fit PATH_MAX bytes, as that's the maximum supported by the
> syscall on FreeBSD anyway.  This way we do what we can to be able to
> benefit from the syscall, but we also won't loop forever if there is a
> real permission issue.
>
> This fixes a regression introduced with 7333ed17 (setup: convert
> setup_git_directory_gently_1 et al. to strbuf, 2014-07-28) for paths
> longer than 127 bytes with components that miss read or execute
> permissions (e.g. 0711 on /home for privacy reasons); we used a fixed
> PATH_MAX-sized buffer before.
>
> Reported-by: Zenobiusz Kunegunda <zenobiusz.kunegunda@interia.pl>
> Signed-off-by: Rene Scharfe <l.s.r@web.de>
> ---
Nicely analysed and fixed (or is the right word "worked around"?)
Thanks, will queue.
Zenobiusz Kunegunda· Mar 27, 2017, 05:55 UTC · re: Junio C Hamano · lore

Re: [PATCH] strbuf: support long paths w/o read rights in strbuf_getcwd() on FreeBSD

Od: "Junio C Hamano" <gitster@pobox.com>
Do: "René Scharfe" <l.s.r@web.de>;
Wysłane: 2:40 Poniedziałek 2017-03-27
Temat: Re: [PATCH] strbuf: support long paths w/o read rights in strbuf_getcwd() on FreeBSD
Show 5 quoted lines
> 
> Nicely analysed and fixed (or is the right word "worked around"?)
> 
> Thanks, will queue.
> 
Is this patch going to be included in next git version ( or sooner ) by any chance?
Thank you, everyone,  for your attention to the problem.
Junio C Hamano· Mar 27, 2017, 18:40 UTC · re: Zenobiusz Kunegunda · lore

Re: [PATCH] strbuf: support long paths w/o read rights in strbuf_getcwd() on FreeBSD

Zenobiusz Kunegunda <zenobiusz.kunegunda@interia.pl> writes:
Show 11 quoted lines
> Od: "Junio C Hamano" <gitster@pobox.com>
> Do: "René Scharfe" <l.s.r@web.de>;
> Wysłane: 2:40 Poniedziałek 2017-03-27
> Temat: Re: [PATCH] strbuf: support long paths w/o read rights in strbuf_getcwd() on FreeBSD
>
>> 
>> Nicely analysed and fixed (or is the right word "worked around"?)
>> 
>> Thanks, will queue.
>
> Is this patch going to be included in next git version ( or sooner ) by any chance?

Absolutely. Thanks for your initial report and sticking with us during the session to identify the root cause that led to this solution.

Again, René, thanks for your superb analysis and solution.
Christian Couder· Mar 28, 2017, 21:15 UTC · re: René Scharfe · lore

Re: [PATCH] strbuf: support long paths w/o read rights in strbuf_getcwd() on FreeBSD

On Sun, Mar 26, 2017 at 3:43 PM, René Scharfe <l.s.r@web.de> wrote:
Show 11 quoted lines
> FreeBSD implements getcwd(3) as a syscall, but falls back to a version
> based on readdir(3) if it fails for some reason.  The latter requires
> permissions to read and execute path components, while the former does
> not.  That means that if our buffer is too small and we're missing
> rights we could get EACCES, but we may succeed with a bigger buffer.
>
> Keep retrying if getcwd(3) indicates lack of permissions until our
> buffer can fit PATH_MAX bytes, as that's the maximum supported by the
> syscall on FreeBSD anyway.  This way we do what we can to be able to
> benefit from the syscall, but we also won't loop forever if there is a
> real permission issue.

Sorry to be late and maybe I missed something obvious, but the above and the patch seem complex to me compared with something like:

Show changes to strbuf.c +1 −1
diff --git a/strbuf.c b/strbuf.c
index ace58e7367..25eadcbedc 100644
--- a/strbuf.c
+++ b/strbuf.c
@@ -441,7 +441,7 @@ int strbuf_readlink(struct strbuf *sb, const char
*path, size_t hint)
 int strbuf_getcwd(struct strbuf *sb)
 {
        size_t oldalloc = sb->alloc;
-       size_t guessed_len = 128;
+       size_t guessed_len = PATH_MAX > 128 ? PATH_MAX : 128;

        for (;; guessed_len *= 2) {
                strbuf_grow(sb, guessed_len);
Stefan Beller· Mar 28, 2017, 21:24 UTC · re: Christian Couder · lore

Re: [PATCH] strbuf: support long paths w/o read rights in strbuf_getcwd() on FreeBSD

On Tue, Mar 28, 2017 at 2:15 PM, Christian Couder <christian.couder@gmail.com> wrote:

Show 30 quoted lines
> On Sun, Mar 26, 2017 at 3:43 PM, René Scharfe <l.s.r@web.de> wrote:
>> FreeBSD implements getcwd(3) as a syscall, but falls back to a version
>> based on readdir(3) if it fails for some reason.  The latter requires
>> permissions to read and execute path components, while the former does
>> not.  That means that if our buffer is too small and we're missing
>> rights we could get EACCES, but we may succeed with a bigger buffer.
>>
>> Keep retrying if getcwd(3) indicates lack of permissions until our
>> buffer can fit PATH_MAX bytes, as that's the maximum supported by the
>> syscall on FreeBSD anyway.  This way we do what we can to be able to
>> benefit from the syscall, but we also won't loop forever if there is a
>> real permission issue.
>
> Sorry to be late and maybe I missed something obvious, but the above
> and the patch seem complex to me compared with something like:
>
> diff --git a/strbuf.c b/strbuf.c
> index ace58e7367..25eadcbedc 100644
> --- a/strbuf.c
> +++ b/strbuf.c
> @@ -441,7 +441,7 @@ int strbuf_readlink(struct strbuf *sb, const char
> *path, size_t hint)
>  int strbuf_getcwd(struct strbuf *sb)
>  {
>         size_t oldalloc = sb->alloc;
> -       size_t guessed_len = 128;
> +       size_t guessed_len = PATH_MAX > 128 ? PATH_MAX : 128;
>
>         for (;; guessed_len *= 2) {
>                 strbuf_grow(sb, guessed_len);
From f22a76e911 (strbuf: add strbuf_getcwd(), 2014-07-28):
    Because it doesn't use a fixed-size buffer it supports
    arbitrarily long paths, provided the platform's getcwd() does as well.
    At least on Linux and FreeBSD it handles paths longer than PATH_MAX
    just fine.

So with your patch, we'd still see the original issue for paths > PATH_MAX IIUC.

Thanks, Stefan

Christian Couder· Mar 28, 2017, 21:47 UTC · re: Stefan Beller · lore

Re: [PATCH] strbuf: support long paths w/o read rights in strbuf_getcwd() on FreeBSD

On Tue, Mar 28, 2017 at 11:24 PM, Stefan Beller <sbeller@google.com> wrote:
Show 38 quoted lines
> On Tue, Mar 28, 2017 at 2:15 PM, Christian Couder
> <christian.couder@gmail.com> wrote:
>> On Sun, Mar 26, 2017 at 3:43 PM, René Scharfe <l.s.r@web.de> wrote:
>>> FreeBSD implements getcwd(3) as a syscall, but falls back to a version
>>> based on readdir(3) if it fails for some reason.  The latter requires
>>> permissions to read and execute path components, while the former does
>>> not.  That means that if our buffer is too small and we're missing
>>> rights we could get EACCES, but we may succeed with a bigger buffer.
>>>
>>> Keep retrying if getcwd(3) indicates lack of permissions until our
>>> buffer can fit PATH_MAX bytes, as that's the maximum supported by the
>>> syscall on FreeBSD anyway.  This way we do what we can to be able to
>>> benefit from the syscall, but we also won't loop forever if there is a
>>> real permission issue.
>>
>> Sorry to be late and maybe I missed something obvious, but the above
>> and the patch seem complex to me compared with something like:
>>
>> diff --git a/strbuf.c b/strbuf.c
>> index ace58e7367..25eadcbedc 100644
>> --- a/strbuf.c
>> +++ b/strbuf.c
>> @@ -441,7 +441,7 @@ int strbuf_readlink(struct strbuf *sb, const char
>> *path, size_t hint)
>>  int strbuf_getcwd(struct strbuf *sb)
>>  {
>>         size_t oldalloc = sb->alloc;
>> -       size_t guessed_len = 128;
>> +       size_t guessed_len = PATH_MAX > 128 ? PATH_MAX : 128;
>>
>>         for (;; guessed_len *= 2) {
>>                 strbuf_grow(sb, guessed_len);
>
> From f22a76e911 (strbuf: add strbuf_getcwd(), 2014-07-28):
>     Because it doesn't use a fixed-size buffer it supports
>     arbitrarily long paths, provided the platform's getcwd() does as well.
>     At least on Linux and FreeBSD it handles paths longer than PATH_MAX
>     just fine.
Well René's patch says:
>>> Keep retrying if getcwd(3) indicates lack of permissions until our
>>> buffer can fit PATH_MAX bytes, as that's the maximum supported by the
>>> syscall on FreeBSD anyway.
So it says that FreeBSD syscall doesn't handle paths longer than PATH_MAX.
> So with your patch, we'd still see the original issue for paths > PATH_MAX
> IIUC.
Also, René's patch just adds:
+               if (errno == EACCES && guessed_len < PATH_MAX)
+                       continue;

so if the length of the path is > PATH_MAX, then guessed_len will have to be > PATH_MAX and then René's patch will do nothing.

Jeff King· Mar 28, 2017, 21:49 UTC · re: Christian Couder · lore

Re: [PATCH] strbuf: support long paths w/o read rights in strbuf_getcwd() on FreeBSD

On Tue, Mar 28, 2017 at 11:15:12PM +0200, Christian Couder wrote:
Show 30 quoted lines
> On Sun, Mar 26, 2017 at 3:43 PM, René Scharfe <l.s.r@web.de> wrote:
> > FreeBSD implements getcwd(3) as a syscall, but falls back to a version
> > based on readdir(3) if it fails for some reason.  The latter requires
> > permissions to read and execute path components, while the former does
> > not.  That means that if our buffer is too small and we're missing
> > rights we could get EACCES, but we may succeed with a bigger buffer.
> >
> > Keep retrying if getcwd(3) indicates lack of permissions until our
> > buffer can fit PATH_MAX bytes, as that's the maximum supported by the
> > syscall on FreeBSD anyway.  This way we do what we can to be able to
> > benefit from the syscall, but we also won't loop forever if there is a
> > real permission issue.
> 
> Sorry to be late and maybe I missed something obvious, but the above
> and the patch seem complex to me compared with something like:
> 
> diff --git a/strbuf.c b/strbuf.c
> index ace58e7367..25eadcbedc 100644
> --- a/strbuf.c
> +++ b/strbuf.c
> @@ -441,7 +441,7 @@ int strbuf_readlink(struct strbuf *sb, const char
> *path, size_t hint)
>  int strbuf_getcwd(struct strbuf *sb)
>  {
>         size_t oldalloc = sb->alloc;
> -       size_t guessed_len = 128;
> +       size_t guessed_len = PATH_MAX > 128 ? PATH_MAX : 128;
> 
>         for (;; guessed_len *= 2) {
>                 strbuf_grow(sb, guessed_len);

I think the main reason is just that we do not have to pay the price to allocate PATH_MAX-sized buffers when they are rarely used.

I doubt it matters all that much in practice, though.
-Peff
Christian Couder· Mar 29, 2017, 04:54 UTC · re: Jeff King · lore

Re: [PATCH] strbuf: support long paths w/o read rights in strbuf_getcwd() on FreeBSD

On Tue, Mar 28, 2017 at 11:49 PM, Jeff King <peff@peff.net> wrote:
Show 37 quoted lines
> On Tue, Mar 28, 2017 at 11:15:12PM +0200, Christian Couder wrote:
>
>> On Sun, Mar 26, 2017 at 3:43 PM, René Scharfe <l.s.r@web.de> wrote:
>> > FreeBSD implements getcwd(3) as a syscall, but falls back to a version
>> > based on readdir(3) if it fails for some reason.  The latter requires
>> > permissions to read and execute path components, while the former does
>> > not.  That means that if our buffer is too small and we're missing
>> > rights we could get EACCES, but we may succeed with a bigger buffer.
>> >
>> > Keep retrying if getcwd(3) indicates lack of permissions until our
>> > buffer can fit PATH_MAX bytes, as that's the maximum supported by the
>> > syscall on FreeBSD anyway.  This way we do what we can to be able to
>> > benefit from the syscall, but we also won't loop forever if there is a
>> > real permission issue.
>>
>> Sorry to be late and maybe I missed something obvious, but the above
>> and the patch seem complex to me compared with something like:
>>
>> diff --git a/strbuf.c b/strbuf.c
>> index ace58e7367..25eadcbedc 100644
>> --- a/strbuf.c
>> +++ b/strbuf.c
>> @@ -441,7 +441,7 @@ int strbuf_readlink(struct strbuf *sb, const char
>> *path, size_t hint)
>>  int strbuf_getcwd(struct strbuf *sb)
>>  {
>>         size_t oldalloc = sb->alloc;
>> -       size_t guessed_len = 128;
>> +       size_t guessed_len = PATH_MAX > 128 ? PATH_MAX : 128;
>>
>>         for (;; guessed_len *= 2) {
>>                 strbuf_grow(sb, guessed_len);
>
> I think the main reason is just that we do not have to pay the price to
> allocate PATH_MAX-sized buffers when they are rarely used.
>
> I doubt it matters all that much in practice, though.
Yeah, I can understand that.

But I also wonder if René's patch relies on PATH_MAX being a multiple of 128 on FreeBSD and what would happen for a path between 129 and PATH_MAX if PATH_MAX was something like 254.

René Scharfe· Mar 30, 2017, 18:01 UTC · re: Christian Couder · lore

Re: [PATCH] strbuf: support long paths w/o read rights in strbuf_getcwd() on FreeBSD

Am 29.03.2017 um 06:54 schrieb Christian Couder:
Show 40 quoted lines
> On Tue, Mar 28, 2017 at 11:49 PM, Jeff King <peff@peff.net> wrote:
>> On Tue, Mar 28, 2017 at 11:15:12PM +0200, Christian Couder wrote:
>>
>>> On Sun, Mar 26, 2017 at 3:43 PM, René Scharfe <l.s.r@web.de> wrote:
>>>> FreeBSD implements getcwd(3) as a syscall, but falls back to a version
>>>> based on readdir(3) if it fails for some reason.  The latter requires
>>>> permissions to read and execute path components, while the former does
>>>> not.  That means that if our buffer is too small and we're missing
>>>> rights we could get EACCES, but we may succeed with a bigger buffer.
>>>>
>>>> Keep retrying if getcwd(3) indicates lack of permissions until our
>>>> buffer can fit PATH_MAX bytes, as that's the maximum supported by the
>>>> syscall on FreeBSD anyway.  This way we do what we can to be able to
>>>> benefit from the syscall, but we also won't loop forever if there is a
>>>> real permission issue.
>>>
>>> Sorry to be late and maybe I missed something obvious, but the above
>>> and the patch seem complex to me compared with something like:
>>>
>>> diff --git a/strbuf.c b/strbuf.c
>>> index ace58e7367..25eadcbedc 100644
>>> --- a/strbuf.c
>>> +++ b/strbuf.c
>>> @@ -441,7 +441,7 @@ int strbuf_readlink(struct strbuf *sb, const char
>>> *path, size_t hint)
>>>  int strbuf_getcwd(struct strbuf *sb)
>>>  {
>>>         size_t oldalloc = sb->alloc;
>>> -       size_t guessed_len = 128;
>>> +       size_t guessed_len = PATH_MAX > 128 ? PATH_MAX : 128;
>>>
>>>         for (;; guessed_len *= 2) {
>>>                 strbuf_grow(sb, guessed_len);
>>
>> I think the main reason is just that we do not have to pay the price to
>> allocate PATH_MAX-sized buffers when they are rarely used.
>>
>> I doubt it matters all that much in practice, though.
>
> Yeah, I can understand that.

Right, using PATH_MAX as initial size (no ?: operator needed) is the easiest fix. And yes, I wanted to avoid wasting memory just to fix an odd special case.

Given that the function isn't called very often the impact is probably not that high either way, but increasing memory usage by 3200% on Linux just didn't sit right with me -- even though 4KB isn't much in absolute terms, of course.

> But I also wonder if René's patch relies on PATH_MAX being a multiple
> of 128 on FreeBSD and what would happen for a path between 129 and
> PATH_MAX if PATH_MAX was something like 254.

We can use any value bigger than zero to initialize guessed_len. getcwd() won't have any problems fitting e.g. a path with a length of 130 bytes into a buffer of 256 bytes in the second round of the loop.

René

← back to recent threads