Re: [PATCH 1/6] add open_nofollow() helper

From: Jeff King <peff@peff.net>
Date: Feb 16, 2021, 16:02 UTC
Message-ID: <YCvsj2Teym2K06wa@coredump.intra.peff.net>
In-Reply-To: <YCvoQBhmCeeR9Cup@nand.local>

On Tue, Feb 16, 2021 at 10:44:00AM -0500, Taylor Blau wrote:

Show 12 quoted lines

> On Tue, Feb 16, 2021 at 09:54:24AM -0500, Jeff King wrote:
> > This is all on Linux, of course. Perhaps other systems with slower
> > syscalls may be more impacted.
> 
> I timed it on macOS, which (as you know) I don't use for daily
> development, but it's a useful testbed from time to time.
> 
> On your branch, 'git check-attr -a' took 193.4ms with O_NOFOLLOW, and
> 245.3ms without. After touching every .gitattributes file, those numbers
> shot up to 340.9ms and 346.6ms, respectively.
> 
> (All numbers on linux.git, of course).

That seems...really weird. Your "after touching every file" case is slightly slower, which is totally expected. But why would the _before_ case be so different? It should have made exactly one extra lstat() call, and replaced a bunch of open(NOFOLLOW) calls with lstat() (and if those were so expensive, we'd have presumably seen it in the "touching every file" case which is running _both_ syscalls).

Can you double-check your initial timings?

> There isn't an apples-to-apples comparison between my numbers and yours
> (since my laptop is much slower than yours), but the relative numbers
> are quite clear that only doing a single syscall is worth it in the
> non-pathological case.

I guess in a sense it doesn't matter that much how macos performs. If it has O_NOFOLLOW, then it's a no-brainer to use it. The bigger question is: how many platforms don't have it, and what are _their_ syscalls like (and are they niche enough that we can accept a small slowdown)?

The obvious one we'd care the most about is Windows, but I still hold out hope that there's some equivalent mechanism there we can use.

-Peff