[PATCH v3] wrapper: NetBSD gives EFTYPE and FreeBSD gives EMFILE where POSIX uses ELOOP

From: Collin Funk <collin.funk1@gmail.com>
Date: May 6, 2025, 01:08 UTC
Message-ID: <20250506010946.212068-1-collin.funk1@gmail.com>
In-Reply-To: <20250503041718.42195-1-collin.funk1@gmail.com>

As documented on NetBSD's man page, open with the O_NOFOLLOW flag and a symlink returns -1 and sets errno to EFTYPE which differs from POSIX.

This patch fixes the following test failure:

    $ sh t0602-reffiles-fsck.sh --verbose
    --- expect	2025-05-02 23:05:23.920890147 +0000
    +++ err	2025-05-02 23:05:23.916794959 +0000
    @@ -1 +1 @@
    -error: packed-refs: badRefFiletype: not a regular file but a symlink
    +error: unable to open '.git/packed-refs': Inappropriate file type or format
    not ok 12 - the filetype of packed-refs should be checked

FreeBSD has the same issue for EMLINK instead of EFTYPE.

This portability issue was introduced in cfea2f2da8 (packed-backend: check whether the "packed-refs" is regular file, 2025-02-28)

Signed-off-by: Collin Funk <collin.funk1@gmail.com>
---
 wrapper.c | 21 ++++++++++++++++++++-
 1 file changed, 20 insertions(+), 1 deletion(-)

diff --git a/wrapper.c b/wrapper.c
index 8b98593149..38fce5327a 100644
--- a/wrapper.c
+++ b/wrapper.c
@@ -737,7 +737,26 @@ int is_empty_or_missing_file(const char *filename)
 int open_nofollow(const char *path, int flags)
 {
 #ifdef O_NOFOLLOW
-	return open(path, flags | O_NOFOLLOW);
+	int ret = open(path, flags | O_NOFOLLOW);
+	/*
+	 * NetBSD sets errno to EFTYPE when path is a symlink. The only other
+	 * time this errno occurs when O_REGULAR is used. Since we don't use
+	 * it anywhere we can avoid an lstat here. FreeBSD does the same with
+	 * EMLINK.
+	 */
+# ifdef __NetBSD__
+#  define SYMLINK_ERRNO EFTYPE
+# elif defined(__FreeBSD__)
+#  define SYMLINK_ERRNO EMLINK
+# endif
+# if SYMLINK_ERRNO
+	if (ret < 0 && errno == SYMLINK_ERRNO) {
+		errno = ELOOP;
+		return -1;
+	}
+#  undef SYMLINK_ERRNO
+# endif
+	return ret;
 #else
 	struct stat st;
 	if (lstat(path, &st) < 0)

-- 
2.49.0