threads / patch / 60943

patch, 4 partsosxkeychain: bring in line with other credential helpers

Subject: [PATCH 0/4] osxkeychain: bring in line with other credential helpers

## tl;dr

19 messages between Feb 17, 2024 and Apr 2, 2024. Diffs are folded; open one to read it.

replies: 18people: 6as markdown or json

Bo Anderson via GitGitGadget· Feb 17, 2024, 23:34 UTC · lore

git-credential-osxkeychain has largely fallen behind other external credential helpers in the features it supports, and hasn't received any functional changes since 2013. As it stood, osxkeychain failed seven tests in the external credential helper test suite:

not ok 8 - helper (osxkeychain) overwrites on store not ok 9 - helper (osxkeychain) can forget host not ok 11 - helper (osxkeychain) does not erase a password distinct from input not ok 15 - helper (osxkeychain) erases all matching credentials not ok 18 - helper (osxkeychain) gets password_expiry_utc not ok 19 - helper (osxkeychain) overwrites when password_expiry_utc changes not ok 21 - helper (osxkeychain) gets oauth_refresh_token

osxkeychain also made use of macOS APIs that had been deprecated since 2014. Replacement API was able to be used without regressing the minimum supported macOS established in 5747c8072b (contrib/credential: avoid fixed-size buffer in osxkeychain, 2023-05-01).

After this set of patches, osxkeychain passes all tests in the external credential helper test suite.

Bo Anderson (4):
  osxkeychain: replace deprecated SecKeychain API
  osxkeychain: erase all matching credentials
  osxkeychain: erase matching passwords only
  osxkeychain: store new attributes
 contrib/credential/osxkeychain/Makefile       |   3 +-
 .../osxkeychain/git-credential-osxkeychain.c  | 376 ++++++++++++++----
 2 files changed, 310 insertions(+), 69 deletions(-)
base-commit: 3e0d3cd5c7def4808247caf168e17f2bbf47892b
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1667%2FBo98%2Fosxkeychain-update-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1667/Bo98/osxkeychain-update-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/1667
-- 
gitgitgadget
Bo Anderson via GitGitGadget· Feb 17, 2024, 23:34 UTC · re: Bo Anderson via GitGitGadget · lore

[PATCH 1/4] osxkeychain: replace deprecated SecKeychain API

From: Bo Anderson <mail@boanderson.me>

The SecKeychain API was deprecated in macOS 10.10, nearly 10 years ago. The replacement SecItem API however is available as far back as macOS 10.6.

While supporting older macOS was perhaps prevously a concern, git-credential-osxkeychain already requires a minimum of macOS 10.7 since 5747c8072b (contrib/credential: avoid fixed-size buffer in osxkeychain, 2023-05-01) so using the newer API should not regress the range of macOS versions supported.

Adapting to use the newer SecItem API also happens to fix two test failures in osxkeychain:

    8 - helper (osxkeychain) overwrites on store
    9 - helper (osxkeychain) can forget host
The new API is compatible with credentials saved with the older API.
Signed-off-by: Bo Anderson <mail@boanderson.me>
---
 contrib/credential/osxkeychain/Makefile       |   3 +-
 .../osxkeychain/git-credential-osxkeychain.c  | 265 +++++++++++++-----
 2 files changed, 199 insertions(+), 69 deletions(-)
Show changes to 2 files +199 −69

contrib/credential/osxkeychain/Makefile, contrib/credential/osxkeychain/git-credential-osxkeychain.c

diff --git a/contrib/credential/osxkeychain/Makefile b/contrib/credential/osxkeychain/Makefile
index 4b3a08a2bac..238f5f8c36f 100644
--- a/contrib/credential/osxkeychain/Makefile
+++ b/contrib/credential/osxkeychain/Makefile
@@ -8,7 +8,8 @@ CFLAGS = -g -O2 -Wall
 -include ../../../config.mak
 
 git-credential-osxkeychain: git-credential-osxkeychain.o
-	$(CC) $(CFLAGS) -o $@ $< $(LDFLAGS) -Wl,-framework -Wl,Security
+	$(CC) $(CFLAGS) -o $@ $< $(LDFLAGS) \
+		-framework Security -framework CoreFoundation
 
 git-credential-osxkeychain.o: git-credential-osxkeychain.c
 	$(CC) -c $(CFLAGS) $<
diff --git a/contrib/credential/osxkeychain/git-credential-osxkeychain.c b/contrib/credential/osxkeychain/git-credential-osxkeychain.c
index 5f2e5f16c88..dc294ae944a 100644
--- a/contrib/credential/osxkeychain/git-credential-osxkeychain.c
+++ b/contrib/credential/osxkeychain/git-credential-osxkeychain.c
@@ -3,14 +3,39 @@
 #include <stdlib.h>
 #include <Security/Security.h>
 
-static SecProtocolType protocol;
-static char *host;
-static char *path;
-static char *username;
-static char *password;
-static UInt16 port;
-
-__attribute__((format (printf, 1, 2)))
+#define ENCODING kCFStringEncodingUTF8
+static CFStringRef protocol; /* Stores constant strings - not memory managed */
+static CFStringRef host;
+static CFStringRef path;
+static CFStringRef username;
+static CFDataRef password;
+static CFNumberRef port;
+
+static void clear_credential(void)
+{
+	if (host) {
+		CFRelease(host);
+		host = NULL;
+	}
+	if (path) {
+		CFRelease(path);
+		path = NULL;
+	}
+	if (username) {
+		CFRelease(username);
+		username = NULL;
+	}
+	if (password) {
+		CFRelease(password);
+		password = NULL;
+	}
+	if (port) {
+		CFRelease(port);
+		port = NULL;
+	}
+}
+
+__attribute__((format (printf, 1, 2), __noreturn__))
 static void die(const char *err, ...)
 {
 	char msg[4096];
@@ -19,70 +44,135 @@ static void die(const char *err, ...)
 	vsnprintf(msg, sizeof(msg), err, params);
 	fprintf(stderr, "%s\n", msg);
 	va_end(params);
+	clear_credential();
 	exit(1);
 }
 
-static void *xstrdup(const char *s1)
+static void *xmalloc(size_t len)
 {
-	void *ret = strdup(s1);
+	void *ret = malloc(len);
 	if (!ret)
 		die("Out of memory");
 	return ret;
 }
 
-#define KEYCHAIN_ITEM(x) (x ? strlen(x) : 0), x
-#define KEYCHAIN_ARGS \
-	NULL, /* default keychain */ \
-	KEYCHAIN_ITEM(host), \
-	0, NULL, /* account domain */ \
-	KEYCHAIN_ITEM(username), \
-	KEYCHAIN_ITEM(path), \
-	port, \
-	protocol, \
-	kSecAuthenticationTypeDefault
-
-static void write_item(const char *what, const char *buf, int len)
+static CFDictionaryRef create_dictionary(CFAllocatorRef allocator, ...)
+{
+	va_list args;
+	const void *key;
+	CFMutableDictionaryRef result;
+
+	result = CFDictionaryCreateMutable(allocator,
+					   0,
+					   &kCFTypeDictionaryKeyCallBacks,
+					   &kCFTypeDictionaryValueCallBacks);
+
+
+	va_start(args, allocator);
+	while ((key = va_arg(args, const void *)) != NULL) {
+		const void *value;
+		value = va_arg(args, const void *);
+		if (value)
+			CFDictionarySetValue(result, key, value);
+	}
+	va_end(args);
+
+	return result;
+}
+
+#define CREATE_SEC_ATTRIBUTES(...) \
+	create_dictionary(kCFAllocatorDefault, \
+			  kSecClass, kSecClassInternetPassword, \
+			  kSecAttrServer, host, \
+			  kSecAttrAccount, username, \
+			  kSecAttrPath, path, \
+			  kSecAttrPort, port, \
+			  kSecAttrProtocol, protocol, \
+			  kSecAttrAuthenticationType, \
+			  kSecAttrAuthenticationTypeDefault, \
+			  __VA_ARGS__);
+
+static void write_item(const char *what, const char *buf, size_t len)
 {
 	printf("%s=", what);
 	fwrite(buf, 1, len, stdout);
 	putchar('\n');
 }
 
-static void find_username_in_item(SecKeychainItemRef item)
+static void find_username_in_item(CFDictionaryRef item)
 {
-	SecKeychainAttributeList list;
-	SecKeychainAttribute attr;
+	CFStringRef account_ref;
+	char *username_buf;
+	CFIndex buffer_len;
 
-	list.count = 1;
-	list.attr = &attr;
-	attr.tag = kSecAccountItemAttr;
+	account_ref = CFDictionaryGetValue(item, kSecAttrAccount);
+	if (!account_ref)
+	{
+		write_item("username", "", 0);
+		return;
+	}
 
-	if (SecKeychainItemCopyContent(item, NULL, &list, NULL, NULL))
+	username_buf = (char *)CFStringGetCStringPtr(account_ref, ENCODING);
+	if (username_buf)
+	{
+		write_item("username", username_buf, strlen(username_buf));
 		return;
+	}
 
-	write_item("username", attr.data, attr.length);
-	SecKeychainItemFreeContent(&list, NULL);
+	/* If we can't get a CString pointer then
+	 * we need to allocate our own buffer */
+	buffer_len = CFStringGetMaximumSizeForEncoding(
+			CFStringGetLength(account_ref), ENCODING) + 1;
+	username_buf = xmalloc(buffer_len);
+	if (CFStringGetCString(account_ref,
+				username_buf,
+				buffer_len,
+				ENCODING)) {
+		write_item("username", username_buf, buffer_len - 1);
+	}
+	free(username_buf);
 }
 
-static void find_internet_password(void)
+static OSStatus find_internet_password(void)
 {
-	void *buf;
-	UInt32 len;
-	SecKeychainItemRef item;
+	CFDictionaryRef attrs;
+	CFDictionaryRef item;
+	CFDataRef data;
+	OSStatus result;
 
-	if (SecKeychainFindInternetPassword(KEYCHAIN_ARGS, &len, &buf, &item))
-		return;
+	attrs = CREATE_SEC_ATTRIBUTES(kSecMatchLimit, kSecMatchLimitOne,
+				      kSecReturnAttributes, kCFBooleanTrue,
+				      kSecReturnData, kCFBooleanTrue,
+				      NULL);
+	result = SecItemCopyMatching(attrs, (CFTypeRef *)&item);
+	if (result) {
+		goto out;
+	}
 
-	write_item("password", buf, len);
+	data = CFDictionaryGetValue(item, kSecValueData);
+
+	write_item("password",
+		   (const char *)CFDataGetBytePtr(data),
+		   CFDataGetLength(data));
 	if (!username)
 		find_username_in_item(item);
 
-	SecKeychainItemFreeContent(NULL, buf);
+	CFRelease(item);
+
+out:
+	CFRelease(attrs);
+
+	/* We consider not found to not be an error */
+	if (result == errSecItemNotFound)
+		result = errSecSuccess;
+
+	return result;
 }
 
-static void delete_internet_password(void)
+static OSStatus delete_internet_password(void)
 {
-	SecKeychainItemRef item;
+	CFDictionaryRef attrs;
+	OSStatus result;
 
 	/*
 	 * Require at least a protocol and host for removal, which is what git
@@ -90,25 +180,42 @@ static void delete_internet_password(void)
 	 * Keychain manager.
 	 */
 	if (!protocol || !host)
-		return;
+		return -1;
 
-	if (SecKeychainFindInternetPassword(KEYCHAIN_ARGS, 0, NULL, &item))
-		return;
+	attrs = CREATE_SEC_ATTRIBUTES(NULL);
+	result = SecItemDelete(attrs);
+	CFRelease(attrs);
+
+	/* We consider not found to not be an error */
+	if (result == errSecItemNotFound)
+		result = errSecSuccess;
 
-	SecKeychainItemDelete(item);
+	return result;
 }
 
-static void add_internet_password(void)
+static OSStatus add_internet_password(void)
 {
+	CFDictionaryRef attrs;
+	OSStatus result;
+
 	/* Only store complete credentials */
 	if (!protocol || !host || !username || !password)
-		return;
+		return -1;
 
-	if (SecKeychainAddInternetPassword(
-	      KEYCHAIN_ARGS,
-	      KEYCHAIN_ITEM(password),
-	      NULL))
-		return;
+	attrs = CREATE_SEC_ATTRIBUTES(kSecValueData, password,
+				      NULL);
+
+	result = SecItemAdd(attrs, NULL);
+	if (result == errSecDuplicateItem) {
+		CFDictionaryRef query;
+		query = CREATE_SEC_ATTRIBUTES(NULL);
+		result = SecItemUpdate(query, attrs);
+		CFRelease(query);
+	}
+
+	CFRelease(attrs);
+
+	return result;
 }
 
 static void read_credential(void)
@@ -131,36 +238,52 @@ static void read_credential(void)
 
 		if (!strcmp(buf, "protocol")) {
 			if (!strcmp(v, "imap"))
-				protocol = kSecProtocolTypeIMAP;
+				protocol = kSecAttrProtocolIMAP;
 			else if (!strcmp(v, "imaps"))
-				protocol = kSecProtocolTypeIMAPS;
+				protocol = kSecAttrProtocolIMAPS;
 			else if (!strcmp(v, "ftp"))
-				protocol = kSecProtocolTypeFTP;
+				protocol = kSecAttrProtocolFTP;
 			else if (!strcmp(v, "ftps"))
-				protocol = kSecProtocolTypeFTPS;
+				protocol = kSecAttrProtocolFTPS;
 			else if (!strcmp(v, "https"))
-				protocol = kSecProtocolTypeHTTPS;
+				protocol = kSecAttrProtocolHTTPS;
 			else if (!strcmp(v, "http"))
-				protocol = kSecProtocolTypeHTTP;
+				protocol = kSecAttrProtocolHTTP;
 			else if (!strcmp(v, "smtp"))
-				protocol = kSecProtocolTypeSMTP;
-			else /* we don't yet handle other protocols */
+				protocol = kSecAttrProtocolSMTP;
+			else {
+				/* we don't yet handle other protocols */
+				clear_credential();
 				exit(0);
+			}
 		}
 		else if (!strcmp(buf, "host")) {
 			char *colon = strchr(v, ':');
 			if (colon) {
+				UInt16 port_i;
 				*colon++ = '\0';
-				port = atoi(colon);
+				port_i = atoi(colon);
+				port = CFNumberCreate(kCFAllocatorDefault,
+						      kCFNumberShortType,
+						      &port_i);
 			}
-			host = xstrdup(v);
+			host = CFStringCreateWithCString(kCFAllocatorDefault,
+							 v,
+							 ENCODING);
 		}
 		else if (!strcmp(buf, "path"))
-			path = xstrdup(v);
+			path = CFStringCreateWithCString(kCFAllocatorDefault,
+							 v,
+							 ENCODING);
 		else if (!strcmp(buf, "username"))
-			username = xstrdup(v);
+			username = CFStringCreateWithCString(
+					kCFAllocatorDefault,
+					v,
+					ENCODING);
 		else if (!strcmp(buf, "password"))
-			password = xstrdup(v);
+			password = CFDataCreate(kCFAllocatorDefault,
+						(UInt8 *)v,
+						strlen(v));
 		/*
 		 * Ignore other lines; we don't know what they mean, but
 		 * this future-proofs us when later versions of git do
@@ -173,6 +296,7 @@ static void read_credential(void)
 
 int main(int argc, const char **argv)
 {
+	OSStatus result = 0;
 	const char *usage =
 		"usage: git credential-osxkeychain <get|store|erase>";
 
@@ -182,12 +306,17 @@ int main(int argc, const char **argv)
 	read_credential();
 
 	if (!strcmp(argv[1], "get"))
-		find_internet_password();
+		result = find_internet_password();
 	else if (!strcmp(argv[1], "store"))
-		add_internet_password();
+		result = add_internet_password();
 	else if (!strcmp(argv[1], "erase"))
-		delete_internet_password();
+		result = delete_internet_password();
 	/* otherwise, ignore unknown action */
 
+	if (result)
+		die("failed to %s: %d", argv[1], (int)result);
+
+	clear_credential();
+
 	return 0;
 }
-- 
gitgitgadget
Eric Sunshine· Feb 18, 2024, 06:08 UTC · re: Bo Anderson via GitGitGadget · lore

Re: [PATCH 1/4] osxkeychain: replace deprecated SecKeychain API

On Sat, Feb 17, 2024 at 6:35 PM Bo Anderson via GitGitGadget <gitgitgadget@gmail.com> wrote:

Show 19 quoted lines
> The SecKeychain API was deprecated in macOS 10.10, nearly 10 years ago.
> The replacement SecItem API however is available as far back as macOS
> 10.6.
>
> While supporting older macOS was perhaps prevously a concern,
> git-credential-osxkeychain already requires a minimum of macOS 10.7
> since 5747c8072b (contrib/credential: avoid fixed-size buffer in
> osxkeychain, 2023-05-01) so using the newer API should not regress the
> range of macOS versions supported.
>
> Adapting to use the newer SecItem API also happens to fix two test
> failures in osxkeychain:
>
>     8 - helper (osxkeychain) overwrites on store
>     9 - helper (osxkeychain) can forget host
>
> The new API is compatible with credentials saved with the older API.
>
> Signed-off-by: Bo Anderson <mail@boanderson.me>

I haven't studied the SecItem API, so I can't comment on the meat of the patch, but I can make a few generic observations...

Show 18 quoted lines
> diff --git a/contrib/credential/osxkeychain/git-credential-osxkeychain.c b/contrib/credential/osxkeychain/git-credential-osxkeychain.c
> @@ -3,14 +3,39 @@
> -__attribute__((format (printf, 1, 2)))
> +#define ENCODING kCFStringEncodingUTF8
> +static CFStringRef protocol; /* Stores constant strings - not memory managed */
> +static CFStringRef host;
> [...]
> +
> +static void clear_credential(void)
> +{
> +       if (host) {
> +               CFRelease(host);
> +               host = NULL;
> +       }
> +       [...]
> +}
> +
> +__attribute__((format (printf, 1, 2), __noreturn__))

The addition of `__noreturn__` to the `__attribute__` seems unrelated to the stated purpose of this patch. As such, it typically would be placed in its own patch. If it really is too minor for a separate patch, mentioning it in the commit message as a "While at it..." would be helpful.

Show 13 quoted lines
> @@ -19,70 +44,135 @@ static void die(const char *err, ...)
> +static CFDictionaryRef create_dictionary(CFAllocatorRef allocator, ...)
> +{
> +       va_list args;
> +       const void *key;
> +       CFMutableDictionaryRef result;
> +
> +       result = CFDictionaryCreateMutable(allocator,
> +                                          0,
> +                                          &kCFTypeDictionaryKeyCallBacks,
> +                                          &kCFTypeDictionaryValueCallBacks);
> +
> +
Style: one blank line is preferred over two
Show 8 quoted lines
> +       va_start(args, allocator);
> +       while ((key = va_arg(args, const void *)) != NULL) {
> +               const void *value;
> +               value = va_arg(args, const void *);
> +               if (value)
> +                       CFDictionarySetValue(result, key, value);
> +       }
> +       va_end(args);
A couple related comments...

If va_arg() ever returns NULL for `value`, the next iteration of the loop will call va_arg() again, but calling va_arg() again after it has already returned NULL is likely undefined behavior. At minimum, I would have expected this to be written as:

    while (...) {
        ...
        if (!value)
            break;
        CFDictionarySetValue(...);
    }

However, isn't it a programmer error if va_arg() returns NULL for `value`? If so, I'd think we'd want to scream loudly about that rather than silently ignoring it. So, perhaps something like this:

    while (...) {
        ...
        if (!value) {
            fprintf(stderr, "BUG: ...");
            abort();
        }
        CFDictionarySetValue(...);
   }

Or, perhaps just call the existing die() function in this file with a suitable "BUG ..." message.

Show 12 quoted lines
> +static void find_username_in_item(CFDictionaryRef item)
>  {
> +       CFStringRef account_ref;
> +       char *username_buf;
> +       CFIndex buffer_len;
>
> +       account_ref = CFDictionaryGetValue(item, kSecAttrAccount);
> +       if (!account_ref)
> +       {
> +               write_item("username", "", 0);
> +               return;
> +       }
Style: opening brace sticks to the `if` line:
    if !(account_ref) {
        ...
    }
Same comment applies to the `if` below.
Show 6 quoted lines
> +       username_buf = (char *)CFStringGetCStringPtr(account_ref, ENCODING);
> +       if (username_buf)
> +       {
> +               write_item("username", username_buf, strlen(username_buf));
>                 return;
> +       }

According to the documentation for CFStringGetCStringPtr(), the returned C-string is not newly-allocated, so the caller does not have to free it. Therefore, can `username_buf` be declared `const char *` rather than `char *` to make it clear to readers that nothing is being leaked here? Same comment about the `(char *)` cast.

> +       /* If we can't get a CString pointer then
> +        * we need to allocate our own buffer */
Style:
    /*
     * Multi-line comments
     * are formatted like this.
     */
Show 10 quoted lines
> +       buffer_len = CFStringGetMaximumSizeForEncoding(
> +                       CFStringGetLength(account_ref), ENCODING) + 1;
> +       username_buf = xmalloc(buffer_len);
> +       if (CFStringGetCString(account_ref,
> +                               username_buf,
> +                               buffer_len,
> +                               ENCODING)) {
> +               write_item("username", username_buf, buffer_len - 1);
> +       }
> +       free(username_buf);

Okay, this explains why `username_buf` is declared `char *` rather than `const char *`. Typically, when we have a situation in which a value may or may not need freeing, we use a `to_free` variable like this:

    const char *username_buf;
    char *to_free = NULL;
    ...
    username_buf = (const char *)CFStringGetCStringPtr(...);
    if (username_buf) {
        ...
        return;
    }
    ...
    username_buf = to_free = xmalloc(buffer_len);
    if (CFStringGetCString(...))
        ...
    free(to_free);

But that may be overkill for this simple case, and what you have here may be "good enough" for anyone already familiar with the API and who knows that the `return` after CFStringGetCStringPtr() isn't leaking.

Show 13 quoted lines
> +static OSStatus find_internet_password(void)
>  {
> +       CFDictionaryRef attrs;
> +       [...]
>
> +       attrs = CREATE_SEC_ATTRIBUTES(kSecMatchLimit, kSecMatchLimitOne,
> +                                     kSecReturnAttributes, kCFBooleanTrue,
> +                                     kSecReturnData, kCFBooleanTrue,
> +                                     NULL);
> +       result = SecItemCopyMatching(attrs, (CFTypeRef *)&item);
> +       if (result) {
> +               goto out;
> +       }
We omit braces when the body is a single statement:
    if (result)
        goto out;
(Same comment applies to other code in this patch.)
Show 5 quoted lines
> +       data = CFDictionaryGetValue(item, kSecValueData);
> +       [...]
> +
> +out:
> +       CFRelease(attrs);
Good, `attrs` is released in all cases.
Show 15 quoted lines
> +static OSStatus add_internet_password(void)
>  {
> +       [...]
> +       attrs = CREATE_SEC_ATTRIBUTES(kSecValueData, password,
> +                                     NULL);
> +       result = SecItemAdd(attrs, NULL);
> +       if (result == errSecDuplicateItem) {
> +               CFDictionaryRef query;
> +               query = CREATE_SEC_ATTRIBUTES(NULL);
> +               result = SecItemUpdate(query, attrs);
> +               CFRelease(query);
> +       }
> +       CFRelease(attrs);
> +       return result;
>  }
Good, `attrs` and `query` are released in all cases.
Bo Anderson· Feb 18, 2024, 14:48 UTC · re: Eric Sunshine · lore

Re: [PATCH 1/4] osxkeychain: replace deprecated SecKeychain API

> On 18 Feb 2024, at 06:08, Eric Sunshine <sunshine@sunshineco.com> wrote:
> 
> I haven't studied the SecItem API, so I can't comment on the meat of
> the patch, but I can make a few generic observations...
Thanks for taking a look!
Show 24 quoted lines
>> diff --git a/contrib/credential/osxkeychain/git-credential-osxkeychain.c b/contrib/credential/osxkeychain/git-credential-osxkeychain.c
>> @@ -3,14 +3,39 @@
>> -__attribute__((format (printf, 1, 2)))
>> +#define ENCODING kCFStringEncodingUTF8
>> +static CFStringRef protocol; /* Stores constant strings - not memory managed */
>> +static CFStringRef host;
>> [...]
>> +
>> +static void clear_credential(void)
>> +{
>> +       if (host) {
>> +               CFRelease(host);
>> +               host = NULL;
>> +       }
>> +       [...]
>> +}
>> +
>> +__attribute__((format (printf, 1, 2), __noreturn__))
> 
> The addition of `__noreturn__` to the `__attribute__` seems unrelated
> to the stated purpose of this patch. As such, it typically would be
> placed in its own patch. If it really is too minor for a separate
> patch, mentioning it in the commit message as a "While at it..." would
> be helpful.
Acknowledged. It is indeed a bit of a nothing change that doesn’t really do much on its own, but when paired with the port variable reorder could potentially make a “minor code cleanup” commit.
Show 39 quoted lines
>> +       va_start(args, allocator);
>> +       while ((key = va_arg(args, const void *)) != NULL) {
>> +               const void *value;
>> +               value = va_arg(args, const void *);
>> +               if (value)
>> +                       CFDictionarySetValue(result, key, value);
>> +       }
>> +       va_end(args);
> 
> A couple related comments...
> 
> If va_arg() ever returns NULL for `value`, the next iteration of the
> loop will call va_arg() again, but calling va_arg() again after it has
> already returned NULL is likely undefined behavior. At minimum, I
> would have expected this to be written as:
> 
> while (...) {
>     ...
>     if (!value)
>         break;
>     CFDictionarySetValue(...);
> }
> 
> However, isn't it a programmer error if va_arg() returns NULL for
> `value`? If so, I'd think we'd want to scream loudly about that rather
> than silently ignoring it. So, perhaps something like this:
> 
> while (...) {
>     ...
>     if (!value) {
>         fprintf(stderr, "BUG: ...");
>         abort();
>     }
>     CFDictionarySetValue(...);
> }
> 
> Or, perhaps just call the existing die() function in this file with a
> suitable "BUG ..." message.
> 
In this case it’s by design to accept and check for NULL values as it greatly simplifies the code. Inputs to the credential helpers have various optional fields, such as port and path. It is programmer error to pass NULL to the SecItem API (runtime crash) so in order to simplify having to check each individual field in all of the callers (and probably ditch varargs since you can’t really do dynamic varargs), I check the value here instead. That means you can do something like:
 create_dictionary(kCFAllocatorDefault,
     kSecAttrServer, host,
     kSecAttrPath, path, \
     kSecAttrPort, port,
     NULL)
And it will only include the key-value pairs that have non-NULL values.
It would indeed be programmer error to not pass key-value pairs, though it is equally programmer error to not have a terminating NULL.
Show 56 quoted lines
>> +       username_buf = (char *)CFStringGetCStringPtr(account_ref, ENCODING);
>> +       if (username_buf)
>> +       {
>> +               write_item("username", username_buf, strlen(username_buf));
>>             return;
>> +       }
> 
> According to the documentation for CFStringGetCStringPtr(), the
> returned C-string is not newly-allocated, so the caller does not have
> to free it. Therefore, can `username_buf` be declared `const char *`
> rather than `char *` to make it clear to readers that nothing is being
> leaked here? Same comment about the `(char *)` cast.
> 
>> +       /* If we can't get a CString pointer then
>> +        * we need to allocate our own buffer */
> 
> Style:
> 
> /*
>  * Multi-line comments
>  * are formatted like this.
>  */
> 
>> +       buffer_len = CFStringGetMaximumSizeForEncoding(
>> +                       CFStringGetLength(account_ref), ENCODING) + 1;
>> +       username_buf = xmalloc(buffer_len);
>> +       if (CFStringGetCString(account_ref,
>> +                               username_buf,
>> +                               buffer_len,
>> +                               ENCODING)) {
>> +               write_item("username", username_buf, buffer_len - 1);
>> +       }
>> +       free(username_buf);
> 
> Okay, this explains why `username_buf` is declared `char *` rather
> than `const char *`. Typically, when we have a situation in which a
> value may or may not need freeing, we use a `to_free` variable like
> this:
> 
> const char *username_buf;
> char *to_free = NULL;
> ...
> username_buf = (const char *)CFStringGetCStringPtr(...);
> if (username_buf) {
>     ...
>     return;
> }
> ...
> username_buf = to_free = xmalloc(buffer_len);
> if (CFStringGetCString(...))
>     ...
> free(to_free);
> 
> But that may be overkill for this simple case, and what you have here
> may be "good enough" for anyone already familiar with the API and who
> knows that the `return` after CFStringGetCStringPtr() isn't leaking.
Would it make sense to just have a comment paired with the CFStringGetCStringPtr return explaining why it doesn’t need to be freed there? I’m OK with the to_free variable however if that’s clearer. Idea in my mind was pairing it based on `xmalloc` but I can see why pairing based on variable is clearer.
Eric Sunshine· Feb 18, 2024, 18:39 UTC · re: Bo Anderson · lore

Re: [PATCH 1/4] osxkeychain: replace deprecated SecKeychain API

On Sun, Feb 18, 2024 at 9:48 AM Bo Anderson <mail@boanderson.me> wrote:
Show 35 quoted lines
> > On 18 Feb 2024, at 06:08, Eric Sunshine <sunshine@sunshineco.com> wrote:
> >> +    va_start(args, allocator);
> >> +    while ((key = va_arg(args, const void *)) != NULL) {
> >> +        const void *value;
> >> +        value = va_arg(args, const void *);
> >> +        if (value)
> >> +            CFDictionarySetValue(result, key, value);
> >> +    }
> >> +    va_end(args);
> >
> > However, isn't it a programmer error if va_arg() returns NULL for
> > `value`? If so, I'd think we'd want to scream loudly about that rather
> > than silently ignoring it. So, perhaps something like this: [...]
>
> In this case it’s by design to accept and check for NULL values as
> it greatly simplifies the code. Inputs to the credential helpers
> have various optional fields, such as port and path. It is
> programmer error to pass NULL to the SecItem API (runtime crash) so
> in order to simplify having to check each individual field in all of
> the callers (and probably ditch varargs since you can’t really do
> dynamic varargs), I check the value here instead. That means you can
> do something like:
>
> create_dictionary(kCFAllocatorDefault,
>   kSecAttrServer, host,
>   kSecAttrPath, path, \
>   kSecAttrPort, port,
>   NULL)
>
> And it will only include the key-value pairs that have non-NULL
> values.
>
> It would indeed be programmer error to  not pass key-value pairs,
> though it is equally programmer  error to  not have a terminating
> NULL.

Okay. I had thought that this check was merely protecting against programmer error, but the described use-case to avoid passing NULL to SecItem API makes perfect sense. It might be helpful to future readers to explain this either as a function-level comment (explaining how to call the function) or as an in-code comment.

Show 38 quoted lines
> >> +    username_buf = (char *)CFStringGetCStringPtr(account_ref, ENCODING);
> >> +    if (username_buf)
> >> +    {
> >> +        write_item("username", username_buf, strlen(username_buf));
> >>       return;
> >> +    }
> >
> > According to the documentation for CFStringGetCStringPtr(), the
> > returned C-string is not newly-allocated, so the caller does not have
> > to free it. Therefore, can `username_buf` be declared `const char *`
> > rather than `char *` to make it clear to readers that nothing is being
> > leaked here? Same comment about the `(char *)` cast.
> >
> >> +    buffer_len = CFStringGetMaximumSizeForEncoding(
> >> +            CFStringGetLength(account_ref), ENCODING) + 1;
> >> +    username_buf = xmalloc(buffer_len);
> >> +    if (CFStringGetCString(account_ref,
> >> +                username_buf,
> >> +                buffer_len,
> >> +                ENCODING)) {
> >> +        write_item("username", username_buf, buffer_len - 1);
> >> +    }
> >> +    free(username_buf);
> >
> > Okay, this explains why `username_buf` is declared `char *` rather
> > than `const char *`. Typically, when we have a situation in which a
> > value may or may not need freeing, we use a `to_free` variable like
> > this: [...]
> >
> > But that may be overkill for this simple case, and what you have here
> > may be "good enough" for anyone already familiar with the API and who
> > knows that the `return` after CFStringGetCStringPtr() isn't leaking.
>
> Would it make sense to just have a comment paired with the
> CFStringGetCStringPtr return explaining why it doesn’t need to be
> freed there? I’m OK with the to_free variable however if that’s
> clearer. Idea in my mind was pairing it based on `xmalloc` but I can
> see why pairing based on variable is clearer.

Most likely, anyone working on this code is already familiar with the CoreFoundation API, thus would understand implicitly that this isn't leaking. But, yes, a simple comment should be plenty sufficient for everyone else if you are re-rolling anyhow.

Bo Anderson via GitGitGadget· Feb 17, 2024, 23:34 UTC · re: Bo Anderson via GitGitGadget · lore

[PATCH 2/4] osxkeychain: erase all matching credentials

From: Bo Anderson <mail@boanderson.me>

Other credential managers erased all matching credentials, as indicated by a test case that osxkeychain failed:

    15 - helper (osxkeychain) erases all matching credentials
Signed-off-by: Bo Anderson <mail@boanderson.me>
---
 contrib/credential/osxkeychain/git-credential-osxkeychain.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
Show changes to contrib/credential/osxkeychain/git-credential-osxkeychain.c +2 −1
diff --git a/contrib/credential/osxkeychain/git-credential-osxkeychain.c b/contrib/credential/osxkeychain/git-credential-osxkeychain.c
index dc294ae944a..e9cee3aed45 100644
--- a/contrib/credential/osxkeychain/git-credential-osxkeychain.c
+++ b/contrib/credential/osxkeychain/git-credential-osxkeychain.c
@@ -182,7 +182,8 @@ static OSStatus delete_internet_password(void)
 	if (!protocol || !host)
 		return -1;
 
-	attrs = CREATE_SEC_ATTRIBUTES(NULL);
+	attrs = CREATE_SEC_ATTRIBUTES(kSecMatchLimit, kSecMatchLimitAll,
+				      NULL);
 	result = SecItemDelete(attrs);
 	CFRelease(attrs);
 
-- 
gitgitgadget
Bo Anderson via GitGitGadget· Feb 17, 2024, 23:34 UTC · re: Bo Anderson via GitGitGadget · lore

[PATCH 3/4] osxkeychain: erase matching passwords only

From: Bo Anderson <mail@boanderson.me>

Other credential helpers support deleting credentials that match a specified password. See 7144dee3ec (credential/libsecret: erase matching creds only, 2023-07-26) and cb626f8e5c (credential/wincred: erase matching creds only, 2023-07-26).

Support this in osxkeychain too by extracting, decrypting and comparing the stored password before deleting.

Fixes the following test failure with osxkeychain:
    11 - helper (osxkeychain) does not erase a password distinct from
    input
Signed-off-by: Bo Anderson <mail@boanderson.me>
---
 .../osxkeychain/git-credential-osxkeychain.c  | 56 ++++++++++++++++++-
 1 file changed, 55 insertions(+), 1 deletion(-)
Show changes to contrib/credential/osxkeychain/git-credential-osxkeychain.c +55 −1
diff --git a/contrib/credential/osxkeychain/git-credential-osxkeychain.c b/contrib/credential/osxkeychain/git-credential-osxkeychain.c
index e9cee3aed45..9e742796336 100644
--- a/contrib/credential/osxkeychain/git-credential-osxkeychain.c
+++ b/contrib/credential/osxkeychain/git-credential-osxkeychain.c
@@ -169,9 +169,55 @@ static OSStatus find_internet_password(void)
 	return result;
 }
 
+static OSStatus delete_ref(const void *itemRef)
+{
+	CFArrayRef item_ref_list;
+	CFDictionaryRef delete_query;
+	OSStatus result;
+
+	item_ref_list = CFArrayCreate(kCFAllocatorDefault,
+				      &itemRef,
+				      1,
+				      &kCFTypeArrayCallBacks);
+	delete_query = create_dictionary(kCFAllocatorDefault,
+					 kSecClass, kSecClassInternetPassword,
+					 kSecMatchItemList, item_ref_list,
+					 NULL);
+
+	if (password) {
+		/* We only want to delete items with a matching password */
+		CFIndex capacity;
+		CFMutableDictionaryRef query;
+		CFDataRef data;
+
+		capacity = CFDictionaryGetCount(delete_query) + 1;
+		query = CFDictionaryCreateMutableCopy(kCFAllocatorDefault,
+						      capacity,
+						      delete_query);
+		CFDictionarySetValue(query, kSecReturnData, kCFBooleanTrue);
+		result = SecItemCopyMatching(query, (CFTypeRef *)&data);
+		if (!result) {
+			if (CFEqual(data, password))
+				result = SecItemDelete(delete_query);
+
+			CFRelease(data);
+		}
+
+		CFRelease(query);
+	} else {
+		result = SecItemDelete(delete_query);
+	}
+
+	CFRelease(delete_query);
+	CFRelease(item_ref_list);
+
+	return result;
+}
+
 static OSStatus delete_internet_password(void)
 {
 	CFDictionaryRef attrs;
+	CFArrayRef refs;
 	OSStatus result;
 
 	/*
@@ -183,10 +229,18 @@ static OSStatus delete_internet_password(void)
 		return -1;
 
 	attrs = CREATE_SEC_ATTRIBUTES(kSecMatchLimit, kSecMatchLimitAll,
+				      kSecReturnRef, kCFBooleanTrue,
 				      NULL);
-	result = SecItemDelete(attrs);
+	result = SecItemCopyMatching(attrs, (CFTypeRef *)&refs);
 	CFRelease(attrs);
 
+	if (!result) {
+		for (CFIndex i = 0; !result && i < CFArrayGetCount(refs); i++)
+			result = delete_ref(CFArrayGetValueAtIndex(refs, i));
+
+		CFRelease(refs);
+	}
+
 	/* We consider not found to not be an error */
 	if (result == errSecItemNotFound)
 		result = errSecSuccess;
-- 
gitgitgadget
Bo Anderson via GitGitGadget· Feb 17, 2024, 23:34 UTC · re: Bo Anderson via GitGitGadget · lore

[PATCH 4/4] osxkeychain: store new attributes

From: Bo Anderson <mail@boanderson.me>

d208bfdfef (credential: new attribute password_expiry_utc, 2023-02-18) and a5c76569e7 (credential: new attribute oauth_refresh_token, 2023-04-21) introduced new credential attributes but support was missing from git-credential-osxkeychain.

Support these attributes by appending the data to the password in the keychain, separated by line breaks. Line breaks cannot appear in a git credential password so it is an appropriate separator.

Fixes the remaining test failures with osxkeychain:
    18 - helper (osxkeychain) gets password_expiry_utc
    19 - helper (osxkeychain) overwrites when password_expiry_utc
    changes
    21 - helper (osxkeychain) gets oauth_refresh_token
Signed-off-by: Bo Anderson <mail@boanderson.me>
---
 .../osxkeychain/git-credential-osxkeychain.c  | 68 +++++++++++++++++--
 1 file changed, 62 insertions(+), 6 deletions(-)
Show changes to contrib/credential/osxkeychain/git-credential-osxkeychain.c +62 −6
diff --git a/contrib/credential/osxkeychain/git-credential-osxkeychain.c b/contrib/credential/osxkeychain/git-credential-osxkeychain.c
index 9e742796336..6a40917b1ef 100644
--- a/contrib/credential/osxkeychain/git-credential-osxkeychain.c
+++ b/contrib/credential/osxkeychain/git-credential-osxkeychain.c
@@ -6,10 +6,12 @@
 #define ENCODING kCFStringEncodingUTF8
 static CFStringRef protocol; /* Stores constant strings - not memory managed */
 static CFStringRef host;
+static CFNumberRef port;
 static CFStringRef path;
 static CFStringRef username;
 static CFDataRef password;
-static CFNumberRef port;
+static CFDataRef password_expiry_utc;
+static CFDataRef oauth_refresh_token;
 
 static void clear_credential(void)
 {
@@ -17,6 +19,10 @@ static void clear_credential(void)
 		CFRelease(host);
 		host = NULL;
 	}
+	if (port) {
+		CFRelease(port);
+		port = NULL;
+	}
 	if (path) {
 		CFRelease(path);
 		path = NULL;
@@ -29,12 +35,18 @@ static void clear_credential(void)
 		CFRelease(password);
 		password = NULL;
 	}
-	if (port) {
-		CFRelease(port);
-		port = NULL;
+	if (password_expiry_utc) {
+		CFRelease(password_expiry_utc);
+		password_expiry_utc = NULL;
+	}
+	if (oauth_refresh_token) {
+		CFRelease(oauth_refresh_token);
+		oauth_refresh_token = NULL;
 	}
 }
 
+#define STRING_WITH_LENGTH(s) s, sizeof(s) - 1
+
 __attribute__((format (printf, 1, 2), __noreturn__))
 static void die(const char *err, ...)
 {
@@ -197,9 +209,27 @@ static OSStatus delete_ref(const void *itemRef)
 		CFDictionarySetValue(query, kSecReturnData, kCFBooleanTrue);
 		result = SecItemCopyMatching(query, (CFTypeRef *)&data);
 		if (!result) {
-			if (CFEqual(data, password))
+			CFDataRef kc_password;
+			const UInt8 *raw_data;
+			const UInt8 *line;
+
+			/* Don't match appended metadata */
+			raw_data = CFDataGetBytePtr(data);
+			line = memchr(raw_data, '\n', CFDataGetLength(data));
+			if (line)
+				kc_password = CFDataCreateWithBytesNoCopy(
+						kCFAllocatorDefault,
+						raw_data,
+						line - raw_data,
+						kCFAllocatorNull);
+			else
+				kc_password = data;
+
+			if (CFEqual(kc_password, password))
 				result = SecItemDelete(delete_query);
 
+			if (line)
+				CFRelease(kc_password);
 			CFRelease(data);
 		}
 
@@ -250,6 +280,7 @@ static OSStatus delete_internet_password(void)
 
 static OSStatus add_internet_password(void)
 {
+	CFMutableDataRef data;
 	CFDictionaryRef attrs;
 	OSStatus result;
 
@@ -257,7 +288,23 @@ static OSStatus add_internet_password(void)
 	if (!protocol || !host || !username || !password)
 		return -1;
 
-	attrs = CREATE_SEC_ATTRIBUTES(kSecValueData, password,
+	data = CFDataCreateMutableCopy(kCFAllocatorDefault, 0, password);
+	if (password_expiry_utc) {
+		CFDataAppendBytes(data,
+		    (const UInt8 *)STRING_WITH_LENGTH("\npassword_expiry_utc="));
+		CFDataAppendBytes(data,
+				  CFDataGetBytePtr(password_expiry_utc),
+				  CFDataGetLength(password_expiry_utc));
+	}
+	if (oauth_refresh_token) {
+		CFDataAppendBytes(data,
+		    (const UInt8 *)STRING_WITH_LENGTH("\noauth_refresh_token="));
+		CFDataAppendBytes(data,
+				  CFDataGetBytePtr(oauth_refresh_token),
+				  CFDataGetLength(oauth_refresh_token));
+	}
+
+	attrs = CREATE_SEC_ATTRIBUTES(kSecValueData, data,
 				      NULL);
 
 	result = SecItemAdd(attrs, NULL);
@@ -268,6 +315,7 @@ static OSStatus add_internet_password(void)
 		CFRelease(query);
 	}
 
+	CFRelease(data);
 	CFRelease(attrs);
 
 	return result;
@@ -339,6 +387,14 @@ static void read_credential(void)
 			password = CFDataCreate(kCFAllocatorDefault,
 						(UInt8 *)v,
 						strlen(v));
+		else if (!strcmp(buf, "password_expiry_utc"))
+			password_expiry_utc = CFDataCreate(kCFAllocatorDefault,
+							   (UInt8 *)v,
+							   strlen(v));
+		else if (!strcmp(buf, "oauth_refresh_token"))
+			oauth_refresh_token = CFDataCreate(kCFAllocatorDefault,
+							   (UInt8 *)v,
+							   strlen(v));
 		/*
 		 * Ignore other lines; we don't know what they mean, but
 		 * this future-proofs us when later versions of git do
-- 
gitgitgadget
Eric Sunshine· Feb 18, 2024, 06:31 UTC · re: Bo Anderson via GitGitGadget · lore

Re: [PATCH 4/4] osxkeychain: store new attributes

On Sat, Feb 17, 2024 at 6:35 PM Bo Anderson via GitGitGadget <gitgitgadget@gmail.com> wrote:

Show 26 quoted lines
> d208bfdfef (credential: new attribute password_expiry_utc, 2023-02-18)
> and a5c76569e7 (credential: new attribute oauth_refresh_token,
> 2023-04-21) introduced new credential attributes but support was missing
> from git-credential-osxkeychain.
> [...]
> Signed-off-by: Bo Anderson <mail@boanderson.me>
> ---
> diff --git a/contrib/credential/osxkeychain/git-credential-osxkeychain.c b/contrib/credential/osxkeychain/git-credential-osxkeychain.c
> @@ -6,10 +6,12 @@
>  static CFStringRef host;
> +static CFNumberRef port;
>  static CFStringRef path;
> -static CFNumberRef port;
> @@ -17,6 +19,10 @@ static void clear_credential(void)
> +       if (port) {
> +               CFRelease(port);
> +               port = NULL;
> +       }
> @@ -29,12 +35,18 @@ static void clear_credential(void)
> -       if (port) {
> -               CFRelease(port);
> -               port = NULL;
> +       if (password_expiry_utc) {
> +               CFRelease(password_expiry_utc);
> +               password_expiry_utc = NULL;
> +       }

The relocation of `port` is unrelated to the stated purpose of this patch. We would normally avoid this sort of "noise" change since it obscures the "real" changes made by the patch, and would instead place it in its own patch. That said, it's such a minor issue, I doubt that it's worth a reroll.

Eric Sunshine· Feb 18, 2024, 06:38 UTC · re: Bo Anderson via GitGitGadget · lore

Re: [PATCH 0/4] osxkeychain: bring in line with other credential helpers

On Sat, Feb 17, 2024 at 6:35 PM Bo Anderson via GitGitGadget <gitgitgadget@gmail.com> wrote:

Show 8 quoted lines
> git-credential-osxkeychain has largely fallen behind other external
> credential helpers in the features it supports, and hasn't received any
> functional changes since 2013. [...]
>
> osxkeychain also made use of macOS APIs that had been deprecated since 2014.
> Replacement API was able to be used without regressing the minimum supported
> macOS established in 5747c8072b (contrib/credential: avoid fixed-size buffer
> in osxkeychain, 2023-05-01).

Although I'm not familiar with the SecItem API nor with the Git keychain API, I gave this series a readthrough and left a few minor comments. Aside from a few very minor style nits, perhaps the only substantive comment was that patch [1/4] could do a slightly better job of protecting against future programmer error, but even that is minor in that it doesn't impact the functionality actually implemented by the patch, thus may not be worth a reroll.

Overall, despite not being familiar with the APIs in question, everything I read in the patches made sense and was cleanly implemented. Nicely done.

M Hickford· Feb 18, 2024, 20:40 UTC · re: Bo Anderson via GitGitGadget · lore

Re: [PATCH 0/4] osxkeychain: bring in line with other credential helpers

Show 14 quoted lines
> git-credential-osxkeychain has largely fallen behind other external
> credential helpers in the features it supports, and hasn't received any
> functional changes since 2013. As it stood, osxkeychain failed seven tests
> in the external credential helper test suite:
> 
> not ok 8 - helper (osxkeychain) overwrites on store
> not ok 9 - helper (osxkeychain) can forget host
> not ok 11 - helper (osxkeychain) does not erase a password distinct from input
> not ok 15 - helper (osxkeychain) erases all matching credentials
> not ok 18 - helper (osxkeychain) gets password_expiry_utc
> not ok 19 - helper (osxkeychain) overwrites when password_expiry_utc changes
> not ok 21 - helper (osxkeychain) gets oauth_refresh_token
>
> After this set of patches, osxkeychain passes all tests in the external
credential helper test suite.
Great work!
Could these tests run as part of macOS CI?
Bo Anderson· Feb 18, 2024, 23:23 UTC · re: M Hickford · lore

Re: [PATCH 0/4] osxkeychain: bring in line with other credential helpers

Show 21 quoted lines
> On 18 Feb 2024, at 20:40, M Hickford <mirth.hickford@gmail.com> wrote:
> 
>> git-credential-osxkeychain has largely fallen behind other external
>> credential helpers in the features it supports, and hasn't received any
>> functional changes since 2013. As it stood, osxkeychain failed seven tests
>> in the external credential helper test suite:
>> 
>> not ok 8 - helper (osxkeychain) overwrites on store
>> not ok 9 - helper (osxkeychain) can forget host
>> not ok 11 - helper (osxkeychain) does not erase a password distinct from input
>> not ok 15 - helper (osxkeychain) erases all matching credentials
>> not ok 18 - helper (osxkeychain) gets password_expiry_utc
>> not ok 19 - helper (osxkeychain) overwrites when password_expiry_utc changes
>> not ok 21 - helper (osxkeychain) gets oauth_refresh_token
>> 
>> After this set of patches, osxkeychain passes all tests in the external
> credential helper test suite.
> 
> Great work!
> 
> Could these tests run as part of macOS CI?
Do we do so for any of the other external credential helpers?
It definitely makes sense in principle. Though the concern perhaps will be that any new features added to the credential helpers and thus its test suite would need adding to each credential helper simultaneously to avoid failing CI. Ideally we would do exactly that, though that requires knowledge on each of the keystore APIs used in each of the credential helpers.
M Hickford· Mar 4, 2024, 08:00 UTC · re: Bo Anderson · lore

Re: [PATCH 0/4] osxkeychain: bring in line with other credential helpers

On Sun, 18 Feb 2024 at 23:24, Bo Anderson <mail@boanderson.me> wrote:
Show 8 quoted lines
>
> > On 18 Feb 2024, at 20:40, M Hickford <mirth.hickford@gmail.com> wrote:
> >
> >
> > Could these tests run as part of macOS CI?
>
> Do we do so for any of the other external credential helpers?
>
We don't.
> It definitely makes sense in principle. Though the concern perhaps will be that any new features added to the credential helpers and thus its test suite would need adding to each credential helper simultaneously to avoid failing CI. Ideally we would do exactly that, though that requires knowledge on each of the keystore APIs used in each of the credential helpers.
Good point.
Jeff King· Mar 7, 2024, 09:47 UTC · re: M Hickford · lore

Re: [PATCH 0/4] osxkeychain: bring in line with other credential helpers

On Mon, Mar 04, 2024 at 08:00:00AM +0000, M Hickford wrote:
Show 8 quoted lines
> > It definitely makes sense in principle. Though the concern perhaps
> > will be that any new features added to the credential helpers and
> > thus its test suite would need adding to each credential helper
> > simultaneously to avoid failing CI. Ideally we would do exactly
> > that, though that requires knowledge on each of the keystore APIs
> > used in each of the credential helpers.
> 
> Good point.

I think we suffer from that somewhat already. You cannot run t0303 successfully against credential-store anymore, as of 0ce02e2fec (credential/libsecret: store new attributes, 2023-06-16).

There is some prior art in the GIT_TEST_CREDENTIAL_HELPER_TIMEOUT variable, as time is not a concept to every helper (like store, for example). Other new tests like the password-expiry and oauth features could be gated on similar variables. That would help non-CI users testing helpers manually, and then CI jobs could set the appropriate switches for each helper that they cover.

All that said, I'd be surprised if testing osxkeychain in the CI environment worked. Back when I worked on it in 2011, I found that I had to actually run the tests in a local terminal; even a remote ssh login could not access the keychain. It's possible that things have changed since then, though, or perhaps I was imply ignorant of how to configure things correctly.

-Peff
Robert Coup· Apr 2, 2024, 13:21 UTC · re: Jeff King · lore

Re: [PATCH 0/4] osxkeychain: bring in line with other credential helpers

Hi all,
Show 6 quoted lines
> All that said, I'd be surprised if testing osxkeychain in the CI
> environment worked. Back when I worked on it in 2011, I found that I had
> to actually run the tests in a local terminal; even a remote ssh login
> could not access the keychain. It's possible that things have changed
> since then, though, or perhaps I was imply ignorant of how to configure
> things correctly.

I have gotten keychain working in Github Actions before: there's some helpers for it, but you can also basically do it manually via the steps from [1]. Basically anyone who needs to do Apple code-signing in CI has to make it work.

@Bo, how are you actually testing this manually? Following these steps:

$ make $ (cd contrib/credential/osxkeychain && make) $ ln -s contrib/credential/osxkeychain/git-credential-osxkeychain . $ cd t $ make GIT_TEST_CREDENTIAL_HELPER=osxkeychain t0303-credential-external.sh

I get 'A keychain cannot be found to store "store-user".' in a popup dialog when #2 runs; then similar for other tests in 0303. For #14 I get a slight alternative with "A keychain cannot be found". There's a "Reset To Defaults" button, but that wipes everything. AFAIK I have a relatively normal setup, with a login keychain as default. macOS 14.3.1; arm64.

$ security list-keychains
    "/Users/rc/Library/Keychains/login.keychain-db"
    "/Library/Keychains/System.keychain"
$ security default-keychain
    "/Users/rc/Library/Keychains/login.keychain-db"
$ security unlock-keychain
password to unlock default: ...

I don't see any settings or code for setting which keychain the credential helper uses, so I guess it's the default one?

Cheers,
Rob :)
[1] https://docs.github.com/en/actions/deployment/deploying-xcode-applications/installing-an-apple-certificate-on-macos-runners-for-xcode-development
Bo Anderson· Apr 2, 2024, 13:53 UTC · re: Robert Coup · lore

Re: [PATCH 0/4] osxkeychain: bring in line with other credential helpers

The test script does not interact well with the env filtering. This was the case before this change too.
To interact with your default keychain, you will need:
GIT_TEST_CREDENTIAL_HELPER_SETUP="export HOME=$HOME”
This is because the default macOS user keychain is local to your home directory - that’s why it’s giving errors about not finding any.
Bo
Show 47 quoted lines
> On 2 Apr 2024, at 14:21, Robert Coup <robert.coup@koordinates.com> wrote:
> 
> Hi all,
> 
>> All that said, I'd be surprised if testing osxkeychain in the CI
>> environment worked. Back when I worked on it in 2011, I found that I had
>> to actually run the tests in a local terminal; even a remote ssh login
>> could not access the keychain. It's possible that things have changed
>> since then, though, or perhaps I was imply ignorant of how to configure
>> things correctly.
> 
> I have gotten keychain working in Github Actions before: there's some
> helpers for it, but you can also basically do it manually via the
> steps from [1]. Basically anyone who needs to do Apple code-signing in
> CI has to make it work.
> 
> @Bo, how are you actually testing this manually? Following these steps:
> 
> $ make
> $ (cd contrib/credential/osxkeychain && make)
> $ ln -s contrib/credential/osxkeychain/git-credential-osxkeychain .
> $ cd t
> $ make GIT_TEST_CREDENTIAL_HELPER=osxkeychain t0303-credential-external.sh
> 
> I get 'A keychain cannot be found to store "store-user".' in a popup
> dialog when #2 runs; then similar for other tests in 0303. For #14 I
> get a slight alternative with "A keychain cannot be found". There's a
> "Reset To Defaults" button, but that wipes everything. AFAIK I have a
> relatively normal setup, with a login keychain as default. macOS
> 14.3.1; arm64.
> 
> $ security list-keychains
>    "/Users/rc/Library/Keychains/login.keychain-db"
>    "/Library/Keychains/System.keychain"
> $ security default-keychain
>    "/Users/rc/Library/Keychains/login.keychain-db"
> $ security unlock-keychain
> password to unlock default: ...
> 
> I don't see any settings or code for setting which keychain the
> credential helper uses, so I guess it's the default one?
> 
> Cheers,
> 
> Rob :)
> 
> [1] https://docs.github.com/en/actions/deployment/deploying-xcode-applications/installing-an-apple-certificate-on-macos-runners-for-xcode-development
Robert Coup· Apr 2, 2024, 14:54 UTC · re: Bo Anderson · lore

Re: [PATCH 0/4] osxkeychain: bring in line with other credential helpers

Hi Bo,
On Tue, 2 Apr 2024 at 14:53, Bo Anderson <mail@boanderson.me> wrote:
>
> The test script does not interact well with the env filtering. This was the case before this change too.

I guess without writing a helper-specific test or having some per-helper-setup thing it's a bit tricky.

Show 5 quoted lines
> To interact with your default keychain, you will need:
>
> GIT_TEST_CREDENTIAL_HELPER_SETUP="export HOME=$HOME”
>
> This is because the default macOS user keychain is local to your home directory - that’s why it’s giving errors about not finding any.
And with that, the tests all pass :-) Comparing with master where 7/21 failed.
Tested-by: Robert Coup <robert.coup@koordinates.com>

Could we document that setup step somewhere? I guess the simplest is probably just to put it in the header of t/t0303-credential-external.sh; maybe along the lines of the patch below.

Rob :)
Show changes to t/t0303-credential-external.sh +7 −0
diff --git a/t/t0303-credential-external.sh b/t/t0303-credential-external.sh
index 095574bfc6..e4e693b233 100755
--- a/t/t0303-credential-external.sh
+++ b/t/t0303-credential-external.sh
@@ -27,6 +27,13 @@ timeout, you can test that feature with:
 If your helper requires additional setup before the tests are started,
 you can set GIT_TEST_CREDENTIAL_HELPER_SETUP to a sequence of shell
 commands.
+
+- osxkeychain:
+
+  Because the default macOS user keychain is local to your home
+  directory, you will need:
+
+    GIT_TEST_CREDENTIAL_HELPER_SETUP="export HOME=$HOME”
 '

 . ./test-lib.sh
M Hickford· Apr 1, 2024, 21:40 UTC · re: Bo Anderson via GitGitGadget · lore

Re: [PATCH 0/4] osxkeychain: bring in line with other credential helpers

Show 46 quoted lines
> From: "Bo Anderson via GitGitGadget" <gitgitgadget@gmail.com>
> To: git@vger.kernel.org
> Cc: Bo Anderson <mail@boanderson.me>
> Subject: [PATCH 0/4] osxkeychain: bring in line with other credential helpers
> Date: Sat, 17 Feb 2024 23:34:52 +0000	[thread overview]
> Message-ID: <pull.1667.git.1708212896.gitgitgadget@gmail.com> (raw)
> 
> git-credential-osxkeychain has largely fallen behind other external
> credential helpers in the features it supports, and hasn't received any
> functional changes since 2013. As it stood, osxkeychain failed seven tests
> in the external credential helper test suite:
> 
> not ok 8 - helper (osxkeychain) overwrites on store
> not ok 9 - helper (osxkeychain) can forget host
> not ok 11 - helper (osxkeychain) does not erase a password distinct from input
> not ok 15 - helper (osxkeychain) erases all matching credentials
> not ok 18 - helper (osxkeychain) gets password_expiry_utc
> not ok 19 - helper (osxkeychain) overwrites when password_expiry_utc changes
> not ok 21 - helper (osxkeychain) gets oauth_refresh_token
> 
> 
> osxkeychain also made use of macOS APIs that had been deprecated since 2014.
> Replacement API was able to be used without regressing the minimum supported
> macOS established in 5747c8072b (contrib/credential: avoid fixed-size buffer
> in osxkeychain, 2023-05-01).
> 
> After this set of patches, osxkeychain passes all tests in the external
> credential helper test suite.
> 
> Bo Anderson (4):
>   osxkeychain: replace deprecated SecKeychain API
>   osxkeychain: erase all matching credentials
>   osxkeychain: erase matching passwords only
>   osxkeychain: store new attributes
> 
>  contrib/credential/osxkeychain/Makefile       |   3 +-
>  .../osxkeychain/git-credential-osxkeychain.c  | 376 ++++++++++++++----
>  2 files changed, 310 insertions(+), 69 deletions(-)
> 
> 
> base-commit: 3e0d3cd5c7def4808247caf168e17f2bbf47892b
> Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1667%2FBo98%2Fosxkeychain-update-v1
> Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1667/Bo98/osxkeychain-update-v1
> Pull-Request: https://github.com/gitgitgadget/git/pull/1667
> -- 
> gitgitgadget
Hi. Is this patch ready to cook in seen?
Junio C Hamano· Apr 1, 2024, 22:16 UTC · re: M Hickford · lore

Re: [PATCH 0/4] osxkeychain: bring in line with other credential helpers

M Hickford <mirth.hickford@gmail.com> writes:
Show 9 quoted lines
>> From: "Bo Anderson via GitGitGadget" <gitgitgadget@gmail.com>
>> To: git@vger.kernel.org
>> Cc: Bo Anderson <mail@boanderson.me>
>> Subject: [PATCH 0/4] osxkeychain: bring in line with other credential helpers
>> Date: Sat, 17 Feb 2024 23:34:52 +0000	[thread overview]
>> Message-ID: <pull.1667.git.1708212896.gitgitgadget@gmail.com> (raw)
>>  ...
>> git-credential-osxkeychain has largely fallen behind other external
> Hi. Is this patch ready to cook in seen?

A better nudge would be for somebody who uses macOS to resend the patches with their "Tested-by:" trailers added ;-).

Thanks.

← back to recent threads