Re: [PATCH] send-email: If the ca path is not specified, use the defaults

On Jan 16, 2014, at 15:19, Junio C Hamano wrote:

I had written up this draft email that describes the situation on OS X and decided it might not be exactly on topic and wasn't going to send it, but, since you asked... ;)

OpenSSL's default location can be found with:

   openssl version -d

On my Ubuntu system I get this:

   $ openssl version -d
   OPENSSLDIR: "/usr/lib/ssl"

Then if I look in there like so:

   $ ls -l /usr/lib/ssl
   total 8
   lrwxrwxrwx 1 root root   14 certs -> /etc/ssl/certs
   drwxr-xr-x 2 root root 4096 engines
   drwxr-xr-x 2 root root 4096 misc
   lrwxrwxrwx 1 root root   20 openssl.cnf -> /etc/ssl/openssl.cnf
   lrwxrwxrwx 1 root root   16 private -> /etc/ssl/private

And that's how it ends up using /etc/ssl/certs. From the SSL_CTX_load_verify_locations man page:

   "When looking up CA certificates, the OpenSSL library will first  
search the certificates in CAfile, then those in CApath."

The default CAfile is "cert.pem" and the default CApath is "certs" both located in the openssl version -d directory. See the crypto/ cryptlib.h header [1].

On my OS X platform depending on which version of OpenSSL I'm using, the OPENSSLDIR path would be one of these:

   /System/Library/OpenSSL
   /opt/local/etc/openssl

And neither of those uses a "certs" directory, they both use a "cert.pem" bundle instead:

   $ ls -l /System/Library/OpenSSL
   total 32
   lrwxrwxrwx  1 root  wheel    42 cert.pem -> ../../../usr/share/curl/ 
curl-ca-bundle.crt
   drwxr-xr-x  2 root  wheel    68 certs
   drwxr-xr-x  8 root  wheel   272 misc
   -rw-r--r--  1 root  wheel  9381 openssl.cnf
   drwxr-xr-x  2 root  wheel    68 private
   # the certs directory is empty

   $ ls -l /opt/local/etc/openssl
   total 32
   lrwxrwxrwx   1 root  admin    35 cert.pem@ -> ../../share/curl/curl- 
ca-bundle.crt
   drwxr-xr-x   9 root  admin   306 misc/
   -rw-r--r--   1 root  admin 10835 openssl.cnf

Notice neither of those refers to /etc/ssl/certs at all.

So the short answer is, yes, hard-coding /etc/ssl/certs as the path on OS X is incorrect and if setting /etc/ssl/certs as the path has the effect of replacing the default locations the verification will fail. Of course Apple patches the included version of OpenSSL starting with OS X 10.6 to look in Apple's keychain, but if you're using a MacPorts-built version that won't happen and still /etc/ssl/ certs will be wrong in both cases.

As for the hint in that wiki/CACertificates link above, that does seem odd to me as well.

A much better solution would be (if python simply MUST have a file) to export the system's list of trusted root certificates like so:

   security export -k \
     /System/Library/Keychains/SystemRootCertificates.keychain \
     -t certs -f pemseq > rootcerts.pem

   # the generated rootcerts.pem file is suitable for use with the
   # openssl verify -CAfile option (root certs)

   # Substituting SystemCACertificates for SystemRootCertificates
   # produces a file suitable for use with the
   # openssl verify -untrusted option (intermediate certs)

and then point python to that rootcerts.pem file. This file [2] may be helpful in understanding what's in SystemRootCertificates.keychain and SystemCACertificates.keychain. The intermediate certs may also need to be exported to a file and pointed to as well (a quick glance at the hgrc docs did not turn up an option for this).

--Kyle

[1] http://git.openssl.org/gitweb/?p=openssl.git;a=blob;f=crypto/cryptlib.h#l84
[2] <http://www.opensource.apple.com/source/security_certificates/security_certificates-32641/CertificateInstructions.rtf 
 >