[PATCH v4] http: add http.sslVerifyStatus to check stapled OCSP responses

From: Grayson Gordon <graysongordon1@gmail.com>

git asks libcurl to verify the peer certificate and the hostname, but it never sets CURLOPT_SSL_VERIFYSTATUS, so the "Certificate Status Request" TLS extension is never requested and any stapled OCSP response the server does send is ignored.

On an OpenSSL-linked build this is silent. OpenSSL hands the stapled response to the application and takes no view on it: SSL_CTX_set_tlsext_status_cb(3) says the callback "should determine whether the returned OCSP response(s) are acceptable or not", and libcurl only installs that callback when CURLOPT_SSL_VERIFYSTATUS is set. So git will fetch from a server whose own staple says its certificate has been revoked.

A GnuTLS-linked build behaves differently, and the difference does not come from curl. GnuTLS consults a stapled response inside gnutls_certificate_verify_peers(), so the failure surfaces through the verifypeer branch of curl's GnuTLS backend (lib/vtls/gtls.c) whether or not CURLOPT_SSL_VERIFYSTATUS was ever set. The same git, against the same server, therefore enforces revocation or not depending only on how its libcurl was built. That difference is documented here rather than papered over: this option turns the check on where the backend needs asking, and setting it to false does not turn the check off on GnuTLS.

Add an http.sslVerifyStatus boolean that sets CURLOPT_SSL_VERIFYSTATUS. Because http_options() is the collect_fn of a urlmatch config, the per-URL form works with no further changes:

    git config http.https://example.com/.sslVerifyStatus true

It defaults to false, and has to. The option is fail-closed: libcurl fails verification when the server staples nothing at all, so turning this on globally would break every remote that does not staple.

Leaving the default to libcurl is not an option either. The same complaint was raised there in https://github.com/curl/curl/issues/15483 and closed as intentional ("Marked as enhancement since this was done on purpose"), with the observation that stapling is expected to see less use as Let's Encrypt drops OCSP support. If the check is to be reachable at all, the lever has to come from the application.

If the TLS backend cannot check the staple, curl_easy_setopt() returns CURLE_NOT_BUILT_IN. Fail loudly there rather than carrying on, since silently not checking is precisely what this option exists to prevent.

CURLOPT_SSL_VERIFYSTATUS has been available since libcurl 7.41.0, well below the 7.61.0 floor documented in INSTALL, so no version guard is needed.

The tests go in t5551 and run only in its https pass, which t5559 provides by sourcing t5551 with LIB_HTTPD_SSL set; that is the only https server the suite has. They exercise the fail-closed path, which needs no CA and no OCSP responder: lib-httpd's server staples nothing, so enabling the option has to turn a working fetch into a failing one. Verified against an unpatched build, where exactly the two assertions that depend on the new option fail and the two controls still pass, and against OpenSSL, GnuTLS and mbedTLS-linked builds of libcurl.

Signed-off-by: Grayson Gordon <graysongordon1@gmail.com>
---

v4: drop the new test script. The four tests now live in t5551, keyed on $HTTPD_PROTO so they run in the https pass (t5559) only.

To answer the question I skipped past in v3: no, we do not need a new script. t5559 is t5551 run with LIB_HTTPD_SSL set, and it is the only https server the suite has, so a new script would have had to stand up a second one to reach the same place. Nothing is left over from t5567 or t5568 and no test number is spent.

That also means the t/meson.build hunk is not squashed in. t5551 is already listed there. Adding t5568 to the list now would break configure the other way round, since the list is checked against ls in both directions and errors with "Test files configured, but not found".

On the lost exit status: changed, but to test_might_fail rather than a bare &&. The ls-remote in the prerequisite is expected to fail, that is the premise of the test, so && short-circuits on the expected failure and leaves the prerequisite unsatisfied. Run against the https server both ways:

    bare &&              ok 51 # skip http.sslVerifyStatus=true fails
                                 without a staple (missing SSL_VERIFYSTATUS)
    test_might_fail      ok 51 - http.sslVerifyStatus=true fails
                                 without a staple

The first still reports "passed all 61 test(s)", which is the failure mode the prerequisite was written to avoid. test_might_fail keeps the chain intact and says the status is ignored on purpose.

Also dropped the "ls-remote succeeds with http.sslVerifyStatus unset" test. It was a control for the standalone script, and in t5551 the surrounding tests already exercise that URL throughout.

The rationale that sat in an in-code comment in v3 is in the log now, per the earlier review.

Verified: t5551 over plain http and t5559 over https both pass all 61
tests, with the four new ones skipping on the former and running on the
latter.
 Documentation/config/http.adoc | 17 +++++++++++++++++
 http.c                         | 10 ++++++++++
 t/t5551-http-fetch-smart.sh    | 29 +++++++++++++++++++++++++++++
 3 files changed, 56 insertions(+)