patchverify-tag/verify-commit should exit unsuccessfully when signature is not trusted

Hello,

me and my colleague are struggling with automation of verifying git repositories and we have encountered that git verify-commit and verify-tag accepts untrusted signatures and exit successfully.

We have done some investigation of the GPG verification changes in git repository which I includes in this patch message. GPG results `TRUST_NEVER` and `TRUST_UNDEFINED` in raw output is treated as untrusted in git (U) and should not be accepted in verify-commit and verify-tag command.

In 434060ec6d verify-tag and verify-commit was centralized into check_signature function and good (G) and untrusted (U) signatures were marked as valid and exited successfully. In this commit it is incorrectly stated that this behavior is adopted from older verify-tag function however original verify-tag behavior was to return exit code from gpg process itself (removed in a4cc18f29).

Also rejecting untrusted (U) signature is the pull/merge with --verify-signatures behavior (defined in builtin/merge.c cmd_merge function and presented in eb307ae7bb).

The behavior of merge/pull --verify-signatures and verify-commit/verify-tag should be the same.

With regards, Vojtech Myslivec and Karel Koci

From c9c7b555da284c4f67fe36dc95d592644089544a Mon Sep 17 00:00:00 2001
From: Vojtech Myslivec <vojtech.myslivec@nic.cz>
Date: Tue, 31 Jul 2018 20:32:32 +0200
Subject: [PATCH] gpg-interface: Do not accept untrusted signatures

In 434060ec6d verify-tag and verify-commit was centralized into check_signature function and good (G) and untrusted (U) signatures were marked as valid and exited successfully. In this commit it is incorrectly stated that this behavior is adopted from older verify-tag function however original verify-tag behavior was to return exit code from gpg process itself (removed in a4cc18f29).

Also rejecting untrusted (U) signature is the pull/merge with --verify-signatures behavior (defined in builtin/merge.c cmd_merge function and presented in eb307ae7bb).

The behavior of merge/pull --verify-signatures and
verify-commit/verify-tag should be the same.
---
 gpg-interface.c          | 2 +-
 t/t7030-verify-tag.sh    | 4 ++--
 t/t7510-signed-commit.sh | 6 +++---
 3 files changed, 6 insertions(+), 6 deletions(-)

On Tue, Jul 31, 2018 at 10:05:22PM +0200, Vojtech Myslivec wrote:

I don't have strong feelings on your change one way or the other, but for automation it may be useful to use the --raw flag, which gives you the raw gpg output and much greater control. For example, you can require that a subkey is or is not used or require certain algorithms.

I will say that most signatures are untrusted in my experience, so unless people are using TOFU mode or making local signatures, git will exit nonzero for most signatures. I think the current status is to exit on a good signature, even if it isn't necessarily a valid signature.

I'm interested to hear others' thoughts on this.

On Wed, Aug 01, 2018 at 12:19:42AM +0000, brian m. carlson wrote:

I'd find it odd that we deviate from the gpg behavior, that returns 0 when verifyng an untrusted signatures. Tooling around gpg is generally difficult for this reason, but using the raw output should be enough to discard signatures with untrusted keys.

Another alternative is to use a keyring with trusted keys *only* and disable fetching keys from hkp servers. This way signature verification should fail.

Thanks, -Santiago.

On Tue, Jul 31, 2018 at 08:25:47PM -0400, Santiago Torres wrote:

This is not a deviation. GPG correctly recognizes difference between trusted, untrusted and unknown levels. git on the other hand does not. Well it did until the commit a4cc18f29. That one removed GPG exit code propagation.

I think that core of the problem is that git considers both 'TRUST_NEVER' and 'TRUST_UNDEFINED' as being the same. In git they both should result in error or at lest 'TRUST_NEVER' should (the same way as it does with just GPG).

There is an illustration of difference of untrusted and unknown key verification at the end of this mail. You can test it just by using your own GPG key (just use our fingerprint instead).

My and my colleague's expectations are that 'git verify-commit branch' would handle the commit on the tip of `branch` the same way as 'git merge --verify-signature branch'.

It is also confusing that untrusted key is ok but expired or missing one results in error. GPG's primary usage is for building web of trust. And I think that untrusted keys are more problematic in that sense than expired ones.

I also disagree with idea that using --raw should be only way how to check if we trust a key. It is handy in scripts for sure if some additional info about signature is required but I think that it should not be primary way to check for signature validity. Exit code should serve that purpose the same way as in case of GPG itself.

Small example would be that to replace current GPG behavior in portable way (shells such as dash or ash) it is required to do at least this:

set -e RES="$(git verify-tag --raw v1.1 2>&1)" ! echo "$RES" | grep -q "^\[GNUPG:\] TRUST_NEVER"

And that will for sure get more complicated when I use other trust levels and/or trust database and different trust model but direct.

Overall I thing that deviation from returning GPG exit code was not the best and I don't understand why it was done. Decision if signature validity is reported as error should be on GPG not on git specially when there is almost no configuration possibility for GPG in git.

Note that our proposed change is conservative and drawn as a unification of behavior of pull/merge and verify-*. To make it really optimal for automation it would be best to return exit code of GPG command.

With regards Karel Kočí

Illustration of difference between unknown and untrusted key:

$ gpg --sign file
$ mkdir -m 700 gpgt
$ export GNUPGHOME=$PWD/gpgt
$ echo 'trust-model:0:"direct' | gpgconfg --change-options gpg
$ gpg --recv-keys 2B1F70F95F1B48DA2265A7FAA6BC8B8CEB31659B
$ gpg --list-key
/home/cynerd/gpgt/pubring.kbx
-----------------------------
pub   rsa2048 2016-07-07 [SC]
      2B1F70F95F1B48DA2265A7FAA6BC8B8CEB31659B
uid           [ unknown] Karel Kočí (nic.cz) <karel.koci@nic.cz>
uid           [ unknown] Karel Kočí (cynerd.cz) <cynerd@email.cz>
uid           [ unknown] Karel Kočí (sh.cvut.cz) <k.koci@sh.cvut.cz>
uid           [ unknown] Karel Kočí (fel.cvut.cz) <kocikare@fel.cvut.cz>
sub   rsa2048 2016-07-07 [E]
sub   rsa4096 2016-09-27 [S]
$ gpg --verify file.gpg
gpg: Signature made Fri 03 Aug 2018 02:01:49 PM CEST
gpg:                using RSA key 46D715A0ED0E0C433CBF5963D83BD732AC2BD828
gpg:                issuer "karel.koci@nic.cz"
gpg: Good signature from "Karel Kočí (nic.cz) <karel.koci@nic.cz>" [marginal]
gpg:                 aka "Karel Kočí (cynerd.cz) <cynerd@email.cz>" [marginal]
gpg:                 aka "Karel Kočí (sh.cvut.cz) <k.koci@sh.cvut.cz>" [marginal]
gpg:                 aka "Karel Kočí (fel.cvut.cz) <kocikare@fel.cvut.cz>" [marginal]
gpg: WARNING: This key is not certified with sufficiently trusted signatures!
gpg:          It is not certain that the signature belongs to the owner.
$ echo $?
0
$ echo "2B1F70F95F1B48DA2265A7FAA6BC8B8CEB31659B:4:" | gpg --import-ownertrust
$ gpg --list-key
/home/cynerd/gpgt/pubring.kbx
-----------------------------
pub   rsa2048 2016-07-07 [SC]
      2B1F70F95F1B48DA2265A7FAA6BC8B8CEB31659B
uid           [  never ] Karel Kočí (nic.cz) <karel.koci@nic.cz>
uid           [  never ] Karel Kočí (cynerd.cz) <cynerd@email.cz>
uid           [  never ] Karel Kočí (sh.cvut.cz) <k.koci@sh.cvut.cz>
uid           [  never ] Karel Kočí (fel.cvut.cz) <kocikare@fel.cvut.cz>
sub   rsa2048 2016-07-07 [E]
sub   rsa4096 2016-09-27 [S]
$ gpg --verify file.gpg
gpg: WARNING: unsafe permissions on homedir '/home/cynerd/gpgt'
gpg: Signature made Fri 03 Aug 2018 02:01:49 PM CEST
gpg:                using RSA key 46D715A0ED0E0C433CBF5963D83BD732AC2BD828
gpg:                issuer "karel.koci@nic.cz"
gpg: Good signature from "Karel Kočí (nic.cz) <karel.koci@nic.cz>" [never]
gpg:                 aka "Karel Kočí (cynerd.cz) <cynerd@email.cz>" [never]
gpg:                 aka "Karel Kočí (sh.cvut.cz) <k.koci@sh.cvut.cz>" [never]
gpg:                 aka "Karel Kočí (fel.cvut.cz) <kocikare@fel.cvut.cz>" [never]
gpg: WARNING: We do NOT trust this key!
gpg:          The signature is probably a FORGERY.
$ echo $?
1

Oh wow. Sorry my assumption parted from looking at the code back in the day where this happens. I assumed git was quietly propagating the gpg error code and took it from there.

Now that I think about it though, verify tag can verify more than one tag. I assume that this would make it difficult to propagate individual errors in trusting. I honestly don't know what's the best way to modify this behavior then.

Thanks, -Santiago

On Fri, Aug 03, 2018 at 11:43:44AM -0400, Santiago Torres wrote:

I think the only sensible thing is to err on the conservative side, and return non-zero if we saw _any_ invalid signature.

I will note, though, that just checking the exit code of `verify-tag` isn't really that thorough. It shows that there was _a_ signature, but we don't know:

  - if it was an identity the user would expect to be signing tags

  - if it even matches the refname we used to find the tag

So I'd argue that any real verification needs to either have a human in the loop, or implement a custom policy based on reading the full output.

I know we (and you specifically Santiago) talked about this a while ago, and we ended up providing ways to get more information out of verify-tag, so that a tool could sit on top of that and implement more project-specific policy. I don't know offhand of any reusable tools that do so, though.

-Peff

On Fri, Aug 03, 2018 at 12:06:34PM -0400, Jeff King wrote:

I think that it would be even legit to exit on first tag verification failure. If someone wants to really verify all tags then it can be done with simple for loop. git that way does not have to solve problem of error combination.

That can be done just by using trust levels.

Can you explain this more? You mean that string (such as v1.1) used to lookup tag object is not verified as part of that object?

OK I thing that it was enough of abstract concepts from me. Let me explain you what am I trying to achieve. I am implementing feeds (in other words git repositories with packages) and package sources verification for OpenWRT. We (project Turris by CZ.NIC) are signing all our commits and all our tags. Now we are using small script that is verifying our repositories just before we run build. That is against keyring maintained on our server. I am trying to extend that to whole OpenWRT tree. That introduces problem of having a lot of keys and a lot of packages sharing same allowed keys. Fetching all allowed keys for every package from key servers is just slow because of that I have to share those between packages. In general there are two options. First one is to have cache of already fetched keys in armor format. Second one is to have one keyring and by setting all keys explicitly as never trusted with package given exception. Unfortunately first option can't be used because of one other request that is from our team. We don't want to be forced to update list of allowed contributors to our projects every time we have new colleague. Solution we come up with is to have central PGP key that signs our whole team and then verification is done by allowing GPG to fetch additional keys with max-cert-depth 1. That brings me to git verify-commit/tag that won't exit with zero code when signature is not trusted.

I have a solution for my problem (calling git verify-* twice and grep). That is not the point of this email nor this contribution. The point is that although GPG's behavior of exiting with 0 code when trust level is unknown is unexpected but in the end understandable, git's behavior of exiting with 0 code even if key is explicitly untrusted is just counterintuitive. I think that few people are still going to get nasty surprise when I consider that this change was introduced mid 2014 just after v2.4.0 and Ubuntu 14.04 lts (running even on part of our infrastructure) still contains version 1.9.1 and in that release it was acknowledging GPG exit code.

K.K.

On Sat, Aug 04, 2018 at 10:43:46AM +0200, Karel Kočí wrote:

Yeah, I'd be fine with that.

I may be showing my PGP cluelessness here, but I thought trust levels were about saying "to what degree do I think this uid matches this key". Or are you proposing feeding gpg a fixed trust-db pre-seeded with keys that are allowed to sign?

I suppose that would work, but:

  - is it much easier than just verifying the uid from gpg output
    against a trusted list?

  - it mixes authentication and authorization. I.e., you lose the
    ability to know about a case of "yes, this signature is valid by
    this person, but they are not an authorized tagger".

Definitely for some use cases that is fine (and easier still is to just not even have disallowed keys in your keyring). But I don't think it's a general solution.

Yes. The signature is just over the tag object contents itself. That object does contain a "tag" field, but it may or may not be the tag you're expecting. Git _could_ confirm that when you looked up a tag via refs/tags/v1.1 that the tag object we found has "tag v1.1" in it. But that's not always possible (you might feed Git a resolved name, for example).

OK, that makes some sense (and I guess answers my "how would you use the trustdb" question above).

FWIW, I'm on board with returning non-zero in any case where gpg would.

-Peff

On Wed, Aug 08, 2018 at 07:04:56PM -0400, Jeff King wrote:

I think that's probably the best solution overall. There's a bug report in Debian (https://bugs.debian.org/895048) that requests that behavior instead of the status quo, and also it's the behavior that's documented:

       gpg.program
           Use this custom program instead of "gpg" found on $PATH when
           making or verifying a PGP signature. The program must support
           the same command-line interface as GPG, namely, to verify a
           detached signature, "gpg --verify $file - <$signature" is
           run, and the program is expected to signal a good signature
           by exiting with code 0, and to generate an ASCII-armored
           detached signature, the standard input of "gpg -bsau $key" is
           fed with the contents to be signed, and the program is
           expected to send the result to its standard output.

"brian m. carlson" <sandals@crustytoothpaste.net> writes:

FWIW, I am not married to the current behaviour. I would not be surprised if it mostly came by accident and not designed.

The last bit is a bit questionable; I think you are reading too much into the description.

A substitute for gpg.program MUST signal good (or not good) signature the same way as gpg would with its exit code---that is all the description says. It does not say anything about how that exit code affects the exit status of "tag --verify" and friends that called gpg.program.

On Wed, Aug 08, 2018 at 05:59:43PM -0700, Junio C Hamano wrote:

Since apparently I was the author of the commit that changed the behavior originally, let me simply say that I was not aware that gpg signalled the correctness of a signature by its exit status when I wrote that patch. If I had known that, I would have deferred to gpg in my change. My goal was consistency between verify-tag and verify-commit, and in retrospect I probably made the wrong decision.

I agree that the description doesn't specifically say that. In fact, it doesn't explicitly say that it must exit nonzero on a bad signature, although I agree with your interpretation that that would be logical (and, AIUI, the behavior of gpg).

However, I would assert that we do want Git's verification to exit successfully on a good signature and unsuccessfully on a bad signature, and deferring to gpg may be the most robust way of ensuring that.

On Thu, Aug 09, 2018 at 01:43:02AM +0000, brian m. carlson wrote:

OK, so it seems like we're all in agreement now.

What next?

There was a patch at the start of this thread, but it specifically checks for "sigc->result == U". That's probably OK, since I think it restores the behavior in earlier versions of Git. But I wonder if we should simply be storing the fact that gpg exited non-zero and relaying that. That would fix this problem and truly make the rule "if gpg reported an error, we propagate that".

-Peff

Jeff King <peff@peff.net> writes:

Yeah, I like that.  Something like this, perhaps?  Points to note:

 * status gets the return value from verify_signed_buffer(), which
   essentially is what wait_or_whine() gives us for the "gpg
   --verify" process.

 * Even if status says "failed", we still need to parse the output
   to set sigc->result.  We used to use sigc->result as the sole
   source of our return value, but now we turn 'status' into 'bad'
   (i.e. non-zero) after parsing and finding it is not mechanically
   good (which is the same criteria as we have always used before).
   An already bad status is left as bad.

 * And we return 'status'.

If we choose to blindly trust the exit status of "gpg --verify" and not interpret the result ourselves, we can lose the "smudge status to be bad if not G/U" bit, which I offhand do not think makes much difference either way. I just left it there because showing what can be removed and saying it can be dropped is easier than showing the result of removal and saying it can be added--simply because I need to describe "it" if I go the latter route.

 gpg-interface.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

On Thu, Aug 09, 2018 at 08:30:25AM -0700, Junio C Hamano wrote:

Yeah, this is exactly what I had in mind. And the size of the code change is much smaller than I feared. The case that I thought might be complicated is still reading the output after we've seen the non-zero status, but the existing "if (status && !gpg_output.len)" covers that.

I guess leaving it serves as a sort of cross-check if gpg would return a zero exit code but indicate in the status result that the signature was not good. Sort of a belt-and-suspenders, I guess (which might not be that implausible if we think about somebody wrapping gpg with a sloppy bit of shell code that loses the exit code -- it's their fault, but it might be nice for us to err on the conservative side).

Probably it should go back to just "result != G" then, though (thus bringing the whole conversation full circle :) ).

I could live with or without it, though.

-Peff

Jeff King <peff@peff.net> writes:

OK, this time a real log message.

-- >8 --
Subject: [PATCH] gpg-interface: propagate exit status from gpg back to the callers

When gpg-interface API unified support for signature verification codepaths for signed tags and signed commits in mid 2015 at around v2.6.0-rc0~114, we accidentally loosened the GPG signature verification.

Before that change, signed commits were verified by looking for "G"ood signature from GPG, while ignoring the exit status of "gpg --verify" process, while signed tags were verified by simply passing the exit status of "gpg --verify" through. The unified code we currently have ignores the exit status of "gpg --verify" and returns successful verification when the signature matches an unexpired key regardless of the trust placed on the key (i.e. in addition to "G"ood ones, we accept "U"ntrusted ones).

Make these commands signal failure with their exit status when underlying "gpg --verify" (or the custom command specified by "gpg.program" configuration variable) does so. This essentially changes their behaviour in a backward incompatible way to reject signatures that have been made with untrusted keys even if they correctly verify, as that is how "gpg --verify" behaves.

Note that the code still overrides a zero exit status obtained from "gpg" (or gpg.program) if the output does not say the signature is good or computes correctly but made with untrusted keys, to catch a poorly written wrapper around "gpg" the user may give us.

We could exclude "U"ntrusted support from this fallback code, but that would be making two backward incompatible changes in a single commit, so let's avoid that for now. A follow-up change could do so if desired.

Helped-by: Vojtech Myslivec <vojtech.myslivec@nic.cz>
Helped-by: brian m. carlson <sandals@crustytoothpaste.net>
Helped-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 gpg-interface.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

On Thu, Aug 09, 2018 at 11:40:27AM -0700, Junio C Hamano wrote:

Thanks, the explanation and the patch look good to me.

I'm on the fence over whether a follow-up patch to take away the "U" is worth it. In practice the code should never trigger either way, since gpg would already have exited non-zero in such a case.

-Peff

On Thu, Aug 09, 2018 at 11:40:27AM -0700, Junio C Hamano wrote:

This looks great to me.  Thanks.

On 9.8.2018 20:40, Junio C Hamano wrote:

Now it is possible to achieve git verify-tag/verify-commit exits unsuccessfully when signatures are not trusted. I would like to introduce some tests for this behavior to prevent this problem in the future.

Thank you all for discussion and for solving the issue.

Vojtech and Karel

Santiago Torres <santiago@nyu.edu> writes:

I am not sure if changing the behaviour is warranted to begin with. Especially when somebody wants to get more than a single bit. I think our single bit signals "does signature compute correctly?" without taking "how much trust do we place in that particular key?" into account. As Brian mentioned in an earlier response, those who want to take different factors like the level of trust etc. into account can read from "--raw", using the exit code only for "does signature compute correctly?" and nothing else. That would allow them to perform any validation underlying GNUPG let them to do.