{"thread":{"id":"27247","subject":"[Tagging Commits] feedback / discussion request","startedAt":"2011-05-03T23:36:51Z","lastAt":"2011-05-05T20:30:16Z","messageCount":7,"participants":["Richard Peterson","Sverre Rabbelier","Jeff King","Michael J Gruber"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"166979","messageId":"BANLkTik5-Ygh0YwN=j+ibLhP6==ots_MXQ@mail.gmail.com","threadId":"27247","inReplyTo":null,"subject":"[Tagging Commits] feedback / discussion request","fromName":"Richard Peterson","fromEmail":"richard@rcpeterson.com","sentAt":"2011-05-03T23:36:51Z","receivedAt":"2011-05-03T23:36:51Z","isPatch":false,"sender":{"key":"richard@rcpeterson.com","avatar":"https://gravatar.com/avatar/cc6d791c99e8302288a850cd4477a28bb7782b1133b03a45f7b7e0f93624390f?d=mp&s=160"},"body":"This is a different idea from that discussed in\nhttp://marc.info/?t=123879411100002&r=1&w=2.\n\nI'm going to present some use cases for signing commits (instead of just\ntags). Then I'll present an idea for implementation. I hope to implement\nthis as described, along with Eric Ritz. We would appreciate any insight you\nmay have - we want to help, not waste our time or anybody else's.\n\nFirst, Linus has argued against signing commits in this thread:\nhttp://marc.info/?t=123879411100002&r=1&w=2.  He claims it is pointless to\nsign individual commits, as opposed to signing just the tip of the tree\n(tags). For many use cases, that's true. But read on.\n\n\nHere are some possible semantics you could assign to signing a commit hash:\n\n* Making a verifiable claim of authorship of a commit\n* Making a verifiable claim to have reviewed a commit or set of commits\n* Making a verifiable claim to have approved a commit or set of commits for\nsome purpose\n* Making some other verifiable claim about a commit TBD by your workflow\n* Making a verifiable claim to have reviewed or approved the entire tree\nunder the commit\n\nClaiming to have reviewed or approved the entire tree is useful in many\ncases. It's great for something like the Linux kernel.  If you've got a tip\nsigned by Linus, you've got the kernel. You don't need to care what's merged\nin under that, as long as it's signed at the top. It's like an MD5 checksum\non a download. You don't care what mirror you download an ISO from, as long\nas the computed hash matches the authoritative hash.\n\nSemantically, someone who signs a tree takes responsibility for everything\nincluded in that tree, to whatever degree that applies in their project.\n\nNow *technically* signing the tree may be equivalent to signing an\nindividual commit, but don't get wrapped up in that. Stick to the semantics\nwith me.\n\nImagine the following scenario to help justify the other use cases above.\n\nThere are 200 developers working on a financial trading system, and each of\nthem has the opportunity to slip malicious code into the project. When the\nfinal release is prepared, the project lead signs the tip commit, thus\nsigning the whole tree. Now it is discovered that someone did slip some\nmalicious code in.  How do you audit the system? Could higher levels of\nindividual accountability have discouraged this scenario?\n\nI've seen it argued that a proper SSH setup and user management are the key.\nThese are good for security and access control, but not for some durable\nform of accountability.\n\nIf each commit is individually signed, the authorship claims have teeth. In\nour scenario above, a single signature at the tip of the tree did no good in\nterms of accountability. You can blame the guy on top, but is it really\nreasonable that he review every line? However, if each commit were signed,\ntracing the malicious code would be simple. If a reviewer had been required\nto sign every commit, or maybe every range of commits (signing\n186fa861..8645b061, for instance), then there could be a double layer of\naccountability. This kind of \"hard\" accountability can be valuable in\nsensitive projects. I work on such projects.\n\nSome people might not see the use of this kind of auditability. I'll tell\nyou though - I work in a large organization that uses SVN because of some\nkind of perceived auditability. They shy away from Git because it's\n\"distributed\" and therefore not auditable.  Of course that's a\nphony-baloney, ignorant argument, but the point is that the need for\nauditability is there.\n\n\nSo how do these semantics line up with Git?\n\nIt seems that creating a signed tag is the same as signing a commit.  There\nare a few problems, though.  Tags don't provide a secure means of asserting\nthe type of signature being applied to the commit hash. That is - is the\nhash signed because someone is claiming authorship? Because they are\nasserting the integrity of the entire tree? Because they have reviewed the\ncode? Because they reviewed a certain subset of the tree? Of course there's\nalso the issue that tags live in a cluttered namespace. Signing a commit is\nessentially a different thing from providing a name for a commit. Using tags\njust to sign commits requires a glut of tag names.\n\nI propose expanding the concept of tags, or alternately creating a new\nconcept which subsumes the existing tag concept. I'll call this new concept\na \"sig\" for the purposes of this discussion. The concept of a sig cross-cuts\nthe concept of a tag.\n\nA tag signs the commit hash. A sig signs a SHA1-based absolute commit\nreference with a (possibly null) string concatenated to it. For instance, a\nsig might sign the following string:\n\n\"0b9deecf625677cf44058a42c2abd7add5167e81^0 author\"\nwhich would mean that the signor is claiming authorship of that individual\ncommit. (Suggestions for notating a single commit are welcome. \"^0\" seemed\nnatural.)\n\nor\n\n\"5ae6f5ca2f70bd7d5ca88c20f2be62bf3844af73..0b9deecf625677cf44058a42c2abd7add5167e81\nreviewer\"\nwhich means the signor has reviewed that particular chain of commits.\n\nSigning the string\n\"0b9deecf625677cf44058a42c2abd7add5167e81\"\nwould be the same as signing the entire tree for that hash, which is what\nhappens in a tag.\n\nA signed tag would essentially be a name associated with a sig - not too far\noff from how it works now. A lightweight tag would be a name not associated\nwith a sig - again, about how it is now.\n\n\nThis strategy has several benefits:\n\n* It is separable from the commit tree.  Linus argued against including\ncommit signatures within commits. This solution doesn't do that. If someone\nwants to ditch all signatures and publish the tree, they can.\n\n* It is extensible. Standard strings like \"author\" and \"reviewer\" might get\nbuilt-in support, but there is nothing preventing adding custom signature\ntypes to meet your workflow needs.  Someone could even add something like a\ndatestamp, if the need arose. In addition, there would be no limit to the\nnumber of \"author\" or \"reviewer\" sigs that could point to a single commit.\nGreat for pair programming, group code reviews, or other workflows.\n\n* It can probably be implemented cleanly without breaking the existing tags.\n\nI see a few potential issues:\n\n* What on earth does it mean to tag a range of commits? With commit ranges\nbeing siggable, and tags containing sigs, what does it mean to tag a range\nof 10 commits, for instance? Is that desirable? Does it make any sense\nwhatsoever? Does it hurt anything if it happens?\n\n* Performance? I think it would be extremely quick to verify a bunch of\nsigs, but I don't know. Maybe I'm not thinking clearly about it.\nFortunately, sigs can be ignored entirely and need not affect things.\n\n* Any others issues?\n\nThank you, and please give feedback. I'm no git pro - just a guy with an\nidea. Based on your feedback, Eric and I will steer our implementation.\n\nRichard Peterson\n"},{"id":"166981","messageId":"BANLkTimTmkufMnY5dJtDD6BWxs=vsDTygA@mail.gmail.com","threadId":"27247","inReplyTo":"BANLkTik5-Ygh0YwN=j+ibLhP6==ots_MXQ@mail.gmail.com","subject":"Re: [Tagging Commits] feedback / discussion request","fromName":"Sverre Rabbelier","fromEmail":"srabbelier@gmail.com","sentAt":"2011-05-03T23:49:46Z","receivedAt":"2011-05-03T23:49:46Z","isPatch":false,"sender":{"key":"srabbelier@gmail.com","avatar":"https://avatars.githubusercontent.com/u/3098?v=4"},"body":"Heya,\n\nOn Wed, May 4, 2011 at 01:36, Richard Peterson <richard@rcpeterson.com> wrote:\n> Thank you, and please give feedback. I'm no git pro - just a guy with an\n> idea. Based on your feedback, Eric and I will steer our implementation.\n\nHave you looked at git notes? They seem relevant. You could use them\nto sign commits after the fact, and by multiple people, etc.\n\n-- \nCheers,\n\nSverre Rabbelier\n"},{"id":"167000","messageId":"20110504084212.GB8512@sigill.intra.peff.net","threadId":"27247","inReplyTo":"BANLkTik5-Ygh0YwN=j+ibLhP6==ots_MXQ@mail.gmail.com","subject":"Re: [Tagging Commits] feedback / discussion request","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2011-05-04T08:42:13Z","receivedAt":"2011-05-04T08:42:13Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, May 03, 2011 at 07:36:51PM -0400, Richard Peterson wrote:\n\n> Here are some possible semantics you could assign to signing a commit hash:\n> \n> * Making a verifiable claim of authorship of a commit\n> * Making a verifiable claim to have reviewed a commit or set of commits\n> * Making a verifiable claim to have approved a commit or set of commits for\n> some purpose\n> * Making some other verifiable claim about a commit TBD by your workflow\n> * Making a verifiable claim to have reviewed or approved the entire tree\n> under the commit\n\nYeah, all of those make sense in certain workflows. But with the\nexception of authorship verification, they are not things you would want\nto do at _commit_ time, but rather something you say later about a\ncommit. So I think fundamentally you are not interested in adding\nsignatures to git commits themselves, but rather about making statements\nabout commits that happen to be signed. Which is good, because your\nproblem is much easier. :)\n\nThe nice thing is that git gives you a stable, cryptographically\nverifiable identifier for the commit. So all you have to do is mention\nit along with some metadata, sign it, and then store it somewhere.\n\nThe first two parts can be as simple as something like:\n\n  (git rev-parse --verify HEAD\n   echo \"I reviewed this and it meets some standard X.\"\n  ) | gpg --sign\n\nwhere probably you would want to define some kind of parsable metadata\nformat for your particular workflow.\n\nFor storage, you basically have three options:\n\n  1. Somewhere completely outside of git. There's no reason this needs\n     to be stored in git at all, depending on your workflow. It may be\n     simpler to keep it in some database related to your review system\n     (in fact, you may not doing anything cryptographic at all, but\n     merely have a separate review system with a central database that\n     mentions commits by sha1).\n\n  2. In git tags. You can already do this with:\n\n       git tag -s -m \"I reviewed this\" HEAD\n\n     But tags aren't a good fit for a workflow that signs every commit\n     (some of them perhaps even multiple times!). You end up with lots\n     of tag refs.\n\n  3. In git notes. You can do something like:\n\n       (git rev-parse --verify HEAD\n        echo \"I reviewed this\"\n       ) | gpg --sign -a |\n       git notes add -F - HEAD\n\n     though you'd probably want to be a little more complex, and handle\n     lists of signed notes for each commit. And you may want to store\n     these in a separate notes ref from the default one.\n\n     The advantage of notes are that they are designed for lots of\n     per-commit storage, and can be accessed fairly efficiently.\n\nSo now you have your review storage system (or authorship, or whatever\nmetadata you want to stick in there). You can peek at it manually, of\ncourse, when you suspect something is not right. But you probably also\nwant to do automatic things, like making sure nothing goes into some\nbranch \"foo\" that isn't signed with an authorship note.\n\nAssuming you are storing with git notes (if you are using some external\nsystem, replace the call to git-notes below with whatever database\nlookup you would want), you could use a pre-receive hook that did\nsomething like:\n\n  git rev-list $old..$new |\n  while read commit; do\n    git notes show $commit >tmp\n    gpg --verify tmp >data 2>siginfo || die \"$commit: signature is bad\"\n    # ugh, is there really no better way to get this info from gpg?\n    perl -lne 'print $1 if /Good signature from \"(.*)\"/ siginfo >signer\n    git show --format=\"%an <%ae>\" $commit >author\n    cmp author signer || die \"$commit: signer and committer don't match\"\n    test \"`head -n 1 data`\" = $commit ||\n      die \"$commit: signed commit does not match\"\n  done\n\nAnd obviously that is hacked together and you would want something more\nrobust, and you'd need to handle the web of trust for the signing keys\nsomehow (though I think that is external to this script, and is about\nsetting up the desired keyring). But I hope it gives a sense of what you\ncan do. You could also replace gpg completely with something like\nopenssl using x.509 certs, if that makes more sense to your\norganization.\n\nDevelopers would have to make a note and push their notes tree first,\nand then push their actual commits into a branch (and you might want to\ndo some verification on the notes they push, like checking that entries\nfor commit $X actually contains signatures for $X, or that the signer\nidentity matches some ssh credential, or that the pusher isn't deleting\nany signatures or erasing note history).\n\nI suspect you already thought through some of this already. But I wanted\nto start with first principles, because I really don't think this is a\n_git_ problem as much as it is a _workflow_ problem. So it's important\nto first define the workflow you want, and then think about how git can\nhelp. Stable commit identifiers already provide much of the basis. I\nthink notes provide a nice storage format that is efficient and\npush-able to other repos (though in a centralized shop, some other\ndatabase might make sense, too). What really remains to be done is:\n\n  1. Define the metadata format that encapsulates what you want to say\n     about commits.\n\n  2. Write scripts to help developers and reviewers make these notes,\n     and verify them.  Write hooks to implement policy on letting\n     commits into certain branches, as above.\n\nAnd both of those happen outside of git (though if you write them in a\ngeneric enough form, I'm sure people on the list would be very happy to\nsee them shared).\n\n> There are 200 developers working on a financial trading system, and each of\n> them has the opportunity to slip malicious code into the project. When the\n> final release is prepared, the project lead signs the tip commit, thus\n> signing the whole tree. Now it is discovered that someone did slip some\n> malicious code in.  How do you audit the system? Could higher levels of\n> individual accountability have discouraged this scenario?\n\nI like this example. It shows that signing a commit is not really\nmeaningful by itself; you have to understand the semantics of that\nsignature (and maybe they're included as comments in the tag object, or\nmaybe it is assumed by your organization's workflow).\n\nIn the case of the kernel, Linus signing a commit with a tag implicitly\nmeans \"I think what is in this tree and everything before it is good, so\nyou should feel comfortable using it\" (or at least insofar as you trust\nLinus).\n\nBut it doesn't have to be that way. Your project lead signing may mean\n\"this is good and we should ship it\". But developers signing commits may\nsimply mean \"I promise that I wrote the changes between this commit's\ntree and its parent\". Those are all signatures of commits, but they mean\nvery different things; the key is adding metadata to know which is\nwhich.\n\n> I've seen it argued that a proper SSH setup and user management are the key.\n> These are good for security and access control, but not for some durable\n> form of accountability.\n\nRight. You are trusting the server's records, not cryptography. The main\nadvantage is that it's efficient and easy to set up. :)\n\n> It seems that creating a signed tag is the same as signing a commit.  There\n> are a few problems, though.  Tags don't provide a secure means of asserting\n> the type of signature being applied to the commit hash. That is - is the\n> hash signed because someone is claiming authorship? Because they are\n> asserting the integrity of the entire tree? Because they have reviewed the\n> code? Because they reviewed a certain subset of the tree? Of course there's\n> also the issue that tags live in a cluttered namespace. Signing a commit is\n> essentially a different thing from providing a name for a commit. Using tags\n> just to sign commits requires a glut of tag names.\n\nAgain, metadata. Say what you mean in the free-form content of the tag.\nFor the kernel, there is nothing to be said. Linus signing tags has a\nwell-known meaning in the community. But in an organization signing for\na lot of different reasons, you would want the signed data to say why it\nwas signed.\n\n> I propose expanding the concept of tags, or alternately creating a new\n> concept which subsumes the existing tag concept. I'll call this new concept\n> a \"sig\" for the purposes of this discussion. The concept of a sig cross-cuts\n> the concept of a tag.\n> \n> A tag signs the commit hash. A sig signs a SHA1-based absolute commit\n> reference with a (possibly null) string concatenated to it. For instance, a\n> sig might sign the following string:\n\nA tag can already include arbitrary data.\n\nIn fact, tags basically do what you want already; it's just that storing\none tag ref per commit is going to be ugly. It might make sense to\nreplace the ad-hoc gpg signatures I used in my examples above with tag\nobjects, and then store the tag object in the notes tree.\n\n> \"0b9deecf625677cf44058a42c2abd7add5167e81^0 author\"\n> which would mean that the signor is claiming authorship of that individual\n> commit. (Suggestions for notating a single commit are welcome. \"^0\" seemed\n> natural.)\n\nSee? You're defining metadata now. :)\n\n> * What on earth does it mean to tag a range of commits? With commit ranges\n> being siggable, and tags containing sigs, what does it mean to tag a range\n> of 10 commits, for instance? Is that desirable? Does it make any sense\n> whatsoever? Does it hurt anything if it happens?\n\nIt's slightly more efficient. If I wrote 10 commits, I can either sign\neach individually saying \"I wrote this\", or I can make a single\nsignature showing them all. The tradeoff is that parsing and verifying\nmetadata becomes a lot more complex. But crytographically speaking, a\nrange is not ambiguous;\n\n> * Performance? I think it would be extremely quick to verify a bunch of\n> sigs, but I don't know. Maybe I'm not thinking clearly about it.\n> Fortunately, sigs can be ignored entirely and need not affect things.\n\nCompared to usual git operations, no, it's not quick. But you don't have\nto verify all the time. You can verify commits when they enter your\nrepo, or when you're interested in some aspect of them, or when you\nsuspect something fishy is going on. You don't have to do it on every\nrev-list.\n\n-Peff\n"},{"id":"167006","messageId":"4DC11A8F.5020408@drmicha.warpmail.net","threadId":"27247","inReplyTo":"BANLkTimTmkufMnY5dJtDD6BWxs=vsDTygA@mail.gmail.com","subject":"Re: [Tagging Commits] feedback / discussion request","fromName":"Michael J Gruber","fromEmail":"git@drmicha.warpmail.net","sentAt":"2011-05-04T09:21:19Z","receivedAt":"2011-05-04T09:21:19Z","isPatch":false,"sender":{"key":"git@grubix.eu","avatar":"https://avatars.githubusercontent.com/u/233215?v=4"},"body":"Sverre Rabbelier venit, vidit, dixit 04.05.2011 01:49:\n> Heya,\n> \n> On Wed, May 4, 2011 at 01:36, Richard Peterson <richard@rcpeterson.com> wrote:\n>> Thank you, and please give feedback. I'm no git pro - just a guy with an\n>> idea. Based on your feedback, Eric and I will steer our implementation.\n> \n> Have you looked at git notes? They seem relevant. You could use them\n> to sign commits after the fact, and by multiple people, etc.\n> \n\nExactly. Sign and store sig in refs/notes/sigs:\n\ngit rev-parse <commit> | gpg -sa | git notes --ref=sigs append -F- <commit>\n\nVerify:\n\ngit notes --ref=sigs show <commit> | gpg\n\nYou can sign any object (blob, tree...) that way, of course.\n\nEverything else (meaning of this sig, just like the meaning of a signed\ntag or a s-o-b line) is a matter of project policy.\n\nMichael\n"},{"id":"167115","messageId":"BANLkTinCxzXCmmtxXSM7=+yeve2hhLSYNQ@mail.gmail.com","threadId":"27247","inReplyTo":"20110504084212.GB8512@sigill.intra.peff.net","subject":"Re: [Tagging Commits] feedback / discussion request","fromName":"Richard Peterson","fromEmail":"richard@rcpeterson.com","sentAt":"2011-05-05T15:39:41Z","receivedAt":"2011-05-05T15:39:41Z","isPatch":false,"sender":{"key":"richard@rcpeterson.com","avatar":"https://gravatar.com/avatar/cc6d791c99e8302288a850cd4477a28bb7782b1133b03a45f7b7e0f93624390f?d=mp&s=160"},"body":"First off, thanks for the awesome response, Peff, and Sverre and\nMichael as well. Great stuff, and plenty that I had not thought of.\n\nOn Wed, May 4, 2011 at 4:42 AM, Jeff King <peff@peff.net> wrote:\n> On Tue, May 03, 2011 at 07:36:51PM -0400, Richard Peterson wrote:\n>\n>> Here are some possible semantics you could assign to signing a commit hash:\n>>\n>> * Making a verifiable claim of authorship of a commit\n>> * Making a verifiable claim to have reviewed a commit or set of commits\n>> * Making a verifiable claim to have approved a commit or set of commits for\n>> some purpose\n>> * Making some other verifiable claim about a commit TBD by your workflow\n>> * Making a verifiable claim to have reviewed or approved the entire tree\n>> under the commit\n>\n> Yeah, all of those make sense in certain workflows. But with the\n> exception of authorship verification, they are not things you would want\n> to do at _commit_ time,\n\nEven authorship could be claimed after commit time too, for that matter.\n\n> but rather something you say later about a\n> commit. So I think fundamentally you are not interested in adding\n> signatures to git commits themselves, but rather about making statements\n> about commits that happen to be signed. Which is good, because your\n> problem is much easier. :)\n>\n> The nice thing is that git gives you a stable, cryptographically\n> verifiable identifier for the commit. So all you have to do is mention\n> it along with some metadata, sign it, and then store it somewhere.\n>\n> The first two parts can be as simple as something like:\n>\n>  (git rev-parse --verify HEAD\n>   echo \"I reviewed this and it meets some standard X.\"\n>  ) | gpg --sign\n>\n> where probably you would want to define some kind of parsable metadata\n> format for your particular workflow.\n>\n> For storage, you basically have three options:\n>\n>  1. Somewhere completely outside of git. There's no reason this needs\n>     to be stored in git at all, depending on your workflow. It may be\n>     simpler to keep it in some database related to your review system\n>     (in fact, you may not doing anything cryptographic at all, but\n>     merely have a separate review system with a central database that\n>     mentions commits by sha1).\n\nI see this as a useful option considering some poor souls in my\norganization use Subversion, and we could factor out the audit / review\nworkflow to not depend on a single version control system.\n\nOn the other hand, it makes sense to keep data actually within git if it\nuses a git internal identifier as a key, and its useful to operate on it with\nthe git tool set.\n\n>\n>  2. In git tags. You can already do this with:\n>\n>       git tag -s -m \"I reviewed this\" HEAD\n>\n>     But tags aren't a good fit for a workflow that signs every commit\n>     (some of them perhaps even multiple times!). You end up with lots\n>     of tag refs.\n\nRight - one of the reasons I don't like tags for this. Tags really just don't\nfit the bill, unfortunately.\n\n>\n>  3. In git notes. You can do something like:\n>\n>       (git rev-parse --verify HEAD\n>        echo \"I reviewed this\"\n>       ) | gpg --sign -a |\n>       git notes add -F - HEAD\n>\n>     though you'd probably want to be a little more complex, and handle\n>     lists of signed notes for each commit. And you may want to store\n>     these in a separate notes ref from the default one.\n\nI had looked at this option, but had failed to see the usefulness of using\na different ref. I was worried about cluttering things up, overloading the\nintended purpose of notes, and so forth. I wasn't really sure if notes were\nintended to be general purpose storage for systematic, structured data.\n\nMy inclination was to do this outside notes, or even in a parallel\nimplementation to notes, factoring out the common parts. I suppose that\nlooking at notes as somewhat of a free-for-all obviates this need. Is this\nreally what notes are for?\n\n>\n>     The advantage of notes are that they are designed for lots of\n>     per-commit storage, and can be accessed fairly efficiently.\n\nThat was my other concern about notes - performance. Not sure how\nnotes are stored, but I certainly trust you that they're efficient.\n\n>\n> So now you have your review storage system (or authorship, or whatever\n> metadata you want to stick in there). You can peek at it manually, of\n> course, when you suspect something is not right. But you probably also\n> want to do automatic things, like making sure nothing goes into some\n> branch \"foo\" that isn't signed with an authorship note.\n>\n> Assuming you are storing with git notes (if you are using some external\n> system, replace the call to git-notes below with whatever database\n> lookup you would want), you could use a pre-receive hook that did\n> something like:\n>\n>  git rev-list $old..$new |\n>  while read commit; do\n>    git notes show $commit >tmp\n>    gpg --verify tmp >data 2>siginfo || die \"$commit: signature is bad\"\n>    # ugh, is there really no better way to get this info from gpg?\n\nSee? We need functions for this stuff! I'll share whatever I come up with,\nand maybe it will be useful in general.\n\n>    perl -lne 'print $1 if /Good signature from \"(.*)\"/ siginfo >signer\n>    git show --format=\"%an <%ae>\" $commit >author\n>    cmp author signer || die \"$commit: signer and committer don't match\"\n\nYes this needs to be handled robustly. The signer would need to be told\nat sign-time if his signature didn't match.\n\n>    test \"`head -n 1 data`\" = $commit ||\n>      die \"$commit: signed commit does not match\"\n>  done\n>\n> And obviously that is hacked together and you would want something more\n> robust,\n\nThank you - this is all solid stuff to get me started.\n\n> and you'd need to handle the web of trust for the signing keys\n> somehow (though I think that is external to this script, and is about\n> setting up the desired keyring). But I hope it gives a sense of what you\n> can do. You could also replace gpg completely with something like\n> openssl using x.509 certs, if that makes more sense to your\n> organization.\n\nYou read my mind. Everybody in my organization has a set of x.509 certs\non smartcards. That's phase 2 of my project.\n\n>\n> Developers would have to make a note and push their notes tree first,\n\nYou mean for hook / verification purposes? Or is there some underlying\nreason to push notes first?\n\n> and then push their actual commits into a branch (and you might want to\n> do some verification on the notes they push, like checking that entries\n> for commit $X actually contains signatures for $X, or that the signer\n> identity matches some ssh credential, or that the pusher isn't deleting\n> any signatures or erasing note history).\n>\n> I suspect you already thought through some of this already. But I wanted\n> to start with first principles, because I really don't think this is a\n> _git_ problem as much as it is a _workflow_ problem. So it's important\n> to first define the workflow you want, and then think about how git can\n> help. Stable commit identifiers already provide much of the basis. I\n> think notes provide a nice storage format that is efficient and\n> push-able to other repos (though in a centralized shop, some other\n> database might make sense, too). What really remains to be done is:\n>\n>  1. Define the metadata format that encapsulates what you want to say\n>     about commits.\n>\n>  2. Write scripts to help developers and reviewers make these notes,\n>     and verify them.  Write hooks to implement policy on letting\n>     commits into certain branches, as above.\n>\n> And both of those happen outside of git (though if you write them in a\n> generic enough form, I'm sure people on the list would be very happy to\n> see them shared).\n\nI'll be sure to share.\n\n\n>\n>> There are 200 developers working on a financial trading system, and each of\n>> them has the opportunity to slip malicious code into the project. When the\n>> final release is prepared, the project lead signs the tip commit, thus\n>> signing the whole tree. Now it is discovered that someone did slip some\n>> malicious code in.  How do you audit the system? Could higher levels of\n>> individual accountability have discouraged this scenario?\n>\n> I like this example. It shows that signing a commit is not really\n> meaningful by itself; you have to understand the semantics of that\n> signature (and maybe they're included as comments in the tag object, or\n> maybe it is assumed by your organization's workflow).\n>\n> In the case of the kernel, Linus signing a commit with a tag implicitly\n> means \"I think what is in this tree and everything before it is good, so\n> you should feel comfortable using it\" (or at least insofar as you trust\n> Linus).\n>\n> But it doesn't have to be that way. Your project lead signing may mean\n> \"this is good and we should ship it\". But developers signing commits may\n> simply mean \"I promise that I wrote the changes between this commit's\n> tree and its parent\". Those are all signatures of commits, but they mean\n> very different things; the key is adding metadata to know which is\n> which.\n>\n>> I've seen it argued that a proper SSH setup and user management are the key.\n>> These are good for security and access control, but not for some durable\n>> form of accountability.\n>\n> Right. You are trusting the server's records, not cryptography. The main\n> advantage is that it's efficient and easy to set up. :)\n\nThe main reason this doesn't work for me is that codebases are passed around\nmy organization like hand-me-down clothes. It is not unheard of to get the\nentire repository for a critical application delivered from one shop to another\non CD. We need to be able to verify the integrity of a repository entirely\nindependently of any outside information.  The only centralized source of trust\nin our organization is the certificate authority.\n\nNow my big question to ponder: what do do when the CA expires a cert? Hmm...\n\n\n>\n>> It seems that creating a signed tag is the same as signing a commit.  There\n>> are a few problems, though.  Tags don't provide a secure means of asserting\n>> the type of signature being applied to the commit hash. That is - is the\n>> hash signed because someone is claiming authorship? Because they are\n>> asserting the integrity of the entire tree? Because they have reviewed the\n>> code? Because they reviewed a certain subset of the tree? Of course there's\n>> also the issue that tags live in a cluttered namespace. Signing a commit is\n>> essentially a different thing from providing a name for a commit. Using tags\n>> just to sign commits requires a glut of tag names.\n>\n> Again, metadata. Say what you mean in the free-form content of the tag.\n> For the kernel, there is nothing to be said. Linus signing tags has a\n> well-known meaning in the community. But in an organization signing for\n> a lot of different reasons, you would want the signed data to say why it\n> was signed.\n>\n>> I propose expanding the concept of tags, or alternately creating a new\n>> concept which subsumes the existing tag concept. I'll call this new concept\n>> a \"sig\" for the purposes of this discussion. The concept of a sig cross-cuts\n>> the concept of a tag.\n>>\n>> A tag signs the commit hash. A sig signs a SHA1-based absolute commit\n>> reference with a (possibly null) string concatenated to it. For instance, a\n>> sig might sign the following string:\n>\n> A tag can already include arbitrary data.\n>\n> In fact, tags basically do what you want already; it's just that storing\n> one tag ref per commit is going to be ugly. It might make sense to\n> replace the ad-hoc gpg signatures I used in my examples above with tag\n> objects, and then store the tag object in the notes tree.\n>\n>> \"0b9deecf625677cf44058a42c2abd7add5167e81^0 author\"\n>> which would mean that the signor is claiming authorship of that individual\n>> commit. (Suggestions for notating a single commit are welcome. \"^0\" seemed\n>> natural.)\n>\n> See? You're defining metadata now. :)\n>\n>> * What on earth does it mean to tag a range of commits? With commit ranges\n>> being siggable, and tags containing sigs, what does it mean to tag a range\n>> of 10 commits, for instance? Is that desirable? Does it make any sense\n>> whatsoever? Does it hurt anything if it happens?\n>\n> It's slightly more efficient. If I wrote 10 commits, I can either sign\n> each individually saying \"I wrote this\", or I can make a single\n> signature showing them all. The tradeoff is that parsing and verifying\n> metadata becomes a lot more complex. But crytographically speaking, a\n> range is not ambiguous;\n>\n>> * Performance? I think it would be extremely quick to verify a bunch of\n>> sigs, but I don't know. Maybe I'm not thinking clearly about it.\n>> Fortunately, sigs can be ignored entirely and need not affect things.\n>\n> Compared to usual git operations, no, it's not quick. But you don't have\n> to verify all the time. You can verify commits when they enter your\n> repo, or when you're interested in some aspect of them, or when you\n> suspect something fishy is going on. You don't have to do it on every\n> rev-list.\n\nGood point. I had thought it would be something to see every time I run\ngit-log, but I suppose it makes perfect sense to do this thing in the nightlies\nor some other rarer occasion.\n\nThanks,\n\nRichard Peterson\n"},{"id":"167150","messageId":"BANLkTinKLnsVp50+d_7U_vSUiaMNtZ-NCA@mail.gmail.com","threadId":"27247","inReplyTo":"BANLkTinCxzXCmmtxXSM7=+yeve2hhLSYNQ@mail.gmail.com","subject":"Re: [Tagging Commits] feedback / discussion request","fromName":"Sverre Rabbelier","fromEmail":"srabbelier@gmail.com","sentAt":"2011-05-05T18:49:39Z","receivedAt":"2011-05-05T18:49:39Z","isPatch":false,"sender":{"key":"srabbelier@gmail.com","avatar":"https://avatars.githubusercontent.com/u/3098?v=4"},"body":"Heya,\n\nOn Thu, May 5, 2011 at 17:39, Richard Peterson <richard@rcpeterson.com> wrote:\n> Now my big question to ponder: what do do when the CA expires a cert? Hmm...\n\nYou could re-sign the commits with the new cert, notes are mutable,\nand they keep history too. So you could just create a commit on the\nnotes history ref \"re-sign commits for expired cert\", optionally\nremoving the old signature. The hook verifying that no-one is\ntampering with the notes might get complex if you do that kind of\nstuff though (might be easier to just append the new signature and\nkeep the old one in place).\n\n-- \nCheers,\n\nSverre Rabbelier\n"},{"id":"167183","messageId":"20110505203016.GD1770@sigill.intra.peff.net","threadId":"27247","inReplyTo":"BANLkTinCxzXCmmtxXSM7=+yeve2hhLSYNQ@mail.gmail.com","subject":"Re: [Tagging Commits] feedback / discussion request","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2011-05-05T20:30:16Z","receivedAt":"2011-05-05T20:30:16Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, May 05, 2011 at 11:39:41AM -0400, Richard Peterson wrote:\n\n> >  3. In git notes. You can do something like:\n> >\n> >       (git rev-parse --verify HEAD\n> >        echo \"I reviewed this\"\n> >       ) | gpg --sign -a |\n> >       git notes add -F - HEAD\n> >\n> >     though you'd probably want to be a little more complex, and handle\n> >     lists of signed notes for each commit. And you may want to store\n> >     these in a separate notes ref from the default one.\n> \n> I had looked at this option, but had failed to see the usefulness of using\n> a different ref. I was worried about cluttering things up, overloading the\n> intended purpose of notes, and so forth. I wasn't really sure if notes were\n> intended to be general purpose storage for systematic, structured data.\n\nThey are definitely intended to be general purpose storage. For one such\n(ab)use, see the textconv-caching subsystem. It maps binary blobs into\ntheir converted text counterparts. So we are keying on blobs (not\ncommits!), and storing arbitrarily gigantic data in the notes values.\nAnd the nice thing is that because notes use git objects for storage, we\nget all the usual delta compression benefits on the result.\n\n> My inclination was to do this outside notes, or even in a parallel\n> implementation to notes, factoring out the common parts. I suppose that\n> looking at notes as somewhat of a free-for-all obviates this need. Is this\n> really what notes are for?\n\nYep. Definitely use notes if you are going to do the storage in git.\n\n> >     The advantage of notes are that they are designed for lots of\n> >     per-commit storage, and can be accessed fairly efficiently.\n> \n> That was my other concern about notes - performance. Not sure how\n> notes are stored, but I certainly trust you that they're efficient.\n\nEach notes tree is stored as a git tree full of entries representing the\ncommit (or other object) hashes. And each entry points to a blob which\nis the note's value. And then as the notes change over time, we version\nthem with commit objects. So you can make notes and your coworker can\nmake notes, and you can merge them together.\n\nFor fun, you can do:\n\n  # make a repo to play around in\n  git clone /path/to/some/repo notes-test\n  cd notes-test\n\n  # make some notes. You could also use \"notes add -F\"\n  # to add notes with arbitrary binary content.\n  git notes --ref=foo add -m \"this is note 1\" HEAD\n  git notes --ref=foo add -m \"this is note 2\" HEAD^\n\n  # check them out in context\n  git log --show-notes=foo -2\n\n  # and then see how they're stored\n  git checkout refs/notes/foo\n  grep . *\n\n> > Developers would have to make a note and push their notes tree first,\n> \n> You mean for hook / verification purposes? Or is there some underlying\n> reason to push notes first?\n\nYeah, for a pre-receive hook. You need to first tell the server \"here\nare some signatures\" by pushing the notes, and then it can verify those\nsignatures when trying to put commits on actual branches.\n\n> I'll be sure to share.\n\nGreat. I look forward to seeing the result.\n\n-Peff\n"}]}