{"thread":{"id":"51768","subject":"Is git clone followed by git verify-tag meaningful?","startedAt":"2019-08-28T20:32:28Z","lastAt":"2019-08-29T14:10:13Z","messageCount":6,"participants":["Konstantin Ryabitsev","brian m. carlson","Jeff King","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"381470","messageId":"20190828203224.GE26001@chatter.i7.local","threadId":"51768","inReplyTo":null,"subject":"Is git clone followed by git verify-tag meaningful?","fromName":"Konstantin Ryabitsev","fromEmail":"konstantin@linuxfoundation.org","sentAt":"2019-08-28T20:32:24Z","receivedAt":"2019-08-28T20:32:28Z","isPatch":false,"sender":{"key":"konstantin@linuxfoundation.org","avatar":"https://gravatar.com/avatar/7cb8827c6de56e1bd2dea16508c6708aa43feed3bf3813bcdacecdf96ceadd79?d=mp&s=160"},"body":"Hi, all:\n\nIf I know that a project uses tag signing, would \"git clone\" followed by \n\"git verify-tag\" be meaningful without a \"git fsck\" in-between? I.e. if \nan attacker has control over the remote server, can they sneak in any \nbadness into any of the resulting files and still have the clone, \ncheckout, and verify-tag return success unless the repository is fsck'd \nbefore verify-tag?\n\nI assume that it would break during the checkout stage, but I wanted to \nverify my assumptions.\n\nThanks,\n-K\n"},{"id":"381481","messageId":"20190828232718.GE11334@genre.crustytoothpaste.net","threadId":"51768","inReplyTo":"20190828203224.GE26001@chatter.i7.local","subject":"Re: Is git clone followed by git verify-tag meaningful?","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2019-08-28T23:27:18Z","receivedAt":"2019-08-28T23:27:28Z","isPatch":false,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2019-08-28 at 20:32:24, Konstantin Ryabitsev wrote:\n> Hi, all:\n> \n> If I know that a project uses tag signing, would \"git clone\" followed by\n> \"git verify-tag\" be meaningful without a \"git fsck\" in-between? I.e. if an\n> attacker has control over the remote server, can they sneak in any badness\n> into any of the resulting files and still have the clone, checkout, and\n> verify-tag return success unless the repository is fsck'd before verify-tag?\n> \n> I assume that it would break during the checkout stage, but I wanted to\n> verify my assumptions.\n\nWe pass the entire tag buffer to GnuPG, which means that we verify\nexactly what is in the tag: no more, no less.  Whether that represents a\nvalid, usable tag with meaningful data is not verified, although of\ncourse it can't be changed once written.\n\nIf you trust the signer to produce valid data, then you can verify the\ntag and know that the data is correct.  If not, then you probably need\ngit fsck to verify that the data is usable and meets Git's standards.\n-- \nbrian m. carlson: Houston, Texas, US\nOpenPGP: https://keybase.io/bk2204\n"},{"id":"381483","messageId":"20190828234706.GB25355@sigill.intra.peff.net","threadId":"51768","inReplyTo":"20190828203224.GE26001@chatter.i7.local","subject":"Re: Is git clone followed by git verify-tag meaningful?","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2019-08-28T23:47:06Z","receivedAt":"2019-08-28T23:47:08Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Aug 28, 2019 at 04:32:24PM -0400, Konstantin Ryabitsev wrote:\n\n> If I know that a project uses tag signing, would \"git clone\" followed by\n> \"git verify-tag\" be meaningful without a \"git fsck\" in-between? I.e. if an\n> attacker has control over the remote server, can they sneak in any badness\n> into any of the resulting files and still have the clone, checkout, and\n> verify-tag return success unless the repository is fsck'd before verify-tag?\n\nIt depends on your definition of badness. :)\n\nGenerally, Git clients do not trust the server much at all (not only to\nbe no malicious, but also not to accidentally introduce bit errors).\n\nEven without the fsck, we will compute the sha1 of each object (we must,\nbecause the other side doesn't send it at all), and that we have all\nobjects reachable from the refs. So verifying the tag at that point\ndemonstrates a signature on the tag object, which refers to probably\nsome commit via sha1, which refers to actual trees and blobs by a chain\nof sha1s. If you believe in the integrity of sha1, then it has\neffectively signed all of that content.\n\nLikewise, Git does not necessarily trust what is in the objects. A\nmalicious repository could claim to store an entry for \".git/config\" or\n\"/etc/passwd\". Without any further action from you, we'd detect and\nreject those during a checkout.\n\nIf you want to analyze each object for such malformed bits before the\ncheckout, you can do so with \"git fsck\". But consider instead setting\ntransfer.fsckObjects to check the objects while they're being indexed by\nthe initial clone (i.e., having their sha1's computed). It's effectively\nfree to do it at that point, whereas a later fsck has to access each\nobject again (this takes on the order of minutes of CPU for the kernel).\n\nI don't think there's any real safety in doing so for the case you've\ndescribed (there's no bad pattern that fsck knows about that the actual\ncheckout code does not).  But it does give you an early warning, and is\nespecially help if you're not planning to check things out yourself, but\nwant to avoid hosting malicious repos.\n\n-Peff\n"},{"id":"381491","messageId":"xmqq36hk1vw1.fsf@gitster-ct.c.googlers.com","threadId":"51768","inReplyTo":"20190828203224.GE26001@chatter.i7.local","subject":"Re: Is git clone followed by git verify-tag meaningful?","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2019-08-29T03:41:02Z","receivedAt":"2019-08-29T03:41:07Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Konstantin Ryabitsev <konstantin@linuxfoundation.org> writes:\n\n> If I know that a project uses tag signing, would \"git clone\" followed\n> by \"git verify-tag\" be meaningful without a \"git fsck\" in-between?\n> I.e. if an attacker has control over the remote server, can they sneak\n> in any badness into any of the resulting files and still have the\n> clone, checkout, and verify-tag return success unless the repository\n> is fsck'd before verify-tag?\n>\n> I assume that it would break during the checkout stage, but I wanted\n> to verify my assumptions.\n\nWhat you are trusting and what you are trying to protect?\n\nI am assuming that you are cloning and a commit that has a signed\ntag is at the tip of the default branch, which gets checked out and\nyou want to make sure that what you see in the working tree after\ncheckout is healthy.  I also assume that you trust your local\nmachine, its Git and GPG binary included, and also you trust that\nthe underlying hash function Git uses has not been exploited for\nthis particular repository.\n\nverify-tag would tell you that the tag you specify is signed by\nwhich GPG key you have in your keychain.  Since tag records the\ncommit object name, you can check it against the HEAD.  As long\nas you trust the underlying hash function and your local Git,\nthe trust flows from the HEAD's commit object name down to each\nand every file checked out to the working tree.  As long as you\ndid not get any error from checkout, no fsck is needed here.\n\nIf your project is high-valued target like the Linux kernel, it is\nprobably a good idea to enable fetch.fsckobjects so that the\nincoming objects are automatically checked while receiving over the\nwire.\n\n\n"},{"id":"381505","messageId":"20190829133457.GA26173@chatter.i7.local","threadId":"51768","inReplyTo":"20190828234706.GB25355@sigill.intra.peff.net","subject":"Re: Is git clone followed by git verify-tag meaningful?","fromName":"Konstantin Ryabitsev","fromEmail":"konstantin@linuxfoundation.org","sentAt":"2019-08-29T13:34:57Z","receivedAt":"2019-08-29T13:35:03Z","isPatch":false,"sender":{"key":"konstantin@linuxfoundation.org","avatar":"https://gravatar.com/avatar/7cb8827c6de56e1bd2dea16508c6708aa43feed3bf3813bcdacecdf96ceadd79?d=mp&s=160"},"body":"On Wed, Aug 28, 2019 at 07:47:06PM -0400, Jeff King wrote:\n>On Wed, Aug 28, 2019 at 04:32:24PM -0400, Konstantin Ryabitsev wrote:\n>\n>> If I know that a project uses tag signing, would \"git clone\" followed by\n>> \"git verify-tag\" be meaningful without a \"git fsck\" in-between? I.e. if an\n>> attacker has control over the remote server, can they sneak in any badness\n>> into any of the resulting files and still have the clone, checkout, and\n>> verify-tag return success unless the repository is fsck'd before verify-tag?\n>\n>It depends on your definition of badness. :)\n\nAs you know, for the Linux kernel we provide both tag signatures and \ndetached PGP signatures on tarballs (and the same is true for git). The \nargument I hear frequently is that providing detached tarball signatures \nis redundant[*] when tags are already PGP-signed, so I wanted to \ndouble-check that all checksums are computed and matched on the client \nin the process of \"git checkout\" and we're not just verifying a \nsignature of a non-verified checksum.\n\nIn other words, I needed to double-check that what we get in the end is \nassurance that \"all files in this repository are exactly the same as on \nthe developer's system at the time when they ran 'git tag -s'.\"\n\n>Generally, Git clients do not trust the server much at all (not only to\n>be no malicious, but also not to accidentally introduce bit errors).\n>\n>Even without the fsck, we will compute the sha1 of each object (we must,\n>because the other side doesn't send it at all), and that we have all\n>objects reachable from the refs. So verifying the tag at that point\n>demonstrates a signature on the tag object, which refers to probably\n>some commit via sha1, which refers to actual trees and blobs by a chain\n>of sha1s. If you believe in the integrity of sha1, then it has\n>effectively signed all of that content.\n\nSo, the client will actually calculate those checksums during the \ncheckout stage to make sure that all content in the repository matches \nthe hash of the commit being checked out, correct? \n\n>If you want to analyze each object for such malformed bits before the\n>checkout, you can do so with \"git fsck\". But consider instead setting\n>transfer.fsckObjects to check the objects while they're being indexed by\n>the initial clone (i.e., having their sha1's computed). It's effectively\n>free to do it at that point, whereas a later fsck has to access each\n>object again (this takes on the order of minutes of CPU for the kernel).\n>\n>I don't think there's any real safety in doing so for the case you've\n>described (there's no bad pattern that fsck knows about that the actual\n>checkout code does not).  But it does give you an early warning, and is\n>especially help if you're not planning to check things out yourself, but\n>want to avoid hosting malicious repos.\n\nRight, but it's not something end-users are going to do if they just \nwant to check out a repository and access code from it. The \"git clone \n&& git verify-tag\" workflow is now used by some distros that are \npackaging Github releases, and they aren't setting transfer.fsckObjects \nbefore \"git clone\" starts, pretty sure.\n\nThanks for your help!\n\n-K\n\n[*] Tarball signatures may be redundant in cryptographic sense, but for \nrepositories like linux.git, which are now around 1.2 GB in size, it \nmakes significant difference whether someone downloads the full git tree \nor just a highly compressed tarball that is only 100MB. I know that it's \npossible to clone with --depth 1 to reduce the amount of downloaded \nhistory, but that's hard on the servers and not something I really want \nto widely advertise as a mechanism for getting the kernel. :) In \naddition to that, distributing static content like tarballs is much \neasier logistically than git repositories, and it's much harder to \nintroduce accidental corruption to a bunch of static files. Disk is \ncheap, but CPU and admin time aren't.\n"},{"id":"381509","messageId":"20190829141010.GD1797@sigill.intra.peff.net","threadId":"51768","inReplyTo":"20190829133457.GA26173@chatter.i7.local","subject":"Re: Is git clone followed by git verify-tag meaningful?","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2019-08-29T14:10:11Z","receivedAt":"2019-08-29T14:10:13Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Aug 29, 2019 at 09:34:57AM -0400, Konstantin Ryabitsev wrote:\n\n> As you know, for the Linux kernel we provide both tag signatures and\n> detached PGP signatures on tarballs (and the same is true for git). The\n> argument I hear frequently is that providing detached tarball signatures is\n> redundant[*] when tags are already PGP-signed, so I wanted to double-check\n> that all checksums are computed and matched on the client in the process of\n> \"git checkout\" and we're not just verifying a signature of a non-verified\n> checksum.\n> \n> In other words, I needed to double-check that what we get in the end is\n> assurance that \"all files in this repository are exactly the same as on the\n> developer's system at the time when they ran 'git tag -s'.\"\n\nThen yes, there is no need to fsck. When the objects were received on\nthe server side (by push) and then again when you got them from the\nserver (by clone), their sha1s were recomputed from scratch, not\ntrusting the sender at all in either case.\n\n(Again, assuming you trust sha1; I think you should, especially since we\nuse the collision-detecting sha1 by default, but I wanted to make that\npart clear).\n\n> > Even without the fsck, we will compute the sha1 of each object (we must,\n> > because the other side doesn't send it at all), and that we have all\n> > objects reachable from the refs. So verifying the tag at that point\n> > demonstrates a signature on the tag object, which refers to probably\n> > some commit via sha1, which refers to actual trees and blobs by a chain\n> > of sha1s. If you believe in the integrity of sha1, then it has\n> > effectively signed all of that content.\n> \n> So, the client will actually calculate those checksums during the checkout\n> stage to make sure that all content in the repository matches the hash of\n> the commit being checked out, correct?\n\nIt's not during the checkout itself, but rather during the transfer of\nobjects into the receiving repository. I.e., there is no need to even\nhave a checkout. E.g., you could verify the tag and then use \"git\narchive\".\n\nDo note that both archive and checkout can modify files from their\nin-repository state using gitattributes (e.g., to do line-ending\nconversion, or using export-subst to add things like the commit ID into\nthe generated tarball). So it's possible that a tarball (either\ngenerated from git-archive or from checked out contents) may not be\nbyte-for-byte identical.\n\nDepending on your use case, that can range from an annoyance to ignore\n(if a developer is using those features, tell them not to do that) to a\nsecurity issue (if you are somehow certifying the tarball contents based\non the tag signature, there is room for a malicious signer to tweak the\ntarball contents).\n\nBut I think your question is mostly just \"if I clone the repo and verify\nthe tag, is it what the original person signed?\". And that answer is\nyes.\n\n-Peff\n"}]}