{"thread":{"id":"22912","subject":"Question about scm security holes","startedAt":"2010-03-04T20:09:41Z","lastAt":"2010-03-05T22:33:12Z","messageCount":13,"participants":["walt","Avery Pennarun","John Tapsell","Andreas Krey","Johannes Schindelin","Jakub Narebski","Daniel Barkalow"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"136176","messageId":"hmp427$d6h$1@dough.gmane.org","threadId":"22912","inReplyTo":null,"subject":"Question about scm security holes","fromName":"walt","fromEmail":"w41ter@gmail.com","sentAt":"2010-03-04T20:09:41Z","receivedAt":"2010-03-04T20:09:41Z","isPatch":false,"sender":{"key":"w41ter@gmail.com","avatar":null},"body":"I just saw this article about the \"google hackers\" exploiting weaknesses in scms,\nPerforce in particular:\n\nhttp://www.wired.com/threatlevel/2010/03/source-code-hacks/?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+wired%2Findex+%28Wired%3A+Index+3+%28Top+Stories+2%29%29\n\nI guess google didn't take Linus's advice to dump Perforce :)\n\nI can't tell from the article if Perforce is any worse than any other scm for\nsecurity holes, in fact it seems to imply that others haven't been tested in\nthe same way.\n\nJust curious if anyone here has any thoughts about how the article may or may\nnot have any relevance for git (git being the scm I use most, by far, which is\nthe reason I'm interested).\n\nThanks\n"},{"id":"136177","messageId":"32541b131003041803q9abf6baq4cf9ffcca990b51c@mail.gmail.com","threadId":"22912","inReplyTo":"hmp427$d6h$1@dough.gmane.org","subject":"Re: Question about scm security holes","fromName":"Avery Pennarun","fromEmail":"apenwarr@gmail.com","sentAt":"2010-03-05T02:03:08Z","receivedAt":"2010-03-05T02:03:08Z","isPatch":false,"sender":{"key":"apenwarr@gmail.com","avatar":"https://avatars.githubusercontent.com/u/20592?v=4"},"body":"On Thu, Mar 4, 2010 at 3:09 PM, walt wrote:\n> I can't tell from the article if Perforce is any worse than any other scm\n> for security holes, in fact it seems to imply that others haven't been tested in\n> the same way.\n>\n> Just curious if anyone here has any thoughts about how the article may or\n> may not have any relevance for git (git being the scm I use most, by far, which\n> is the reason I'm interested).\n\nThe attack was uninteresting. The paper seems to go on and on about\ndifferent ways that an attacker can steal source code by accessing a\npoorly-secured SCM server. This discussion is kind of moot for git,\nwhere every single developer workstation has a complete copy of the\nentire project history anyway.\n\nAn attack in which someone untraceably modified the repo to contain\nmodified code would be a little more interesting. git makes this sort\nof thing pretty much impossible to do without it being *noticeable* at\nleast. Traceable, not so much, because you can create a commit with\nwhatever committer/author names you want and then push them in.\nCommits aren't GPG-signed, only tags are, so there are lots of ways to\nforge a commit from someone else and mess up the audit log. At least\nyou can't edit old commits without people noticing, though.\n\nHave fun,\n\nAvery\n"},{"id":"136178","messageId":"43d8ce651003041900x66000be4s9a15ab0cde3a0fe7@mail.gmail.com","threadId":"22912","inReplyTo":"32541b131003041803q9abf6baq4cf9ffcca990b51c@mail.gmail.com","subject":"Re: Question about scm security holes","fromName":"John Tapsell","fromEmail":"johnflux@gmail.com","sentAt":"2010-03-05T03:00:10Z","receivedAt":"2010-03-05T03:00:10Z","isPatch":false,"sender":{"key":"johnflux@gmail.com","avatar":"https://gravatar.com/avatar/25f70d4c0f96396b84a2e34bcd9bdc233462c7b4be29b5fdca8266fc53f30b0c?d=mp&s=160"},"body":"On 5 March 2010 02:03, Avery Pennarun wrote:\n> modified code would be a little more interesting.  git makes this sort\n> of thing pretty much impossible to do without it being *noticeable* at\n> least.  Traceable, not so much, because you can create a commit with\n> whatever committer/author names you want and then push them in.\n\nWhich is why you simply record the username of whoever pushed them in.\n This is what gitorious.org does etc.\n\nJohn\n"},{"id":"136181","messageId":"32541b131003041919u6a477b46s447a6aeb18f3b393@mail.gmail.com","threadId":"22912","inReplyTo":"43d8ce651003041900x66000be4s9a15ab0cde3a0fe7@mail.gmail.com","subject":"Re: Question about scm security holes","fromName":"Avery Pennarun","fromEmail":"apenwarr@gmail.com","sentAt":"2010-03-05T03:19:13Z","receivedAt":"2010-03-05T03:19:13Z","isPatch":false,"sender":{"key":"apenwarr@gmail.com","avatar":"https://avatars.githubusercontent.com/u/20592?v=4"},"body":"On Thu, Mar 4, 2010 at 10:00 PM, John Tapsell wrote:\n> On 5 March 2010 02:03, Avery Pennarun wrote:\n>> modified code would be a little more interesting.  git makes this sort\n>> of thing pretty much impossible to do without it being *noticeable* at\n>> least.  Traceable, not so much, because you can create a commit with\n>> whatever committer/author names you want and then push them in.\n>\n> Which is why you simply record the username of whoever pushed them in.\n>  This is what gitorious.org does etc.\n\nNot bad, but it's still very hard to trace properly. Imagine I pull\nfrom a peer, then push my combined branch into the central repo.\nIt'll say I'm pushing in patches from me *and* my friend. Did I forge\nthem or are they real?\n\nAvery\n"},{"id":"136180","messageId":"4B907884.5080501@gmail.com","threadId":"22912","inReplyTo":"32541b131003041803q9abf6baq4cf9ffcca990b51c@mail.gmail.com","subject":"Re: Question about scm security holes","fromName":"walt","fromEmail":"w41ter@gmail.com","sentAt":"2010-03-05T03:20:36Z","receivedAt":"2010-03-05T03:20:36Z","isPatch":false,"sender":{"key":"w41ter@gmail.com","avatar":null},"body":"On 03/04/2010 06:03 PM, Avery Pennarun wrote:\n\n> ...you can create a commit with\n> whatever committer/author names you want and then push them in.\n> Commits aren't GPG-signed, only tags are, so there are lots of ways to\n> forge a commit from someone else and mess up the audit log...\n\nThanks, that's the kind of reply I was hoping for. Do you think there\nshould be a way to sign the commits themselves, at least as an option?\n\nI certainly wouldn't bother, but OTOH nobody wants to steal my code :-/\n\nDo you suppose the devs at Adobe carry the complete source repository\nhome on their laptops every night? (Not if they use Perforce, of course,\nbut they might if they adopted git as their scm.)\n"},{"id":"136183","messageId":"32541b131003041928m50aee3d0jcde58f3f4ff63a8b@mail.gmail.com","threadId":"22912","inReplyTo":"4B907884.5080501@gmail.com","subject":"Re: Question about scm security holes","fromName":"Avery Pennarun","fromEmail":"apenwarr@gmail.com","sentAt":"2010-03-05T03:28:38Z","receivedAt":"2010-03-05T03:28:38Z","isPatch":false,"sender":{"key":"apenwarr@gmail.com","avatar":"https://avatars.githubusercontent.com/u/20592?v=4"},"body":"On Thu, Mar 4, 2010 at 10:20 PM, walt wrote:\n> On 03/04/2010 06:03 PM, Avery Pennarun wrote:\n>> ...you can create a commit with\n>> whatever committer/author names you want and then push them in.\n>> Commits aren't GPG-signed, only tags are, so there are lots of ways to\n>> forge a commit from someone else and mess up the audit log...\n>\n> Thanks, that's the kind of reply I was hoping for.  Do you think there\n> should be a way to sign the commits themselves, at least as an option?\n>\n> I certainly wouldn't bother, but OTOH nobody wants to steal my code :-/\n\nThe whole thing is a bit overblown. One of my friends once took me on\na tour of Microsoft on a weekend. The place was mostly deserted, but\ntons of developers left their workstations unlocked overnight, and\neveryone had a private office. And with tens of thousands of\ndevelopers on the campus, nobody would know if you're supposed to be\nthere or not.\n\nIt would have been easy to walk off with the source code to Windows\nfrom one of those workstations. The fact is, nobody really *wants*\nthe source code to Windows, except probably to look at it and be\nhorrified.\n\nWhat would you do if you stole the source code to Adobe's flash\nplayer? It's illegal (in the U.S. anyway) to reverse engineer it and\nit's illegal to steal it, so you're on the wrong side of the law no\nmatter how you pretend you managed to figure out a way around their\nDRM or whatever.\n\nPeople describe source code as a company's \"crown jewels,\" but that's\na bit of a joke. I can barely get our interns to figure out how to\ncompile and understand our code. Expecting a thief to do it, with\nnothing but a raw repo and hundreds of gigabytes of crap, is pure\nparanoia.\n\nSneaking in patches? Yeah, watch out for that. But you should be\nreviewing patch changelogs anyway. At least git prevents people from\n*retroactively* changing stuff; they can only add patches on top, so\nit's easy to review after a break-in.\n\nHave fun,\n\nAvery\n"},{"id":"136184","messageId":"43d8ce651003042007o4e41e527j8f64c898e7492d70@mail.gmail.com","threadId":"22912","inReplyTo":"32541b131003041919u6a477b46s447a6aeb18f3b393@mail.gmail.com","subject":"Re: Question about scm security holes","fromName":"John Tapsell","fromEmail":"johnflux@gmail.com","sentAt":"2010-03-05T04:07:08Z","receivedAt":"2010-03-05T04:07:08Z","isPatch":false,"sender":{"key":"johnflux@gmail.com","avatar":"https://gravatar.com/avatar/25f70d4c0f96396b84a2e34bcd9bdc233462c7b4be29b5fdca8266fc53f30b0c?d=mp&s=160"},"body":"On 5 March 2010 03:19, Avery Pennarun wrote:\n> On Thu, Mar 4, 2010 at 10:00 PM, John Tapsell wrote:\n>> On 5 March 2010 02:03, Avery Pennarun wrote:\n>>> modified code would be a little more interesting.  git makes this sort\n>>> of thing pretty much impossible to do without it being *noticeable* at\n>>> least.  Traceable, not so much, because you can create a commit with\n>>> whatever committer/author names you want and then push them in.\n>>\n>> Which is why you simply record the username of whoever pushed them in.\n>>  This is what gitorious.org does etc.\n>\n> Not bad, but it's still very hard to trace properly.  Imagine I pull\n> from a peer, then push my combined branch into the central repo.\n> It'll say I'm pushing in patches from me *and* my friend.  Did I forge\n> them or are they real?\n\nWhile true, it's still traceable back to you. You did the push, so\nyou are responsible for that code. It wouldn't be any different to\njust pushing a bad commit yourself.\n"},{"id":"136185","messageId":"20100305073642.GA16131@inner.home.ulmdo.de","threadId":"22912","inReplyTo":"32541b131003041803q9abf6baq4cf9ffcca990b51c@mail.gmail.com","subject":"Re: Question about scm security holes","fromName":"Andreas Krey","fromEmail":"a.krey@gmx.de","sentAt":"2010-03-05T07:36:42Z","receivedAt":"2010-03-05T07:36:42Z","isPatch":false,"sender":{"key":"a.krey@gmx.de","avatar":"https://avatars.githubusercontent.com/u/37810?v=4"},"body":"On Thu, 04 Mar 2010 21:03:08 +0000, Avery Pennarun wrote:\n...\n> where every single developer workstation has a complete copy of the\n> entire project history anyway.\n\nIt's the point of a dev workstation to have access to the code,\nso McAfees whining about SCMs letting that happen is moot.\n\nWhat would be helping here is a separation between internet-facing\nand local work into separate machines.\n\n> least. Traceable, not so much, because you can create a commit with\n> whatever committer/author names you want and then push them in.\n\nYou can still log who pushed what into your blessed repo,\nand hold that person accountable.\n\nAndreas\n"},{"id":"136193","messageId":"alpine.DEB.1.00.1003050953580.20986@pacific.mpi-cbg.de","threadId":"22912","inReplyTo":"32541b131003041803q9abf6baq4cf9ffcca990b51c@mail.gmail.com","subject":"Re: Question about scm security holes","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2010-03-05T09:25:35Z","receivedAt":"2010-03-05T09:25:35Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Thu, 4 Mar 2010, Avery Pennarun wrote:\n\n> An attack in which someone untraceably modified the repo to contain \n> modified code would be a little more interesting.\n\nI disagree that stealing the code in this particular case is \nuninteresting. You know that there are billions in the business how to \nmanipulate Google's search results. If you can see how Google rates \nwebsites, you can prepare your xxx sites for it, and nobody would be able \nto know, let alone prove, that you \"reverse-engineered\" the system.\n\n> git makes this sort of thing pretty much impossible to do without it \n> being *noticeable* at least.\n\nThat is not true in all cases.\n\nIf you're talking about a workflow as git.git has it, you're right, there \nis a maintainer, and a refused push would ring all kinds of alarm bells \nthere.\n\nExcept, of course, when the maintainer happens to work on different \nmachines, and is likely to pull from her main repository quite often. \nThink \"get something compiling on an obscure platform while developing \nsomething different on your main computer, then do a criss-cross merge at \nthe end\".\n\nIt gets even much, much worse in the common setup of companies: a central \nrepository. (The two main reasons why a central repository is used are: \ntradition (we did it with Subversion, too), and bottleneck problems: a \nsingle maintainer reviewing all changes is often deemed too expensive \nand slow.)\n\nSo in the regular case, it is _very_ easy to sneak in a code-change \nunnoticedly.\n\nThe trick now is to craft the commit in such a manner that it will not be \nnoticed retro-actively. This is a simple case of social engineering: you \nhave to imitate the style of the committer/author you are impersonating. \nThe commit message must look like the usual ones (typos, preferred words, \ngrammar, length of paragraphs, comprehensibility, etc)\n\nLikewise, the code has to be analyzed for style, and obviously for most \nlikely targets of a backdoor (both in terms of \"it is a perfect spot for \na backdoor\" and \"it is not uncommon for the author to touch that \npart of the code\").\n\nCrafting the commit message and the backdoor needs some time, and it needs \nto be done _after_ succeeding with the break-in, as you can only then \nstart analyzing style (and most likely workflow -- whether there is a \nsingle maintainer or whether everybody pushes to a single repository).\n\nThe most likely route, therefore is to have _two_ break-ins. One for \nreconaissance, the second for the actual change.\n\nConclusion: there are no technical reasons why Git should be better than \nPerforce when it comes to a break-in.\n\nShort version: it's a social problem, so it needs a social solution.\n\nCiao,\nDscho\n\nP.S.: Sorry for the overly long mail. I did not have time to make it \nshort.\n"},{"id":"136198","messageId":"m3lje7kpr9.fsf@localhost.localdomain","threadId":"22912","inReplyTo":"alpine.DEB.1.00.1003050953580.20986@pacific.mpi-cbg.de","subject":"Re: Question about scm security holes","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2010-03-05T10:49:46Z","receivedAt":"2010-03-05T10:49:46Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"Johannes Schindelin writes:\n> On Thu, 4 Mar 2010, Avery Pennarun wrote:\n> \n> > An attack in which someone untraceably modified the repo to contain \n> > modified code would be a little more interesting.\n\n> > git makes this sort of thing pretty much impossible to do without it \n> > being *noticeable* at least.\n> \n> That is not true in all cases.\n> \n> If you're talking about a workflow as git.git has it, you're right, there \n> is a maintainer, and a refused push would ring all kinds of alarm bells \n> there.\n\n[...]\n> It gets even much, much worse in the common setup of companies: a central \n> repository. (The two main reasons why a central repository is used are: \n> tradition (we did it with Subversion, too), and bottleneck problems: a \n> single maintainer reviewing all changes is often deemed too expensive \n> and slow.)\n\nAbout \"bottleneck problem\". Frederick Brooks wrote in his seminal\nbook \"The Mythical Man-Month\" that recommended way of organizing teams\nis *with a maintainer*. But this is less known that his most famous\nstatement: \"Adding manpower to a late software project makes it\nlater.\" (The Brooks's Law)... and I guess companies do not know about\nthis one either :-)\n\n-- \nJakub Narebski\nPoland\nShadeHawk on #git\n"},{"id":"136216","messageId":"alpine.LNX.2.00.1003051103490.14365@iabervon.org","threadId":"22912","inReplyTo":"hmp427$d6h$1@dough.gmane.org","subject":"Re: Question about scm security holes","fromName":"Daniel Barkalow","fromEmail":"barkalow@iabervon.org","sentAt":"2010-03-05T17:47:33Z","receivedAt":"2010-03-05T17:47:33Z","isPatch":false,"sender":{"key":"barkalow@iabervon.org","avatar":"https://avatars.githubusercontent.com/u/55364219?v=4"},"body":"On Thu, 4 Mar 2010, walt wrote:\n\n> I just saw this article about the \"google hackers\" exploiting weaknesses in\n> scms,\n> Perforce in particular:\n> \n> http://www.wired.com/threatlevel/2010/03/source-code-hacks/?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+wired%2Findex+%28Wired%3A+Index+3+%28Top+Stories+2%29%29\n> \n> I guess google didn't take Linus's advice to dump Perforce :)\n> \n> I can't tell from the article if Perforce is any worse than any other scm for\n> security holes, in fact it seems to imply that others haven't been tested in\n> the same way.\n> \n> Just curious if anyone here has any thoughts about how the article may or may\n> not have any relevance for git (git being the scm I use most, by far, which is\n> the reason I'm interested).\n\nI took a look at the white paper the article links to. I had to ignore a \nlot of the introductory sections (yes, the most secure system would be to\nprevent people from doing any work that might be stolen or released after \nit was corrupted), but I assume that the \"findings\" are the actually \nrelevant part. Comparing git and Perforce here:\n\n - The Perforce server software for Windows installs to run as root. I'm \n not sure what the norm is for git central repositories on Windows, but \n it's probably better. I don't know if people actually run Perforce \n servers on Windows in practice, either.\n\n - Perforce has built-in authorization and authentication. By default, it \n allows unauthenticated people to create users without any specific \n authorization. It transmits passwords in cleartext in some cases. It \n discloses a lot of information about the authorization and \n authentication in force to arbitrary people, including users of the \n internal web site who do not have protocol access at all. It issues \n login tickets that last a long time. The authorization controls are not \n applied reliably to operations that modify the authorization and \n authentication information in some of the server software. The initial \n configuration with respect to access control is completely \n unrestrictive. Git does not have built-in authorization or \n authentication, so avoiding or making these mistakes is outside git's \n scope.\n\n - Perforce sends all of content over the network in cleartext. This is \n essentially true of git as well, but in order to get any sort of access \n control with git, you need to use some wrapping method, which will \n generally provide encryption as well.\n\n - Perforce stores, on the server, the location of the working directory \n on the client, and this is used by the client to place files. Git does \n not store this information at all.\n\nIn general, they seem to have found numerous flaws due to the fact that \nPerforce includes security-related code while not being designed by \nsecurity specialists. Git is designed not to include security-related \ncode, and to have properly developed security code control access to it. \nIt is possible to run Perforce in a configuration where access control is \nexternal to Perforce, but it's not easy or standard.\n\nOn the other hand, I don't see any indication that the attack they were \ninvestigating used any of the problems they found, or any problems of a \nsimilar class. The actual attack seemed to involve a successful attack on \nthe workstation of someone with legitimate priviledges, which the \nattackers then used. It's hard to say if any security measures on the part \nof the SCM could have any effect other than limiting the choice of the \nuser to target.\n\n\t-Daniel\n*This .sig left intentionally blank*\n"},{"id":"136217","messageId":"32541b131003051022oe64428bsa387e64e30bbeaab@mail.gmail.com","threadId":"22912","inReplyTo":"alpine.DEB.1.00.1003050953580.20986@pacific.mpi-cbg.de","subject":"Re: Question about scm security holes","fromName":"Avery Pennarun","fromEmail":"apenwarr@gmail.com","sentAt":"2010-03-05T18:22:19Z","receivedAt":"2010-03-05T18:22:19Z","isPatch":false,"sender":{"key":"apenwarr@gmail.com","avatar":"https://avatars.githubusercontent.com/u/20592?v=4"},"body":"On Fri, Mar 5, 2010 at 4:25 AM, Johannes Schindelin\n wrote:\n> The trick now is to craft the commit in such a manner that it will not be\n> noticed retro-actively. This is a simple case of social engineering: you\n> have to imitate the style of the committer/author you are impersonating.\n> The commit message must look like the usual ones (typos, preferred words,\n> grammar, length of paragraphs, comprehensibility, etc)\n>\n> Likewise, the code has to be analyzed for style, and obviously for most\n> likely targets of a backdoor (both in terms of \"it is a perfect spot for\n> a backdoor\" and \"it is not uncommon for the author to touch that\n> part of the code\").\n\nThere is still one major advantage to preventing modification of past\ncommits: once you find out there's been a breach, you can just go back\nthrough the commits *since* the breach and double-check them. Without\nthat guarantee, you have to recheck *every* commit, which is much more\nwork.\n\nNot to say that a sneaky commit would be easy to detect, though. I\noften add bugs to my own code without even trying to hide them, and\nthey're still pretty hard to find afterward.\n\nHave fun,\n\nAvery\n"},{"id":"136223","messageId":"alpine.DEB.1.00.1003052331140.20986@pacific.mpi-cbg.de","threadId":"22912","inReplyTo":"32541b131003051022oe64428bsa387e64e30bbeaab@mail.gmail.com","subject":"Re: Question about scm security holes","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2010-03-05T22:33:12Z","receivedAt":"2010-03-05T22:33:12Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Fri, 5 Mar 2010, Avery Pennarun wrote:\n\n> On Fri, Mar 5, 2010 at 4:25 AM, Johannes Schindelin\n> wrote:\n> > The trick now is to craft the commit in such a manner that it will not be\n> > noticed retro-actively. This is a simple case of social engineering: you\n> > have to imitate the style of the committer/author you are impersonating.\n> > The commit message must look like the usual ones (typos, preferred words,\n> > grammar, length of paragraphs, comprehensibility, etc)\n> >\n> > Likewise, the code has to be analyzed for style, and obviously for most\n> > likely targets of a backdoor (both in terms of \"it is a perfect spot for\n> > a backdoor\" and \"it is not uncommon for the author to touch that\n> > part of the code\").\n> \n> There is still one major advantage to preventing modification of past\n> commits: once you find out there's been a breach, you can just go back\n> through the commits *since* the breach and double-check them.\n\nIf you find out which commit it was in the past, you can always revert it. \nIt does not take Git to do it.\n\nI am all in favor of Git, yes, but let's be honest: Git does not prevent \nan intelligent break-in.\n\nTo repeat, as I seem to not have made the point before: a break-in is a \nsocial problem, so it requires a social solution.\n\nCiao,\nDscho\n"}]}