{"thread":{"id":"34892","subject":"[Proposal] Clonable scripts","startedAt":"2013-09-09T20:48:42Z","lastAt":"2013-09-10T12:55:14Z","messageCount":7,"participants":["Niels Basjes","Hilco Wijbenga","Ramkumar Ramachandra","Sitaram Chamarty","Andreas Krey"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"227259","messageId":"CADoiZqpec6rPOgPLPQFFfLdE+Cc4ZKtWs0Q0VSfKGm3b1Lai2g@mail.gmail.com","threadId":"34892","inReplyTo":null,"subject":"[Proposal] Clonable scripts","fromName":"Niels Basjes","fromEmail":"niels@basjes.nl","sentAt":"2013-09-09T20:48:42Z","receivedAt":"2013-09-09T20:48:42Z","isPatch":false,"sender":{"key":"niels@basjes.nl","avatar":"https://gravatar.com/avatar/1a1e9443d32983849de0a7a64aaad3fd2e238ca69bccbaa71a914d8edbd84ed5?d=mp&s=160"},"body":"Hi,\n\nAs we all know the hooks ( in .git/hooks ) are not cloned along with\nthe code of a project.\nNow this is a correct approach for the scripts that do stuff like\nemailing the people responsible for releases or submitting the commit\nto a CI system.\n\nFor several other things it makes a lot of sense to give the developer\nimmediate feedback. Things like the format of the commit message (i.e.\nit must start with an issue tracker id) or compliance with a coding\nstandard.\n\nInitially I wanted to propose introducing fully clonable (pre-commit)\nhook scripts.\nHowever I can imagine that a malicious opensource coder can create a\ngithub repo and try to hack the computer of a contributer via those\nscripts. So having such scripts is a 'bad idea'.\n\nIf those scripts were how ever written in a language that is build\ninto the git program and the script are run in such a way that they\ncan only interact with the files in the local git (and _nothing_\noutside of that) this would be solved.\n\nAlso have a builtin scripting language also means that this would run\non all operating systems (yes, even Windows).\n\nSo I propose the following new feature:\n\n1) A scripting language is put inside git. Perhaps a version of python\nor ruby or go or ... (no need for a 'new' language)\n\n2) If a project contains a folder called .githooks in the root of the\ncode base then the rules/scripts that are present there are executed\nONLY on the system doing the actual commit. These scripts are run in\nsuch a limited way that they can only read the files in the\nrepository, they cannot do any networking/write to disk/etc and they\ncan only do a limited set op actions against the current operation at\nhand (i.e. do checks, parse messages, etc).\n\n3) For the regular hooks this language is also support and when\nlocated in the (not cloned!) .git/hooks directory they are just as\npowerful as a normal script (i.e. can control CI, send emails, etc.).\n\nLike I said, this is just a proposal and I would like to know what you\nguys think.\n\n-- \nBest regards / Met vriendelijke groeten,\n\nNiels Basjes\n"},{"id":"227263","messageId":"CAE1pOi0TioYa2pWCe=8kFbrSNp847rHDUbxXdxBAe=jN3BkWxg@mail.gmail.com","threadId":"34892","inReplyTo":"CADoiZqpec6rPOgPLPQFFfLdE+Cc4ZKtWs0Q0VSfKGm3b1Lai2g@mail.gmail.com","subject":"Re: [Proposal] Clonable scripts","fromName":"Hilco Wijbenga","fromEmail":"hilco.wijbenga@gmail.com","sentAt":"2013-09-09T21:13:00Z","receivedAt":"2013-09-09T21:13:00Z","isPatch":false,"sender":{"key":"hilco.wijbenga@gmail.com","avatar":null},"body":"On 9 September 2013 13:48, Niels Basjes wrote:\n> If those scripts were how ever written in a language that is build\n> into the git program and the script are run in such a way that they\n> can only interact with the files in the local git (and _nothing_\n> outside of that) this would be solved.\n\nThat sounds interesting.\n\n> Also have a builtin scripting language also means that this would run\n> on all operating systems (yes, even Windows).\n\nThis would be *very* helpful. It's a total pain trying to get hooks\nworking across different OSes.\n\n> So I propose the following new feature:\n>\n> 1) A scripting language is put inside git. Perhaps a version of python\n> or ruby or go or ... (no need for a 'new' language)\n\nThat sounds nice but ...\n\n> 2) If a project contains a folder called .githooks in the root of the\n> code base then the rules/scripts that are present there are executed\n> ONLY on the system doing the actual commit. These scripts are run in\n> such a limited way that they can only read the files in the\n> repository, they cannot do any networking/write to disk/etc and they\n> can only do a limited set op actions against the current operation at\n> hand (i.e. do checks, parse messages, etc).\n\n... how would you prevent Ruby/Python/Go/$GeneralProgLang from\nexecuting arbitrary code?\n\n> Like I said, this is just a proposal and I would like to know what you\n> guys think.\n\nI love the idea but I'm not sure how feasible it is. I think you would\nbe forced to copy an existing language and somehow \"make it secure\"\n(seems like a maintenance nightmare) or to create your own language\n(potentially a lot of work). But perhaps something more declarative\nmight be usable?\n"},{"id":"227264","messageId":"CADoiZqqKSX+=yHjU=tTz=0iP_JKzkQvsDancdVXQ3rjkbch9eg@mail.gmail.com","threadId":"34892","inReplyTo":"CAE1pOi0TioYa2pWCe=8kFbrSNp847rHDUbxXdxBAe=jN3BkWxg@mail.gmail.com","subject":"Re: [Proposal] Clonable scripts","fromName":"Niels Basjes","fromEmail":"niels@basjes.nl","sentAt":"2013-09-09T21:23:19Z","receivedAt":"2013-09-09T21:23:19Z","isPatch":false,"sender":{"key":"niels@basjes.nl","avatar":"https://gravatar.com/avatar/1a1e9443d32983849de0a7a64aaad3fd2e238ca69bccbaa71a914d8edbd84ed5?d=mp&s=160"},"body":"On Mon, Sep 9, 2013 at 11:13 PM, Hilco Wijbenga\n wrote:\n> On 9 September 2013 13:48, Niels Basjes wrote:\n>> So I propose the following new feature:\n>>\n>> 1) A scripting language is put inside git. Perhaps a version of python\n>> or ruby or go or ... (no need for a 'new' language)\n>\n> That sounds nice but ...\n\n>> 2) If a project contains a folder called .githooks in the root of the\n>> code base then the rules/scripts that are present there are executed\n>> ONLY on the system doing the actual commit. These scripts are run in\n>> such a limited way that they can only read the files in the\n>> repository, they cannot do any networking/write to disk/etc and they\n>> can only do a limited set op actions against the current operation at\n>> hand (i.e. do checks, parse messages, etc).\n\n> ... how would you prevent Ruby/Python/Go/$GeneralProgLang from\n> executing arbitrary code?\n\nSome kind of sandbox?\n\n>> Like I said, this is just a proposal and I would like to know what you\n>> guys think.\n>\n> I love the idea but I'm not sure how feasible it is. I think you would\n> be forced to copy an existing language and somehow \"make it secure\"\n> (seems like a maintenance nightmare) or to create your own language\n> (potentially a lot of work). But perhaps something more declarative\n> might be usable?\n\nAs far as I'm concerned it should be the 'best suitable' language for\nthe task at hand.\n\n-- \nBest regards / Met vriendelijke groeten,\n\nNiels Basjes\n"},{"id":"227269","messageId":"CALkWK0nk1KfL7YuTKuKJecPSxmu7gXmo67i2Ezb2QY5qmdN+rA@mail.gmail.com","threadId":"34892","inReplyTo":"CADoiZqpec6rPOgPLPQFFfLdE+Cc4ZKtWs0Q0VSfKGm3b1Lai2g@mail.gmail.com","subject":"Re: [Proposal] Clonable scripts","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2013-09-09T22:18:32Z","receivedAt":"2013-09-09T22:18:32Z","isPatch":false,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"Niels Basjes wrote:\n> As we all know the hooks ( in .git/hooks ) are not cloned along with\n> the code of a project.\n> Now this is a correct approach for the scripts that do stuff like\n> emailing the people responsible for releases or submitting the commit\n> to a CI system.\n\nMore often than not, maintainers come with these hooks and they keep\nthem private.\n\n> For several other things it makes a lot of sense to give the developer\n> immediate feedback. Things like the format of the commit message (i.e.\n> it must start with an issue tracker id) or compliance with a coding\n> standard.\n\ni.e. tracker ID. Compliance is simply a request. The developer must be\nable to pick it up from surrounding style.\n\n> Initially I wanted to propose introducing fully clonable (pre-commit)\n> hook scripts.\n> However I can imagine that a malicious opensource coder can create a\n> github repo and try to hack the computer of a contributer via those\n> scripts. So having such scripts is a 'bad idea'.\n\nI think it's a good idea, since the contributer can look through the scripts.\n\n> If those scripts were how ever written in a language that is build\n> into the git program and the script are run in such a way that they\n> can only interact with the files in the local git (and _nothing_\n> outside of that) this would be solved.\n\nGNU make.\n\n> Also have a builtin scripting language also means that this would run\n> on all operating systems (yes, even Windows).\n\nkbuild tends to get complicated.\n\n> So I propose the following new feature:\n>\n> 1) A scripting language is put inside git. Perhaps a version of python\n> or ruby or go or ... (no need for a 'new' language)\n\nmake + go sounds like a good alternative.\n\n> 2) If a project contains a folder called .githooks in the root of the\n> code base then the rules/scripts that are present there are executed\n> ONLY on the system doing the actual commit. These scripts are run in\n> such a limited way that they can only read the files in the\n> repository, they cannot do any networking/write to disk/etc and they\n> can only do a limited set op actions against the current operation at\n> hand (i.e. do checks, parse messages, etc).\n\nSubmodules and url..insteadOf come in handy here.\n\n> 3) For the regular hooks this language is also support and when\n> located in the (not cloned!) .git/hooks directory they are just as\n> powerful as a normal script (i.e. can control CI, send emails, etc.).\n\nI'm confused now; how can .git/hooks be as powerful as .githooks? The\nformer users should consider uploading their code on GitHub.\n\n> Like I said, this is just a proposal and I would like to know what you\n> guys think.\n\n> Best regards / Met vriendelijke groeten,\n\nWhich reminds me that we need to have GitTogethers. Thanks for this!\n"},{"id":"227311","messageId":"CADoiZqpT1BLpcQmTDT=mmJ=5gmQ8b-uZC_qmw8BK2db57YLOmg@mail.gmail.com","threadId":"34892","inReplyTo":"CALkWK0nk1KfL7YuTKuKJecPSxmu7gXmo67i2Ezb2QY5qmdN+rA@mail.gmail.com","subject":"Re: [Proposal] Clonable scripts","fromName":"Niels Basjes","fromEmail":"niels@basjes.nl","sentAt":"2013-09-10T08:35:13Z","receivedAt":"2013-09-10T08:35:13Z","isPatch":false,"sender":{"key":"niels@basjes.nl","avatar":"https://gravatar.com/avatar/1a1e9443d32983849de0a7a64aaad3fd2e238ca69bccbaa71a914d8edbd84ed5?d=mp&s=160"},"body":"Hi,\n\nOn Tue, Sep 10, 2013 at 12:18 AM, Ramkumar Ramachandra\n wrote:\n> Niels Basjes wrote:\n>> As we all know the hooks ( in .git/hooks ) are not cloned along with\n>> the code of a project.\n>> Now this is a correct approach for the scripts that do stuff like\n>> emailing the people responsible for releases or submitting the commit\n>> to a CI system.\n>\n> More often than not, maintainers come with these hooks and they keep\n> them private.\n\nYes.\n\n>> Initially I wanted to propose introducing fully clonable (pre-commit)\n>> hook scripts.\n>> However I can imagine that a malicious opensource coder can create a\n>> github repo and try to hack the computer of a contributer via those\n>> scripts. So having such scripts is a 'bad idea'.\n>\n> I think it's a good idea, since the contributer can look through the scripts.\n\nWhat I meant to say is that having \"fully functional unrestricted\nscripts\" that are cloned is a \"bad idea\".\nHaving \"restricted cloned scripts\" to me is a \"goog idea\" (or atleast,\nthat is what I propose here).\n\n\n>> 3) For the regular hooks this language is also support and when\n>> located in the (not cloned!) .git/hooks directory they are just as\n>> powerful as a normal script (i.e. can control CI, send emails, etc.).\n>\n> I'm confused now; how can .git/hooks be as powerful as .githooks? The\n> former users should consider uploading their code on GitHub.\n\nThe way I envisioned is is that the scripting language in .git/hooks\nis \"pick any language you like\" with the builtin language as a new\naddition.\nIn the .githooks (which is under version control in the code base and\ncloned) is a the same builtin language, yet constrained in a sandbox.\n\n> Which reminds me that we need to have GitTogethers. Thanks for this!\n\nYou're welcome.\n\n-- \nBest regards / Met vriendelijke groeten,\n\nNiels Basjes\n"},{"id":"227313","messageId":"522EEA53.3020308@gmail.com","threadId":"34892","inReplyTo":"CADoiZqpec6rPOgPLPQFFfLdE+Cc4ZKtWs0Q0VSfKGm3b1Lai2g@mail.gmail.com","subject":"Re: [Proposal] Clonable scripts","fromName":"Sitaram Chamarty","fromEmail":"sitaramc@gmail.com","sentAt":"2013-09-10T09:45:55Z","receivedAt":"2013-09-10T09:45:55Z","isPatch":false,"sender":{"key":"sitaramc@gmail.com","avatar":"https://avatars.githubusercontent.com/u/43316?v=4"},"body":"On 09/10/2013 02:18 AM, Niels Basjes wrote:\n\n> As we all know the hooks ( in .git/hooks ) are not cloned along with\n> the code of a project.\n> Now this is a correct approach for the scripts that do stuff like\n> emailing the people responsible for releases or submitting the commit\n> to a CI system.\n> \n> For several other things it makes a lot of sense to give the developer\n> immediate feedback. Things like the format of the commit message (i.e.\n> it must start with an issue tracker id) or compliance with a coding\n> standard.\n> \n> Initially I wanted to propose introducing fully clonable (pre-commit)\n> hook scripts.\n> However I can imagine that a malicious opensource coder can create a\n> github repo and try to hack the computer of a contributer via those\n> scripts. So having such scripts is a 'bad idea'.\n> \n> If those scripts were how ever written in a language that is build\n> into the git program and the script are run in such a way that they\n> can only interact with the files in the local git (and _nothing_\n> outside of that) this would be solved.\n> \n> Also have a builtin scripting language also means that this would run\n> on all operating systems (yes, even Windows).\n> \n> So I propose the following new feature:\n> \n> 1) A scripting language is put inside git. Perhaps a version of python\n> or ruby or go or ... (no need for a 'new' language)\n> \n> 2) If a project contains a folder called .githooks in the root of the\n> code base then the rules/scripts that are present there are executed\n> ONLY on the system doing the actual commit. These scripts are run in\n> such a limited way that they can only read the files in the\n> repository, they cannot do any networking/write to disk/etc and they\n> can only do a limited set op actions against the current operation at\n> hand (i.e. do checks, parse messages, etc).\n> \n> 3) For the regular hooks this language is also support and when\n> located in the (not cloned!) .git/hooks directory they are just as\n> powerful as a normal script (i.e. can control CI, send emails, etc.).\n> \n> Like I said, this is just a proposal and I would like to know what you\n> guys think.\n\nI am not in favour of any idea like this. It will end in some sort of\ncompromise (in both sense of the word!)\n\nIt has to be voluntary, but we can make it easier. I suggest something\nlike this:\n\n - some special directory can have normal hook files, but it's just a\n place holder.\n\n - each hook code file comes with some meta data at the top, say\n githook name, hook name, version, remote-name. I'll use these\n examples:\n\n pre-commit crlf-check 1.1 origin\n\n - on a clone/pull, if there is a change to any of these code files\n when compared to the previous HEAD, and if the program is running\n interactively, then you can ask and setup these hooks.\n\n The purpose of the remote name in the stored metadata is that we\n don't want to bother updating when we pull from some other repo,\n like when merging a feature branch.\n\n The purpose of the version number is so you can do some intelligent\n things, even silently upgrade under certain conditions.\n\nAll we're doing is making things easier compared to what you can already\ndo even now (which is completely manual and instructions based).\n\nI don't think anything more intrusive or forced is wise.\n\nAnd people who say it is OK, I'm going to seriously wonder if you work\nfor the NSA (directly or indirectly). Sadly, that is not meant to be a\njoke question; such is life now.\n"},{"id":"227317","messageId":"20130910125514.GA22078@inner.h.apk.li","threadId":"34892","inReplyTo":"CADoiZqpec6rPOgPLPQFFfLdE+Cc4ZKtWs0Q0VSfKGm3b1Lai2g@mail.gmail.com","subject":"Re: [Proposal] Clonable scripts","fromName":"Andreas Krey","fromEmail":"a.krey@gmx.de","sentAt":"2013-09-10T12:55:14Z","receivedAt":"2013-09-10T12:55:14Z","isPatch":false,"sender":{"key":"a.krey@gmx.de","avatar":"https://avatars.githubusercontent.com/u/37810?v=4"},"body":"On Mon, 09 Sep 2013 22:48:42 +0000, Niels Basjes wrote:\n...\n> However I can imagine that a malicious opensource coder can create a\n> github repo and try to hack the computer of a contributer via those\n> scripts. So having such scripts is a 'bad idea'.\n\nGiven that half the repos out there are cloned to 'make install' in\nthem...it's still a bad idea.\n\n> If those scripts were how ever written in a language that is build\n> into the git program and the script are run in such a way that they\n> can only interact with the files in the local git (and _nothing_\n> outside of that) this would be solved.\n\nI still think this is a nightmare of maintenance. You'd need a restricted\nversion of a language that doesn't allow access outside the repo (and\nno TCP either), and someone will always miss some module...\n\nNot that it wouldn't be cool, yet.\n\n...\n> Like I said, this is just a proposal and I would like to know what you\n> guys think.\n\nI think there are generally two use cases:\n\n- Many people working on repos in an organization. Give them a wrapper\n script that does the clone (and also knows the clone URL already),\n that will set up hooks and configuration as needed.\n\n- github-style cooperation. Add a make hooks to your Makefile that sets\n up the hooks your project seems to want. After all, this is for the\n developers to pre-check what they will submit, so it is in their own\n interest to have (and cross-read) the hooks.\n\nAndreas\n\n-- \n\"Totally trivial. Famous last words.\"\nFrom: Linus Torvalds \nDate: Fri, 22 Jan 2010 07:29:21 -0800\n"}]}