threads / discuss / 34892

[Proposal] Clonable scripts

Subject: [Proposal] Clonable scripts

## tl;dr

7 messages between Sep 9, 2013 and Sep 10, 2013.

replies: 6people: 5as markdown or json

Niels Basjes· Sep 9, 2013, 20:48 UTC · lore
Hi,

As we all know the hooks ( in .git/hooks ) are not cloned along with the code of a project. Now this is a correct approach for the scripts that do stuff like emailing the people responsible for releases or submitting the commit to a CI system.

For several other things it makes a lot of sense to give the developer immediate feedback. Things like the format of the commit message (i.e. it must start with an issue tracker id) or compliance with a coding standard.

Initially I wanted to propose introducing fully clonable (pre-commit) hook scripts. However I can imagine that a malicious opensource coder can create a github repo and try to hack the computer of a contributer via those scripts. So having such scripts is a 'bad idea'.

If those scripts were how ever written in a language that is build into the git program and the script are run in such a way that they can only interact with the files in the local git (and _nothing_ outside of that) this would be solved.

Also have a builtin scripting language also means that this would run on all operating systems (yes, even Windows).

So I propose the following new feature:
1) A scripting language is put inside git. Perhaps a version of python
or ruby or go or ... (no need for a 'new' language)
2) If a project contains a folder called .githooks in the root of the
code base then the rules/scripts that are present there are executed
ONLY on the system doing the actual commit. These scripts are run in
such a limited way that they can only read the files in the
repository, they cannot do any networking/write to disk/etc and they
can only do a limited set op actions against the current operation at
hand (i.e. do checks, parse messages, etc).
3) For the regular hooks this language is also support and when
located in the (not cloned!) .git/hooks directory they are just as
powerful as a normal script (i.e. can control CI, send emails, etc.).

Like I said, this is just a proposal and I would like to know what you guys think.

-- 
Best regards / Met vriendelijke groeten,

Niels Basjes
Hilco Wijbenga· Sep 9, 2013, 21:13 UTC · re: Niels Basjes · lore

Re: [Proposal] Clonable scripts

On 9 September 2013 13:48, Niels Basjes <Niels@basjes.nl> wrote:
> If those scripts were how ever written in a language that is build
> into the git program and the script are run in such a way that they
> can only interact with the files in the local git (and _nothing_
> outside of that) this would be solved.
That sounds interesting.
> Also have a builtin scripting language also means that this would run
> on all operating systems (yes, even Windows).

This would be *very* helpful. It's a total pain trying to get hooks working across different OSes.

> So I propose the following new feature:
>
> 1) A scripting language is put inside git. Perhaps a version of python
> or ruby or go or ... (no need for a 'new' language)
That sounds nice but ...
Show 7 quoted lines
> 2) If a project contains a folder called .githooks in the root of the
> code base then the rules/scripts that are present there are executed
> ONLY on the system doing the actual commit. These scripts are run in
> such a limited way that they can only read the files in the
> repository, they cannot do any networking/write to disk/etc and they
> can only do a limited set op actions against the current operation at
> hand (i.e. do checks, parse messages, etc).

... how would you prevent Ruby/Python/Go/$GeneralProgLang from executing arbitrary code?

> Like I said, this is just a proposal and I would like to know what you
> guys think.

I love the idea but I'm not sure how feasible it is. I think you would be forced to copy an existing language and somehow "make it secure" (seems like a maintenance nightmare) or to create your own language (potentially a lot of work). But perhaps something more declarative might be usable?

Niels Basjes· Sep 9, 2013, 21:23 UTC · re: Hilco Wijbenga · lore

Re: [Proposal] Clonable scripts

On Mon, Sep 9, 2013 at 11:13 PM, Hilco Wijbenga <hilco.wijbenga@gmail.com> wrote:

Show 7 quoted lines
> On 9 September 2013 13:48, Niels Basjes <Niels@basjes.nl> wrote:
>> So I propose the following new feature:
>>
>> 1) A scripting language is put inside git. Perhaps a version of python
>> or ruby or go or ... (no need for a 'new' language)
>
> That sounds nice but ...
Show 7 quoted lines
>> 2) If a project contains a folder called .githooks in the root of the
>> code base then the rules/scripts that are present there are executed
>> ONLY on the system doing the actual commit. These scripts are run in
>> such a limited way that they can only read the files in the
>> repository, they cannot do any networking/write to disk/etc and they
>> can only do a limited set op actions against the current operation at
>> hand (i.e. do checks, parse messages, etc).
> ... how would you prevent Ruby/Python/Go/$GeneralProgLang from
> executing arbitrary code?
Some kind of sandbox?
Show 8 quoted lines
>> Like I said, this is just a proposal and I would like to know what you
>> guys think.
>
> I love the idea but I'm not sure how feasible it is. I think you would
> be forced to copy an existing language and somehow "make it secure"
> (seems like a maintenance nightmare) or to create your own language
> (potentially a lot of work). But perhaps something more declarative
> might be usable?

As far as I'm concerned it should be the 'best suitable' language for the task at hand.

-- 
Best regards / Met vriendelijke groeten,

Niels Basjes
Ramkumar Ramachandra· Sep 9, 2013, 22:18 UTC · re: Niels Basjes · lore

Re: [Proposal] Clonable scripts

Niels Basjes wrote:
Show 5 quoted lines
> As we all know the hooks ( in .git/hooks ) are not cloned along with
> the code of a project.
> Now this is a correct approach for the scripts that do stuff like
> emailing the people responsible for releases or submitting the commit
> to a CI system.

More often than not, maintainers come with these hooks and they keep them private.

> For several other things it makes a lot of sense to give the developer
> immediate feedback. Things like the format of the commit message (i.e.
> it must start with an issue tracker id) or compliance with a coding
> standard.

i.e. tracker ID. Compliance is simply a request. The developer must be able to pick it up from surrounding style.

Show 5 quoted lines
> Initially I wanted to propose introducing fully clonable (pre-commit)
> hook scripts.
> However I can imagine that a malicious opensource coder can create a
> github repo and try to hack the computer of a contributer via those
> scripts. So having such scripts is a 'bad idea'.
I think it's a good idea, since the contributer can look through the scripts.
> If those scripts were how ever written in a language that is build
> into the git program and the script are run in such a way that they
> can only interact with the files in the local git (and _nothing_
> outside of that) this would be solved.
GNU make.
> Also have a builtin scripting language also means that this would run
> on all operating systems (yes, even Windows).
kbuild tends to get complicated.
> So I propose the following new feature:
>
> 1) A scripting language is put inside git. Perhaps a version of python
> or ruby or go or ... (no need for a 'new' language)
make + go sounds like a good alternative.
Show 7 quoted lines
> 2) If a project contains a folder called .githooks in the root of the
> code base then the rules/scripts that are present there are executed
> ONLY on the system doing the actual commit. These scripts are run in
> such a limited way that they can only read the files in the
> repository, they cannot do any networking/write to disk/etc and they
> can only do a limited set op actions against the current operation at
> hand (i.e. do checks, parse messages, etc).
Submodules and url.<url>.insteadOf come in handy here.
> 3) For the regular hooks this language is also support and when
> located in the (not cloned!) .git/hooks directory they are just as
> powerful as a normal script (i.e. can control CI, send emails, etc.).

I'm confused now; how can .git/hooks be as powerful as .githooks? The former users should consider uploading their code on GitHub.

> Like I said, this is just a proposal and I would like to know what you
> guys think.
> Best regards / Met vriendelijke groeten,
Which reminds me that we need to have GitTogethers. Thanks for this!
Niels Basjes· Sep 10, 2013, 08:35 UTC · re: Ramkumar Ramachandra · lore

Re: [Proposal] Clonable scripts

Hi,

On Tue, Sep 10, 2013 at 12:18 AM, Ramkumar Ramachandra <artagnon@gmail.com> wrote:

Show 9 quoted lines
> Niels Basjes wrote:
>> As we all know the hooks ( in .git/hooks ) are not cloned along with
>> the code of a project.
>> Now this is a correct approach for the scripts that do stuff like
>> emailing the people responsible for releases or submitting the commit
>> to a CI system.
>
> More often than not, maintainers come with these hooks and they keep
> them private.
Yes.
Show 7 quoted lines
>> Initially I wanted to propose introducing fully clonable (pre-commit)
>> hook scripts.
>> However I can imagine that a malicious opensource coder can create a
>> github repo and try to hack the computer of a contributer via those
>> scripts. So having such scripts is a 'bad idea'.
>
> I think it's a good idea, since the contributer can look through the scripts.

What I meant to say is that having "fully functional unrestricted scripts" that are cloned is a "bad idea". Having "restricted cloned scripts" to me is a "goog idea" (or atleast, that is what I propose here).

Show 6 quoted lines
>> 3) For the regular hooks this language is also support and when
>> located in the (not cloned!) .git/hooks directory they are just as
>> powerful as a normal script (i.e. can control CI, send emails, etc.).
>
> I'm confused now; how can .git/hooks be as powerful as .githooks? The
> former users should consider uploading their code on GitHub.

The way I envisioned is is that the scripting language in .git/hooks is "pick any language you like" with the builtin language as a new addition. In the .githooks (which is under version control in the code base and cloned) is a the same builtin language, yet constrained in a sandbox.

> Which reminds me that we need to have GitTogethers. Thanks for this!
You're welcome.
-- 
Best regards / Met vriendelijke groeten,

Niels Basjes
Sitaram Chamarty· Sep 10, 2013, 09:45 UTC · re: Niels Basjes · lore

Re: [Proposal] Clonable scripts

On 09/10/2013 02:18 AM, Niels Basjes wrote:
Show 44 quoted lines
> As we all know the hooks ( in .git/hooks ) are not cloned along with
> the code of a project.
> Now this is a correct approach for the scripts that do stuff like
> emailing the people responsible for releases or submitting the commit
> to a CI system.
> 
> For several other things it makes a lot of sense to give the developer
> immediate feedback. Things like the format of the commit message (i.e.
> it must start with an issue tracker id) or compliance with a coding
> standard.
> 
> Initially I wanted to propose introducing fully clonable (pre-commit)
> hook scripts.
> However I can imagine that a malicious opensource coder can create a
> github repo and try to hack the computer of a contributer via those
> scripts. So having such scripts is a 'bad idea'.
> 
> If those scripts were how ever written in a language that is build
> into the git program and the script are run in such a way that they
> can only interact with the files in the local git (and _nothing_
> outside of that) this would be solved.
> 
> Also have a builtin scripting language also means that this would run
> on all operating systems (yes, even Windows).
> 
> So I propose the following new feature:
> 
> 1) A scripting language is put inside git. Perhaps a version of python
> or ruby or go or ... (no need for a 'new' language)
> 
> 2) If a project contains a folder called .githooks in the root of the
> code base then the rules/scripts that are present there are executed
> ONLY on the system doing the actual commit. These scripts are run in
> such a limited way that they can only read the files in the
> repository, they cannot do any networking/write to disk/etc and they
> can only do a limited set op actions against the current operation at
> hand (i.e. do checks, parse messages, etc).
> 
> 3) For the regular hooks this language is also support and when
> located in the (not cloned!) .git/hooks directory they are just as
> powerful as a normal script (i.e. can control CI, send emails, etc.).
> 
> Like I said, this is just a proposal and I would like to know what you
> guys think.

I am not in favour of any idea like this. It will end in some sort of compromise (in both sense of the word!)

It has to be voluntary, but we can make it easier. I suggest something like this:

  - some special directory can have normal hook files, but it's just a
    place holder.
  - each hook code file comes with some meta data at the top, say
    githook name, hook name, version, remote-name.  I'll use these
    examples:
        pre-commit  crlf-check  1.1     origin
  - on a clone/pull, if there is a change to any of these code files
    when compared to the previous HEAD, and if the program is running
    interactively, then you can ask and setup these hooks.
    The purpose of the remote name in the stored metadata is that we
    don't want to bother updating when we pull from some other repo,
    like when merging a feature branch.
    The purpose of the version number is so you can do some intelligent
    things, even silently upgrade under certain conditions.

All we're doing is making things easier compared to what you can already do even now (which is completely manual and instructions based).

I don't think anything more intrusive or forced is wise.

And people who say it is OK, I'm going to seriously wonder if you work for the NSA (directly or indirectly). Sadly, that is not meant to be a joke question; such is life now.

Andreas Krey· Sep 10, 2013, 12:55 UTC · re: Niels Basjes · lore

Re: [Proposal] Clonable scripts

On Mon, 09 Sep 2013 22:48:42 +0000, Niels Basjes wrote: ...

> However I can imagine that a malicious opensource coder can create a
> github repo and try to hack the computer of a contributer via those
> scripts. So having such scripts is a 'bad idea'.

Given that half the repos out there are cloned to 'make install' in them...it's still a bad idea.

> If those scripts were how ever written in a language that is build
> into the git program and the script are run in such a way that they
> can only interact with the files in the local git (and _nothing_
> outside of that) this would be solved.

I still think this is a nightmare of maintenance. You'd need a restricted version of a language that doesn't allow access outside the repo (and no TCP either), and someone will always miss some module...

Not that it wouldn't be cool, yet.
...
> Like I said, this is just a proposal and I would like to know what you
> guys think.
I think there are generally two use cases:
- Many people working on repos in an organization. Give them a wrapper
  script that does the clone (and also knows the clone URL already),
  that will set up hooks and configuration as needed.
- github-style cooperation. Add a make hooks to your Makefile that sets
  up the hooks your project seems to want. After all, this is for the
  developers to pre-check what they will submit, so it is in their own
  interest to have (and cross-read) the hooks.
Andreas
-- 
"Totally trivial. Famous last words."
From: Linus Torvalds <torvalds@*.org>
Date: Fri, 22 Jan 2010 07:29:21 -0800

← back to recent threads