{"thread":{"id":"44417","subject":"[PATCH] transport: add core.allowProtocol config option","startedAt":"2016-11-02T22:21:19Z","lastAt":"2016-12-15T17:29:40Z","messageCount":117,"participants":["Brandon Williams","Stefan Beller","Jeff King","Jonathan Nieder","Blake Burkhart","Junio C Hamano","Jacob Keller"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"305323","messageId":"1478125247-62372-1-git-send-email-bmwill@google.com","threadId":"44417","inReplyTo":null,"subject":"[PATCH] transport: add core.allowProtocol config option","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-11-02T22:20:47Z","receivedAt":"2016-11-02T22:21:19Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"Add configuration option 'core.allowProtocol' to allow users to create a\nwhitelist of allowed protocols for fetch/push/clone in their gitconfig.\n\nFor git-submodule.sh, fallback to default whitelist only if the user\nhasn't explicitly set `GIT_ALLOW_PROTOCOL` or doesn't have a whitelist\nin their gitconfig.\n\nSigned-off-by: Brandon Williams \n---\n Documentation/config.txt | 9 +++++++++\n git-submodule.sh | 3 ++-\n transport.c | 2 +-\n 3 files changed, 12 insertions(+), 2 deletions(-)\n\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex 27069ac..7f83e40 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -455,6 +455,15 @@ core.sshCommand::\n \tthe `GIT_SSH_COMMAND` environment variable and is overridden\n \twhen the environment variable is set.\n \n+core.allowProtocol::\n+\tProvide a colon-separated list of protocols which are allowed to be\n+\tused with fetch/push/clone. This is useful to restrict recursive\n+\tsubmodule initialization from an untrusted repository. Any protocol not\n+\tmentioned will be disallowed (i.e., this is a whitelist, not a\n+\tblacklist). If the variable is not set at all, all protocols are\n+\tenabled. If the `GIT_ALLOW_PROTOCOL` enviornment variable is set, it is\n+\tused as the protocol whitelist instead of this config option.\n+\n core.ignoreStat::\n \tIf true, Git will avoid using lstat() calls to detect if files have\n \tchanged by setting the \"assume-unchanged\" bit for those tracked files\ndiff --git a/git-submodule.sh b/git-submodule.sh\nindex a024a13..ad94c75 100755\n--- a/git-submodule.sh\n+++ b/git-submodule.sh\n@@ -27,7 +27,8 @@ cd_to_toplevel\n #\n # If the user has already specified a set of allowed protocols,\n # we assume they know what they're doing and use that instead.\n-: ${GIT_ALLOW_PROTOCOL=file:git:http:https:ssh}\n+config_whitelist=$(git config core.allowProtocol)\n+: ${GIT_ALLOW_PROTOCOL=${config_whitelist:-file:git:http:https:ssh}}\n export GIT_ALLOW_PROTOCOL\n \n command=\ndiff --git a/transport.c b/transport.c\nindex d57e8de..b1098cd 100644\n--- a/transport.c\n+++ b/transport.c\n@@ -652,7 +652,7 @@ static const struct string_list *protocol_whitelist(void)\n \n \tif (enabled < 0) {\n \t\tconst char *v = getenv(\"GIT_ALLOW_PROTOCOL\");\n-\t\tif (v) {\n+\t\tif (v || !git_config_get_value(\"core.allowProtocol\", &v)) {\n \t\t\tstring_list_split(&allowed, v, ':', -1);\n \t\t\tstring_list_sort(&allowed);\n \t\t\tenabled = 1;\n-- \n2.8.0.rc3.226.g39d4020\n\n"},{"id":"305326","messageId":"CAGZ79kbytopD1UgvU6R0X18Ge6JsJ46K2KQ-SFLh+hu5tprTmA@mail.gmail.com","threadId":"44417","inReplyTo":"1478125247-62372-1-git-send-email-bmwill@google.com","subject":"Re: [PATCH] transport: add core.allowProtocol config option","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2016-11-02T22:41:02Z","receivedAt":"2016-11-02T22:41:08Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Wed, Nov 2, 2016 at 3:20 PM, Brandon Williams wrote:\n> Add configuration option 'core.allowProtocol' to allow users to create a\n> whitelist of allowed protocols for fetch/push/clone in their gitconfig.\n>\n> For git-submodule.sh, fallback to default whitelist only if the user\n> hasn't explicitly set `GIT_ALLOW_PROTOCOL` or doesn't have a whitelist\n> in their gitconfig.\n>\n> Signed-off-by: Brandon Williams \n> ---\n> Documentation/config.txt | 9 +++++++++\n> git-submodule.sh | 3 ++-\n> transport.c | 2 +-\n> 3 files changed, 12 insertions(+), 2 deletions(-)\n>\n> diff --git a/Documentation/config.txt b/Documentation/config.txt\n> index 27069ac..7f83e40 100644\n> --- a/Documentation/config.txt\n> +++ b/Documentation/config.txt\n> @@ -455,6 +455,15 @@ core.sshCommand::\n> the `GIT_SSH_COMMAND` environment variable and is overridden\n> when the environment variable is set.\n>\n> +core.allowProtocol::\n> + Provide a colon-separated list of protocols which are allowed to be\n> + used with fetch/push/clone.\n\nok.\n\n> This is useful to restrict recursive\n> + submodule initialization from an untrusted repository.\n\nok. Though as a user submodules may not spring to mind immediately here.\nI think this is generally useful, too. e.g. an admin could put this in\nthe system wide\nconfig to prevent certain protocols from being used.\n\n> Any protocol not\n> + mentioned will be disallowed\n\nFor the regular fetch/clone/pull case. For the submodule case we still\nfall back to\nthe hardcoded list of known good things?\n\n> (i.e., this is a whitelist, not a\n> + blacklist).\n\nThat is very explicit, I'd drop it. However this inspires bike\nshedding on the name:\nWhat about core.protocolWhitelist instead?\n\n> If the variable is not set at all, all protocols are\n> + enabled. If the `GIT_ALLOW_PROTOCOL` enviornment variable is set, it is\n> + used as the protocol whitelist instead of this config option.\n\nSo the env var is of higher priority than this config.\n\n> +\n> core.ignoreStat::\n> If true, Git will avoid using lstat() calls to detect if files have\n> changed by setting the \"assume-unchanged\" bit for those tracked files\n> diff --git a/git-submodule.sh b/git-submodule.sh\n> index a024a13..ad94c75 100755\n> --- a/git-submodule.sh\n> +++ b/git-submodule.sh\n> @@ -27,7 +27,8 @@ cd_to_toplevel\n> #\n> # If the user has already specified a set of allowed protocols,\n> # we assume they know what they're doing and use that instead.\n> -: ${GIT_ALLOW_PROTOCOL=file:git:http:https:ssh}\n> +config_whitelist=$(git config core.allowProtocol)\n\nSo first we lookup the configured protocols.\n\n> +: ${GIT_ALLOW_PROTOCOL=${config_whitelist:-file:git:http:https:ssh}}\n\nThen if they are not configured use the current hard coded white list.\n\n> export GIT_ALLOW_PROTOCOL\n>\n> command=\n> diff --git a/transport.c b/transport.c\n> index d57e8de..b1098cd 100644\n> --- a/transport.c\n> +++ b/transport.c\n> @@ -652,7 +652,7 @@ static const struct string_list *protocol_whitelist(void)\n>\n> if (enabled < 0) {\n> const char *v = getenv(\"GIT_ALLOW_PROTOCOL\");\n> - if (v) {\n> + if (v || !git_config_get_value(\"core.allowProtocol\", &v)) {\n\nThis implementation matches what the config promised, I would think.\n\nDo we have any tests for this that could be extended? (Otherwise we'd\nmaybe want to add a test for both the regular case as well as a forbidden\nsubmodule?)\n\n\n> string_list_split(&allowed, v, ':', -1);\n> string_list_sort(&allowed);\n> enabled = 1;\n> --\n> 2.8.0.rc3.226.g39d4020\n>\n"},{"id":"305327","messageId":"20161102224736.GA183367@google.com","threadId":"44417","inReplyTo":"CAGZ79kbytopD1UgvU6R0X18Ge6JsJ46K2KQ-SFLh+hu5tprTmA@mail.gmail.com","subject":"Re: [PATCH] transport: add core.allowProtocol config option","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-11-02T22:47:36Z","receivedAt":"2016-11-02T22:47:43Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 11/02, Stefan Beller wrote:\n\n> > This is useful to restrict recursive\n> > + submodule initialization from an untrusted repository.\n> \n> ok. Though as a user submodules may not spring to mind immediately here.\n> I think this is generally useful, too. e.g. an admin could put this in\n> the system wide\n> config to prevent certain protocols from being used.\n\nOh I pretty much copied the description from what exists for\n`GIT_ALLOW_PROTOCOL` which included this bit about submodules.\n\n> > Any protocol not\n> > + mentioned will be disallowed\n> \n> For the regular fetch/clone/pull case. For the submodule case we still\n> fall back to\n> the hardcoded list of known good things?\n\nYep! This is done by explicitly setting GIT_ALLOW_PROTOCOL to the\nhardcoded list if the user hasn't supplied a whitelist.\n\n> \n> > (i.e., this is a whitelist, not a\n> > + blacklist).\n> \n> That is very explicit, I'd drop it. However this inspires bike\n> shedding on the name:\n> What about core.protocolWhitelist instead?\n\nSimply to keep the name similar to the env variable that already exists\nfor this functionality.\n\n> So the env var is of higher priority than this config.\n\n\n> > -: ${GIT_ALLOW_PROTOCOL=file:git:http:https:ssh}\n> > +config_whitelist=$(git config core.allowProtocol)\n> \n> So first we lookup the configured protocols.\n> \n> > +: ${GIT_ALLOW_PROTOCOL=${config_whitelist:-file:git:http:https:ssh}}\n> \n> Then if they are not configured use the current hard coded white list.\n\nThe lookup of the configured whitelist is done first but wont be used\nunless GIT_ALLOW_PROTOCOL is unset. If neither is set it will fallback\nto the hardcoded list.\n\n> Do we have any tests for this that could be extended? (Otherwise we'd\n> maybe want to add a test for both the regular case as well as a forbidden\n> submodule?)\n> \n\nI can write a couple tests for a v2 of the patch.\n\n-- \nBrandon Williams\n"},{"id":"305328","messageId":"20161102230538.jx3jwa4hqgrrltno@sigill.intra.peff.net","threadId":"44417","inReplyTo":"1478125247-62372-1-git-send-email-bmwill@google.com","subject":"Re: [PATCH] transport: add core.allowProtocol config option","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-11-02T23:05:39Z","receivedAt":"2016-11-02T23:05:47Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Nov 02, 2016 at 03:20:47PM -0700, Brandon Williams wrote:\n\n> Add configuration option 'core.allowProtocol' to allow users to create a\n> whitelist of allowed protocols for fetch/push/clone in their gitconfig.\n> \n> For git-submodule.sh, fallback to default whitelist only if the user\n> hasn't explicitly set `GIT_ALLOW_PROTOCOL` or doesn't have a whitelist\n> in their gitconfig.\n\nThis says \"what\", but not \"why\". What's the use case?\n\nI can see somebody wanting to pare down the whitelist further (e.g.,\nbecause they are carrying ssh credentials that they don't want to use on\nbehalf of a malicious repo). But in general I'd expect this setting to\nbe a function of the environment you're operating in, and not the\non-disk config.\n\nOr is the intent to broaden it for cases where you have a clone that\nuses some non-standard protocol, and you want it to Just Work on\nsubsequent recursive fetches?\n\n> +core.allowProtocol::\n> +\tProvide a colon-separated list of protocols which are allowed to be\n> +\tused with fetch/push/clone. This is useful to restrict recursive\n> +\tsubmodule initialization from an untrusted repository. Any protocol not\n> +\tmentioned will be disallowed (i.e., this is a whitelist, not a\n> +\tblacklist). If the variable is not set at all, all protocols are\n> +\tenabled. If the `GIT_ALLOW_PROTOCOL` enviornment variable is set, it is\n> +\tused as the protocol whitelist instead of this config option.\n\nThe \"not set at all, all protocols are enabled\" bit is not quite\ncorrect, is it? It is true for a top-level fetch, but not for submodule\nrecursion (and especially since you are talking about submodule\nrecursion immediately before, it is rather confusing).\n\n> --- a/git-submodule.sh\n> +++ b/git-submodule.sh\n> @@ -27,7 +27,8 @@ cd_to_toplevel\n> #\n> # If the user has already specified a set of allowed protocols,\n> # we assume they know what they're doing and use that instead.\n> -: ${GIT_ALLOW_PROTOCOL=file:git:http:https:ssh}\n> +config_whitelist=$(git config core.allowProtocol)\n> +: ${GIT_ALLOW_PROTOCOL=${config_whitelist:-file:git:http:https:ssh}}\n\nThe original uses \"=\" without a \":\" so that an empty variable takes\nprecedence over the stock list (i.e., allowing nothing). Would you want\nthe same behavior for the config variable? I.e.:\n\n # this should probably allow nothing, right?\n git config core.allowProtocol \"\"\n\nI think you'd have to check the return code of \"git config\" to\ndistinguish those cases.\n\n> diff --git a/transport.c b/transport.c\n> index d57e8de..b1098cd 100644\n> --- a/transport.c\n> +++ b/transport.c\n> @@ -652,7 +652,7 @@ static const struct string_list *protocol_whitelist(void)\n> \n> \tif (enabled < 0) {\n> \t\tconst char *v = getenv(\"GIT_ALLOW_PROTOCOL\");\n> -\t\tif (v) {\n> +\t\tif (v || !git_config_get_value(\"core.allowProtocol\", &v)) {\n> \t\t\tstring_list_split(&allowed, v, ':', -1);\n> \t\t\tstring_list_sort(&allowed);\n> \t\t\tenabled = 1;\n\nI thought at first we'd have to deal with leaking \"v\", but \"get_value\"\nis the \"raw\" version that gives you the uninterpreted value. I think\nthat means it may give you NULL, though if we see an implicit bool like:\n\n [core]\n allowProtocol\n\nThat's nonsense, of course, but we would still segfault. I\nthink the easiest way to test is:\n\n git -c core.allowProtocol fetch\n\nwhich seems to segfault for me with this patch.\n\n-Peff\n"},{"id":"305329","messageId":"20161102230853.3xdk6ksnzzote5ud@sigill.intra.peff.net","threadId":"44417","inReplyTo":"20161102230538.jx3jwa4hqgrrltno@sigill.intra.peff.net","subject":"Re: [PATCH] transport: add core.allowProtocol config option","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-11-02T23:08:53Z","receivedAt":"2016-11-02T23:09:00Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Nov 02, 2016 at 07:05:39PM -0400, Jeff King wrote:\n\n> > +core.allowProtocol::\n> > +\tProvide a colon-separated list of protocols which are allowed to be\n> > +\tused with fetch/push/clone. This is useful to restrict recursive\n> > +\tsubmodule initialization from an untrusted repository. Any protocol not\n> > +\tmentioned will be disallowed (i.e., this is a whitelist, not a\n> > +\tblacklist). If the variable is not set at all, all protocols are\n> > +\tenabled. If the `GIT_ALLOW_PROTOCOL` enviornment variable is set, it is\n> > +\tused as the protocol whitelist instead of this config option.\n> \n> The \"not set at all, all protocols are enabled\" bit is not quite\n> correct, is it? It is true for a top-level fetch, but not for submodule\n> recursion (and especially since you are talking about submodule\n> recursion immediately before, it is rather confusing).\n\nHeh, just saw that you copied this straight from the discussion of\nGIT_ALLOW_PROTOCOL. What idiot wrote the original? :)\n\nIt might be worth fixing both places (or possibly just fixing the\noriginal and phrasing this one as \"If GIT_ALLOW_PROTOCOL is not set, use\nthis as the default value; see git(1) for details\").\n\n-Peff\n"},{"id":"305334","messageId":"20161102233304.GB183367@google.com","threadId":"44417","inReplyTo":"20161102230538.jx3jwa4hqgrrltno@sigill.intra.peff.net","subject":"Re: [PATCH] transport: add core.allowProtocol config option","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-11-02T23:33:04Z","receivedAt":"2016-11-02T23:33:11Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 11/02, Jeff King wrote:\n> On Wed, Nov 02, 2016 at 03:20:47PM -0700, Brandon Williams wrote:\n> \n> > Add configuration option 'core.allowProtocol' to allow users to create a\n> > whitelist of allowed protocols for fetch/push/clone in their gitconfig.\n> > \n> > For git-submodule.sh, fallback to default whitelist only if the user\n> > hasn't explicitly set `GIT_ALLOW_PROTOCOL` or doesn't have a whitelist\n> > in their gitconfig.\n> \n> This says \"what\", but not \"why\". What's the use case?\n> \n> I can see somebody wanting to pare down the whitelist further (e.g.,\n> because they are carrying ssh credentials that they don't want to use on\n> behalf of a malicious repo). But in general I'd expect this setting to\n> be a function of the environment you're operating in, and not the\n> on-disk config.\n> \n> Or is the intent to broaden it for cases where you have a clone that\n> uses some non-standard protocol, and you want it to Just Work on\n> subsequent recursive fetches?\n> \n> > +core.allowProtocol::\n> > +\tProvide a colon-separated list of protocols which are allowed to be\n> > +\tused with fetch/push/clone. This is useful to restrict recursive\n> > +\tsubmodule initialization from an untrusted repository. Any protocol not\n> > +\tmentioned will be disallowed (i.e., this is a whitelist, not a\n> > +\tblacklist). If the variable is not set at all, all protocols are\n> > +\tenabled. If the `GIT_ALLOW_PROTOCOL` enviornment variable is set, it is\n> > +\tused as the protocol whitelist instead of this config option.\n> \n> The \"not set at all, all protocols are enabled\" bit is not quite\n> correct, is it? It is true for a top-level fetch, but not for submodule\n> recursion (and especially since you are talking about submodule\n> recursion immediately before, it is rather confusing).\n\nYeah stefan mentioned this to me. I simply copied the documentaion from\nGIT_ALLOW_PROTOCOL, perhaps that should be updated as well?\n\n> \n> > --- a/git-submodule.sh\n> > +++ b/git-submodule.sh\n> > @@ -27,7 +27,8 @@ cd_to_toplevel\n> > #\n> > # If the user has already specified a set of allowed protocols,\n> > # we assume they know what they're doing and use that instead.\n> > -: ${GIT_ALLOW_PROTOCOL=file:git:http:https:ssh}\n> > +config_whitelist=$(git config core.allowProtocol)\n> > +: ${GIT_ALLOW_PROTOCOL=${config_whitelist:-file:git:http:https:ssh}}\n> \n> The original uses \"=\" without a \":\" so that an empty variable takes\n> precedence over the stock list (i.e., allowing nothing). Would you want\n> the same behavior for the config variable? I.e.:\n> \n> # this should probably allow nothing, right?\n> git config core.allowProtocol \"\"\n> \n> I think you'd have to check the return code of \"git config\" to\n> distinguish those cases.\n\nOh, I didn't think of that case. That can be done easy enough, just\nmakes the code a bit more verbose.\n\n> \n> > diff --git a/transport.c b/transport.c\n> > index d57e8de..b1098cd 100644\n> > --- a/transport.c\n> > +++ b/transport.c\n> > @@ -652,7 +652,7 @@ static const struct string_list *protocol_whitelist(void)\n> > \n> > \tif (enabled < 0) {\n> > \t\tconst char *v = getenv(\"GIT_ALLOW_PROTOCOL\");\n> > -\t\tif (v) {\n> > +\t\tif (v || !git_config_get_value(\"core.allowProtocol\", &v)) {\n> > \t\t\tstring_list_split(&allowed, v, ':', -1);\n> > \t\t\tstring_list_sort(&allowed);\n> > \t\t\tenabled = 1;\n> \n> I thought at first we'd have to deal with leaking \"v\", but \"get_value\"\n> is the \"raw\" version that gives you the uninterpreted value. I think\n> that means it may give you NULL, though if we see an implicit bool like:\n> \n> [core]\n> allowProtocol\n> \n> That's nonsense, of course, but we would still segfault. I\n> think the easiest way to test is:\n> \n> git -c core.allowProtocol fetch\n> \n> which seems to segfault for me with this patch.\n\nwhat is the desired behavior when a user provides a config in a way that\nisn't intended?\n\n-- \nBrandon Williams\n"},{"id":"305335","messageId":"20161102233455.GC183367@google.com","threadId":"44417","inReplyTo":"20161102230853.3xdk6ksnzzote5ud@sigill.intra.peff.net","subject":"Re: [PATCH] transport: add core.allowProtocol config option","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-11-02T23:34:55Z","receivedAt":"2016-11-02T23:35:03Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 11/02, Jeff King wrote:\n> On Wed, Nov 02, 2016 at 07:05:39PM -0400, Jeff King wrote:\n> \n> > > +core.allowProtocol::\n> > > +\tProvide a colon-separated list of protocols which are allowed to be\n> > > +\tused with fetch/push/clone. This is useful to restrict recursive\n> > > +\tsubmodule initialization from an untrusted repository. Any protocol not\n> > > +\tmentioned will be disallowed (i.e., this is a whitelist, not a\n> > > +\tblacklist). If the variable is not set at all, all protocols are\n> > > +\tenabled. If the `GIT_ALLOW_PROTOCOL` enviornment variable is set, it is\n> > > +\tused as the protocol whitelist instead of this config option.\n> > \n> > The \"not set at all, all protocols are enabled\" bit is not quite\n> > correct, is it? It is true for a top-level fetch, but not for submodule\n> > recursion (and especially since you are talking about submodule\n> > recursion immediately before, it is rather confusing).\n> \n> Heh, just saw that you copied this straight from the discussion of\n> GIT_ALLOW_PROTOCOL. What idiot wrote the original? :)\n> \n> It might be worth fixing both places (or possibly just fixing the\n> original and phrasing this one as \"If GIT_ALLOW_PROTOCOL is not set, use\n> this as the default value; see git(1) for details\").\n> \n> -Peff\n\nhaha K I'll fix the original as well.\n\n-- \nBrandon Williams\n"},{"id":"305336","messageId":"20161102234613.GD183367@google.com","threadId":"44417","inReplyTo":"20161102233304.GB183367@google.com","subject":"Re: [PATCH] transport: add core.allowProtocol config option","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-11-02T23:46:13Z","receivedAt":"2016-11-02T23:46:20Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"> > > diff --git a/transport.c b/transport.c\n> > > index d57e8de..b1098cd 100644\n> > > --- a/transport.c\n> > > +++ b/transport.c\n> > > @@ -652,7 +652,7 @@ static const struct string_list *protocol_whitelist(void)\n> > > \n> > > \tif (enabled < 0) {\n> > > \t\tconst char *v = getenv(\"GIT_ALLOW_PROTOCOL\");\n> > > -\t\tif (v) {\n> > > +\t\tif (v || !git_config_get_value(\"core.allowProtocol\", &v)) {\n> > > \t\t\tstring_list_split(&allowed, v, ':', -1);\n> > > \t\t\tstring_list_sort(&allowed);\n> > > \t\t\tenabled = 1;\n> > \n> > I thought at first we'd have to deal with leaking \"v\", but \"get_value\"\n> > is the \"raw\" version that gives you the uninterpreted value. I think\n> > that means it may give you NULL, though if we see an implicit bool like:\n> > \n> > [core]\n> > allowProtocol\n> > \n> > That's nonsense, of course, but we would still segfault. I\n> > think the easiest way to test is:\n> > \n> > git -c core.allowProtocol fetch\n> > \n> > which seems to segfault for me with this patch.\n> \n> what is the desired behavior when a user provides a config in a way that\n> isn't intended?\n\noh...I can just drop in git_config_get_string_const() instead.\n\n-- \nBrandon Williams\n"},{"id":"305337","messageId":"20161103000839.fwmqqggvmhzo7zmx@sigill.intra.peff.net","threadId":"44417","inReplyTo":"20161102234613.GD183367@google.com","subject":"Re: [PATCH] transport: add core.allowProtocol config option","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-11-03T00:08:39Z","receivedAt":"2016-11-03T00:08:45Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Nov 02, 2016 at 04:46:13PM -0700, Brandon Williams wrote:\n\n> > > I thought at first we'd have to deal with leaking \"v\", but \"get_value\"\n> > > is the \"raw\" version that gives you the uninterpreted value. I think\n> > > that means it may give you NULL, though if we see an implicit bool like:\n> > > \n> > > [core]\n> > > allowProtocol\n> > > \n> > > That's nonsense, of course, but we would still segfault. I\n> > > think the easiest way to test is:\n> > > \n> > > git -c core.allowProtocol fetch\n> > > \n> > > which seems to segfault for me with this patch.\n> > \n> > what is the desired behavior when a user provides a config in a way that\n> > isn't intended?\n> \n> oh...I can just drop in git_config_get_string_const() instead.\n\nYes, it will call git_config_string(), which will make sure there's an\nactual value and die otherwise. But note that it also duplicates the\nstring, so you'd have to deal with freeing it.\n\n-Peff\n"},{"id":"305338","messageId":"20161103002225.GA13369@google.com","threadId":"44417","inReplyTo":"1478125247-62372-1-git-send-email-bmwill@google.com","subject":"Re: [PATCH] transport: add core.allowProtocol config option","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2016-11-03T00:22:25Z","receivedAt":"2016-11-03T00:23:02Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"(+peff and bburky, who introduced GIT_ALLOW_PROTOCOL)\nBrandon Williams wrote:\n\n> Add configuration option 'core.allowProtocol' to allow users to create a\n> whitelist of allowed protocols for fetch/push/clone in their gitconfig.\n\nOoh.\n\nThis would be especially useful at $DAYJOB, where there is a custom\nsso:// protocol that is often used by submodules. Using an envvar to\nwhitelist it globally is painful because\n\n - it disables other protocols even when explicitly requested on a\n plain \"git clone\" command line by the user. By comparison, the\n built-in git-submodule.sh whitelist only applies to submodules.\n\n - platform-specific instructions to set an environment variable can\n be more difficult than \"just set this git configuration\"\n\nAnother difficulty with setting GIT_ALLOW_PROTOCOL globally is that it\nrequires copy/pasting the default value from upstream and then adding\nthe values I want. There's no straightforward way to get the current\nvalue and add to it, in case I want to benefit from future upstream\nfixes to the default list.\n\nThat is, would it be possible to use something like\n\n\t[protocol \"sso\"]\n\t\tallow = always\n\ninstead of\n\n\t[core]\n\t\tallowProtocol = file:git:http:https:....:sso\n\n?\n\n[...]\n> --- a/git-submodule.sh\n> +++ b/git-submodule.sh\n> @@ -27,7 +27,8 @@ cd_to_toplevel\n> #\n> # If the user has already specified a set of allowed protocols,\n> # we assume they know what they're doing and use that instead.\n> -: ${GIT_ALLOW_PROTOCOL=file:git:http:https:ssh}\n> +config_whitelist=$(git config core.allowProtocol)\n> +: ${GIT_ALLOW_PROTOCOL=${config_whitelist:-file:git:http:https:ssh}}\n\noptional: To avoid config parsing when GIT_ALLOW_PROTOCOL is already\nset, could do something like\n\n if ! test \"${GIT_ALLOW_PROTOCOL+set}\"\n then\n\tGIT_ALLOW_PROTOCOL=$(\n\t\tgit config --name-only --get-regexp 'protocol\\..*\\.allow' always |\n\t\tsed -e 's/^protocol.//' -e 's/.allow$//' |\n\t\ttr '\\n' ':'\n\t)\n\tGIT_ALLOW_PROTOCOL=${GIT_ALLOW_PROTOCOL%:}\n\t: ${GIT_ALLOW_PROTOCOL:=file:git:http:https:ssh}\n fi\n\n[...]\n> --- a/transport.c\n> +++ b/transport.c\n> @@ -652,7 +652,7 @@ static const struct string_list *protocol_whitelist(void)\n> \n> \tif (enabled < 0) {\n> \t\tconst char *v = getenv(\"GIT_ALLOW_PROTOCOL\");\n> -\t\tif (v) {\n> +\t\tif (v || !git_config_get_value(\"core.allowProtocol\", &v)) {\n> \t\t\tstring_list_split(&allowed, v, ':', -1);\n\nThis has the effect of always disabling other protocols when\ncore.allowProtocol is set. Is that intended?\n\nLike the default list used by submodule, I'd be happiest if this only\napplied to repositories cloned implicitly instead of those passed\ndirectly to 'git clone'.\n\nThat reminds me: external tools also set GIT_ALLOW_PROTOCOL when the\nuser hasn't set it explicitly, like git-submodule.sh does. E.g.\nrepo ,\nmercurial .\nOther external tools consume GIT_ALLOW_PROTOCOL, like 'go get'\n.\nCan we make it more convenient for them to support this configuration\ntoo?\n\nAn example approach would be a GIT_ALLOW_PROTOCOL var returned by\n\"git var\".\n\nThat way git-submodule.sh could do\n\n\t: ${GIT_ALLOW_PROTOCOL=$(git var GIT_ALLOW_PROTOCOL)}\n\nand it would just work. Other tools could do the same, with a\nfallback to the current default until new enough git is in widespread\nuse.\n\nThanks and hope that helps,\nJonathan\n"},{"id":"305340","messageId":"CAP3OtXjrwqxPHGxdc7P_Dv+S9sWgtqE61jzQ8rVByphYC5D47g@mail.gmail.com","threadId":"44417","inReplyTo":"20161103002225.GA13369@google.com","subject":"Re: [PATCH] transport: add core.allowProtocol config option","fromName":"Blake Burkhart","fromEmail":"bburky@bburky.com","sentAt":"2016-11-03T00:41:16Z","receivedAt":"2016-11-03T00:41:41Z","isPatch":true,"sender":{"key":"bburky@bburky.com","avatar":"https://avatars.githubusercontent.com/u/140526?v=4"},"body":"Thanks for CCing me.\n\nI haven't looked at this implementation in detail, but it would be\ngood to move this configuration into the config system because I think\nwe can more easily provide a default safe configuration.\n\nIt would be nice to use this to introduce a default list of\nwhitelisted protocols that even applies to `git clone`. I strongly\nthink we need to find a way to have git-remote-ext disabled by\ndefault. This could be a way to do it.\n\nOn Wed, Nov 2, 2016 at 7:22 PM, Jonathan Nieder wrote:\n> That reminds me: external tools also set GIT_ALLOW_PROTOCOL when the\n> user hasn't set it explicitly, like git-submodule.sh does. E.g.\n> repo ,\n> mercurial .\n> Other external tools consume GIT_ALLOW_PROTOCOL, like 'go get'\n> .\n> Can we make it more convenient for them to support this configuration\n> too?\n\nMost of these are my fault too. I encouraged git-repo and mercurial to\nuse GIT_ALLOW_PROTOCOL to avoid security issues from git-remote-ext.\n\n-- \nBlake Burkhart\n"},{"id":"305341","messageId":"1478134215-61707-1-git-send-email-bmwill@google.com","threadId":"44417","inReplyTo":"1478125247-62372-1-git-send-email-bmwill@google.com","subject":"[PATCH v2] transport: add core.allowProtocol config option","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-11-03T00:50:15Z","receivedAt":"2016-11-03T00:50:42Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"Add configuration option 'core.allowProtocol' to allow users to create a\nwhitelist of allowed protocols for fetch/push/clone in their gitconfig.\n\nFor git-submodule.sh, fallback to default whitelist only if the user\nhasn't explicitly set `GIT_ALLOW_PROTOCOL` or doesn't have a whitelist\nin their gitconfig.\n\nSigned-off-by: Brandon Williams \n---\n Documentation/config.txt | 8 ++++++++\n Documentation/git.txt | 6 ++++--\n git-submodule.sh | 3 ++-\n t/lib-proto-disable.sh | 27 +++++++++++++++++++++++++++\n t/t5815-submodule-protos.sh | 22 ++++++++++++++++++++++\n transport.c | 6 ++++++\n 6 files changed, 69 insertions(+), 3 deletions(-)\n\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex 27069ac..78c97e3 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -455,6 +455,14 @@ core.sshCommand::\n \tthe `GIT_SSH_COMMAND` environment variable and is overridden\n \twhen the environment variable is set.\n \n+core.allowProtocol::\n+\tProvide a colon-separated list of protocols which are allowed to be\n+\tused with fetch/push/clone. Any protocol not mentioned will be\n+\tdisallowed (i.e., this is a whitelist, not a blacklist). If the\n+\t`GIT_ALLOW_PROTOCOL` environment variable is set, it is used as the\n+\tprotocol whitelist instead of this config option. If neither is set,\n+\tall protocols are enabled. See git(1) for more details.\n+\n core.ignoreStat::\n \tIf true, Git will avoid using lstat() calls to detect if files have\n \tchanged by setting the \"assume-unchanged\" bit for those tracked files\ndiff --git a/Documentation/git.txt b/Documentation/git.txt\nindex ab7215e..a86ec16 100644\n--- a/Documentation/git.txt\n+++ b/Documentation/git.txt\n@@ -1155,8 +1155,10 @@ of clones and fetches.\n \trestrict recursive submodule initialization from an untrusted\n \trepository. Any protocol not mentioned will be disallowed (i.e.,\n \tthis is a whitelist, not a blacklist). If the variable is not\n-\tset at all, all protocols are enabled. The protocol names\n-\tcurrently used by git are:\n+\tset at all, all protocols are enabled. The exception to this is when\n+\trunning `git-submodule` which will use a default list of known-safe\n+\tprotocols (file:git:http:https:ssh) in the event no whitelist is\n+\tprovided. The protocol names currently used by git are:\n \n \t - `file`: any local file-based path (including `file://` URLs,\n \t or local paths)\ndiff --git a/git-submodule.sh b/git-submodule.sh\nindex a024a13..bfbfb8e 100755\n--- a/git-submodule.sh\n+++ b/git-submodule.sh\n@@ -27,7 +27,8 @@ cd_to_toplevel\n #\n # If the user has already specified a set of allowed protocols,\n # we assume they know what they're doing and use that instead.\n-: ${GIT_ALLOW_PROTOCOL=file:git:http:https:ssh}\n+whitelist=$(git config core.allowProtocol) || whitelist=file:git:http:https:ssh\n+: ${GIT_ALLOW_PROTOCOL=$whitelist}\n export GIT_ALLOW_PROTOCOL\n \n command=\ndiff --git a/t/lib-proto-disable.sh b/t/lib-proto-disable.sh\nindex b0917d9..e8820a6 100644\n--- a/t/lib-proto-disable.sh\n+++ b/t/lib-proto-disable.sh\n@@ -62,6 +62,33 @@ test_proto () {\n \t\t\ttest_must_fail git clone --bare \"$url\" tmp.git\n \t\t)\n \t'\n+\n+\t# Run tests again using the gitconfig method for setting a whitelist\n+\ttest_expect_success \"clone $1 (enabled)\" '\n+\t\trm -rf tmp.git &&\n+\t\tgit -c core.allowProtocol=\"$proto\" clone --bare \"$url\" tmp.git\n+\t'\n+\n+\ttest_expect_success \"fetch $1 (enabled)\" '\n+\t\tgit -C tmp.git -c core.allowProtocol=\"$proto\" fetch\n+\t'\n+\n+\ttest_expect_success \"push $1 (enabled)\" '\n+\t\tgit -C tmp.git -c core.allowProtocol=\"$proto\" push origin HEAD:pushed\n+\t'\n+\n+\ttest_expect_success \"push $1 (disabled)\" '\n+\t\ttest_must_fail git -C tmp.git -c core.allowProtocol=none push origin HEAD:pushed\n+\t'\n+\n+\ttest_expect_success \"fetch $1 (disabled)\" '\n+\t\ttest_must_fail git -C tmp.git -c core.allowProtocol=none fetch\n+\t'\n+\n+\ttest_expect_success \"clone $1 (disabled)\" '\n+\t\trm -rf tmp.git &&\n+\t\ttest_must_fail git -C tmp.git -c core.allowProtocol=none clone --bare \"$url\" tmp.git\n+\t'\n }\n \n # set up an ssh wrapper that will access $host/$repo in the\ndiff --git a/t/t5815-submodule-protos.sh b/t/t5815-submodule-protos.sh\nindex 06f55a1..f85e71c 100755\n--- a/t/t5815-submodule-protos.sh\n+++ b/t/t5815-submodule-protos.sh\n@@ -40,4 +40,26 @@ test_expect_success 'user can override whitelist' '\n \tGIT_ALLOW_PROTOCOL=ext git -C dst submodule update ext-module\n '\n \n+test_expect_success 'reset dst repo for config tests' '\n+\trm -rf dst &&\n+\tgit clone . dst &&\n+\tgit -C dst submodule init\n+'\n+\n+test_expect_success 'update of ssh not allowed when not in config whitelist' '\n+\ttest_must_fail git -C dst -c core.allowProtocol=none submodule update ssh-module\n+'\n+\n+test_expect_success 'update of ssh allowed via config whitelist' '\n+\tgit -C dst -c core.allowProtocol=\"ssh:http:https\" submodule update ssh-module\n+'\n+\n+test_expect_success 'update of ext not allowed' '\n+\ttest_must_fail git -C dst -c core.allowProtocol=ssh submodule update ext-module\n+'\n+\n+test_expect_success 'user can override whitelist' '\n+\tgit -C dst -c core.allowProtocol=ext submodule update ext-module\n+'\n+\n test_done\ndiff --git a/transport.c b/transport.c\nindex d57e8de..e3d8e88 100644\n--- a/transport.c\n+++ b/transport.c\n@@ -652,10 +652,16 @@ static const struct string_list *protocol_whitelist(void)\n \n \tif (enabled < 0) {\n \t\tconst char *v = getenv(\"GIT_ALLOW_PROTOCOL\");\n+\t\tchar *w = NULL;\n \t\tif (v) {\n \t\t\tstring_list_split(&allowed, v, ':', -1);\n \t\t\tstring_list_sort(&allowed);\n \t\t\tenabled = 1;\n+\t\t} else if (!git_config_get_string(\"core.allowProtocol\", &w)) {\n+\t\t\tstring_list_split(&allowed, w, ':', -1);\n+\t\t\tstring_list_sort(&allowed);\n+\t\t\tenabled = 1;\n+\t\t\tfree(w);\n \t\t} else {\n \t\t\tenabled = 0;\n \t\t}\n-- \n2.8.0.rc3.226.g39d4020\n\n"},{"id":"305342","messageId":"xmqqtwbpe0yi.fsf@gitster.mtv.corp.google.com","threadId":"44417","inReplyTo":"20161103002225.GA13369@google.com","subject":"Re: [PATCH] transport: add core.allowProtocol config option","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-11-03T02:44:53Z","receivedAt":"2016-11-03T02:45:01Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jonathan Nieder writes:\n\n> That is, would it be possible to use something like\n>\n> \t[protocol \"sso\"]\n> \t\tallow = always\n>\n> instead of\n>\n> \t[core]\n> \t\tallowProtocol = file:git:http:https:....:sso\n>\n> ?\n\nThat sounds like quite a large usability improvement to me.\n\n"},{"id":"305348","messageId":"20161103143806.hce4msk3dhxtgpre@sigill.intra.peff.net","threadId":"44417","inReplyTo":"20161103002225.GA13369@google.com","subject":"Re: [PATCH] transport: add core.allowProtocol config option","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-11-03T14:38:06Z","receivedAt":"2016-11-03T14:38:14Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Nov 02, 2016 at 05:22:25PM -0700, Jonathan Nieder wrote:\n\n> Another difficulty with setting GIT_ALLOW_PROTOCOL globally is that it\n> requires copy/pasting the default value from upstream and then adding\n> the values I want. There's no straightforward way to get the current\n> value and add to it, in case I want to benefit from future upstream\n> fixes to the default list.\n\nI agree that this is a big drawback of the current scheme, and it would\nbe nice to be able to say \"also allow\".\n\n> That is, would it be possible to use something like\n> \n> \t[protocol \"sso\"]\n> \t\tallow = always\n> \n> instead of\n> \n> \t[core]\n> \t\tallowProtocol = file:git:http:https:....:sso\n> \n> ?\n\nOne complication is that the whitelist has multiple states:\n\n 1. if it's not used at all, anything goes\n\n 2. if it exists and has zero or more entries, only those entries are\n allowed\n\nAnd then submodules are an exception to (1), because it's not anything\ngoes. It's \"this default safe whitelist\".\n\nSo when does protocol.sso.allow kick in? We wouldn't want it to trigger\ncase (2) for things like fetch (disabling other non-allowed protocols).\nNor do I think we'd only want it for the submodule case, as I would\nassume that \"protocol.sso.allow = false\" should disable it.\n\nSo I think this probably needs to be a separate parallel system where\neach protocol can be white- or black-listed in a context-specific way.\nLike:\n\n protocol.X.allow = always | user | never\n\nWhere \"user\" specifies that the protocol is OK coming directly from the\nuser, but not from other sources. And we default known-common-and-good\nones like protocol.http.allow to \"always\", unknown ones (like \"foo\"\nwhich runs \"remote-foo\") to \"user\"), and possibly known-scary ones like\n\"ext\") to \"never\".\n\nThen we need some way of telling git \"you are in a context where the URL\nparameter is not coming from the user\". Probably via the environment in\nGIT_PROTOCOL_FROM_USER or similar. Which git-submodule would sent when\nrecursing clones, along with things like \"go get\".\n\nIn other words, stop asking git-submodule or \"go get\" to specify policy,\nand let them specify context that can be used to implement policy that\nthe user specifies (and have git provide a sane default policy).\n\nI think this would all take a backseat to GIT_ALLOW_PROTOCOL, for\nbackwards compatibility, and then GIT_ALLOW_PROTOCOL could slowly die\noff over time.\n\n> That reminds me: external tools also set GIT_ALLOW_PROTOCOL when the\n> user hasn't set it explicitly, like git-submodule.sh does. E.g.\n> repo ,\n> mercurial .\n> Other external tools consume GIT_ALLOW_PROTOCOL, like 'go get'\n> .\n> Can we make it more convenient for them to support this configuration\n> too?\n\nI think under my proposal above this ugliness just goes away, as all\nthey have to do is say \"trust my URLs less; they come from an automated\nsource\" without specifying policy themselves.\n\nBut we may still want...\n\n> An example approach would be a GIT_ALLOW_PROTOCOL var returned by\n> \"git var\".\n> \n> That way git-submodule.sh could do\n> \n> \t: ${GIT_ALLOW_PROTOCOL=$(git var GIT_ALLOW_PROTOCOL)}\n> \n> and it would just work. Other tools could do the same, with a\n> fallback to the current default until new enough git is in widespread\n> use.\n\n...some automated way to say \"is this protocol supported\"? I think it is\nnot just \"give me ALLOW_PROTOCOL\" anymore, though, but \"apply your rules\nto this protocol, and tell me if it is supported\".\n\nI don't think things like \"go get\" would need it, but you would if you\nhad a porcelain built around git that was accessing a URL _not_ via\ngit-fetch, but wanted to apply git's rules. That could come as a step 2\nlater, though.\n\n-Peff\n"},{"id":"305367","messageId":"20161103172515.GA182568@google.com","threadId":"44417","inReplyTo":"20161103143806.hce4msk3dhxtgpre@sigill.intra.peff.net","subject":"Re: [PATCH] transport: add core.allowProtocol config option","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-11-03T17:25:15Z","receivedAt":"2016-11-03T17:25:23Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 11/03, Jeff King wrote:\n> On Wed, Nov 02, 2016 at 05:22:25PM -0700, Jonathan Nieder wrote:\n> \n> > Another difficulty with setting GIT_ALLOW_PROTOCOL globally is that it\n> > requires copy/pasting the default value from upstream and then adding\n> > the values I want. There's no straightforward way to get the current\n> > value and add to it, in case I want to benefit from future upstream\n> > fixes to the default list.\n> \n> I agree that this is a big drawback of the current scheme, and it would\n> be nice to be able to say \"also allow\".\n> \n> > That is, would it be possible to use something like\n> > \n> > \t[protocol \"sso\"]\n> > \t\tallow = always\n> > \n> > instead of\n> > \n> > \t[core]\n> > \t\tallowProtocol = file:git:http:https:....:sso\n> > \n> > ?\n> \n> One complication is that the whitelist has multiple states:\n> \n> 1. if it's not used at all, anything goes\n> \n> 2. if it exists and has zero or more entries, only those entries are\n> allowed\n> \n> And then submodules are an exception to (1), because it's not anything\n> goes. It's \"this default safe whitelist\".\n> \n> So when does protocol.sso.allow kick in? We wouldn't want it to trigger\n> case (2) for things like fetch (disabling other non-allowed protocols).\n> Nor do I think we'd only want it for the submodule case, as I would\n> assume that \"protocol.sso.allow = false\" should disable it.\n> \n> So I think this probably needs to be a separate parallel system where\n> each protocol can be white- or black-listed in a context-specific way.\n> Like:\n> \n> protocol.X.allow = always | user | never\n\nIt sounds like there is interest for this sort of behavior, it would\ndefinitely require a larger change than what I initially proposed. One\nproblem I see though is that with this we have support for both a\nblacklist and a whitelist. Which wins? Or do we simply generate a\nwhitelist of allowed protocols which includes all protocols with allow\nset to 'always' and if it is set to 'never' then it just isn't included\nin the whitelist?\n\nI don't know if I'm sold on a 'user' state just yet, perhaps that's just\nbecause I view a whitelist or blacklist as well black and white and\nhaving this user state adds in a gray area.\n\n> > An example approach would be a GIT_ALLOW_PROTOCOL var returned by\n> > \"git var\".\n> > \n> > That way git-submodule.sh could do\n> > \n> > \t: ${GIT_ALLOW_PROTOCOL=$(git var GIT_ALLOW_PROTOCOL)}\n> > \n> > and it would just work. Other tools could do the same, with a\n> > fallback to the current default until new enough git is in widespread\n> > use.\n> \n> ...some automated way to say \"is this protocol supported\"? I think it is\n> not just \"give me ALLOW_PROTOCOL\" anymore, though, but \"apply your rules\n> to this protocol, and tell me if it is supported\".\n\nI agree, if we do add different states to a protocol then we couldn't\nsimply ask for a whitelist/blacklist of protocols anymore since its more\nof a graylist :) (if such a thing exits). \n\n-- \nBrandon Williams\n"},{"id":"305368","messageId":"CAGZ79kbvs+ryGRjCkHYO=3iNK4tPPhBhYjRMZaH7rP0QMrULhg@mail.gmail.com","threadId":"44417","inReplyTo":"20161103172515.GA182568@google.com","subject":"Re: [PATCH] transport: add core.allowProtocol config option","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2016-11-03T17:39:35Z","receivedAt":"2016-11-03T17:39:41Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":">> protocol.X.allow = always | user | never\n>\n> It sounds like there is interest for this sort of behavior, it would\n> definitely require a larger change than what I initially proposed. One\n> problem I see though is that with this we have support for both a\n> blacklist and a whitelist. Which wins?\n\nFor the submodule operations we'll use a whitelist, because we want to\nprovide security and for the other case we can offer a blacklist as a bandaid.\n\nMy opinion on blacklists is roughly aligned with e.g. :\nhttps://blog.codinghorror.com/blacklists-dont-work/\nhttp://blog.deepinstinct.com/2016/02/04/when-blacklists-dont-really-work/\n\nSo IMHO we could drop the \"never\" and substitute it with a \"warn\" or\n\"ask-user\", such that this configuration becomes a white list for both cases:\n\n protocol.X.allow = always | user | warn\n\n> Or do we simply generate a\n> whitelist of allowed protocols which includes all protocols with allow\n> set to 'always' and if it is set to 'never' then it just isn't included\n> in the whitelist?\n\nSo you're suggesting that setting it to \"never\" doesn't have any effect\nexcept for cluttering the config file?\nI don't think we should do that; each setting should have an impact.\nSo maybe the \"never\" would be there to disallow protocols of the hardcoded\nwhite list (e.g. http)\n\n>\n> I don't know if I'm sold on a 'user' state just yet, perhaps that's just\n> because I view a whitelist or blacklist as well black and white and\n> having this user state adds in a gray area.\n\nWell the \"user\" state is to differentiate between the\n* \"I consciously typed `git clone ...` (and e.g. I know what happens as\n I know the server admin and they are trustworthy.)\n* a repository contains a possible hostile .gitmodules file such\n that I am not aware of the network connection.\n"},{"id":"305370","messageId":"20161103175131.GB182568@google.com","threadId":"44417","inReplyTo":"CAGZ79kbvs+ryGRjCkHYO=3iNK4tPPhBhYjRMZaH7rP0QMrULhg@mail.gmail.com","subject":"Re: [PATCH] transport: add core.allowProtocol config option","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-11-03T17:51:31Z","receivedAt":"2016-11-03T17:51:42Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 11/03, Stefan Beller wrote:\n> >> protocol.X.allow = always | user | never\n> >\n> > It sounds like there is interest for this sort of behavior, it would\n> > definitely require a larger change than what I initially proposed. One\n> > problem I see though is that with this we have support for both a\n> > blacklist and a whitelist. Which wins?\n> \n> For the submodule operations we'll use a whitelist, because we want to\n> provide security and for the other case we can offer a blacklist as a bandaid.\n> \n> My opinion on blacklists is roughly aligned with e.g. :\n> https://blog.codinghorror.com/blacklists-dont-work/\n> http://blog.deepinstinct.com/2016/02/04/when-blacklists-dont-really-work/\n> \n> So IMHO we could drop the \"never\" and substitute it with a \"warn\" or\n> \"ask-user\", such that this configuration becomes a white list for both cases:\n> \n> protocol.X.allow = always | user | warn\n> \n> > Or do we simply generate a\n> > whitelist of allowed protocols which includes all protocols with allow\n> > set to 'always' and if it is set to 'never' then it just isn't included\n> > in the whitelist?\n> \n> So you're suggesting that setting it to \"never\" doesn't have any effect\n> except for cluttering the config file?\n> I don't think we should do that; each setting should have an impact.\n> So maybe the \"never\" would be there to disallow protocols of the hardcoded\n> white list (e.g. http)\n\nThats what I meant, if a protocol is listed as 'never' then it just\nremoves that protocol from the whitelist. That way we still have the\nbenefit of using a whitelist vs a blacklist. Also, if we move in this\ndirection should we setup a default whitelist of allowed protocols?\n\n> >\n> > I don't know if I'm sold on a 'user' state just yet, perhaps that's just\n> > because I view a whitelist or blacklist as well black and white and\n> > having this user state adds in a gray area.\n> \n> Well the \"user\" state is to differentiate between the\n> * \"I consciously typed `git clone ...` (and e.g. I know what happens as\n> I know the server admin and they are trustworthy.)\n> * a repository contains a possible hostile .gitmodules file such\n> that I am not aware of the network connection.\n\nThis is still a gray area to me. I think that if we have a whitelist of\nprotocols then it should be a true whitelist and not have some means of\ngoing around it. It just seems like something that could be exploited.\n\n-- \nBrandon Williams\n"},{"id":"305371","messageId":"20161103175327.nn2yasvlsxsy22be@sigill.intra.peff.net","threadId":"44417","inReplyTo":"20161103172515.GA182568@google.com","subject":"Re: [PATCH] transport: add core.allowProtocol config option","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-11-03T17:53:27Z","receivedAt":"2016-11-03T17:53:34Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Nov 03, 2016 at 10:25:15AM -0700, Brandon Williams wrote:\n\n> > So I think this probably needs to be a separate parallel system where\n> > each protocol can be white- or black-listed in a context-specific way.\n> > Like:\n> > \n> > protocol.X.allow = always | user | never\n> \n> It sounds like there is interest for this sort of behavior, it would\n> definitely require a larger change than what I initially proposed. One\n> problem I see though is that with this we have support for both a\n> blacklist and a whitelist. Which wins? Or do we simply generate a\n> whitelist of allowed protocols which includes all protocols with allow\n> set to 'always' and if it is set to 'never' then it just isn't included\n> in the whitelist?\n\nI think trying to combine the two or generate the whitelist from the\nmore flexible format is a recipe for madness.\n\nI'd design the new system from scratch, and have it kick in _only_ when\nGIT_ALLOW_PROTOCOL is not set. That lets existing callers continue to\nhave the safe behavior until they are ready to move to the new format.\n\nSomething like the patch below (which is just for illustration, and not\ntested beyond compilation).\n\n> I don't know if I'm sold on a 'user' state just yet, perhaps that's just\n> because I view a whitelist or blacklist as well black and white and\n> having this user state adds in a gray area.\n\nThe lack of understanding the \"user\" context is what makes the current\nsystem so painful. The only way a caller can influence the system is to\nhand over the policy directly: this is allowed, this is not. But systems\nlike \"go get\" should not be setting policy. They should be giving us a\nhint about the context, and letting git implement the policy.\n\n-- >8 --\ndiff --git a/transport.c b/transport.c\nindex d57e8dec2..0d5b382db 100644\n--- a/transport.c\n+++ b/transport.c\n@@ -664,10 +664,69 @@ static const struct string_list *protocol_whitelist(void)\n \treturn enabled ? &allowed : NULL;\n }\n \n+enum protocol_allow_config {\n+\tPROTOCOL_ALLOW_NEVER,\n+\tPROTOCOL_ALLOW_USER_ONLY,\n+\tPROTOCOL_ALLOW_ALWAYS\n+};\n+\n+static enum protocol_allow_config parse_protocol_config(const char *key,\n+\t\t\t\t\t\t\tconst char *value)\n+{\n+\tif (!strcasecmp(value, \"always\"))\n+\t\treturn PROTOCOL_ALLOW_ALWAYS;\n+\telse if (!strcasecmp(value, \"never\"))\n+\t\treturn PROTOCOL_ALLOW_NEVER;\n+\telse if (!strcasecmp(value, \"user\"))\n+\t\treturn PROTOCOL_ALLOW_USER_ONLY;\n+\n+\t/* XXX maybe also interpret git_config_bool() here? */\n+\tdie(\"unknown value for config '%s': %s\", key, value);\n+}\n+\n+static enum protocol_allow_config get_protocol_config(const char *type)\n+{\n+\tchar *key = xstrfmt(\"protocol.%s.allow\", type);\n+\tchar *value;\n+\n+\tif (!git_config_get_string(key, &value)) {\n+\t\tenum protocol_allow_config ret =\n+\t\t\tparse_protocol_config(key, value);\n+\t\tfree(key);\n+\t\tfree(value);\n+\t\treturn ret;\n+\t}\n+\tfree(key);\n+\n+\t/* known safe */\n+\tif (!strcmp(type, \"http\") ||\n+\t !strcmp(type, \"https\") ||\n+\t !strcmp(type, \"git\") ||\n+\t !strcmp(type, \"ssh\"))\n+\t\treturn PROTOCOL_ALLOW_ALWAYS;\n+\n+\t/* known scary; err on the side of caution */\n+\tif (!strcmp(type, \"ext\"))\n+\t\treturn PROTOCOL_ALLOW_NEVER;\n+\n+\t/* unknown; let them be used only directly by the user */\n+\treturn PROTOCOL_ALLOW_USER_ONLY;\n+}\n+\n int is_transport_allowed(const char *type)\n {\n-\tconst struct string_list *allowed = protocol_whitelist();\n-\treturn !allowed || string_list_has_string(allowed, type);\n+\tconst struct string_list *whitelist = protocol_whitelist();\n+\tif (whitelist)\n+\t\treturn string_list_has_string(whitelist, type);\n+\n+\tswitch (get_protocol_config(type)) {\n+\tcase PROTOCOL_ALLOW_ALWAYS:\n+\t\treturn 1;\n+\tcase PROTOCOL_ALLOW_NEVER:\n+\t\treturn 0;\n+\tcase PROTOCOL_ALLOW_USER_ONLY:\n+\t\treturn git_env_bool(\"GIT_PROTOCOL_FROM_USER\", 1);\n+\t}\n }\n \n void transport_check_allowed(const char *type)\n"},{"id":"305372","messageId":"20161103180017.x2hjb5wm2u6isajm@sigill.intra.peff.net","threadId":"44417","inReplyTo":"CAGZ79kbvs+ryGRjCkHYO=3iNK4tPPhBhYjRMZaH7rP0QMrULhg@mail.gmail.com","subject":"Re: [PATCH] transport: add core.allowProtocol config option","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-11-03T18:00:17Z","receivedAt":"2016-11-03T18:00:25Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Nov 03, 2016 at 10:39:35AM -0700, Stefan Beller wrote:\n\n> >> protocol.X.allow = always | user | never\n> >\n> > It sounds like there is interest for this sort of behavior, it would\n> > definitely require a larger change than what I initially proposed. One\n> > problem I see though is that with this we have support for both a\n> > blacklist and a whitelist. Which wins?\n> \n> For the submodule operations we'll use a whitelist, because we want to\n> provide security and for the other case we can offer a blacklist as a bandaid.\n> \n> My opinion on blacklists is roughly aligned with e.g. :\n> https://blog.codinghorror.com/blacklists-dont-work/\n> http://blog.deepinstinct.com/2016/02/04/when-blacklists-dont-really-work/\n> \n> So IMHO we could drop the \"never\" and substitute it with a \"warn\" or\n> \"ask-user\", such that this configuration becomes a white list for both cases:\n> \n> protocol.X.allow = always | user | warn\n\nI don't think blacklists work in the general case, because they grow out\nof date and fail-open. But you want to have _some_ blacklisting\nmechanism, in order override a decision of the whitelist.\n\nFor instance, the default submodule whitelist would probably include\nhttps and ssh. But if I'm cloning potentially malicious repos and I\ndon't ever want them to trigger ssh (because I don't want them to use my\nssh keys, whereas I have explicitly set up my credentials such that http\nis safe to use), I would want to be able to do:\n\n git config protocol.ssh.allow never\n\n(or \"git -c\", or whatever).\n\nTrue, a whitelist is safer. If we add a new \"foo\" protocol that also\nlooks at your ssh keys, you're screwed. And that's why I designed it as\na pure-whitelist in the first place. But it comes at the price of\nconvenience, because you have to manually add each new innocent protocol\nto the whitelist.\n\n> So you're suggesting that setting it to \"never\" doesn't have any effect\n> except for cluttering the config file?\n> I don't think we should do that; each setting should have an impact.\n> So maybe the \"never\" would be there to disallow protocols of the hardcoded\n> white list (e.g. http)\n\nExactly.\n\n> > I don't know if I'm sold on a 'user' state just yet, perhaps that's just\n> > because I view a whitelist or blacklist as well black and white and\n> > having this user state adds in a gray area.\n> \n> Well the \"user\" state is to differentiate between the\n> * \"I consciously typed `git clone ...` (and e.g. I know what happens as\n> I know the server admin and they are trustworthy.)\n> * a repository contains a possible hostile .gitmodules file such\n> that I am not aware of the network connection.\n\nRight. I had assumed that we would tell the difference between those\nautomatically (by seeing if we got the URL on the command line), but\nthings like \"go get\" show that the context is often before git is even\ncalled.\n\n-Peff\n"},{"id":"305373","messageId":"20161103180224.7mhpziotpinmd3tr@sigill.intra.peff.net","threadId":"44417","inReplyTo":"20161103175131.GB182568@google.com","subject":"Re: [PATCH] transport: add core.allowProtocol config option","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-11-03T18:02:24Z","receivedAt":"2016-11-03T18:02:30Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Nov 03, 2016 at 10:51:31AM -0700, Brandon Williams wrote:\n\n> > > I don't know if I'm sold on a 'user' state just yet, perhaps that's just\n> > > because I view a whitelist or blacklist as well black and white and\n> > > having this user state adds in a gray area.\n> > \n> > Well the \"user\" state is to differentiate between the\n> > * \"I consciously typed `git clone ...` (and e.g. I know what happens as\n> > I know the server admin and they are trustworthy.)\n> > * a repository contains a possible hostile .gitmodules file such\n> > that I am not aware of the network connection.\n> \n> This is still a gray area to me. I think that if we have a whitelist of\n> protocols then it should be a true whitelist and not have some means of\n> going around it. It just seems like something that could be exploited.\n\nHow do you implement:\n\n git clone --recursive trusted:foo.git\n\nand use your ssh keys for the \"trusted\" server, but not for any servers\nmentioned in .gitmodules?\n\nYou need some way of distinguishing between the two contexts (and\nsetting policy for each).\n\n-Peff\n"},{"id":"305374","messageId":"20161103180817.GC182568@google.com","threadId":"44417","inReplyTo":"20161103180224.7mhpziotpinmd3tr@sigill.intra.peff.net","subject":"Re: [PATCH] transport: add core.allowProtocol config option","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-11-03T18:08:17Z","receivedAt":"2016-11-03T18:08:26Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 11/03, Jeff King wrote:\n> On Thu, Nov 03, 2016 at 10:51:31AM -0700, Brandon Williams wrote:\n> \n> > > > I don't know if I'm sold on a 'user' state just yet, perhaps that's just\n> > > > because I view a whitelist or blacklist as well black and white and\n> > > > having this user state adds in a gray area.\n> > > \n> > > Well the \"user\" state is to differentiate between the\n> > > * \"I consciously typed `git clone ...` (and e.g. I know what happens as\n> > > I know the server admin and they are trustworthy.)\n> > > * a repository contains a possible hostile .gitmodules file such\n> > > that I am not aware of the network connection.\n> > \n> > This is still a gray area to me. I think that if we have a whitelist of\n> > protocols then it should be a true whitelist and not have some means of\n> > going around it. It just seems like something that could be exploited.\n> \n> How do you implement:\n> \n> git clone --recursive trusted:foo.git\n> \n> and use your ssh keys for the \"trusted\" server, but not for any servers\n> mentioned in .gitmodules?\n> \n> You need some way of distinguishing between the two contexts (and\n> setting policy for each).\n> \n> -Peff\n\nInteresting. Ok I can see how this would be a useful now. Thanks for\nthe example :)\n\n-- \nBrandon Williams\n"},{"id":"305375","messageId":"20161103181954.GD182568@google.com","threadId":"44417","inReplyTo":"20161103175327.nn2yasvlsxsy22be@sigill.intra.peff.net","subject":"Re: [PATCH] transport: add core.allowProtocol config option","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-11-03T18:19:54Z","receivedAt":"2016-11-03T18:20:05Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 11/03, Jeff King wrote:\n> +\n> +\t/* unknown; let them be used only directly by the user */\n> +\treturn PROTOCOL_ALLOW_USER_ONLY;\n> +}\n> +\n> int is_transport_allowed(const char *type)\n> {\n> -\tconst struct string_list *allowed = protocol_whitelist();\n> -\treturn !allowed || string_list_has_string(allowed, type);\n> +\tconst struct string_list *whitelist = protocol_whitelist();\n> +\tif (whitelist)\n> +\t\treturn string_list_has_string(whitelist, type);\n> +\n> +\tswitch (get_protocol_config(type)) {\n> +\tcase PROTOCOL_ALLOW_ALWAYS:\n> +\t\treturn 1;\n> +\tcase PROTOCOL_ALLOW_NEVER:\n> +\t\treturn 0;\n> +\tcase PROTOCOL_ALLOW_USER_ONLY:\n> +\t\treturn git_env_bool(\"GIT_PROTOCOL_FROM_USER\", 1);\n> +\t}\n\nI know this is just a rough patch you wiped up but one question:\nWith the 'user' state, how exactly do you envision this env variable\nworking? Do we want the user to have to explicitly set\nGIT_PROTOCOL_FROM_USER in their environment and then have these other\ncommands (like git-submodule) explicitly clear the env var or would we\nrather these subcommands set a variable indicating they aren't coming\nfrom the user and the deafult state (no var set) is a user run command?\n\n-- \nBrandon Williams\n"},{"id":"305376","messageId":"20161103182428.j3r574evsk7ypfie@sigill.intra.peff.net","threadId":"44417","inReplyTo":"20161103175327.nn2yasvlsxsy22be@sigill.intra.peff.net","subject":"Re: [PATCH] transport: add core.allowProtocol config option","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-11-03T18:24:28Z","receivedAt":"2016-11-03T18:24:35Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Nov 03, 2016 at 01:53:27PM -0400, Jeff King wrote:\n\n> I'd design the new system from scratch, and have it kick in _only_ when\n> GIT_ALLOW_PROTOCOL is not set. That lets existing callers continue to\n> have the safe behavior until they are ready to move to the new format.\n> \n> Something like the patch below (which is just for illustration, and not\n> tested beyond compilation).\n\nHere's that same patch with a few tweaks:\n\n - it changes git-submodule to use the new, more flexible system (which\n also gets a it a lot more test coverage)\n\n - it tweaks two tests which use the \"ext\" helper to enable it (since\n it's blacklisted by default; I have mixed feelings on that, but I\n see why Blake wants it, as it would have protected things like \"go\n get\" out of the box).\n\n - it adds \"file://\" as a known-good protocol, even for submodules,\n which matches the current code. I am not sure if this is reasonable\n or not. A malicious repository probably can't do much by pointing\n you to cloning your own repo as a submodule unless you then _also_\n run some arbitrary code to expose it, at which point it's generally\n game-over anyway.\n\n And I'd expect automated services (like GitHub Pages) to already\n have a cut-down whitelist via GIT_ALLOW_PROTOCOL (and I happen to\n know that it goes).\n\nSo this seems like a reasonable direction to me. It obviously needs\ndocumentation and tests. Arguably there should be a fallback \"allow\"\nvalue when a protocol is not mentioned in the config so that you could\nconvert the default from \"user\" to \"never\" if you wanted your config to\nspecify a pure whitelist.\n\nWithout that, I think we'd want to keep GIT_ALLOW_PROTOCOL for the truly\nparanoid (though we should keep it indefinitely either way for backwards\ncompatibility).\n\nDo you have interest in picking this up and running with it?\n\n-Peff\n\ndiff --git a/git-submodule.sh b/git-submodule.sh\nindex a024a135d..0a477b4c9 100755\n--- a/git-submodule.sh\n+++ b/git-submodule.sh\n@@ -21,14 +21,10 @@ require_work_tree\n wt_prefix=$(git rev-parse --show-prefix)\n cd_to_toplevel\n \n-# Restrict ourselves to a vanilla subset of protocols; the URLs\n-# we get are under control of a remote repository, and we do not\n-# want them kicking off arbitrary git-remote-* programs.\n-#\n-# If the user has already specified a set of allowed protocols,\n-# we assume they know what they're doing and use that instead.\n-: ${GIT_ALLOW_PROTOCOL=file:git:http:https:ssh}\n-export GIT_ALLOW_PROTOCOL\n+# Tell the rest of git that any URLs we get don't come\n+# directly from the user, so it can apply policy as appropriate.\n+GIT_PROTOCOL_FROM_USER=0\n+export GIT_PROTOCOL_FROM_USER\n \n command=\n branch=\ndiff --git a/t/t5509-fetch-push-namespaces.sh b/t/t5509-fetch-push-namespaces.sh\nindex bc44ac36d..75c570adc 100755\n--- a/t/t5509-fetch-push-namespaces.sh\n+++ b/t/t5509-fetch-push-namespaces.sh\n@@ -4,6 +4,7 @@ test_description='fetch/push involving ref namespaces'\n . ./test-lib.sh\n \n test_expect_success setup '\n+\tgit config --global protocol.ext.allow user &&\n \ttest_tick &&\n \tgit init original &&\n \t(\ndiff --git a/t/t5802-connect-helper.sh b/t/t5802-connect-helper.sh\nindex b7a7f9d58..c6c266187 100755\n--- a/t/t5802-connect-helper.sh\n+++ b/t/t5802-connect-helper.sh\n@@ -4,6 +4,7 @@ test_description='ext::cmd remote \"connect\" helper'\n . ./test-lib.sh\n \n test_expect_success setup '\n+\tgit config --global protocol.ext.allow user &&\n \ttest_tick &&\n \tgit commit --allow-empty -m initial &&\n \ttest_tick &&\ndiff --git a/transport.c b/transport.c\nindex d57e8dec2..cd603fbf5 100644\n--- a/transport.c\n+++ b/transport.c\n@@ -664,10 +664,70 @@ static const struct string_list *protocol_whitelist(void)\n \treturn enabled ? &allowed : NULL;\n }\n \n+enum protocol_allow_config {\n+\tPROTOCOL_ALLOW_NEVER = 0,\n+\tPROTOCOL_ALLOW_USER_ONLY,\n+\tPROTOCOL_ALLOW_ALWAYS\n+};\n+\n+static enum protocol_allow_config parse_protocol_config(const char *key,\n+\t\t\t\t\t\t\tconst char *value)\n+{\n+\tif (!strcasecmp(value, \"always\"))\n+\t\treturn PROTOCOL_ALLOW_ALWAYS;\n+\telse if (!strcasecmp(value, \"never\"))\n+\t\treturn PROTOCOL_ALLOW_NEVER;\n+\telse if (!strcasecmp(value, \"user\"))\n+\t\treturn PROTOCOL_ALLOW_USER_ONLY;\n+\n+\t/* XXX maybe also interpret git_config_bool() here? */\n+\tdie(\"unknown value for config '%s': %s\", key, value);\n+}\n+\n+static enum protocol_allow_config get_protocol_config(const char *type)\n+{\n+\tchar *key = xstrfmt(\"protocol.%s.allow\", type);\n+\tchar *value;\n+\n+\tif (!git_config_get_string(key, &value)) {\n+\t\tenum protocol_allow_config ret =\n+\t\t\tparse_protocol_config(key, value);\n+\t\tfree(key);\n+\t\tfree(value);\n+\t\treturn ret;\n+\t}\n+\tfree(key);\n+\n+\t/* known safe */\n+\tif (!strcmp(type, \"http\") ||\n+\t !strcmp(type, \"https\") ||\n+\t !strcmp(type, \"git\") ||\n+\t !strcmp(type, \"ssh\") ||\n+\t !strcmp(type, \"file\"))\n+\t\treturn PROTOCOL_ALLOW_ALWAYS;\n+\n+\t/* known scary; err on the side of caution */\n+\tif (!strcmp(type, \"ext\"))\n+\t\treturn PROTOCOL_ALLOW_NEVER;\n+\n+\t/* unknown; let them be used only directly by the user */\n+\treturn PROTOCOL_ALLOW_USER_ONLY;\n+}\n+\n int is_transport_allowed(const char *type)\n {\n-\tconst struct string_list *allowed = protocol_whitelist();\n-\treturn !allowed || string_list_has_string(allowed, type);\n+\tconst struct string_list *whitelist = protocol_whitelist();\n+\tif (whitelist)\n+\t\treturn string_list_has_string(whitelist, type);\n+\n+\tswitch (get_protocol_config(type)) {\n+\tcase PROTOCOL_ALLOW_ALWAYS:\n+\t\treturn 1;\n+\tcase PROTOCOL_ALLOW_NEVER:\n+\t\treturn 0;\n+\tcase PROTOCOL_ALLOW_USER_ONLY:\n+\t\treturn git_env_bool(\"GIT_PROTOCOL_FROM_USER\", 1);\n+\t}\n }\n \n void transport_check_allowed(const char *type)\n"},{"id":"305377","messageId":"20161103182557.jov2sab7jtozqw7t@sigill.intra.peff.net","threadId":"44417","inReplyTo":"20161103181954.GD182568@google.com","subject":"Re: [PATCH] transport: add core.allowProtocol config option","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-11-03T18:25:57Z","receivedAt":"2016-11-03T18:26:04Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Nov 03, 2016 at 11:19:54AM -0700, Brandon Williams wrote:\n\n> On 11/03, Jeff King wrote:\n> > +\n> > +\t/* unknown; let them be used only directly by the user */\n> > +\treturn PROTOCOL_ALLOW_USER_ONLY;\n> > +}\n> > +\n> > int is_transport_allowed(const char *type)\n> > {\n> > -\tconst struct string_list *allowed = protocol_whitelist();\n> > -\treturn !allowed || string_list_has_string(allowed, type);\n> > +\tconst struct string_list *whitelist = protocol_whitelist();\n> > +\tif (whitelist)\n> > +\t\treturn string_list_has_string(whitelist, type);\n> > +\n> > +\tswitch (get_protocol_config(type)) {\n> > +\tcase PROTOCOL_ALLOW_ALWAYS:\n> > +\t\treturn 1;\n> > +\tcase PROTOCOL_ALLOW_NEVER:\n> > +\t\treturn 0;\n> > +\tcase PROTOCOL_ALLOW_USER_ONLY:\n> > +\t\treturn git_env_bool(\"GIT_PROTOCOL_FROM_USER\", 1);\n> > +\t}\n> \n> I know this is just a rough patch you wiped up but one question:\n> With the 'user' state, how exactly do you envision this env variable\n> working? Do we want the user to have to explicitly set\n> GIT_PROTOCOL_FROM_USER in their environment and then have these other\n> commands (like git-submodule) explicitly clear the env var or would we\n> rather these subcommands set a variable indicating they aren't coming\n> from the user and the deafult state (no var set) is a user run command?\n\nSee the follow-up I just posted, but basically, the rules are:\n\n - if you don't say anything, then the URL is from the user\n\n - git-submodule would set it to \"0\" (i.e., tell us to be more careful)\n\n - tools like \"go get\" would similarly set it to \"0\" if they are\n passing untrusted URLs\n\n-Peff\n"},{"id":"305378","messageId":"20161103184538.GE182568@google.com","threadId":"44417","inReplyTo":"20161103182428.j3r574evsk7ypfie@sigill.intra.peff.net","subject":"Re: [PATCH] transport: add core.allowProtocol config option","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-11-03T18:45:38Z","receivedAt":"2016-11-03T18:45:46Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 11/03, Jeff King wrote:\n> \n> So this seems like a reasonable direction to me. It obviously needs\n> documentation and tests. Arguably there should be a fallback \"allow\"\n> value when a protocol is not mentioned in the config so that you could\n> convert the default from \"user\" to \"never\" if you wanted your config to\n> specify a pure whitelist.\n\nYes I agree there should probably be a fallback value of 'never' maybe?\nWhat you currently have preserves the behavior of what git does\nnow, if we did instead have a fallback of 'never' it would break current\nusers who don't already use GIT_ALLOW_PROTOCOL (well only if they use\ncrazy protocols). We could ease into it though and start with default\nto allow and then transition to a true whitelist sometime after this\nchange has been made?\n\n> \n> Without that, I think we'd want to keep GIT_ALLOW_PROTOCOL for the truly\n> paranoid (though we should keep it indefinitely either way for backwards\n> compatibility).\n> \n> Do you have interest in picking this up and running with it?\n\nYep! Thanks for the help in shaping this.\n\n-- \nBrandon Williams\n"},{"id":"305379","messageId":"20161103185052.q46x4hlcwne7kpc5@sigill.intra.peff.net","threadId":"44417","inReplyTo":"20161103184538.GE182568@google.com","subject":"Re: [PATCH] transport: add core.allowProtocol config option","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-11-03T18:50:52Z","receivedAt":"2016-11-03T18:51:15Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Nov 03, 2016 at 11:45:38AM -0700, Brandon Williams wrote:\n\n> On 11/03, Jeff King wrote:\n> > \n> > So this seems like a reasonable direction to me. It obviously needs\n> > documentation and tests. Arguably there should be a fallback \"allow\"\n> > value when a protocol is not mentioned in the config so that you could\n> > convert the default from \"user\" to \"never\" if you wanted your config to\n> > specify a pure whitelist.\n> \n> Yes I agree there should probably be a fallback value of 'never' maybe?\n> What you currently have preserves the behavior of what git does\n> now, if we did instead have a fallback of 'never' it would break current\n> users who don't already use GIT_ALLOW_PROTOCOL (well only if they use\n> crazy protocols). We could ease into it though and start with default\n> to allow and then transition to a true whitelist sometime after this\n> change has been made?\n\nI don't see the value in moving the out-of-the-box install to any\ndefault except \"user\". Right now the experience of using a third-party\nhelper is something like:\n\n cp git-remote-hg /somewhere/in/your/PATH\n git clone hg::whatever\n\nWe restrict its use in submodules by default, which is unlikely to bite\nmany people. But if we started falling back to \"never\" all the time,\nthen that second command would break until you officially \"approve\"\nremote-hg in your config.\n\nI was thinking of just something to let people decide to have that level\nof paranoia themselves (especially if they want to just set up a\nwhole-system white list via the config without bothering with\nenvironment variables). Like:\n\n git config --system protocol.allow never\n git config --system protocol.https.allow always\n\nThat behaves exactly like:\n\n export GIT_ALLOW_PROTOCOL=https\n\nexcept it just works everywhere, without having to tweak the environment\nof every process.\n\n> > Do you have interest in picking this up and running with it?\n> \n> Yep! Thanks for the help in shaping this.\n\nGreat, thanks. I'm happy to review or discuss further as necessary.\n\n-Peff\n"},{"id":"305380","messageId":"20161103185611.GF182568@google.com","threadId":"44417","inReplyTo":"20161103185052.q46x4hlcwne7kpc5@sigill.intra.peff.net","subject":"Re: [PATCH] transport: add core.allowProtocol config option","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-11-03T18:56:11Z","receivedAt":"2016-11-03T18:56:18Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 11/03, Jeff King wrote:\n> On Thu, Nov 03, 2016 at 11:45:38AM -0700, Brandon Williams wrote:\n> \n> > On 11/03, Jeff King wrote:\n> > > \n> > > So this seems like a reasonable direction to me. It obviously needs\n> > > documentation and tests. Arguably there should be a fallback \"allow\"\n> > > value when a protocol is not mentioned in the config so that you could\n> > > convert the default from \"user\" to \"never\" if you wanted your config to\n> > > specify a pure whitelist.\n> > \n> > Yes I agree there should probably be a fallback value of 'never' maybe?\n> > What you currently have preserves the behavior of what git does\n> > now, if we did instead have a fallback of 'never' it would break current\n> > users who don't already use GIT_ALLOW_PROTOCOL (well only if they use\n> > crazy protocols). We could ease into it though and start with default\n> > to allow and then transition to a true whitelist sometime after this\n> > change has been made?\n> \n> I don't see the value in moving the out-of-the-box install to any\n> default except \"user\". Right now the experience of using a third-party\n> helper is something like:\n> \n> cp git-remote-hg /somewhere/in/your/PATH\n> git clone hg::whatever\n> \n> We restrict its use in submodules by default, which is unlikely to bite\n> many people. But if we started falling back to \"never\" all the time,\n> then that second command would break until you officially \"approve\"\n> remote-hg in your config.\n> \n> I was thinking of just something to let people decide to have that level\n> of paranoia themselves (especially if they want to just set up a\n> whole-system white list via the config without bothering with\n> environment variables). Like:\n> \n> git config --system protocol.allow never\n> git config --system protocol.https.allow always\n> \n> That behaves exactly like:\n> \n> export GIT_ALLOW_PROTOCOL=https\n> \n> except it just works everywhere, without having to tweak the environment\n> of every process.\n> \n\nAh ok, so essentially letting the user specify a default behaviour\nthemselves.\n\n-- \nBrandon Williams\n"},{"id":"305411","messageId":"1478292933-7873-1-git-send-email-bmwill@google.com","threadId":"44417","inReplyTo":"1478125247-62372-1-git-send-email-bmwill@google.com","subject":"[PATCH v3] transport: add protocol policy config option","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-11-04T20:55:33Z","receivedAt":"2016-11-04T20:55:48Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"Previously the `GIT_ALLOW_PROTOCOL` environment variable was used to\nspecify a whitelist of protocols to be used in clone/fetch/pull\ncommands. This patch introduces new configuration options for more\nfine-grained control for allowing/disallowing protocols. This also has\nthe added benefit of allowing easier construction of a protocol\nwhitelist on systems where setting an environment variable is\nnon-trivial.\n\nNow users can specify a policy to be used for each type of protocol via\nthe 'protocol..allow' config option. A default policy for all\nunknown protocols can be set with the 'protocol.allow' config option.\nIf no user configured default is made git, by default, will allow\nknown-safe protocols (http, https, git, ssh, file), disallow\nknown-dangerous protocols (ext), and have a default poliy of `user` for\nall other protocols.\n\nThe supported policies are `always`, `never`, and `user`. The `user`\npolicy can be used to configure a protocol to be usable when explicitly\nused by a user, while disallowing it for commands which run\nclone/fetch/pull commands without direct user intervention (e.g.\nrecursive initialization of submodules). Commands which can potentially\nclone/fetch/pull from untrusted repositories without user intervention\ncan export `GIT_PROTOCOL_FROM_USER` with a value of '0' to prevent\nprotocols configured to the `user` policy from being used.\n\nSigned-off-by: Brandon Williams \n---\n Documentation/config.txt | 25 ++++++++\n Documentation/git.txt | 19 +++---\n git-submodule.sh | 12 ++--\n t/lib-proto-disable.sh | 132 +++++++++++++++++++++++++++++++++++----\n t/t5509-fetch-push-namespaces.sh | 1 +\n t/t5802-connect-helper.sh | 1 +\n transport.c | 73 +++++++++++++++++++++-\n 7 files changed, 235 insertions(+), 28 deletions(-)\n\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex 27069ac..5d845c4 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -2308,6 +2308,31 @@ pretty.::\n \tNote that an alias with the same name as a built-in format\n \twill be silently ignored.\n \n+protocol.allow::\n+\tIf set, provide a user defined default policy for all protocols which\n+\tdon't explicitly have a policy (protocol..allow). By default,\n+\tif unset, known-safe protocols (http, https, git, ssh, file) have a\n+\tdefault policy of `always`, known-dangerous protocols (ext) have a\n+\tdefault policy of `never`, and all other protocols have a default policy\n+\tof `user`. Supported policies:\n++\n+--\n+\n+* `always` - protocol is always able to be used.\n+\n+* `never` - protocol is never able to be used.\n+\n+* `user` - protocol is only able to be used when `GIT_PROTOCOL_FROM_USER` is\n+ either unset or has a value of 1. This policy should be used when you want a\n+ protocol to be usable by the user but don't want it used by commands which\n+ execute clone/fetch/pull commands without user input, e.g. recursive\n+ submodule initialization.\n+\n+--\n+\n+protocol..allow::\n+\tSet a policy to be used by protocol with clone/fetch/pull commands.\n+\n pull.ff::\n \tBy default, Git does not create an extra merge commit when merging\n \ta commit that is a descendant of the current commit. Instead, the\ndiff --git a/Documentation/git.txt b/Documentation/git.txt\nindex ab7215e..ab25580 100644\n--- a/Documentation/git.txt\n+++ b/Documentation/git.txt\n@@ -1150,13 +1150,13 @@ of clones and fetches.\n \tcloning a repository to make a backup).\n \n `GIT_ALLOW_PROTOCOL`::\n-\tIf set, provide a colon-separated list of protocols which are\n-\tallowed to be used with fetch/push/clone. This is useful to\n-\trestrict recursive submodule initialization from an untrusted\n-\trepository. Any protocol not mentioned will be disallowed (i.e.,\n-\tthis is a whitelist, not a blacklist). If the variable is not\n-\tset at all, all protocols are enabled. The protocol names\n-\tcurrently used by git are:\n+\tThe new way to configure allowed protocols is done through the config\n+\tinterface, though this setting takes precedences. See\n+\tlinkgit:git-config[1] for more details. If set, provide a\n+\tcolon-separated list of protocols which are allowed to be used with\n+\tfetch/push/clone. Any protocol not mentioned will be disallowed (i.e.,\n+\tthis is a whitelist, not a blacklist). The protocol names currently\n+\tused by git are:\n \n \t - `file`: any local file-based path (including `file://` URLs,\n \t or local paths)\n@@ -1174,6 +1174,11 @@ of clones and fetches.\n \t - any external helpers are named by their protocol (e.g., use\n \t `hg` to allow the `git-remote-hg` helper)\n \n+`GIT_PROTOCOL_FROM_USER`::\n+\tSet to 0 to prevent protocols used by fetch/push/clone which are\n+\tconfigured to the `user` state. This is useful to restrict recursive\n+\tsubmodule initialization from an untrusted repository. See\n+\tlinkgit:git-config[1] for more details.\n \n Discussion[[Discussion]]\n ------------------------\ndiff --git a/git-submodule.sh b/git-submodule.sh\nindex a024a13..0a477b4 100755\n--- a/git-submodule.sh\n+++ b/git-submodule.sh\n@@ -21,14 +21,10 @@ require_work_tree\n wt_prefix=$(git rev-parse --show-prefix)\n cd_to_toplevel\n \n-# Restrict ourselves to a vanilla subset of protocols; the URLs\n-# we get are under control of a remote repository, and we do not\n-# want them kicking off arbitrary git-remote-* programs.\n-#\n-# If the user has already specified a set of allowed protocols,\n-# we assume they know what they're doing and use that instead.\n-: ${GIT_ALLOW_PROTOCOL=file:git:http:https:ssh}\n-export GIT_ALLOW_PROTOCOL\n+# Tell the rest of git that any URLs we get don't come\n+# directly from the user, so it can apply policy as appropriate.\n+GIT_PROTOCOL_FROM_USER=0\n+export GIT_PROTOCOL_FROM_USER\n \n command=\n branch=\ndiff --git a/t/lib-proto-disable.sh b/t/lib-proto-disable.sh\nindex b0917d9..5950fbf 100644\n--- a/t/lib-proto-disable.sh\n+++ b/t/lib-proto-disable.sh\n@@ -1,15 +1,12 @@\n # Test routines for checking protocol disabling.\n \n-# test cloning a particular protocol\n-# $1 - description of the protocol\n-# $2 - machine-readable name of the protocol\n-# $3 - the URL to try cloning\n-test_proto () {\n+# Test clone/fetch/push with GIT_ALLOW_PROTOCOL whitelist\n+test_whitelist () {\n \tdesc=$1\n \tproto=$2\n \turl=$3\n \n-\ttest_expect_success \"clone $1 (enabled)\" '\n+\ttest_expect_success \"clone $desc (enabled)\" '\n \t\trm -rf tmp.git &&\n \t\t(\n \t\t\tGIT_ALLOW_PROTOCOL=$proto &&\n@@ -18,7 +15,7 @@ test_proto () {\n \t\t)\n \t'\n \n-\ttest_expect_success \"fetch $1 (enabled)\" '\n+\ttest_expect_success \"fetch $desc (enabled)\" '\n \t\t(\n \t\t\tcd tmp.git &&\n \t\t\tGIT_ALLOW_PROTOCOL=$proto &&\n@@ -27,7 +24,7 @@ test_proto () {\n \t\t)\n \t'\n \n-\ttest_expect_success \"push $1 (enabled)\" '\n+\ttest_expect_success \"push $desc (enabled)\" '\n \t\t(\n \t\t\tcd tmp.git &&\n \t\t\tGIT_ALLOW_PROTOCOL=$proto &&\n@@ -36,7 +33,7 @@ test_proto () {\n \t\t)\n \t'\n \n-\ttest_expect_success \"push $1 (disabled)\" '\n+\ttest_expect_success \"push $desc (disabled)\" '\n \t\t(\n \t\t\tcd tmp.git &&\n \t\t\tGIT_ALLOW_PROTOCOL=none &&\n@@ -45,7 +42,7 @@ test_proto () {\n \t\t)\n \t'\n \n-\ttest_expect_success \"fetch $1 (disabled)\" '\n+\ttest_expect_success \"fetch $desc (disabled)\" '\n \t\t(\n \t\t\tcd tmp.git &&\n \t\t\tGIT_ALLOW_PROTOCOL=none &&\n@@ -54,7 +51,7 @@ test_proto () {\n \t\t)\n \t'\n \n-\ttest_expect_success \"clone $1 (disabled)\" '\n+\ttest_expect_success \"clone $desc (disabled)\" '\n \t\trm -rf tmp.git &&\n \t\t(\n \t\t\tGIT_ALLOW_PROTOCOL=none &&\n@@ -64,6 +61,119 @@ test_proto () {\n \t'\n }\n \n+test_config () {\n+\tdesc=$1\n+\tproto=$2\n+\turl=$3\n+\n+\t# Test clone/fetch/push with protocol..allow config\n+\ttest_expect_success \"clone $desc (enabled with config)\" '\n+\t\trm -rf tmp.git &&\n+\t\tgit -c protocol.$proto.allow=always clone --bare \"$url\" tmp.git\n+\t'\n+\n+\ttest_expect_success \"fetch $desc (enabled)\" '\n+\t\tgit -C tmp.git -c protocol.$proto.allow=always fetch\n+\t'\n+\n+\ttest_expect_success \"push $desc (enabled)\" '\n+\t\tgit -C tmp.git -c protocol.$proto.allow=always push origin HEAD:pushed\n+\t'\n+\n+\ttest_expect_success \"push $desc (disabled)\" '\n+\t\ttest_must_fail git -C tmp.git -c protocol.$proto.allow=never push origin HEAD:pushed\n+\t'\n+\n+\ttest_expect_success \"fetch $desc (disabled)\" '\n+\t\ttest_must_fail git -C tmp.git -c protocol.$proto.allow=never fetch\n+\t'\n+\n+\ttest_expect_success \"clone $desc (disabled)\" '\n+\t\trm -rf tmp.git &&\n+\t\ttest_must_fail git -c protocol.$proto.allow=never clone --bare \"$url\" tmp.git\n+\t'\n+\n+\t# Test clone/fetch/push with protocol.user.allow and its env var\n+\ttest_expect_success \"clone $desc (enabled)\" '\n+\t\trm -rf tmp.git &&\n+\t\tgit -c protocol.$proto.allow=user clone --bare \"$url\" tmp.git\n+\t'\n+\n+\ttest_expect_success \"fetch $desc (enabled)\" '\n+\t\tgit -C tmp.git -c protocol.$proto.allow=user fetch\n+\t'\n+\n+\ttest_expect_success \"push $desc (enabled)\" '\n+\t\tgit -C tmp.git -c protocol.$proto.allow=user push origin HEAD:pushed\n+\t'\n+\n+\ttest_expect_success \"push $desc (disabled)\" '\n+\t\t(\n+\t\t\tcd tmp.git &&\n+\t\t\tGIT_PROTOCOL_FROM_USER=0 &&\n+\t\t\texport GIT_PROTOCOL_FROM_USER &&\n+\t\t\ttest_must_fail git -c protocol.$proto.allow=user push origin HEAD:pushed\n+\t\t)\n+\t'\n+\n+\ttest_expect_success \"fetch $desc (disabled)\" '\n+\t\t(\n+\t\t\tcd tmp.git &&\n+\t\t\tGIT_PROTOCOL_FROM_USER=0 &&\n+\t\t\texport GIT_PROTOCOL_FROM_USER &&\n+\t\t\ttest_must_fail git -c protocol.$proto.allow=user fetch\n+\t\t)\n+\t'\n+\n+\ttest_expect_success \"clone $desc (disabled)\" '\n+\t\trm -rf tmp.git &&\n+\t\t(\n+\t\t\tGIT_PROTOCOL_FROM_USER=0 &&\n+\t\t\texport GIT_PROTOCOL_FROM_USER &&\n+\t\t\ttest_must_fail git -c protocol.$proto.allow=user clone --bare \"$url\" tmp.git\n+\t\t)\n+\t'\n+\n+\t# Test clone/fetch/push with protocol.allow user defined default\n+\ttest_expect_success \"clone $desc (enabled)\" '\n+\t\trm -rf tmp.git &&\n+\t\tgit config --global protocol.allow always &&\n+\t\tgit clone --bare \"$url\" tmp.git\n+\t'\n+\n+\ttest_expect_success \"fetch $desc (enabled)\" '\n+\t\tgit -C tmp.git fetch\n+\t'\n+\n+\ttest_expect_success \"push $desc (enabled)\" '\n+\t\tgit -C tmp.git push origin HEAD:pushed\n+\t'\n+\n+\ttest_expect_success \"push $desc (disabled)\" '\n+\t\tgit config --global protocol.allow never &&\n+\t\ttest_must_fail git -C tmp.git push origin HEAD:pushed\n+\t'\n+\n+\ttest_expect_success \"fetch $desc (disabled)\" '\n+\t\ttest_must_fail git -C tmp.git fetch\n+\t'\n+\n+\ttest_expect_success \"clone $desc (disabled)\" '\n+\t\trm -rf tmp.git &&\n+\t\ttest_must_fail git clone --bare \"$url\" tmp.git\n+\t'\n+}\n+\n+# test cloning a particular protocol\n+# $1 - description of the protocol\n+# $2 - machine-readable name of the protocol\n+# $3 - the URL to try cloning\n+test_proto () {\n+\ttest_whitelist \"$@\"\n+\n+\ttest_config \"$@\"\n+}\n+\n # set up an ssh wrapper that will access $host/$repo in the\n # trash directory, and enable it for subsequent tests.\n setup_ssh_wrapper () {\ndiff --git a/t/t5509-fetch-push-namespaces.sh b/t/t5509-fetch-push-namespaces.sh\nindex bc44ac3..75c570a 100755\n--- a/t/t5509-fetch-push-namespaces.sh\n+++ b/t/t5509-fetch-push-namespaces.sh\n@@ -4,6 +4,7 @@ test_description='fetch/push involving ref namespaces'\n . ./test-lib.sh\n \n test_expect_success setup '\n+\tgit config --global protocol.ext.allow user &&\n \ttest_tick &&\n \tgit init original &&\n \t(\ndiff --git a/t/t5802-connect-helper.sh b/t/t5802-connect-helper.sh\nindex b7a7f9d..c6c2661 100755\n--- a/t/t5802-connect-helper.sh\n+++ b/t/t5802-connect-helper.sh\n@@ -4,6 +4,7 @@ test_description='ext::cmd remote \"connect\" helper'\n . ./test-lib.sh\n \n test_expect_success setup '\n+\tgit config --global protocol.ext.allow user &&\n \ttest_tick &&\n \tgit commit --allow-empty -m initial &&\n \ttest_tick &&\ndiff --git a/transport.c b/transport.c\nindex d57e8de..c2d0538 100644\n--- a/transport.c\n+++ b/transport.c\n@@ -664,10 +664,79 @@ static const struct string_list *protocol_whitelist(void)\n \treturn enabled ? &allowed : NULL;\n }\n \n+enum protocol_allow_config {\n+\tPROTOCOL_ALLOW_NEVER = 0,\n+\tPROTOCOL_ALLOW_USER_ONLY,\n+\tPROTOCOL_ALLOW_ALWAYS\n+};\n+\n+static enum protocol_allow_config parse_protocol_config(const char *key,\n+\t\t\t\t\t\t\tconst char *value)\n+{\n+\tif (!strcasecmp(value, \"always\"))\n+\t\treturn PROTOCOL_ALLOW_ALWAYS;\n+\telse if (!strcasecmp(value, \"never\"))\n+\t\treturn PROTOCOL_ALLOW_NEVER;\n+\telse if (!strcasecmp(value, \"user\"))\n+\t\treturn PROTOCOL_ALLOW_USER_ONLY;\n+\n+\tdie(\"unknown value for config '%s': %s\", key, value);\n+}\n+\n+static enum protocol_allow_config get_protocol_config(const char *type)\n+{\n+\tchar *key = xstrfmt(\"protocol.%s.allow\", type);\n+\tchar *value;\n+\n+\tif (!git_config_get_string(key, &value)) {\n+\t\tenum protocol_allow_config ret =\n+\t\t\tparse_protocol_config(key, value);\n+\t\tfree(key);\n+\t\tfree(value);\n+\t\treturn ret;\n+\t}\n+\tfree(key);\n+\n+\t/* if defined, use user default for unknown protocols */\n+\tif (!git_config_get_string(\"protocol.allow\", &value)) {\n+\t\tenum protocol_allow_config ret =\n+\t\t\tparse_protocol_config(\"protocol.allow\", value);\n+\t\tfree(value);\n+\t\treturn ret;\n+\t}\n+\n+\t/* known safe */\n+\tif (!strcmp(type, \"http\") ||\n+\t !strcmp(type, \"https\") ||\n+\t !strcmp(type, \"git\") ||\n+\t !strcmp(type, \"ssh\") ||\n+\t !strcmp(type, \"file\"))\n+\t\treturn PROTOCOL_ALLOW_ALWAYS;\n+\n+\t/* known scary; err on the side of caution */\n+\tif (!strcmp(type, \"ext\"))\n+\t\treturn PROTOCOL_ALLOW_NEVER;\n+\n+\t/* unknown; by default let them be used only directly by the user */\n+\treturn PROTOCOL_ALLOW_USER_ONLY;\n+}\n+\n int is_transport_allowed(const char *type)\n {\n-\tconst struct string_list *allowed = protocol_whitelist();\n-\treturn !allowed || string_list_has_string(allowed, type);\n+\tconst struct string_list *whitelist = protocol_whitelist();\n+\tif (whitelist)\n+\t\treturn string_list_has_string(whitelist, type);\n+\n+\tswitch (get_protocol_config(type)) {\n+\tcase PROTOCOL_ALLOW_ALWAYS:\n+\t\treturn 1;\n+\tcase PROTOCOL_ALLOW_NEVER:\n+\t\treturn 0;\n+\tcase PROTOCOL_ALLOW_USER_ONLY:\n+\t\treturn git_env_bool(\"GIT_PROTOCOL_FROM_USER\", 1);\n+\t}\n+\n+\tdie(\"BUG: invalid protocol_allow_config type\");\n }\n \n void transport_check_allowed(const char *type)\n-- \n2.8.0.rc3.226.g39d4020\n\n"},{"id":"305412","messageId":"20161104205815.GA127933@google.com","threadId":"44417","inReplyTo":"1478292933-7873-1-git-send-email-bmwill@google.com","subject":"Re: [PATCH v3] transport: add protocol policy config option","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-11-04T20:58:15Z","receivedAt":"2016-11-04T20:58:35Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 11/04, Brandon Williams wrote:\n> Signed-off-by: Brandon Williams \n\nIs there an acceptable way to give credit to Jeff for helping with this patch?\n\n-- \nBrandon Williams\n"},{"id":"305415","messageId":"CAGZ79kZB9wMgPDktMUpBfcvgJy16N8P74SvLP5740UNftZkbVg@mail.gmail.com","threadId":"44417","inReplyTo":"20161104205815.GA127933@google.com","subject":"Re: [PATCH v3] transport: add protocol policy config option","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2016-11-04T21:35:57Z","receivedAt":"2016-11-04T21:36:12Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Fri, Nov 4, 2016 at 1:58 PM, Brandon Williams wrote:\n> On 11/04, Brandon Williams wrote:\n>> Signed-off-by: Brandon Williams \n>\n> Is there an acceptable way to give credit to Jeff for helping with this patch?\n\nWhat about:\nHelped-by: Jeff King \n\n\n>\n> --\n> Brandon Williams\n"},{"id":"305419","messageId":"CAGZ79ka18s3-nzkCog3+g70NCoAawoqO1t-qghKSZJxjK6jkmw@mail.gmail.com","threadId":"44417","inReplyTo":"1478292933-7873-1-git-send-email-bmwill@google.com","subject":"Re: [PATCH v3] transport: add protocol policy config option","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2016-11-04T22:38:41Z","receivedAt":"2016-11-04T22:38:47Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Fri, Nov 4, 2016 at 1:55 PM, Brandon Williams wrote:\n> Previously the `GIT_ALLOW_PROTOCOL` environment variable was used to\n> specify a whitelist of protocols to be used in clone/fetch/pull\n> commands. This patch introduces new configuration options for more\n> fine-grained control for allowing/disallowing protocols. This also has\n> the added benefit of allowing easier construction of a protocol\n> whitelist on systems where setting an environment variable is\n> non-trivial.\n>\n> Now users can specify a policy to be used for each type of protocol via\n> the 'protocol..allow' config option. A default policy for all\n> unknown protocols can be set with the 'protocol.allow' config option.\n> If no user configured default is made git, by default, will allow\n> known-safe protocols (http, https, git, ssh, file), disallow\n> known-dangerous protocols (ext), and have a default poliy of `user` for\n> all other protocols.\n>\n> The supported policies are `always`, `never`, and `user`. The `user`\n> policy can be used to configure a protocol to be usable when explicitly\n> used by a user, while disallowing it for commands which run\n> clone/fetch/pull commands without direct user intervention (e.g.\n> recursive initialization of submodules). Commands which can potentially\n> clone/fetch/pull from untrusted repositories without user intervention\n> can export `GIT_PROTOCOL_FROM_USER` with a value of '0' to prevent\n> protocols configured to the `user` policy from being used.\n>\n> Signed-off-by: Brandon Williams \n> ---\n> Documentation/config.txt | 25 ++++++++\n> Documentation/git.txt | 19 +++---\n> git-submodule.sh | 12 ++--\n> t/lib-proto-disable.sh | 132 +++++++++++++++++++++++++++++++++++----\n> t/t5509-fetch-push-namespaces.sh | 1 +\n> t/t5802-connect-helper.sh | 1 +\n> transport.c | 73 +++++++++++++++++++++-\n> 7 files changed, 235 insertions(+), 28 deletions(-)\n>\n> diff --git a/Documentation/config.txt b/Documentation/config.txt\n> index 27069ac..5d845c4 100644\n> --- a/Documentation/config.txt\n> +++ b/Documentation/config.txt\n> @@ -2308,6 +2308,31 @@ pretty.::\n> Note that an alias with the same name as a built-in format\n> will be silently ignored.\n>\n> +protocol.allow::\n> + If set, provide a user defined default policy for all protocols which\n> + don't explicitly have a policy (protocol..allow). By default,\n\nUse hyphens (`protocol..allow`) to highlight the config option.\n\nBy default, if unset, ... have a default policy ...\nsounds strange. How about just dropping the first 4 words here:\n\n Known-safe protocols (http, https, git, ssh, file) have a\n default policy of `always`, known-dangerous protocols (ext) have a\n default policy of `never`, and all other protocols have a default policy\n of `user`. Supported policies:\n\n\nWhat happens if protocol.allow is set to true?\n\n\n\n> + if unset, known-safe protocols (http, https, git, ssh, file) have a\n> + default policy of `always`, known-dangerous protocols (ext) have a\n> + default policy of `never`, and all other protocols have a default policy\n> + of `user`. Supported policies:\n> ++\n> +--\n> +\n> +* `always` - protocol is always able to be used.\n> +\n> +* `never` - protocol is never able to be used.\n> +\n> +* `user` - protocol is only able to be used when `GIT_PROTOCOL_FROM_USER` is\n> + either unset or has a value of 1. This policy should be used when you want a\n> + protocol to be usable by the user but don't want it used by commands which\n> + execute clone/fetch/pull commands without user input, e.g. recursive\n> + submodule initialization.\n> +\n> +--\n> +\n> +protocol..allow::\n> + Set a policy to be used by protocol with clone/fetch/pull commands.\n\nHow does this interact with protocol.allow?\n\n When protocol.allow is set, this overrides the specific protocol.\n If protocol is not set, it overrides the specific protocol as well(?)\n\n\n> +\n> pull.ff::\n> By default, Git does not create an extra merge commit when merging\n> a commit that is a descendant of the current commit. Instead, the\n> diff --git a/Documentation/git.txt b/Documentation/git.txt\n> index ab7215e..ab25580 100644\n> --- a/Documentation/git.txt\n> +++ b/Documentation/git.txt\n> @@ -1150,13 +1150,13 @@ of clones and fetches.\n> cloning a repository to make a backup).\n>\n> `GIT_ALLOW_PROTOCOL`::\n> - If set, provide a colon-separated list of protocols which are\n> - allowed to be used with fetch/push/clone. This is useful to\n> - restrict recursive submodule initialization from an untrusted\n> - repository. Any protocol not mentioned will be disallowed (i.e.,\n> - this is a whitelist, not a blacklist). If the variable is not\n> - set at all, all protocols are enabled. The protocol names\n> - currently used by git are:\n> + The new way to configure allowed protocols is done through the config\n\nThis is not the right place to mention what is newer. ;)\nHowever it is useful to know about the config interface, which is\n * (supposedly) easier to use\n * more fine grained\n * taking less priority than this env var.\n\n> + test_expect_success \"clone $desc (disabled)\" '\n> + rm -rf tmp.git &&\n> + test_must_fail git clone --bare \"$url\" tmp.git\n> + '\n\n\nI could not spot a test for GIT_ALLOW_PROTOCOL overriding\nany protocol*allow policy. Is that also worth testing? (for\nbackwards compatibility of tools that make use of GIT_ALLOW_PROTOCOL\nbut the user already setup a policy.\n\n>\n> void transport_check_allowed(const char *type)\n> --\n\nThanks!\nStefan\n"},{"id":"305422","messageId":"20161104230613.epbziphiqyl57bcn@sigill.intra.peff.net","threadId":"44417","inReplyTo":"1478292933-7873-1-git-send-email-bmwill@google.com","subject":"Re: [PATCH v3] transport: add protocol policy config option","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-11-04T23:06:13Z","receivedAt":"2016-11-04T23:06:21Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Fri, Nov 04, 2016 at 01:55:33PM -0700, Brandon Williams wrote:\n\n> Previously the `GIT_ALLOW_PROTOCOL` environment variable was used to\n> specify a whitelist of protocols to be used in clone/fetch/pull\n> commands. This patch introduces new configuration options for more\n> fine-grained control for allowing/disallowing protocols. This also has\n> the added benefit of allowing easier construction of a protocol\n> whitelist on systems where setting an environment variable is\n> non-trivial.\n\nGood rationale.\n\n> Now users can specify a policy to be used for each type of protocol via\n> the 'protocol..allow' config option. A default policy for all\n> unknown protocols can be set with the 'protocol.allow' config option.\n\nI think \"unconfigured\" is a better word here than \"unknown\", as it would\napply to known protocols like \"https\", etc.\n\nThat made me wonder if \"unknown\" would be a better behavior, but I'm\npretty sure it is not. It is harder to explain, and I think would be\nless convenient in practice. I.e., you really do want:\n\n git config protocol.allow never\n git config protocol.https.allow always\n\nto allow nothing but https.\n\n> If no user configured default is made git, by default, will allow\n> known-safe protocols (http, https, git, ssh, file), disallow\n> known-dangerous protocols (ext), and have a default poliy of `user` for\n> all other protocols.\n\nI think this is a good way of thinking about it. The order of\nenforcement becomes:\n\n - GIT_ALLOW_PROTOCOL; environment variables always take precedence\n over config, so this makes sense. And it also is nice to put the\n blunt hammer at the front for backwards-compatibility.\n\n - protocol-specific config\n\n - protocol-generic config\n\n - built-in defaults (known-safe, known-scary, unknown)\n\nwhich seems right.\n\nAlso, s/poliy/policy/.\n\n> The supported policies are `always`, `never`, and `user`. The `user`\n> policy can be used to configure a protocol to be usable when explicitly\n> used by a user, while disallowing it for commands which run\n> clone/fetch/pull commands without direct user intervention (e.g.\n> recursive initialization of submodules). Commands which can potentially\n> clone/fetch/pull from untrusted repositories without user intervention\n> can export `GIT_PROTOCOL_FROM_USER` with a value of '0' to prevent\n> protocols configured to the `user` policy from being used.\n\nMakes sense. I know \"user\" came from me. I don't know if there is a\nbetter word to describe it. I originally called it \"cmdline\", but that\nseemed too obscure (especially when a tool external to git sets it).\nSomething like \"trusted\" might make sense (we allow it only in a\nmore-trusted setting), but it's kind of vague. And it also doesn't leave\nroom for there to be more types of trust in the future. So \"user\" is\nprobably reasonable (or perhaps \"user-only\" or similar).\n\n> diff --git a/Documentation/config.txt b/Documentation/config.txt\n> index 27069ac..5d845c4 100644\n> --- a/Documentation/config.txt\n> +++ b/Documentation/config.txt\n> @@ -2308,6 +2308,31 @@ pretty.::\n> \tNote that an alias with the same name as a built-in format\n> \twill be silently ignored.\n> \n> +protocol.allow::\n> +\tIf set, provide a user defined default policy for all protocols which\n> +\tdon't explicitly have a policy (protocol..allow). By default,\n> +\tif unset, known-safe protocols (http, https, git, ssh, file) have a\n> +\tdefault policy of `always`, known-dangerous protocols (ext) have a\n> +\tdefault policy of `never`, and all other protocols have a default policy\n> +\tof `user`. Supported policies:\n> ++\n> +--\n> +\n> +* `always` - protocol is always able to be used.\n> +\n> +* `never` - protocol is never able to be used.\n> +\n> +* `user` - protocol is only able to be used when `GIT_PROTOCOL_FROM_USER` is\n> + either unset or has a value of 1. This policy should be used when you want a\n> + protocol to be usable by the user but don't want it used by commands which\n> + execute clone/fetch/pull commands without user input, e.g. recursive\n> + submodule initialization.\n\nMakes sense. I wonder if it would be good to emphasize _directly_ usable\nhere. I.e., \"...when you want a protocol to be directly usable by the\nuser but don't want...\".\n\nShould clone/fetch/pull also include push?\n\n> +protocol..allow::\n> +\tSet a policy to be used by protocol with clone/fetch/pull commands.\n> +\n\nNice that this matches protocol.allow, so we don't need to re-explain\nthat.\n\nShould the list of protocols be here? I know they're covered under\nGIT_ALLOW_PROTOCOL already, but if this is the preferred system, we\nshould probably explain them here, and then just have GIT_ALLOW_PROTOCOL\nrefer the user.\n\n> diff --git a/Documentation/git.txt b/Documentation/git.txt\n> index ab7215e..ab25580 100644\n> --- a/Documentation/git.txt\n> +++ b/Documentation/git.txt\n> @@ -1150,13 +1150,13 @@ of clones and fetches.\n> \tcloning a repository to make a backup).\n> \n> `GIT_ALLOW_PROTOCOL`::\n> -\tIf set, provide a colon-separated list of protocols which are\n> -\tallowed to be used with fetch/push/clone. This is useful to\n> -\trestrict recursive submodule initialization from an untrusted\n> -\trepository. Any protocol not mentioned will be disallowed (i.e.,\n> -\tthis is a whitelist, not a blacklist). If the variable is not\n> -\tset at all, all protocols are enabled. The protocol names\n> -\tcurrently used by git are:\n> +\tThe new way to configure allowed protocols is done through the config\n> +\tinterface, though this setting takes precedences. See\n> +\tlinkgit:git-config[1] for more details. If set, provide a\n> +\tcolon-separated list of protocols which are allowed to be used with\n> +\tfetch/push/clone. Any protocol not mentioned will be disallowed (i.e.,\n> +\tthis is a whitelist, not a blacklist). The protocol names currently\n> +\tused by git are:\n\nI wonder if we can explain this in terms of the config system. Something\nlike:\n\n If set to a colon-separated list of zero or more protocols, behave as\n if `protocol.allow` is set to `never`, and each of the listed\n protocols has `protocol.$protocol.allow` set to `always`.\n\n> +`GIT_PROTOCOL_FROM_USER`::\n> +\tSet to 0 to prevent protocols used by fetch/push/clone which are\n> +\tconfigured to the `user` state. This is useful to restrict recursive\n> +\tsubmodule initialization from an untrusted repository. See\n> +\tlinkgit:git-config[1] for more details.\n\nUnder \"this is useful\", it may make sense to make it clear that external\nprograms can use this, too. Something like:\n\n It may also be useful for programs which feed potentially-untrusted\n URLs to git commands.\n\n> diff --git a/t/lib-proto-disable.sh b/t/lib-proto-disable.sh\n> index b0917d9..5950fbf 100644\n> --- a/t/lib-proto-disable.sh\n> +++ b/t/lib-proto-disable.sh\n> @@ -1,15 +1,12 @@\n> # Test routines for checking protocol disabling.\n> \n> -# test cloning a particular protocol\n> -# $1 - description of the protocol\n> -# $2 - machine-readable name of the protocol\n> -# $3 - the URL to try cloning\n> -test_proto () {\n> +# Test clone/fetch/push with GIT_ALLOW_PROTOCOL whitelist\n> +test_whitelist () {\n> \tdesc=$1\n> \tproto=$2\n> \turl=$3\n> \n> -\ttest_expect_success \"clone $1 (enabled)\" '\n> +\ttest_expect_success \"clone $desc (enabled)\" '\n\nYeah, this should have been $desc all along. It makes the diff really\nnoisy, though. Should it be split out into a preparatory change?\n\n> +# test cloning a particular protocol\n> +# $1 - description of the protocol\n> +# $2 - machine-readable name of the protocol\n> +# $3 - the URL to try cloning\n> +test_proto () {\n> +\ttest_whitelist \"$@\"\n> +\n> +\ttest_config \"$@\"\n> +}\n\nThis makes sense. It's probably more testing than we actually need. We\ncould just check the config version per-protocol, and then confirm that\nGIT_ALLOW_PROTOCOL behaves as I described above for at least one\nprotocol. The per-protocol code paths are really just making sure that\nthe protocol is correctly named for each code path.\n\nThat being said, simple and stupid test setup is nice as long as it does\nnot take too long to run.\n\n> diff --git a/t/t5509-fetch-push-namespaces.sh b/t/t5509-fetch-push-namespaces.sh\n> index bc44ac3..75c570a 100755\n> --- a/t/t5509-fetch-push-namespaces.sh\n> +++ b/t/t5509-fetch-push-namespaces.sh\n> @@ -4,6 +4,7 @@ test_description='fetch/push involving ref namespaces'\n> . ./test-lib.sh\n> \n> test_expect_success setup '\n> +\tgit config --global protocol.ext.allow user &&\n> \ttest_tick &&\n> \tgit init original &&\n\nThese remote-ext fixups might be worth a note in the commit message, or\na comment here explaining what is going on.\n\n> +static enum protocol_allow_config get_protocol_config(const char *type)\n> +{\n> +\tchar *key = xstrfmt(\"protocol.%s.allow\", type);\n> +\tchar *value;\n> +\n> +\tif (!git_config_get_string(key, &value)) {\n> +\t\tenum protocol_allow_config ret =\n> +\t\t\tparse_protocol_config(key, value);\n> +\t\tfree(key);\n> +\t\tfree(value);\n> +\t\treturn ret;\n> +\t}\n> +\tfree(key);\n> +\n> +\t/* if defined, use user default for unknown protocols */\n> +\tif (!git_config_get_string(\"protocol.allow\", &value)) {\n> +\t\tenum protocol_allow_config ret =\n> +\t\t\tparse_protocol_config(\"protocol.allow\", value);\n> +\t\tfree(value);\n> +\t\treturn ret;\n> +\t}\n> +\n> +\t/* known safe */\n> [...]\n\nIt's probably worth a comment at this point in the function to follow-up\non your \"if defined\" comment above. So the end result reads something\nlike:\n\n /* first check the per-protocol config */\n ...\n\n /* now fallback to the generic config */\n ...\n\n /* and then fallback to our built-in defaults */\n\n-Peff\n"},{"id":"305423","messageId":"20161104230902.ulkyejwamm6q575a@sigill.intra.peff.net","threadId":"44417","inReplyTo":"CAGZ79kZB9wMgPDktMUpBfcvgJy16N8P74SvLP5740UNftZkbVg@mail.gmail.com","subject":"Re: [PATCH v3] transport: add protocol policy config option","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-11-04T23:09:02Z","receivedAt":"2016-11-04T23:09:09Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Fri, Nov 04, 2016 at 02:35:57PM -0700, Stefan Beller wrote:\n\n> On Fri, Nov 4, 2016 at 1:58 PM, Brandon Williams wrote:\n> > On 11/04, Brandon Williams wrote:\n> >> Signed-off-by: Brandon Williams \n> >\n> > Is there an acceptable way to give credit to Jeff for helping with this patch?\n> \n> What about:\n> Helped-by: Jeff King \n\nThat, or often I would write:\n\n Based on a patch by Jeff King \n\nin the commit message. Basically anything is OK _except_ forging\nsigned-off-by, because it has a very specific meaning. So let me also\nsay that I am happy to give my:\n\n Signed-off-by: Jeff King \n\nto the original (which you should add in, to make clear that the\ncopyright issues are OK).\n\nIn some cases it makes sense to just roll somebody's patch into your\nseries, and then build on top. I'm fine with it all going into a single\npatch here.\n\n-Peff\n"},{"id":"305427","messageId":"20161105001809.GB127933@google.com","threadId":"44417","inReplyTo":"20161104230902.ulkyejwamm6q575a@sigill.intra.peff.net","subject":"Re: [PATCH v3] transport: add protocol policy config option","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-11-05T00:18:09Z","receivedAt":"2016-11-05T00:18:20Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 11/04, Jeff King wrote:\n> On Fri, Nov 04, 2016 at 02:35:57PM -0700, Stefan Beller wrote:\n> \n> > On Fri, Nov 4, 2016 at 1:58 PM, Brandon Williams wrote:\n> > > On 11/04, Brandon Williams wrote:\n> > >> Signed-off-by: Brandon Williams \n> > >\n> > > Is there an acceptable way to give credit to Jeff for helping with this patch?\n> > \n> > What about:\n> > Helped-by: Jeff King \n> \n> That, or often I would write:\n> \n> Based on a patch by Jeff King \n> \n> in the commit message. Basically anything is OK _except_ forging\n> signed-off-by, because it has a very specific meaning. So let me also\n> say that I am happy to give my:\n> \n> Signed-off-by: Jeff King \n> \n> to the original (which you should add in, to make clear that the\n> copyright issues are OK).\n> \n> In some cases it makes sense to just roll somebody's patch into your\n> series, and then build on top. I'm fine with it all going into a single\n> patch here.\n> \n> -Peff\n\nOh if it would be more clear I can easily break it up into two patches.\n\n-- \nBrandon Williams\n"},{"id":"305497","messageId":"20161107181425.GA143723@google.com","threadId":"44417","inReplyTo":"CAGZ79ka18s3-nzkCog3+g70NCoAawoqO1t-qghKSZJxjK6jkmw@mail.gmail.com","subject":"Re: [PATCH v3] transport: add protocol policy config option","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-11-07T18:14:25Z","receivedAt":"2016-11-07T18:48:25Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 11/04, Stefan Beller wrote:\n> By default, if unset, ... have a default policy ...\n> sounds strange. How about just dropping the first 4 words here:\n> \n> Known-safe protocols (http, https, git, ssh, file) have a\n> default policy of `always`, known-dangerous protocols (ext) have a\n> default policy of `never`, and all other protocols have a default policy\n> of `user`. Supported policies:\n> \n> \n> What happens if protocol.allow is set to true?\n\nThat wouldn't be allowed, its an unknown value as the only permitted\nvalues are always, never, and user.\n\n> \n> \n> \n> > + if unset, known-safe protocols (http, https, git, ssh, file) have a\n> > + default policy of `always`, known-dangerous protocols (ext) have a\n> > + default policy of `never`, and all other protocols have a default policy\n> > + of `user`. Supported policies:\n> > ++\n> > +--\n> > +\n> > +* `always` - protocol is always able to be used.\n> > +\n> > +* `never` - protocol is never able to be used.\n> > +\n> > +* `user` - protocol is only able to be used when `GIT_PROTOCOL_FROM_USER` is\n> > + either unset or has a value of 1. This policy should be used when you want a\n> > + protocol to be usable by the user but don't want it used by commands which\n> > + execute clone/fetch/pull commands without user input, e.g. recursive\n> > + submodule initialization.\n> > +\n> > +--\n> > +\n> > +protocol..allow::\n> > + Set a policy to be used by protocol with clone/fetch/pull commands.\n> \n> How does this interact with protocol.allow?\n> \n> When protocol.allow is set, this overrides the specific protocol.\n> If protocol is not set, it overrides the specific protocol as well(?)\n\nprotocol.allow is a default for protocols which don't have a specific\nprotocol..allow entry\n\n> > + test_expect_success \"clone $desc (disabled)\" '\n> > + rm -rf tmp.git &&\n> > + test_must_fail git clone --bare \"$url\" tmp.git\n> > + '\n> \n> \n> I could not spot a test for GIT_ALLOW_PROTOCOL overriding\n> any protocol*allow policy. Is that also worth testing? (for\n> backwards compatibility of tools that make use of GIT_ALLOW_PROTOCOL\n> but the user already setup a policy.\n\nI can add in one quick test for that.\n\n-- \nBrandon Williams\n"},{"id":"305502","messageId":"20161107191754.GB143723@google.com","threadId":"44417","inReplyTo":"20161104230613.epbziphiqyl57bcn@sigill.intra.peff.net","subject":"Re: [PATCH v3] transport: add protocol policy config option","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-11-07T19:17:54Z","receivedAt":"2016-11-07T19:18:02Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 11/04, Jeff King wrote:\n> > diff --git a/Documentation/config.txt b/Documentation/config.txt\n> > index 27069ac..5d845c4 100644\n> > --- a/Documentation/config.txt\n> > +++ b/Documentation/config.txt\n> > @@ -2308,6 +2308,31 @@ pretty.::\n> > \tNote that an alias with the same name as a built-in format\n> > \twill be silently ignored.\n> > \n> > +protocol.allow::\n> > +\tIf set, provide a user defined default policy for all protocols which\n> > +\tdon't explicitly have a policy (protocol..allow). By default,\n> > +\tif unset, known-safe protocols (http, https, git, ssh, file) have a\n> > +\tdefault policy of `always`, known-dangerous protocols (ext) have a\n> > +\tdefault policy of `never`, and all other protocols have a default policy\n> > +\tof `user`. Supported policies:\n> > ++\n> > +--\n> > +\n> > +* `always` - protocol is always able to be used.\n> > +\n> > +* `never` - protocol is never able to be used.\n> > +\n> > +* `user` - protocol is only able to be used when `GIT_PROTOCOL_FROM_USER` is\n> > + either unset or has a value of 1. This policy should be used when you want a\n> > + protocol to be usable by the user but don't want it used by commands which\n> > + execute clone/fetch/pull commands without user input, e.g. recursive\n> > + submodule initialization.\n> \n> Makes sense. I wonder if it would be good to emphasize _directly_ usable\n> here. I.e., \"...when you want a protocol to be directly usable by the\n> user but don't want...\".\n> \n> Should clone/fetch/pull also include push?\n\nYou're right, that should really have been clone/fetch/push.\n\n> \n> > +protocol..allow::\n> > +\tSet a policy to be used by protocol with clone/fetch/pull commands.\n> > +\n> \n> Nice that this matches protocol.allow, so we don't need to re-explain\n> that.\n> \n> Should the list of protocols be here? I know they're covered under\n> GIT_ALLOW_PROTOCOL already, but if this is the preferred system, we\n> should probably explain them here, and then just have GIT_ALLOW_PROTOCOL\n> refer the user.\n\nRight now the list of protocols under GIT_ALLOW_PROTOCOL looks like it\nhas a bit more documentation with how the colon list works. The\nprotocols are also mentioned above with their default behaviour.\n\n> \n> > diff --git a/Documentation/git.txt b/Documentation/git.txt\n> > index ab7215e..ab25580 100644\n> > --- a/Documentation/git.txt\n> > +++ b/Documentation/git.txt\n> > @@ -1150,13 +1150,13 @@ of clones and fetches.\n> > \tcloning a repository to make a backup).\n> > \n> > `GIT_ALLOW_PROTOCOL`::\n> > -\tIf set, provide a colon-separated list of protocols which are\n> > -\tallowed to be used with fetch/push/clone. This is useful to\n> > -\trestrict recursive submodule initialization from an untrusted\n> > -\trepository. Any protocol not mentioned will be disallowed (i.e.,\n> > -\tthis is a whitelist, not a blacklist). If the variable is not\n> > -\tset at all, all protocols are enabled. The protocol names\n> > -\tcurrently used by git are:\n> > +\tThe new way to configure allowed protocols is done through the config\n> > +\tinterface, though this setting takes precedences. See\n> > +\tlinkgit:git-config[1] for more details. If set, provide a\n> > +\tcolon-separated list of protocols which are allowed to be used with\n> > +\tfetch/push/clone. Any protocol not mentioned will be disallowed (i.e.,\n> > +\tthis is a whitelist, not a blacklist). The protocol names currently\n> > +\tused by git are:\n> \n> I wonder if we can explain this in terms of the config system. Something\n> like:\n> \n> If set to a colon-separated list of zero or more protocols, behave as\n> if `protocol.allow` is set to `never`, and each of the listed\n> protocols has `protocol.$protocol.allow` set to `always`.\n\nYeah that makes sense.\n\n> \n> > +`GIT_PROTOCOL_FROM_USER`::\n> > +\tSet to 0 to prevent protocols used by fetch/push/clone which are\n> > +\tconfigured to the `user` state. This is useful to restrict recursive\n> > +\tsubmodule initialization from an untrusted repository. See\n> > +\tlinkgit:git-config[1] for more details.\n> \n> Under \"this is useful\", it may make sense to make it clear that external\n> programs can use this, too. Something like:\n> \n> It may also be useful for programs which feed potentially-untrusted\n> URLs to git commands.\n\nI'll put that in too.\n\n> \n> > diff --git a/t/lib-proto-disable.sh b/t/lib-proto-disable.sh\n> > index b0917d9..5950fbf 100644\n> > --- a/t/lib-proto-disable.sh\n> > +++ b/t/lib-proto-disable.sh\n> > @@ -1,15 +1,12 @@\n> > # Test routines for checking protocol disabling.\n> > \n> > -# test cloning a particular protocol\n> > -# $1 - description of the protocol\n> > -# $2 - machine-readable name of the protocol\n> > -# $3 - the URL to try cloning\n> > -test_proto () {\n> > +# Test clone/fetch/push with GIT_ALLOW_PROTOCOL whitelist\n> > +test_whitelist () {\n> > \tdesc=$1\n> > \tproto=$2\n> > \turl=$3\n> > \n> > -\ttest_expect_success \"clone $1 (enabled)\" '\n> > +\ttest_expect_success \"clone $desc (enabled)\" '\n> \n> Yeah, this should have been $desc all along. It makes the diff really\n> noisy, though. Should it be split out into a preparatory change?\n\nI'll pull it out to make the patch a bit cleaner.\n\n-- \nBrandon Williams\n"},{"id":"305504","messageId":"1478547323-47332-2-git-send-email-bmwill@google.com","threadId":"44417","inReplyTo":"1478547323-47332-1-git-send-email-bmwill@google.com","subject":"[PATCH v4 2/2] transport: add protocol policy config option","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-11-07T19:35:23Z","receivedAt":"2016-11-07T19:42:53Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"Previously the `GIT_ALLOW_PROTOCOL` environment variable was used to\nspecify a whitelist of protocols to be used in clone/fetch/push\ncommands. This patch introduces new configuration options for more\nfine-grained control for allowing/disallowing protocols. This also has\nthe added benefit of allowing easier construction of a protocol\nwhitelist on systems where setting an environment variable is\nnon-trivial.\n\nNow users can specify a policy to be used for each type of protocol via\nthe 'protocol..allow' config option. A default policy for all\nunconfigured protocols can be set with the 'protocol.allow' config\noption. If no user configured default is made git, by default, will\nallow known-safe protocols (http, https, git, ssh, file), disallow\nknown-dangerous protocols (ext), and have a default policy of `user` for\nall other protocols.\n\nThe supported policies are `always`, `never`, and `user`. The `user`\npolicy can be used to configure a protocol to be usable when explicitly\nused by a user, while disallowing it for commands which run\nclone/fetch/push commands without direct user intervention (e.g.\nrecursive initialization of submodules). Commands which can potentially\nclone/fetch/push from untrusted repositories without user intervention\ncan export `GIT_PROTOCOL_FROM_USER` with a value of '0' to prevent\nprotocols configured to the `user` policy from being used.\n\nFix remote-ext tests to use the new config to allow the ext\nprotocol to be tested.\n\nBased on a patch by Jeff King \n\nSigned-off-by: Brandon Williams \n---\n Documentation/config.txt | 25 ++++++++\n Documentation/git.txt | 21 ++++---\n git-submodule.sh | 12 ++--\n t/lib-proto-disable.sh | 129 +++++++++++++++++++++++++++++++++++++--\n t/t5509-fetch-push-namespaces.sh | 1 +\n t/t5802-connect-helper.sh | 1 +\n transport.c | 75 ++++++++++++++++++++++-\n 7 files changed, 242 insertions(+), 22 deletions(-)\n\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex 27069ac..0b1c186 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -2308,6 +2308,31 @@ pretty.::\n \tNote that an alias with the same name as a built-in format\n \twill be silently ignored.\n \n+protocol.allow::\n+\tIf set, provide a user defined default policy for all protocols which\n+\tdon't explicitly have a policy (`protocol..allow`). By default,\n+\tif unset, known-safe protocols (http, https, git, ssh, file) have a\n+\tdefault policy of `always`, known-dangerous protocols (ext) have a\n+\tdefault policy of `never`, and all other protocols have a default\n+\tpolicy of `user`. Supported policies:\n++\n+--\n+\n+* `always` - protocol is always able to be used.\n+\n+* `never` - protocol is never able to be used.\n+\n+* `user` - protocol is only able to be used when `GIT_PROTOCOL_FROM_USER` is\n+ either unset or has a value of 1. This policy should be used when you want a\n+ protocol to be directly usable by the user but don't want it used by commands which\n+ execute clone/fetch/push commands without user input, e.g. recursive\n+ submodule initialization.\n+\n+--\n+\n+protocol..allow::\n+\tSet a policy to be used by protocol with clone/fetch/push commands.\n+\n pull.ff::\n \tBy default, Git does not create an extra merge commit when merging\n \ta commit that is a descendant of the current commit. Instead, the\ndiff --git a/Documentation/git.txt b/Documentation/git.txt\nindex ab7215e..c9823e3 100644\n--- a/Documentation/git.txt\n+++ b/Documentation/git.txt\n@@ -1150,13 +1150,14 @@ of clones and fetches.\n \tcloning a repository to make a backup).\n \n `GIT_ALLOW_PROTOCOL`::\n-\tIf set, provide a colon-separated list of protocols which are\n-\tallowed to be used with fetch/push/clone. This is useful to\n-\trestrict recursive submodule initialization from an untrusted\n-\trepository. Any protocol not mentioned will be disallowed (i.e.,\n-\tthis is a whitelist, not a blacklist). If the variable is not\n-\tset at all, all protocols are enabled. The protocol names\n-\tcurrently used by git are:\n+\tThe preferred way to configure allowed protocols is done through the\n+\tconfig interface, though this setting takes precedences. See\n+\tlinkgit:git-config[1] for more details. If set to a colon-separated\n+\tlist of protocols, behave as if `protocol.allow` is set to `never`, and\n+\teach of the listed protocols has `protocol..allow` set to\n+\t`always`. In other words, any protocol not mentioned will be\n+\tdisallowed (i.e., this is a whitelist, not a blacklist). The protocol\n+\tnames currently used by git are:\n \n \t - `file`: any local file-based path (including `file://` URLs,\n \t or local paths)\n@@ -1174,6 +1175,12 @@ of clones and fetches.\n \t - any external helpers are named by their protocol (e.g., use\n \t `hg` to allow the `git-remote-hg` helper)\n \n+`GIT_PROTOCOL_FROM_USER`::\n+\tSet to 0 to prevent protocols used by fetch/push/clone which are\n+\tconfigured to the `user` state. This is useful to restrict recursive\n+\tsubmodule initialization from an untrusted repository or for programs\n+\twhich feed potentially-untrusted URLS to git commands. See\n+\tlinkgit:git-config[1] for more details.\n \n Discussion[[Discussion]]\n ------------------------\ndiff --git a/git-submodule.sh b/git-submodule.sh\nindex a024a13..0a477b4 100755\n--- a/git-submodule.sh\n+++ b/git-submodule.sh\n@@ -21,14 +21,10 @@ require_work_tree\n wt_prefix=$(git rev-parse --show-prefix)\n cd_to_toplevel\n \n-# Restrict ourselves to a vanilla subset of protocols; the URLs\n-# we get are under control of a remote repository, and we do not\n-# want them kicking off arbitrary git-remote-* programs.\n-#\n-# If the user has already specified a set of allowed protocols,\n-# we assume they know what they're doing and use that instead.\n-: ${GIT_ALLOW_PROTOCOL=file:git:http:https:ssh}\n-export GIT_ALLOW_PROTOCOL\n+# Tell the rest of git that any URLs we get don't come\n+# directly from the user, so it can apply policy as appropriate.\n+GIT_PROTOCOL_FROM_USER=0\n+export GIT_PROTOCOL_FROM_USER\n \n command=\n branch=\ndiff --git a/t/lib-proto-disable.sh b/t/lib-proto-disable.sh\nindex be88e9a..4b2db2f 100644\n--- a/t/lib-proto-disable.sh\n+++ b/t/lib-proto-disable.sh\n@@ -1,10 +1,7 @@\n # Test routines for checking protocol disabling.\n \n-# test cloning a particular protocol\n-# $1 - description of the protocol\n-# $2 - machine-readable name of the protocol\n-# $3 - the URL to try cloning\n-test_proto () {\n+# Test clone/fetch/push with GIT_ALLOW_PROTOCOL whitelist\n+test_whitelist () {\n \tdesc=$1\n \tproto=$2\n \turl=$3\n@@ -62,6 +59,128 @@ test_proto () {\n \t\t\ttest_must_fail git clone --bare \"$url\" tmp.git\n \t\t)\n \t'\n+\n+\ttest_expect_success \"clone $desc (env var has precedence)\" '\n+\t\trm -rf tmp.git &&\n+\t\t(\n+\t\t\tGIT_ALLOW_PROTOCOL=none &&\n+\t\t\texport GIT_ALLOW_PROTOCOL &&\n+\t\t\ttest_must_fail git -c protocol.$proto.allow=always clone --bare \"$url\" tmp.git\n+\t\t)\n+\t'\n+}\n+\n+test_config () {\n+\tdesc=$1\n+\tproto=$2\n+\turl=$3\n+\n+\t# Test clone/fetch/push with protocol..allow config\n+\ttest_expect_success \"clone $desc (enabled with config)\" '\n+\t\trm -rf tmp.git &&\n+\t\tgit -c protocol.$proto.allow=always clone --bare \"$url\" tmp.git\n+\t'\n+\n+\ttest_expect_success \"fetch $desc (enabled)\" '\n+\t\tgit -C tmp.git -c protocol.$proto.allow=always fetch\n+\t'\n+\n+\ttest_expect_success \"push $desc (enabled)\" '\n+\t\tgit -C tmp.git -c protocol.$proto.allow=always push origin HEAD:pushed\n+\t'\n+\n+\ttest_expect_success \"push $desc (disabled)\" '\n+\t\ttest_must_fail git -C tmp.git -c protocol.$proto.allow=never push origin HEAD:pushed\n+\t'\n+\n+\ttest_expect_success \"fetch $desc (disabled)\" '\n+\t\ttest_must_fail git -C tmp.git -c protocol.$proto.allow=never fetch\n+\t'\n+\n+\ttest_expect_success \"clone $desc (disabled)\" '\n+\t\trm -rf tmp.git &&\n+\t\ttest_must_fail git -c protocol.$proto.allow=never clone --bare \"$url\" tmp.git\n+\t'\n+\n+\t# Test clone/fetch/push with protocol.user.allow and its env var\n+\ttest_expect_success \"clone $desc (enabled)\" '\n+\t\trm -rf tmp.git &&\n+\t\tgit -c protocol.$proto.allow=user clone --bare \"$url\" tmp.git\n+\t'\n+\n+\ttest_expect_success \"fetch $desc (enabled)\" '\n+\t\tgit -C tmp.git -c protocol.$proto.allow=user fetch\n+\t'\n+\n+\ttest_expect_success \"push $desc (enabled)\" '\n+\t\tgit -C tmp.git -c protocol.$proto.allow=user push origin HEAD:pushed\n+\t'\n+\n+\ttest_expect_success \"push $desc (disabled)\" '\n+\t\t(\n+\t\t\tcd tmp.git &&\n+\t\t\tGIT_PROTOCOL_FROM_USER=0 &&\n+\t\t\texport GIT_PROTOCOL_FROM_USER &&\n+\t\t\ttest_must_fail git -c protocol.$proto.allow=user push origin HEAD:pushed\n+\t\t)\n+\t'\n+\n+\ttest_expect_success \"fetch $desc (disabled)\" '\n+\t\t(\n+\t\t\tcd tmp.git &&\n+\t\t\tGIT_PROTOCOL_FROM_USER=0 &&\n+\t\t\texport GIT_PROTOCOL_FROM_USER &&\n+\t\t\ttest_must_fail git -c protocol.$proto.allow=user fetch\n+\t\t)\n+\t'\n+\n+\ttest_expect_success \"clone $desc (disabled)\" '\n+\t\trm -rf tmp.git &&\n+\t\t(\n+\t\t\tGIT_PROTOCOL_FROM_USER=0 &&\n+\t\t\texport GIT_PROTOCOL_FROM_USER &&\n+\t\t\ttest_must_fail git -c protocol.$proto.allow=user clone --bare \"$url\" tmp.git\n+\t\t)\n+\t'\n+\n+\t# Test clone/fetch/push with protocol.allow user defined default\n+\ttest_expect_success \"clone $desc (enabled)\" '\n+\t\trm -rf tmp.git &&\n+\t\tgit config --global protocol.allow always &&\n+\t\tgit clone --bare \"$url\" tmp.git\n+\t'\n+\n+\ttest_expect_success \"fetch $desc (enabled)\" '\n+\t\tgit -C tmp.git fetch\n+\t'\n+\n+\ttest_expect_success \"push $desc (enabled)\" '\n+\t\tgit -C tmp.git push origin HEAD:pushed\n+\t'\n+\n+\ttest_expect_success \"push $desc (disabled)\" '\n+\t\tgit config --global protocol.allow never &&\n+\t\ttest_must_fail git -C tmp.git push origin HEAD:pushed\n+\t'\n+\n+\ttest_expect_success \"fetch $desc (disabled)\" '\n+\t\ttest_must_fail git -C tmp.git fetch\n+\t'\n+\n+\ttest_expect_success \"clone $desc (disabled)\" '\n+\t\trm -rf tmp.git &&\n+\t\ttest_must_fail git clone --bare \"$url\" tmp.git\n+\t'\n+}\n+\n+# test cloning a particular protocol\n+# $1 - description of the protocol\n+# $2 - machine-readable name of the protocol\n+# $3 - the URL to try cloning\n+test_proto () {\n+\ttest_whitelist \"$@\"\n+\n+\ttest_config \"$@\"\n }\n \n # set up an ssh wrapper that will access $host/$repo in the\ndiff --git a/t/t5509-fetch-push-namespaces.sh b/t/t5509-fetch-push-namespaces.sh\nindex bc44ac3..75c570a 100755\n--- a/t/t5509-fetch-push-namespaces.sh\n+++ b/t/t5509-fetch-push-namespaces.sh\n@@ -4,6 +4,7 @@ test_description='fetch/push involving ref namespaces'\n . ./test-lib.sh\n \n test_expect_success setup '\n+\tgit config --global protocol.ext.allow user &&\n \ttest_tick &&\n \tgit init original &&\n \t(\ndiff --git a/t/t5802-connect-helper.sh b/t/t5802-connect-helper.sh\nindex b7a7f9d..c6c2661 100755\n--- a/t/t5802-connect-helper.sh\n+++ b/t/t5802-connect-helper.sh\n@@ -4,6 +4,7 @@ test_description='ext::cmd remote \"connect\" helper'\n . ./test-lib.sh\n \n test_expect_success setup '\n+\tgit config --global protocol.ext.allow user &&\n \ttest_tick &&\n \tgit commit --allow-empty -m initial &&\n \ttest_tick &&\ndiff --git a/transport.c b/transport.c\nindex d57e8de..2c0ec76 100644\n--- a/transport.c\n+++ b/transport.c\n@@ -664,10 +664,81 @@ static const struct string_list *protocol_whitelist(void)\n \treturn enabled ? &allowed : NULL;\n }\n \n+enum protocol_allow_config {\n+\tPROTOCOL_ALLOW_NEVER = 0,\n+\tPROTOCOL_ALLOW_USER_ONLY,\n+\tPROTOCOL_ALLOW_ALWAYS\n+};\n+\n+static enum protocol_allow_config parse_protocol_config(const char *key,\n+\t\t\t\t\t\t\tconst char *value)\n+{\n+\tif (!strcasecmp(value, \"always\"))\n+\t\treturn PROTOCOL_ALLOW_ALWAYS;\n+\telse if (!strcasecmp(value, \"never\"))\n+\t\treturn PROTOCOL_ALLOW_NEVER;\n+\telse if (!strcasecmp(value, \"user\"))\n+\t\treturn PROTOCOL_ALLOW_USER_ONLY;\n+\n+\tdie(\"unknown value for config '%s': %s\", key, value);\n+}\n+\n+static enum protocol_allow_config get_protocol_config(const char *type)\n+{\n+\tchar *key = xstrfmt(\"protocol.%s.allow\", type);\n+\tchar *value;\n+\n+\t/* first check the per-protocol config */\n+\tif (!git_config_get_string(key, &value)) {\n+\t\tenum protocol_allow_config ret =\n+\t\t\tparse_protocol_config(key, value);\n+\t\tfree(key);\n+\t\tfree(value);\n+\t\treturn ret;\n+\t}\n+\tfree(key);\n+\n+\t/* if defined, fallback to user-defined default for unknown protocols */\n+\tif (!git_config_get_string(\"protocol.allow\", &value)) {\n+\t\tenum protocol_allow_config ret =\n+\t\t\tparse_protocol_config(\"protocol.allow\", value);\n+\t\tfree(value);\n+\t\treturn ret;\n+\t}\n+\n+\t/* fallback to built-in defaults */\n+\t/* known safe */\n+\tif (!strcmp(type, \"http\") ||\n+\t !strcmp(type, \"https\") ||\n+\t !strcmp(type, \"git\") ||\n+\t !strcmp(type, \"ssh\") ||\n+\t !strcmp(type, \"file\"))\n+\t\treturn PROTOCOL_ALLOW_ALWAYS;\n+\n+\t/* known scary; err on the side of caution */\n+\tif (!strcmp(type, \"ext\"))\n+\t\treturn PROTOCOL_ALLOW_NEVER;\n+\n+\t/* unknown; by default let them be used only directly by the user */\n+\treturn PROTOCOL_ALLOW_USER_ONLY;\n+}\n+\n int is_transport_allowed(const char *type)\n {\n-\tconst struct string_list *allowed = protocol_whitelist();\n-\treturn !allowed || string_list_has_string(allowed, type);\n+\tconst struct string_list *whitelist = protocol_whitelist();\n+\tif (whitelist)\n+\t\treturn string_list_has_string(whitelist, type);\n+\n+\tswitch (get_protocol_config(type)) {\n+\tcase PROTOCOL_ALLOW_ALWAYS:\n+\t\treturn 1;\n+\tcase PROTOCOL_ALLOW_NEVER:\n+\t\treturn 0;\n+\tcase PROTOCOL_ALLOW_USER_ONLY:\n+\t\treturn git_env_bool(\"GIT_PROTOCOL_FROM_USER\", 1);\n+\t}\n+\n+\tdie(\"BUG: invalid protocol_allow_config type\");\n }\n \n void transport_check_allowed(const char *type)\n-- \n2.8.0.rc3.226.g39d4020\n\n"},{"id":"305505","messageId":"1478547323-47332-1-git-send-email-bmwill@google.com","threadId":"44417","inReplyTo":"1478292933-7873-1-git-send-email-bmwill@google.com","subject":"[PATCH v4 1/2] lib-proto-disable: variable name fix","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-11-07T19:35:22Z","receivedAt":"2016-11-07T20:04:18Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"Small fix to use '$desc' instead of '$1' in lib-proto-disable.sh.\n\nSigned-off-by: Brandon Williams \n---\n t/lib-proto-disable.sh | 12 ++++++------\n 1 file changed, 6 insertions(+), 6 deletions(-)\n\ndiff --git a/t/lib-proto-disable.sh b/t/lib-proto-disable.sh\nindex b0917d9..be88e9a 100644\n--- a/t/lib-proto-disable.sh\n+++ b/t/lib-proto-disable.sh\n@@ -9,7 +9,7 @@ test_proto () {\n \tproto=$2\n \turl=$3\n \n-\ttest_expect_success \"clone $1 (enabled)\" '\n+\ttest_expect_success \"clone $desc (enabled)\" '\n \t\trm -rf tmp.git &&\n \t\t(\n \t\t\tGIT_ALLOW_PROTOCOL=$proto &&\n@@ -18,7 +18,7 @@ test_proto () {\n \t\t)\n \t'\n \n-\ttest_expect_success \"fetch $1 (enabled)\" '\n+\ttest_expect_success \"fetch $desc (enabled)\" '\n \t\t(\n \t\t\tcd tmp.git &&\n \t\t\tGIT_ALLOW_PROTOCOL=$proto &&\n@@ -27,7 +27,7 @@ test_proto () {\n \t\t)\n \t'\n \n-\ttest_expect_success \"push $1 (enabled)\" '\n+\ttest_expect_success \"push $desc (enabled)\" '\n \t\t(\n \t\t\tcd tmp.git &&\n \t\t\tGIT_ALLOW_PROTOCOL=$proto &&\n@@ -36,7 +36,7 @@ test_proto () {\n \t\t)\n \t'\n \n-\ttest_expect_success \"push $1 (disabled)\" '\n+\ttest_expect_success \"push $desc (disabled)\" '\n \t\t(\n \t\t\tcd tmp.git &&\n \t\t\tGIT_ALLOW_PROTOCOL=none &&\n@@ -45,7 +45,7 @@ test_proto () {\n \t\t)\n \t'\n \n-\ttest_expect_success \"fetch $1 (disabled)\" '\n+\ttest_expect_success \"fetch $desc (disabled)\" '\n \t\t(\n \t\t\tcd tmp.git &&\n \t\t\tGIT_ALLOW_PROTOCOL=none &&\n@@ -54,7 +54,7 @@ test_proto () {\n \t\t)\n \t'\n \n-\ttest_expect_success \"clone $1 (disabled)\" '\n+\ttest_expect_success \"clone $desc (disabled)\" '\n \t\trm -rf tmp.git &&\n \t\t(\n \t\t\tGIT_ALLOW_PROTOCOL=none &&\n-- \n2.8.0.rc3.226.g39d4020\n\n"},{"id":"305506","messageId":"20161107202637.ooaddlkfntw37xg3@sigill.intra.peff.net","threadId":"44417","inReplyTo":"1478547323-47332-1-git-send-email-bmwill@google.com","subject":"Re: [PATCH v4 1/2] lib-proto-disable: variable name fix","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-11-07T20:26:37Z","receivedAt":"2016-11-07T20:26:46Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Nov 07, 2016 at 11:35:22AM -0800, Brandon Williams wrote:\n\n> Small fix to use '$desc' instead of '$1' in lib-proto-disable.sh.\n\nEven for a trivial fixup like this, I think it's good to say why.\nBecause what seems trivial and obvious to you while working on the patch\nmay not be so to a reviewer, or somebody reading it 6 months later.\n\nJust something simple like:\n\n The test_proto function assigns the positional parameters to named\n variables, but then still refers to \"$desc\" as \"$1\". Using $desc is\n more readable and less error-prone.\n\n-Peff\n"},{"id":"305508","messageId":"20161107204430.z6wrazgad4e7yn66@sigill.intra.peff.net","threadId":"44417","inReplyTo":"1478547323-47332-2-git-send-email-bmwill@google.com","subject":"Re: [PATCH v4 2/2] transport: add protocol policy config option","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-11-07T20:44:31Z","receivedAt":"2016-11-07T20:44:39Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Nov 07, 2016 at 11:35:23AM -0800, Brandon Williams wrote:\n\n> Previously the `GIT_ALLOW_PROTOCOL` environment variable was used to\n> specify a whitelist of protocols to be used in clone/fetch/push\n> commands. This patch introduces new configuration options for more\n> fine-grained control for allowing/disallowing protocols. This also has\n> the added benefit of allowing easier construction of a protocol\n> whitelist on systems where setting an environment variable is\n> non-trivial.\n> \n> Now users can specify a policy to be used for each type of protocol via\n> the 'protocol..allow' config option. A default policy for all\n> unconfigured protocols can be set with the 'protocol.allow' config\n> option. If no user configured default is made git, by default, will\n> allow known-safe protocols (http, https, git, ssh, file), disallow\n\nA minor nit, but in \"If no user configured default is made git, by\ndefault, will...\" the second \"by default\" is redundant. And possibly\nmisleading. This _is_ the default case, there is no other way to change\nit. :)\n\n> +protocol..allow::\n> +\tSet a policy to be used by protocol with clone/fetch/push commands.\n\n`` isn't defined here at all. I still think the list of protocols\nshould go here, but at the very least, you need to point the user to the\nexisting list in git(1).\n\n> `GIT_ALLOW_PROTOCOL`::\n> -\tIf set, provide a colon-separated list of protocols which are\n> -\tallowed to be used with fetch/push/clone. This is useful to\n> -\trestrict recursive submodule initialization from an untrusted\n> -\trepository. Any protocol not mentioned will be disallowed (i.e.,\n> -\tthis is a whitelist, not a blacklist). If the variable is not\n> -\tset at all, all protocols are enabled. The protocol names\n> -\tcurrently used by git are:\n> +\tThe preferred way to configure allowed protocols is done through the\n> +\tconfig interface, though this setting takes precedences. See\n\ns/precedences/precedence/.\n\nI actually wonder if we should even drop \"the preferred way\" here. I had\ninitially thought we would want it just for backwards-compatibility, but\nI actually think it is useful in its own right as a shorthand for more\ncomplicated config (and since we have to keep it around effectively\nforever anyway, there's no real cost to continuing to call it a feature\nversus a deprecated feature).\n\nI'm including a squashable patch at the end of this email with suggested\nwording (and which also moves the protocol list).\n\n> +\ttest_expect_success \"clone $desc (env var has precedence)\" '\n> +\t\trm -rf tmp.git &&\n> +\t\t(\n> +\t\t\tGIT_ALLOW_PROTOCOL=none &&\n> +\t\t\texport GIT_ALLOW_PROTOCOL &&\n> +\t\t\ttest_must_fail git -c protocol.$proto.allow=always clone --bare \"$url\" tmp.git\n> +\t\t)\n> +\t'\n\nThis test is a good addition in this round.\n\nI suppose we could test also that GIT_ALLOW_PROTOCOL overrides\nprotocol.allow, but I'm not sure if there is a point. If git were a\nblack box, it's a thing I might check, but we know from the design that\nthis is an unlikely bug (and that the implementation is unlikely to\nchange in a way to cause it). So I could go either way.\n\n> [...]\n\nThe rest of it looks good to me.\n\nSquashable documentation suggestions are below.\n\n-Peff\n\n---\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex e89b33f9e..a9dc23f82 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -2331,7 +2331,28 @@ protocol.allow::\n --\n \n protocol..allow::\n-\tSet a policy to be used by protocol with clone/fetch/push commands.\n+\tSet a policy to be used by protocol `` with clone/fetch/push\n+\tcommands. See `protocol.allow` above for the available policies.\n++\n+The protocol names currently used by git are:\n++\n+--\n+ - `file`: any local file-based path (including `file://` URLs,\n+\t or local paths)\n+\n+ - `git`: the anonymous git protocol over a direct TCP\n+ connection (or proxy, if configured)\n+\n+ - `ssh`: git over ssh (including `host:path` syntax,\n+ `ssh://`, etc).\n+\n+ - `http`: git over http, both \"smart http\" and \"dumb http\".\n+ Note that this does _not_ include `https`; if you want to configure\n+ both, you must do so individually.\n+\n+ - any external helpers are named by their protocol (e.g., use\n+ `hg` to allow the `git-remote-hg` helper)\n+--\n \n pull.ff::\n \tBy default, Git does not create an extra merge commit when merging\ndiff --git a/Documentation/git.txt b/Documentation/git.txt\nindex c9823e34a..c52cec840 100644\n--- a/Documentation/git.txt\n+++ b/Documentation/git.txt\n@@ -1150,30 +1150,13 @@ of clones and fetches.\n \tcloning a repository to make a backup).\n \n `GIT_ALLOW_PROTOCOL`::\n-\tThe preferred way to configure allowed protocols is done through the\n-\tconfig interface, though this setting takes precedences. See\n-\tlinkgit:git-config[1] for more details. If set to a colon-separated\n-\tlist of protocols, behave as if `protocol.allow` is set to `never`, and\n-\teach of the listed protocols has `protocol..allow` set to\n-\t`always`. In other words, any protocol not mentioned will be\n-\tdisallowed (i.e., this is a whitelist, not a blacklist). The protocol\n-\tnames currently used by git are:\n-\n-\t - `file`: any local file-based path (including `file://` URLs,\n-\t or local paths)\n-\n-\t - `git`: the anonymous git protocol over a direct TCP\n-\t connection (or proxy, if configured)\n-\n-\t - `ssh`: git over ssh (including `host:path` syntax,\n-\t `ssh://`, etc).\n-\n-\t - `http`: git over http, both \"smart http\" and \"dumb http\".\n-\t Note that this does _not_ include `https`; if you want both,\n-\t you should specify both as `http:https`.\n-\n-\t - any external helpers are named by their protocol (e.g., use\n-\t `hg` to allow the `git-remote-hg` helper)\n+\tIf set to a colon-separated list of protocols, behave as if\n+\t`protocol.allow` is set to `never`, and each of the listed\n+\tprotocols has `protocol..allow` set to `always`\n+\t(overriding any existing configuration). In other words, any\n+\tprotocol not mentioned will be disallowed (i.e., this is a\n+\twhitelist, not a blacklist). See the description of\n+\t`protocol.allow` in linkgit:git-config[1] for more details.\n \n `GIT_PROTOCOL_FROM_USER`::\n \tSet to 0 to prevent protocols used by fetch/push/clone which are\n"},{"id":"305509","messageId":"20161107204028.GC143723@google.com","threadId":"44417","inReplyTo":"20161107202637.ooaddlkfntw37xg3@sigill.intra.peff.net","subject":"Re: [PATCH v4 1/2] lib-proto-disable: variable name fix","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-11-07T20:40:28Z","receivedAt":"2016-11-07T20:47:57Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 11/07, Jeff King wrote:\n> On Mon, Nov 07, 2016 at 11:35:22AM -0800, Brandon Williams wrote:\n> \n> > Small fix to use '$desc' instead of '$1' in lib-proto-disable.sh.\n> \n> Even for a trivial fixup like this, I think it's good to say why.\n> Because what seems trivial and obvious to you while working on the patch\n> may not be so to a reviewer, or somebody reading it 6 months later.\n> \n> Just something simple like:\n> \n> The test_proto function assigns the positional parameters to named\n> variables, but then still refers to \"$desc\" as \"$1\". Using $desc is\n> more readable and less error-prone.\n> \n> -Peff\n\nAlright will do. Commit messages don't seem to be an area of strength\nfor me, but I'm working on it! :D\n\n-- \nBrandon Williams\n"},{"id":"305511","messageId":"20161107204838.xm463zdimiw7fx77@sigill.intra.peff.net","threadId":"44417","inReplyTo":"20161107204028.GC143723@google.com","subject":"Re: [PATCH v4 1/2] lib-proto-disable: variable name fix","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-11-07T20:48:39Z","receivedAt":"2016-11-07T20:48:45Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Nov 07, 2016 at 12:40:28PM -0800, Brandon Williams wrote:\n\n> On 11/07, Jeff King wrote:\n> > On Mon, Nov 07, 2016 at 11:35:22AM -0800, Brandon Williams wrote:\n> > \n> > > Small fix to use '$desc' instead of '$1' in lib-proto-disable.sh.\n> > \n> > Even for a trivial fixup like this, I think it's good to say why.\n> > Because what seems trivial and obvious to you while working on the patch\n> > may not be so to a reviewer, or somebody reading it 6 months later.\n> > \n> > Just something simple like:\n> > \n> > The test_proto function assigns the positional parameters to named\n> > variables, but then still refers to \"$desc\" as \"$1\". Using $desc is\n> > more readable and less error-prone.\n> > \n> > -Peff\n> \n> Alright will do. Commit messages don't seem to be an area of strength\n> for me, but I'm working on it! :D\n\nIt's possible that I'm overly picky about my commit messages, but that\ndoes not stop me from trying to train an army of picky-commit-message\nclones. :)\n\n-Peff\n"},{"id":"305513","messageId":"20161107210216.GD143723@google.com","threadId":"44417","inReplyTo":"20161107204430.z6wrazgad4e7yn66@sigill.intra.peff.net","subject":"Re: [PATCH v4 2/2] transport: add protocol policy config option","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-11-07T21:02:16Z","receivedAt":"2016-11-07T21:02:23Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 11/07, Jeff King wrote:\n> > +\ttest_expect_success \"clone $desc (env var has precedence)\" '\n> > +\t\trm -rf tmp.git &&\n> > +\t\t(\n> > +\t\t\tGIT_ALLOW_PROTOCOL=none &&\n> > +\t\t\texport GIT_ALLOW_PROTOCOL &&\n> > +\t\t\ttest_must_fail git -c protocol.$proto.allow=always clone --bare \"$url\" tmp.git\n> > +\t\t)\n> > +\t'\n> \n> This test is a good addition in this round.\n> \n> I suppose we could test also that GIT_ALLOW_PROTOCOL overrides\n> protocol.allow, but I'm not sure if there is a point. If git were a\n> black box, it's a thing I might check, but we know from the design that\n> this is an unlikely bug (and that the implementation is unlikely to\n> change in a way to cause it). So I could go either way.\n\nI'll add in another test for that, no reason not to test it.\n\n> \n> Squashable documentation suggestions are below.\n> \n\nSounds good\n\n-- \nBrandon Williams\n"},{"id":"305517","messageId":"1478555462-132573-2-git-send-email-bmwill@google.com","threadId":"44417","inReplyTo":"1478555462-132573-1-git-send-email-bmwill@google.com","subject":"[PATCH v5 2/2] transport: add protocol policy config option","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-11-07T21:51:02Z","receivedAt":"2016-11-07T21:51:23Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"Previously the `GIT_ALLOW_PROTOCOL` environment variable was used to\nspecify a whitelist of protocols to be used in clone/fetch/push\ncommands. This patch introduces new configuration options for more\nfine-grained control for allowing/disallowing protocols. This also has\nthe added benefit of allowing easier construction of a protocol\nwhitelist on systems where setting an environment variable is\nnon-trivial.\n\nNow users can specify a policy to be used for each type of protocol via\nthe 'protocol..allow' config option. A default policy for all\nunconfigured protocols can be set with the 'protocol.allow' config\noption. If no user configured default is made git will allow known-safe\nprotocols (http, https, git, ssh, file), disallow known-dangerous\nprotocols (ext), and have a default policy of `user` for all other\nprotocols.\n\nThe supported policies are `always`, `never`, and `user`. The `user`\npolicy can be used to configure a protocol to be usable when explicitly\nused by a user, while disallowing it for commands which run\nclone/fetch/push commands without direct user intervention (e.g.\nrecursive initialization of submodules). Commands which can potentially\nclone/fetch/push from untrusted repositories without user intervention\ncan export `GIT_PROTOCOL_FROM_USER` with a value of '0' to prevent\nprotocols configured to the `user` policy from being used.\n\nFix remote-ext tests to use the new config to allow the ext\nprotocol to be tested.\n\nBased on a patch by Jeff King \n\nSigned-off-by: Brandon Williams \n---\n Documentation/config.txt | 46 ++++++++++++++\n Documentation/git.txt | 38 +++++-------\n git-submodule.sh | 12 ++--\n t/lib-proto-disable.sh | 130 +++++++++++++++++++++++++++++++++++++--\n t/t5509-fetch-push-namespaces.sh | 1 +\n t/t5802-connect-helper.sh | 1 +\n transport.c | 75 +++++++++++++++++++++-\n 7 files changed, 264 insertions(+), 39 deletions(-)\n\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex 27069ac..5fe50bc 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -2308,6 +2308,52 @@ pretty.::\n \tNote that an alias with the same name as a built-in format\n \twill be silently ignored.\n \n+protocol.allow::\n+\tIf set, provide a user defined default policy for all protocols which\n+\tdon't explicitly have a policy (`protocol..allow`). By default,\n+\tif unset, known-safe protocols (http, https, git, ssh, file) have a\n+\tdefault policy of `always`, known-dangerous protocols (ext) have a\n+\tdefault policy of `never`, and all other protocols have a default\n+\tpolicy of `user`. Supported policies:\n++\n+--\n+\n+* `always` - protocol is always able to be used.\n+\n+* `never` - protocol is never able to be used.\n+\n+* `user` - protocol is only able to be used when `GIT_PROTOCOL_FROM_USER` is\n+ either unset or has a value of 1. This policy should be used when you want a\n+ protocol to be directly usable by the user but don't want it used by commands which\n+ execute clone/fetch/push commands without user input, e.g. recursive\n+ submodule initialization.\n+\n+--\n+\n+protocol..allow::\n+\tSet a policy to be used by protocol `` with clone/fetch/push\n+\tcommands. See `protocol.allow` above for the available policies.\n++\n+The protocol names currently used by git are:\n++\n+--\n+ - `file`: any local file-based path (including `file://` URLs,\n+ or local paths)\n+\n+ - `git`: the anonymous git protocol over a direct TCP\n+ connection (or proxy, if configured)\n+\n+ - `ssh`: git over ssh (including `host:path` syntax,\n+ `ssh://`, etc).\n+\n+ - `http`: git over http, both \"smart http\" and \"dumb http\".\n+ Note that this does _not_ include `https`; if you want to configure\n+ both, you must do so individually.\n+\n+ - any external helpers are named by their protocol (e.g., use\n+ `hg` to allow the `git-remote-hg` helper)\n+--\n+\n pull.ff::\n \tBy default, Git does not create an extra merge commit when merging\n \ta commit that is a descendant of the current commit. Instead, the\ndiff --git a/Documentation/git.txt b/Documentation/git.txt\nindex ab7215e..c52cec8 100644\n--- a/Documentation/git.txt\n+++ b/Documentation/git.txt\n@@ -1150,30 +1150,20 @@ of clones and fetches.\n \tcloning a repository to make a backup).\n \n `GIT_ALLOW_PROTOCOL`::\n-\tIf set, provide a colon-separated list of protocols which are\n-\tallowed to be used with fetch/push/clone. This is useful to\n-\trestrict recursive submodule initialization from an untrusted\n-\trepository. Any protocol not mentioned will be disallowed (i.e.,\n-\tthis is a whitelist, not a blacklist). If the variable is not\n-\tset at all, all protocols are enabled. The protocol names\n-\tcurrently used by git are:\n-\n-\t - `file`: any local file-based path (including `file://` URLs,\n-\t or local paths)\n-\n-\t - `git`: the anonymous git protocol over a direct TCP\n-\t connection (or proxy, if configured)\n-\n-\t - `ssh`: git over ssh (including `host:path` syntax,\n-\t `ssh://`, etc).\n-\n-\t - `http`: git over http, both \"smart http\" and \"dumb http\".\n-\t Note that this does _not_ include `https`; if you want both,\n-\t you should specify both as `http:https`.\n-\n-\t - any external helpers are named by their protocol (e.g., use\n-\t `hg` to allow the `git-remote-hg` helper)\n-\n+\tIf set to a colon-separated list of protocols, behave as if\n+\t`protocol.allow` is set to `never`, and each of the listed\n+\tprotocols has `protocol..allow` set to `always`\n+\t(overriding any existing configuration). In other words, any\n+\tprotocol not mentioned will be disallowed (i.e., this is a\n+\twhitelist, not a blacklist). See the description of\n+\t`protocol.allow` in linkgit:git-config[1] for more details.\n+\n+`GIT_PROTOCOL_FROM_USER`::\n+\tSet to 0 to prevent protocols used by fetch/push/clone which are\n+\tconfigured to the `user` state. This is useful to restrict recursive\n+\tsubmodule initialization from an untrusted repository or for programs\n+\twhich feed potentially-untrusted URLS to git commands. See\n+\tlinkgit:git-config[1] for more details.\n \n Discussion[[Discussion]]\n ------------------------\ndiff --git a/git-submodule.sh b/git-submodule.sh\nindex a024a13..0a477b4 100755\n--- a/git-submodule.sh\n+++ b/git-submodule.sh\n@@ -21,14 +21,10 @@ require_work_tree\n wt_prefix=$(git rev-parse --show-prefix)\n cd_to_toplevel\n \n-# Restrict ourselves to a vanilla subset of protocols; the URLs\n-# we get are under control of a remote repository, and we do not\n-# want them kicking off arbitrary git-remote-* programs.\n-#\n-# If the user has already specified a set of allowed protocols,\n-# we assume they know what they're doing and use that instead.\n-: ${GIT_ALLOW_PROTOCOL=file:git:http:https:ssh}\n-export GIT_ALLOW_PROTOCOL\n+# Tell the rest of git that any URLs we get don't come\n+# directly from the user, so it can apply policy as appropriate.\n+GIT_PROTOCOL_FROM_USER=0\n+export GIT_PROTOCOL_FROM_USER\n \n command=\n branch=\ndiff --git a/t/lib-proto-disable.sh b/t/lib-proto-disable.sh\nindex be88e9a..02f49cb 100644\n--- a/t/lib-proto-disable.sh\n+++ b/t/lib-proto-disable.sh\n@@ -1,10 +1,7 @@\n # Test routines for checking protocol disabling.\n \n-# test cloning a particular protocol\n-# $1 - description of the protocol\n-# $2 - machine-readable name of the protocol\n-# $3 - the URL to try cloning\n-test_proto () {\n+# Test clone/fetch/push with GIT_ALLOW_PROTOCOL whitelist\n+test_whitelist () {\n \tdesc=$1\n \tproto=$2\n \turl=$3\n@@ -62,6 +59,129 @@ test_proto () {\n \t\t\ttest_must_fail git clone --bare \"$url\" tmp.git\n \t\t)\n \t'\n+\n+\ttest_expect_success \"clone $desc (env var has precedence)\" '\n+\t\trm -rf tmp.git &&\n+\t\t(\n+\t\t\tGIT_ALLOW_PROTOCOL=none &&\n+\t\t\texport GIT_ALLOW_PROTOCOL &&\n+\t\t\ttest_must_fail git -c protocol.allow=always clone --bare \"$url\" tmp.git &&\n+\t\t\ttest_must_fail git -c protocol.$proto.allow=always clone --bare \"$url\" tmp.git\n+\t\t)\n+\t'\n+}\n+\n+test_config () {\n+\tdesc=$1\n+\tproto=$2\n+\turl=$3\n+\n+\t# Test clone/fetch/push with protocol..allow config\n+\ttest_expect_success \"clone $desc (enabled with config)\" '\n+\t\trm -rf tmp.git &&\n+\t\tgit -c protocol.$proto.allow=always clone --bare \"$url\" tmp.git\n+\t'\n+\n+\ttest_expect_success \"fetch $desc (enabled)\" '\n+\t\tgit -C tmp.git -c protocol.$proto.allow=always fetch\n+\t'\n+\n+\ttest_expect_success \"push $desc (enabled)\" '\n+\t\tgit -C tmp.git -c protocol.$proto.allow=always push origin HEAD:pushed\n+\t'\n+\n+\ttest_expect_success \"push $desc (disabled)\" '\n+\t\ttest_must_fail git -C tmp.git -c protocol.$proto.allow=never push origin HEAD:pushed\n+\t'\n+\n+\ttest_expect_success \"fetch $desc (disabled)\" '\n+\t\ttest_must_fail git -C tmp.git -c protocol.$proto.allow=never fetch\n+\t'\n+\n+\ttest_expect_success \"clone $desc (disabled)\" '\n+\t\trm -rf tmp.git &&\n+\t\ttest_must_fail git -c protocol.$proto.allow=never clone --bare \"$url\" tmp.git\n+\t'\n+\n+\t# Test clone/fetch/push with protocol.user.allow and its env var\n+\ttest_expect_success \"clone $desc (enabled)\" '\n+\t\trm -rf tmp.git &&\n+\t\tgit -c protocol.$proto.allow=user clone --bare \"$url\" tmp.git\n+\t'\n+\n+\ttest_expect_success \"fetch $desc (enabled)\" '\n+\t\tgit -C tmp.git -c protocol.$proto.allow=user fetch\n+\t'\n+\n+\ttest_expect_success \"push $desc (enabled)\" '\n+\t\tgit -C tmp.git -c protocol.$proto.allow=user push origin HEAD:pushed\n+\t'\n+\n+\ttest_expect_success \"push $desc (disabled)\" '\n+\t\t(\n+\t\t\tcd tmp.git &&\n+\t\t\tGIT_PROTOCOL_FROM_USER=0 &&\n+\t\t\texport GIT_PROTOCOL_FROM_USER &&\n+\t\t\ttest_must_fail git -c protocol.$proto.allow=user push origin HEAD:pushed\n+\t\t)\n+\t'\n+\n+\ttest_expect_success \"fetch $desc (disabled)\" '\n+\t\t(\n+\t\t\tcd tmp.git &&\n+\t\t\tGIT_PROTOCOL_FROM_USER=0 &&\n+\t\t\texport GIT_PROTOCOL_FROM_USER &&\n+\t\t\ttest_must_fail git -c protocol.$proto.allow=user fetch\n+\t\t)\n+\t'\n+\n+\ttest_expect_success \"clone $desc (disabled)\" '\n+\t\trm -rf tmp.git &&\n+\t\t(\n+\t\t\tGIT_PROTOCOL_FROM_USER=0 &&\n+\t\t\texport GIT_PROTOCOL_FROM_USER &&\n+\t\t\ttest_must_fail git -c protocol.$proto.allow=user clone --bare \"$url\" tmp.git\n+\t\t)\n+\t'\n+\n+\t# Test clone/fetch/push with protocol.allow user defined default\n+\ttest_expect_success \"clone $desc (enabled)\" '\n+\t\trm -rf tmp.git &&\n+\t\tgit config --global protocol.allow always &&\n+\t\tgit clone --bare \"$url\" tmp.git\n+\t'\n+\n+\ttest_expect_success \"fetch $desc (enabled)\" '\n+\t\tgit -C tmp.git fetch\n+\t'\n+\n+\ttest_expect_success \"push $desc (enabled)\" '\n+\t\tgit -C tmp.git push origin HEAD:pushed\n+\t'\n+\n+\ttest_expect_success \"push $desc (disabled)\" '\n+\t\tgit config --global protocol.allow never &&\n+\t\ttest_must_fail git -C tmp.git push origin HEAD:pushed\n+\t'\n+\n+\ttest_expect_success \"fetch $desc (disabled)\" '\n+\t\ttest_must_fail git -C tmp.git fetch\n+\t'\n+\n+\ttest_expect_success \"clone $desc (disabled)\" '\n+\t\trm -rf tmp.git &&\n+\t\ttest_must_fail git clone --bare \"$url\" tmp.git\n+\t'\n+}\n+\n+# test cloning a particular protocol\n+# $1 - description of the protocol\n+# $2 - machine-readable name of the protocol\n+# $3 - the URL to try cloning\n+test_proto () {\n+\ttest_whitelist \"$@\"\n+\n+\ttest_config \"$@\"\n }\n \n # set up an ssh wrapper that will access $host/$repo in the\ndiff --git a/t/t5509-fetch-push-namespaces.sh b/t/t5509-fetch-push-namespaces.sh\nindex bc44ac3..75c570a 100755\n--- a/t/t5509-fetch-push-namespaces.sh\n+++ b/t/t5509-fetch-push-namespaces.sh\n@@ -4,6 +4,7 @@ test_description='fetch/push involving ref namespaces'\n . ./test-lib.sh\n \n test_expect_success setup '\n+\tgit config --global protocol.ext.allow user &&\n \ttest_tick &&\n \tgit init original &&\n \t(\ndiff --git a/t/t5802-connect-helper.sh b/t/t5802-connect-helper.sh\nindex b7a7f9d..c6c2661 100755\n--- a/t/t5802-connect-helper.sh\n+++ b/t/t5802-connect-helper.sh\n@@ -4,6 +4,7 @@ test_description='ext::cmd remote \"connect\" helper'\n . ./test-lib.sh\n \n test_expect_success setup '\n+\tgit config --global protocol.ext.allow user &&\n \ttest_tick &&\n \tgit commit --allow-empty -m initial &&\n \ttest_tick &&\ndiff --git a/transport.c b/transport.c\nindex d57e8de..2c0ec76 100644\n--- a/transport.c\n+++ b/transport.c\n@@ -664,10 +664,81 @@ static const struct string_list *protocol_whitelist(void)\n \treturn enabled ? &allowed : NULL;\n }\n \n+enum protocol_allow_config {\n+\tPROTOCOL_ALLOW_NEVER = 0,\n+\tPROTOCOL_ALLOW_USER_ONLY,\n+\tPROTOCOL_ALLOW_ALWAYS\n+};\n+\n+static enum protocol_allow_config parse_protocol_config(const char *key,\n+\t\t\t\t\t\t\tconst char *value)\n+{\n+\tif (!strcasecmp(value, \"always\"))\n+\t\treturn PROTOCOL_ALLOW_ALWAYS;\n+\telse if (!strcasecmp(value, \"never\"))\n+\t\treturn PROTOCOL_ALLOW_NEVER;\n+\telse if (!strcasecmp(value, \"user\"))\n+\t\treturn PROTOCOL_ALLOW_USER_ONLY;\n+\n+\tdie(\"unknown value for config '%s': %s\", key, value);\n+}\n+\n+static enum protocol_allow_config get_protocol_config(const char *type)\n+{\n+\tchar *key = xstrfmt(\"protocol.%s.allow\", type);\n+\tchar *value;\n+\n+\t/* first check the per-protocol config */\n+\tif (!git_config_get_string(key, &value)) {\n+\t\tenum protocol_allow_config ret =\n+\t\t\tparse_protocol_config(key, value);\n+\t\tfree(key);\n+\t\tfree(value);\n+\t\treturn ret;\n+\t}\n+\tfree(key);\n+\n+\t/* if defined, fallback to user-defined default for unknown protocols */\n+\tif (!git_config_get_string(\"protocol.allow\", &value)) {\n+\t\tenum protocol_allow_config ret =\n+\t\t\tparse_protocol_config(\"protocol.allow\", value);\n+\t\tfree(value);\n+\t\treturn ret;\n+\t}\n+\n+\t/* fallback to built-in defaults */\n+\t/* known safe */\n+\tif (!strcmp(type, \"http\") ||\n+\t !strcmp(type, \"https\") ||\n+\t !strcmp(type, \"git\") ||\n+\t !strcmp(type, \"ssh\") ||\n+\t !strcmp(type, \"file\"))\n+\t\treturn PROTOCOL_ALLOW_ALWAYS;\n+\n+\t/* known scary; err on the side of caution */\n+\tif (!strcmp(type, \"ext\"))\n+\t\treturn PROTOCOL_ALLOW_NEVER;\n+\n+\t/* unknown; by default let them be used only directly by the user */\n+\treturn PROTOCOL_ALLOW_USER_ONLY;\n+}\n+\n int is_transport_allowed(const char *type)\n {\n-\tconst struct string_list *allowed = protocol_whitelist();\n-\treturn !allowed || string_list_has_string(allowed, type);\n+\tconst struct string_list *whitelist = protocol_whitelist();\n+\tif (whitelist)\n+\t\treturn string_list_has_string(whitelist, type);\n+\n+\tswitch (get_protocol_config(type)) {\n+\tcase PROTOCOL_ALLOW_ALWAYS:\n+\t\treturn 1;\n+\tcase PROTOCOL_ALLOW_NEVER:\n+\t\treturn 0;\n+\tcase PROTOCOL_ALLOW_USER_ONLY:\n+\t\treturn git_env_bool(\"GIT_PROTOCOL_FROM_USER\", 1);\n+\t}\n+\n+\tdie(\"BUG: invalid protocol_allow_config type\");\n }\n \n void transport_check_allowed(const char *type)\n-- \n2.8.0.rc3.226.g39d4020\n\n"},{"id":"305533","messageId":"1478555462-132573-1-git-send-email-bmwill@google.com","threadId":"44417","inReplyTo":"1478547323-47332-1-git-send-email-bmwill@google.com","subject":"[PATCH v5 1/2] lib-proto-disable: variable name fix","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-11-07T21:51:01Z","receivedAt":"2016-11-08T02:55:31Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"The test_proto function assigns the positional parameters to named\nvariables, but then still refers to \"$desc\" as \"$1\". Using $desc is\nmore readable and less error-prone.\n\nSigned-off-by: Brandon Williams \n---\n t/lib-proto-disable.sh | 12 ++++++------\n 1 file changed, 6 insertions(+), 6 deletions(-)\n\ndiff --git a/t/lib-proto-disable.sh b/t/lib-proto-disable.sh\nindex b0917d9..be88e9a 100644\n--- a/t/lib-proto-disable.sh\n+++ b/t/lib-proto-disable.sh\n@@ -9,7 +9,7 @@ test_proto () {\n \tproto=$2\n \turl=$3\n \n-\ttest_expect_success \"clone $1 (enabled)\" '\n+\ttest_expect_success \"clone $desc (enabled)\" '\n \t\trm -rf tmp.git &&\n \t\t(\n \t\t\tGIT_ALLOW_PROTOCOL=$proto &&\n@@ -18,7 +18,7 @@ test_proto () {\n \t\t)\n \t'\n \n-\ttest_expect_success \"fetch $1 (enabled)\" '\n+\ttest_expect_success \"fetch $desc (enabled)\" '\n \t\t(\n \t\t\tcd tmp.git &&\n \t\t\tGIT_ALLOW_PROTOCOL=$proto &&\n@@ -27,7 +27,7 @@ test_proto () {\n \t\t)\n \t'\n \n-\ttest_expect_success \"push $1 (enabled)\" '\n+\ttest_expect_success \"push $desc (enabled)\" '\n \t\t(\n \t\t\tcd tmp.git &&\n \t\t\tGIT_ALLOW_PROTOCOL=$proto &&\n@@ -36,7 +36,7 @@ test_proto () {\n \t\t)\n \t'\n \n-\ttest_expect_success \"push $1 (disabled)\" '\n+\ttest_expect_success \"push $desc (disabled)\" '\n \t\t(\n \t\t\tcd tmp.git &&\n \t\t\tGIT_ALLOW_PROTOCOL=none &&\n@@ -45,7 +45,7 @@ test_proto () {\n \t\t)\n \t'\n \n-\ttest_expect_success \"fetch $1 (disabled)\" '\n+\ttest_expect_success \"fetch $desc (disabled)\" '\n \t\t(\n \t\t\tcd tmp.git &&\n \t\t\tGIT_ALLOW_PROTOCOL=none &&\n@@ -54,7 +54,7 @@ test_proto () {\n \t\t)\n \t'\n \n-\ttest_expect_success \"clone $1 (disabled)\" '\n+\ttest_expect_success \"clone $desc (disabled)\" '\n \t\trm -rf tmp.git &&\n \t\t(\n \t\t\tGIT_ALLOW_PROTOCOL=none &&\n-- \n2.8.0.rc3.226.g39d4020\n\n"},{"id":"305534","messageId":"CA+P7+xohFeWb1hsOg1_T1kWenc=AKrt1TNcx=TrVYQ+w3+c63Q@mail.gmail.com","threadId":"44417","inReplyTo":"20161107204838.xm463zdimiw7fx77@sigill.intra.peff.net","subject":"Re: [PATCH v4 1/2] lib-proto-disable: variable name fix","fromName":"Jacob Keller","fromEmail":"jacob.keller@gmail.com","sentAt":"2016-11-08T03:32:44Z","receivedAt":"2016-11-08T03:33:10Z","isPatch":true,"sender":{"key":"jacob.keller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/874719?v=4"},"body":"On Mon, Nov 7, 2016 at 12:48 PM, Jeff King wrote:\n> It's possible that I'm overly picky about my commit messages, but that\n> does not stop me from trying to train an army of picky-commit-message\n> clones. :)\n>\n> -Peff\n\nYou're not the only one ;)\n\nRegards,\nJake\n"},{"id":"305589","messageId":"20161108220426.bqvmmjr54w7btgih@sigill.intra.peff.net","threadId":"44417","inReplyTo":"1478555462-132573-2-git-send-email-bmwill@google.com","subject":"Re: [PATCH v5 2/2] transport: add protocol policy config option","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-11-08T22:04:26Z","receivedAt":"2016-11-08T22:04:32Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Nov 07, 2016 at 01:51:02PM -0800, Brandon Williams wrote:\n\n> Previously the `GIT_ALLOW_PROTOCOL` environment variable was used to\n> specify a whitelist of protocols to be used in clone/fetch/push\n> commands. This patch introduces new configuration options for more\n> fine-grained control for allowing/disallowing protocols. This also has\n> the added benefit of allowing easier construction of a protocol\n> whitelist on systems where setting an environment variable is\n> non-trivial.\n\nThis v5 looks good to me (both patches 1 and 2).\n\n-Peff\n"},{"id":"305590","messageId":"20161108220542.GA86649@google.com","threadId":"44417","inReplyTo":"20161108220426.bqvmmjr54w7btgih@sigill.intra.peff.net","subject":"Re: [PATCH v5 2/2] transport: add protocol policy config option","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-11-08T22:05:42Z","receivedAt":"2016-11-08T22:05:50Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 11/08, Jeff King wrote:\n> On Mon, Nov 07, 2016 at 01:51:02PM -0800, Brandon Williams wrote:\n> \n> > Previously the `GIT_ALLOW_PROTOCOL` environment variable was used to\n> > specify a whitelist of protocols to be used in clone/fetch/push\n> > commands. This patch introduces new configuration options for more\n> > fine-grained control for allowing/disallowing protocols. This also has\n> > the added benefit of allowing easier construction of a protocol\n> > whitelist on systems where setting an environment variable is\n> > non-trivial.\n> \n> This v5 looks good to me (both patches 1 and 2).\n\nThanks again for the help with this series!\n\n-- \nBrandon Williams\n"},{"id":"305613","messageId":"xmqq4m3hb3i8.fsf@gitster.mtv.corp.google.com","threadId":"44417","inReplyTo":"CA+P7+xohFeWb1hsOg1_T1kWenc=AKrt1TNcx=TrVYQ+w3+c63Q@mail.gmail.com","subject":"Re: [PATCH v4 1/2] lib-proto-disable: variable name fix","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-11-08T20:52:54Z","receivedAt":"2016-11-09T05:56:22Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jacob Keller writes:\n\n> On Mon, Nov 7, 2016 at 12:48 PM, Jeff King wrote:\n>> It's possible that I'm overly picky about my commit messages, but that\n>> does not stop me from trying to train an army of picky-commit-message\n>> clones. :)\n>>\n>> -Peff\n>\n> You're not the only one ;)\n\nSomebody seems to have trained y'all very well ;-)\n\n"},{"id":"306717","messageId":"1480621447-52399-1-git-send-email-bmwill@google.com","threadId":"44417","inReplyTo":"1478555462-132573-1-git-send-email-bmwill@google.com","subject":"[PATCH v6 0/4] transport protocol policy configuration","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-12-01T19:44:03Z","receivedAt":"2016-12-01T19:45:53Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"v6 introduces 2 additional patches which address problems with protocols that\nlibcurl is allowed to use for redirection.\n\nBrandon Williams (4):\n lib-proto-disable: variable name fix\n transport: add protocol policy config option\n http: always warn if libcurl version is too old\n transport: check if protocol can be used on a redirect\n\n Documentation/config.txt | 46 +++++++++++++\n Documentation/git.txt | 38 ++++-------\n git-submodule.sh | 12 ++--\n http.c | 13 ++--\n t/lib-proto-disable.sh | 142 ++++++++++++++++++++++++++++++++++++---\n t/t5509-fetch-push-namespaces.sh | 1 +\n t/t5802-connect-helper.sh | 1 +\n transport.c | 82 +++++++++++++++++++---\n transport.h | 13 ++--\n 9 files changed, 281 insertions(+), 67 deletions(-)\n\n-- \n2.8.0.rc3.226.g39d4020\n\n"},{"id":"306718","messageId":"1480621447-52399-5-git-send-email-bmwill@google.com","threadId":"44417","inReplyTo":"1480621447-52399-1-git-send-email-bmwill@google.com","subject":"[PATCH v6 4/4] transport: check if protocol can be used on a redirect","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-12-01T19:44:07Z","receivedAt":"2016-12-01T19:46:01Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"Add a the 'redirect' parameter to 'is_transport_allowed' which allows\ncallers to query if a transport protocol can be used on a redirect.\n\nSigned-off-by: Brandon Williams \n---\n http.c | 8 ++++----\n transport.c | 6 +++---\n transport.h | 7 ++++---\n 3 files changed, 11 insertions(+), 10 deletions(-)\n\ndiff --git a/http.c b/http.c\nindex fee128b..d31ded8 100644\n--- a/http.c\n+++ b/http.c\n@@ -725,13 +725,13 @@ static CURL *get_curl_handle(void)\n \tcurl_easy_setopt(result, CURLOPT_POST301, 1);\n #endif\n #if LIBCURL_VERSION_NUM >= 0x071304\n-\tif (is_transport_allowed(\"http\"))\n+\tif (is_transport_allowed(\"http\", 1))\n \t\tallowed_protocols |= CURLPROTO_HTTP;\n-\tif (is_transport_allowed(\"https\"))\n+\tif (is_transport_allowed(\"https\", 1))\n \t\tallowed_protocols |= CURLPROTO_HTTPS;\n-\tif (is_transport_allowed(\"ftp\"))\n+\tif (is_transport_allowed(\"ftp\", 1))\n \t\tallowed_protocols |= CURLPROTO_FTP;\n-\tif (is_transport_allowed(\"ftps\"))\n+\tif (is_transport_allowed(\"ftps\", 1))\n \t\tallowed_protocols |= CURLPROTO_FTPS;\n \tcurl_easy_setopt(result, CURLOPT_REDIR_PROTOCOLS, allowed_protocols);\n #else\ndiff --git a/transport.c b/transport.c\nindex 186de9a..9fee241 100644\n--- a/transport.c\n+++ b/transport.c\n@@ -723,7 +723,7 @@ static enum protocol_allow_config get_protocol_config(const char *type)\n \treturn PROTOCOL_ALLOW_USER_ONLY;\n }\n \n-int is_transport_allowed(const char *type)\n+int is_transport_allowed(const char *type, int redirect)\n {\n \tconst struct string_list *whitelist = protocol_whitelist();\n \tif (whitelist)\n@@ -735,7 +735,7 @@ int is_transport_allowed(const char *type)\n \tcase PROTOCOL_ALLOW_NEVER:\n \t\treturn 0;\n \tcase PROTOCOL_ALLOW_USER_ONLY:\n-\t\treturn git_env_bool(\"GIT_PROTOCOL_FROM_USER\", 1);\n+\t\treturn git_env_bool(\"GIT_PROTOCOL_FROM_USER\", !redirect);\n \t}\n \n \tdie(\"BUG: invalid protocol_allow_config type\");\n@@ -743,7 +743,7 @@ int is_transport_allowed(const char *type)\n \n void transport_check_allowed(const char *type)\n {\n-\tif (!is_transport_allowed(type))\n+\tif (!is_transport_allowed(type, 0))\n \t\tdie(\"transport '%s' not allowed\", type);\n }\n \ndiff --git a/transport.h b/transport.h\nindex f4998bc..72971ad 100644\n--- a/transport.h\n+++ b/transport.h\n@@ -153,10 +153,11 @@ extern int transport_summary_width(const struct ref *refs);\n struct transport *transport_get(struct remote *, const char *);\n \n /*\n- * Check whether a transport is allowed by the environment. Type should\n- * generally be the URL scheme, as described in Documentation/git.txt\n+ * Check whether a transport is allowed by the environment. Setting 'redirect'\n+ * can be used to querry if the transport can be used in a redirect. Type\n+ * should generally be the URL scheme, as described in Documentation/git.txt\n */\n-int is_transport_allowed(const char *type);\n+int is_transport_allowed(const char *type, int redirect);\n \n /*\n * Check whether a transport is allowed by the environment,\n-- \n2.8.0.rc3.226.g39d4020\n\n"},{"id":"306719","messageId":"1480621447-52399-3-git-send-email-bmwill@google.com","threadId":"44417","inReplyTo":"1480621447-52399-1-git-send-email-bmwill@google.com","subject":"[PATCH v6 2/4] transport: add protocol policy config option","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-12-01T19:44:05Z","receivedAt":"2016-12-01T19:46:04Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"Previously the `GIT_ALLOW_PROTOCOL` environment variable was used to\nspecify a whitelist of protocols to be used in clone/fetch/push\ncommands. This patch introduces new configuration options for more\nfine-grained control for allowing/disallowing protocols. This also has\nthe added benefit of allowing easier construction of a protocol\nwhitelist on systems where setting an environment variable is\nnon-trivial.\n\nNow users can specify a policy to be used for each type of protocol via\nthe 'protocol..allow' config option. A default policy for all\nunconfigured protocols can be set with the 'protocol.allow' config\noption. If no user configured default is made git will allow known-safe\nprotocols (http, https, git, ssh, file), disallow known-dangerous\nprotocols (ext), and have a default policy of `user` for all other\nprotocols.\n\nThe supported policies are `always`, `never`, and `user`. The `user`\npolicy can be used to configure a protocol to be usable when explicitly\nused by a user, while disallowing it for commands which run\nclone/fetch/push commands without direct user intervention (e.g.\nrecursive initialization of submodules). Commands which can potentially\nclone/fetch/push from untrusted repositories without user intervention\ncan export `GIT_PROTOCOL_FROM_USER` with a value of '0' to prevent\nprotocols configured to the `user` policy from being used.\n\nFix remote-ext tests to use the new config to allow the ext\nprotocol to be tested.\n\nBased on a patch by Jeff King \n\nSigned-off-by: Brandon Williams \n---\n Documentation/config.txt | 46 ++++++++++++++\n Documentation/git.txt | 38 +++++-------\n git-submodule.sh | 12 ++--\n t/lib-proto-disable.sh | 130 +++++++++++++++++++++++++++++++++++++--\n t/t5509-fetch-push-namespaces.sh | 1 +\n t/t5802-connect-helper.sh | 1 +\n transport.c | 75 +++++++++++++++++++++-\n 7 files changed, 264 insertions(+), 39 deletions(-)\n\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex 27069ac..5fe50bc 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -2308,6 +2308,52 @@ pretty.::\n \tNote that an alias with the same name as a built-in format\n \twill be silently ignored.\n \n+protocol.allow::\n+\tIf set, provide a user defined default policy for all protocols which\n+\tdon't explicitly have a policy (`protocol..allow`). By default,\n+\tif unset, known-safe protocols (http, https, git, ssh, file) have a\n+\tdefault policy of `always`, known-dangerous protocols (ext) have a\n+\tdefault policy of `never`, and all other protocols have a default\n+\tpolicy of `user`. Supported policies:\n++\n+--\n+\n+* `always` - protocol is always able to be used.\n+\n+* `never` - protocol is never able to be used.\n+\n+* `user` - protocol is only able to be used when `GIT_PROTOCOL_FROM_USER` is\n+ either unset or has a value of 1. This policy should be used when you want a\n+ protocol to be directly usable by the user but don't want it used by commands which\n+ execute clone/fetch/push commands without user input, e.g. recursive\n+ submodule initialization.\n+\n+--\n+\n+protocol..allow::\n+\tSet a policy to be used by protocol `` with clone/fetch/push\n+\tcommands. See `protocol.allow` above for the available policies.\n++\n+The protocol names currently used by git are:\n++\n+--\n+ - `file`: any local file-based path (including `file://` URLs,\n+ or local paths)\n+\n+ - `git`: the anonymous git protocol over a direct TCP\n+ connection (or proxy, if configured)\n+\n+ - `ssh`: git over ssh (including `host:path` syntax,\n+ `ssh://`, etc).\n+\n+ - `http`: git over http, both \"smart http\" and \"dumb http\".\n+ Note that this does _not_ include `https`; if you want to configure\n+ both, you must do so individually.\n+\n+ - any external helpers are named by their protocol (e.g., use\n+ `hg` to allow the `git-remote-hg` helper)\n+--\n+\n pull.ff::\n \tBy default, Git does not create an extra merge commit when merging\n \ta commit that is a descendant of the current commit. Instead, the\ndiff --git a/Documentation/git.txt b/Documentation/git.txt\nindex ab7215e..c52cec8 100644\n--- a/Documentation/git.txt\n+++ b/Documentation/git.txt\n@@ -1150,30 +1150,20 @@ of clones and fetches.\n \tcloning a repository to make a backup).\n \n `GIT_ALLOW_PROTOCOL`::\n-\tIf set, provide a colon-separated list of protocols which are\n-\tallowed to be used with fetch/push/clone. This is useful to\n-\trestrict recursive submodule initialization from an untrusted\n-\trepository. Any protocol not mentioned will be disallowed (i.e.,\n-\tthis is a whitelist, not a blacklist). If the variable is not\n-\tset at all, all protocols are enabled. The protocol names\n-\tcurrently used by git are:\n-\n-\t - `file`: any local file-based path (including `file://` URLs,\n-\t or local paths)\n-\n-\t - `git`: the anonymous git protocol over a direct TCP\n-\t connection (or proxy, if configured)\n-\n-\t - `ssh`: git over ssh (including `host:path` syntax,\n-\t `ssh://`, etc).\n-\n-\t - `http`: git over http, both \"smart http\" and \"dumb http\".\n-\t Note that this does _not_ include `https`; if you want both,\n-\t you should specify both as `http:https`.\n-\n-\t - any external helpers are named by their protocol (e.g., use\n-\t `hg` to allow the `git-remote-hg` helper)\n-\n+\tIf set to a colon-separated list of protocols, behave as if\n+\t`protocol.allow` is set to `never`, and each of the listed\n+\tprotocols has `protocol..allow` set to `always`\n+\t(overriding any existing configuration). In other words, any\n+\tprotocol not mentioned will be disallowed (i.e., this is a\n+\twhitelist, not a blacklist). See the description of\n+\t`protocol.allow` in linkgit:git-config[1] for more details.\n+\n+`GIT_PROTOCOL_FROM_USER`::\n+\tSet to 0 to prevent protocols used by fetch/push/clone which are\n+\tconfigured to the `user` state. This is useful to restrict recursive\n+\tsubmodule initialization from an untrusted repository or for programs\n+\twhich feed potentially-untrusted URLS to git commands. See\n+\tlinkgit:git-config[1] for more details.\n \n Discussion[[Discussion]]\n ------------------------\ndiff --git a/git-submodule.sh b/git-submodule.sh\nindex a024a13..0a477b4 100755\n--- a/git-submodule.sh\n+++ b/git-submodule.sh\n@@ -21,14 +21,10 @@ require_work_tree\n wt_prefix=$(git rev-parse --show-prefix)\n cd_to_toplevel\n \n-# Restrict ourselves to a vanilla subset of protocols; the URLs\n-# we get are under control of a remote repository, and we do not\n-# want them kicking off arbitrary git-remote-* programs.\n-#\n-# If the user has already specified a set of allowed protocols,\n-# we assume they know what they're doing and use that instead.\n-: ${GIT_ALLOW_PROTOCOL=file:git:http:https:ssh}\n-export GIT_ALLOW_PROTOCOL\n+# Tell the rest of git that any URLs we get don't come\n+# directly from the user, so it can apply policy as appropriate.\n+GIT_PROTOCOL_FROM_USER=0\n+export GIT_PROTOCOL_FROM_USER\n \n command=\n branch=\ndiff --git a/t/lib-proto-disable.sh b/t/lib-proto-disable.sh\nindex be88e9a..02f49cb 100644\n--- a/t/lib-proto-disable.sh\n+++ b/t/lib-proto-disable.sh\n@@ -1,10 +1,7 @@\n # Test routines for checking protocol disabling.\n \n-# test cloning a particular protocol\n-# $1 - description of the protocol\n-# $2 - machine-readable name of the protocol\n-# $3 - the URL to try cloning\n-test_proto () {\n+# Test clone/fetch/push with GIT_ALLOW_PROTOCOL whitelist\n+test_whitelist () {\n \tdesc=$1\n \tproto=$2\n \turl=$3\n@@ -62,6 +59,129 @@ test_proto () {\n \t\t\ttest_must_fail git clone --bare \"$url\" tmp.git\n \t\t)\n \t'\n+\n+\ttest_expect_success \"clone $desc (env var has precedence)\" '\n+\t\trm -rf tmp.git &&\n+\t\t(\n+\t\t\tGIT_ALLOW_PROTOCOL=none &&\n+\t\t\texport GIT_ALLOW_PROTOCOL &&\n+\t\t\ttest_must_fail git -c protocol.allow=always clone --bare \"$url\" tmp.git &&\n+\t\t\ttest_must_fail git -c protocol.$proto.allow=always clone --bare \"$url\" tmp.git\n+\t\t)\n+\t'\n+}\n+\n+test_config () {\n+\tdesc=$1\n+\tproto=$2\n+\turl=$3\n+\n+\t# Test clone/fetch/push with protocol..allow config\n+\ttest_expect_success \"clone $desc (enabled with config)\" '\n+\t\trm -rf tmp.git &&\n+\t\tgit -c protocol.$proto.allow=always clone --bare \"$url\" tmp.git\n+\t'\n+\n+\ttest_expect_success \"fetch $desc (enabled)\" '\n+\t\tgit -C tmp.git -c protocol.$proto.allow=always fetch\n+\t'\n+\n+\ttest_expect_success \"push $desc (enabled)\" '\n+\t\tgit -C tmp.git -c protocol.$proto.allow=always push origin HEAD:pushed\n+\t'\n+\n+\ttest_expect_success \"push $desc (disabled)\" '\n+\t\ttest_must_fail git -C tmp.git -c protocol.$proto.allow=never push origin HEAD:pushed\n+\t'\n+\n+\ttest_expect_success \"fetch $desc (disabled)\" '\n+\t\ttest_must_fail git -C tmp.git -c protocol.$proto.allow=never fetch\n+\t'\n+\n+\ttest_expect_success \"clone $desc (disabled)\" '\n+\t\trm -rf tmp.git &&\n+\t\ttest_must_fail git -c protocol.$proto.allow=never clone --bare \"$url\" tmp.git\n+\t'\n+\n+\t# Test clone/fetch/push with protocol.user.allow and its env var\n+\ttest_expect_success \"clone $desc (enabled)\" '\n+\t\trm -rf tmp.git &&\n+\t\tgit -c protocol.$proto.allow=user clone --bare \"$url\" tmp.git\n+\t'\n+\n+\ttest_expect_success \"fetch $desc (enabled)\" '\n+\t\tgit -C tmp.git -c protocol.$proto.allow=user fetch\n+\t'\n+\n+\ttest_expect_success \"push $desc (enabled)\" '\n+\t\tgit -C tmp.git -c protocol.$proto.allow=user push origin HEAD:pushed\n+\t'\n+\n+\ttest_expect_success \"push $desc (disabled)\" '\n+\t\t(\n+\t\t\tcd tmp.git &&\n+\t\t\tGIT_PROTOCOL_FROM_USER=0 &&\n+\t\t\texport GIT_PROTOCOL_FROM_USER &&\n+\t\t\ttest_must_fail git -c protocol.$proto.allow=user push origin HEAD:pushed\n+\t\t)\n+\t'\n+\n+\ttest_expect_success \"fetch $desc (disabled)\" '\n+\t\t(\n+\t\t\tcd tmp.git &&\n+\t\t\tGIT_PROTOCOL_FROM_USER=0 &&\n+\t\t\texport GIT_PROTOCOL_FROM_USER &&\n+\t\t\ttest_must_fail git -c protocol.$proto.allow=user fetch\n+\t\t)\n+\t'\n+\n+\ttest_expect_success \"clone $desc (disabled)\" '\n+\t\trm -rf tmp.git &&\n+\t\t(\n+\t\t\tGIT_PROTOCOL_FROM_USER=0 &&\n+\t\t\texport GIT_PROTOCOL_FROM_USER &&\n+\t\t\ttest_must_fail git -c protocol.$proto.allow=user clone --bare \"$url\" tmp.git\n+\t\t)\n+\t'\n+\n+\t# Test clone/fetch/push with protocol.allow user defined default\n+\ttest_expect_success \"clone $desc (enabled)\" '\n+\t\trm -rf tmp.git &&\n+\t\tgit config --global protocol.allow always &&\n+\t\tgit clone --bare \"$url\" tmp.git\n+\t'\n+\n+\ttest_expect_success \"fetch $desc (enabled)\" '\n+\t\tgit -C tmp.git fetch\n+\t'\n+\n+\ttest_expect_success \"push $desc (enabled)\" '\n+\t\tgit -C tmp.git push origin HEAD:pushed\n+\t'\n+\n+\ttest_expect_success \"push $desc (disabled)\" '\n+\t\tgit config --global protocol.allow never &&\n+\t\ttest_must_fail git -C tmp.git push origin HEAD:pushed\n+\t'\n+\n+\ttest_expect_success \"fetch $desc (disabled)\" '\n+\t\ttest_must_fail git -C tmp.git fetch\n+\t'\n+\n+\ttest_expect_success \"clone $desc (disabled)\" '\n+\t\trm -rf tmp.git &&\n+\t\ttest_must_fail git clone --bare \"$url\" tmp.git\n+\t'\n+}\n+\n+# test cloning a particular protocol\n+# $1 - description of the protocol\n+# $2 - machine-readable name of the protocol\n+# $3 - the URL to try cloning\n+test_proto () {\n+\ttest_whitelist \"$@\"\n+\n+\ttest_config \"$@\"\n }\n \n # set up an ssh wrapper that will access $host/$repo in the\ndiff --git a/t/t5509-fetch-push-namespaces.sh b/t/t5509-fetch-push-namespaces.sh\nindex bc44ac3..75c570a 100755\n--- a/t/t5509-fetch-push-namespaces.sh\n+++ b/t/t5509-fetch-push-namespaces.sh\n@@ -4,6 +4,7 @@ test_description='fetch/push involving ref namespaces'\n . ./test-lib.sh\n \n test_expect_success setup '\n+\tgit config --global protocol.ext.allow user &&\n \ttest_tick &&\n \tgit init original &&\n \t(\ndiff --git a/t/t5802-connect-helper.sh b/t/t5802-connect-helper.sh\nindex b7a7f9d..c6c2661 100755\n--- a/t/t5802-connect-helper.sh\n+++ b/t/t5802-connect-helper.sh\n@@ -4,6 +4,7 @@ test_description='ext::cmd remote \"connect\" helper'\n . ./test-lib.sh\n \n test_expect_success setup '\n+\tgit config --global protocol.ext.allow user &&\n \ttest_tick &&\n \tgit commit --allow-empty -m initial &&\n \ttest_tick &&\ndiff --git a/transport.c b/transport.c\nindex d57e8de..2c0ec76 100644\n--- a/transport.c\n+++ b/transport.c\n@@ -664,10 +664,81 @@ static const struct string_list *protocol_whitelist(void)\n \treturn enabled ? &allowed : NULL;\n }\n \n+enum protocol_allow_config {\n+\tPROTOCOL_ALLOW_NEVER = 0,\n+\tPROTOCOL_ALLOW_USER_ONLY,\n+\tPROTOCOL_ALLOW_ALWAYS\n+};\n+\n+static enum protocol_allow_config parse_protocol_config(const char *key,\n+\t\t\t\t\t\t\tconst char *value)\n+{\n+\tif (!strcasecmp(value, \"always\"))\n+\t\treturn PROTOCOL_ALLOW_ALWAYS;\n+\telse if (!strcasecmp(value, \"never\"))\n+\t\treturn PROTOCOL_ALLOW_NEVER;\n+\telse if (!strcasecmp(value, \"user\"))\n+\t\treturn PROTOCOL_ALLOW_USER_ONLY;\n+\n+\tdie(\"unknown value for config '%s': %s\", key, value);\n+}\n+\n+static enum protocol_allow_config get_protocol_config(const char *type)\n+{\n+\tchar *key = xstrfmt(\"protocol.%s.allow\", type);\n+\tchar *value;\n+\n+\t/* first check the per-protocol config */\n+\tif (!git_config_get_string(key, &value)) {\n+\t\tenum protocol_allow_config ret =\n+\t\t\tparse_protocol_config(key, value);\n+\t\tfree(key);\n+\t\tfree(value);\n+\t\treturn ret;\n+\t}\n+\tfree(key);\n+\n+\t/* if defined, fallback to user-defined default for unknown protocols */\n+\tif (!git_config_get_string(\"protocol.allow\", &value)) {\n+\t\tenum protocol_allow_config ret =\n+\t\t\tparse_protocol_config(\"protocol.allow\", value);\n+\t\tfree(value);\n+\t\treturn ret;\n+\t}\n+\n+\t/* fallback to built-in defaults */\n+\t/* known safe */\n+\tif (!strcmp(type, \"http\") ||\n+\t !strcmp(type, \"https\") ||\n+\t !strcmp(type, \"git\") ||\n+\t !strcmp(type, \"ssh\") ||\n+\t !strcmp(type, \"file\"))\n+\t\treturn PROTOCOL_ALLOW_ALWAYS;\n+\n+\t/* known scary; err on the side of caution */\n+\tif (!strcmp(type, \"ext\"))\n+\t\treturn PROTOCOL_ALLOW_NEVER;\n+\n+\t/* unknown; by default let them be used only directly by the user */\n+\treturn PROTOCOL_ALLOW_USER_ONLY;\n+}\n+\n int is_transport_allowed(const char *type)\n {\n-\tconst struct string_list *allowed = protocol_whitelist();\n-\treturn !allowed || string_list_has_string(allowed, type);\n+\tconst struct string_list *whitelist = protocol_whitelist();\n+\tif (whitelist)\n+\t\treturn string_list_has_string(whitelist, type);\n+\n+\tswitch (get_protocol_config(type)) {\n+\tcase PROTOCOL_ALLOW_ALWAYS:\n+\t\treturn 1;\n+\tcase PROTOCOL_ALLOW_NEVER:\n+\t\treturn 0;\n+\tcase PROTOCOL_ALLOW_USER_ONLY:\n+\t\treturn git_env_bool(\"GIT_PROTOCOL_FROM_USER\", 1);\n+\t}\n+\n+\tdie(\"BUG: invalid protocol_allow_config type\");\n }\n \n void transport_check_allowed(const char *type)\n-- \n2.8.0.rc3.226.g39d4020\n\n"},{"id":"306720","messageId":"1480621447-52399-2-git-send-email-bmwill@google.com","threadId":"44417","inReplyTo":"1480621447-52399-1-git-send-email-bmwill@google.com","subject":"[PATCH v6 1/4] lib-proto-disable: variable name fix","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-12-01T19:44:04Z","receivedAt":"2016-12-01T19:46:05Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"The test_proto function assigns the positional parameters to named\nvariables, but then still refers to \"$desc\" as \"$1\". Using $desc is\nmore readable and less error-prone.\n\nSigned-off-by: Brandon Williams \n---\n t/lib-proto-disable.sh | 12 ++++++------\n 1 file changed, 6 insertions(+), 6 deletions(-)\n\ndiff --git a/t/lib-proto-disable.sh b/t/lib-proto-disable.sh\nindex b0917d9..be88e9a 100644\n--- a/t/lib-proto-disable.sh\n+++ b/t/lib-proto-disable.sh\n@@ -9,7 +9,7 @@ test_proto () {\n \tproto=$2\n \turl=$3\n \n-\ttest_expect_success \"clone $1 (enabled)\" '\n+\ttest_expect_success \"clone $desc (enabled)\" '\n \t\trm -rf tmp.git &&\n \t\t(\n \t\t\tGIT_ALLOW_PROTOCOL=$proto &&\n@@ -18,7 +18,7 @@ test_proto () {\n \t\t)\n \t'\n \n-\ttest_expect_success \"fetch $1 (enabled)\" '\n+\ttest_expect_success \"fetch $desc (enabled)\" '\n \t\t(\n \t\t\tcd tmp.git &&\n \t\t\tGIT_ALLOW_PROTOCOL=$proto &&\n@@ -27,7 +27,7 @@ test_proto () {\n \t\t)\n \t'\n \n-\ttest_expect_success \"push $1 (enabled)\" '\n+\ttest_expect_success \"push $desc (enabled)\" '\n \t\t(\n \t\t\tcd tmp.git &&\n \t\t\tGIT_ALLOW_PROTOCOL=$proto &&\n@@ -36,7 +36,7 @@ test_proto () {\n \t\t)\n \t'\n \n-\ttest_expect_success \"push $1 (disabled)\" '\n+\ttest_expect_success \"push $desc (disabled)\" '\n \t\t(\n \t\t\tcd tmp.git &&\n \t\t\tGIT_ALLOW_PROTOCOL=none &&\n@@ -45,7 +45,7 @@ test_proto () {\n \t\t)\n \t'\n \n-\ttest_expect_success \"fetch $1 (disabled)\" '\n+\ttest_expect_success \"fetch $desc (disabled)\" '\n \t\t(\n \t\t\tcd tmp.git &&\n \t\t\tGIT_ALLOW_PROTOCOL=none &&\n@@ -54,7 +54,7 @@ test_proto () {\n \t\t)\n \t'\n \n-\ttest_expect_success \"clone $1 (disabled)\" '\n+\ttest_expect_success \"clone $desc (disabled)\" '\n \t\trm -rf tmp.git &&\n \t\t(\n \t\t\tGIT_ALLOW_PROTOCOL=none &&\n-- \n2.8.0.rc3.226.g39d4020\n\n"},{"id":"306721","messageId":"1480621447-52399-4-git-send-email-bmwill@google.com","threadId":"44417","inReplyTo":"1480621447-52399-1-git-send-email-bmwill@google.com","subject":"[PATCH v6 3/4] http: always warn if libcurl version is too old","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-12-01T19:44:06Z","receivedAt":"2016-12-01T19:46:06Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"Now that there are default \"known-good\" and \"known-bad\" protocols which\nare allowed/disallowed by 'is_transport_allowed' we should always warn\nthe user that older versions of libcurl can't respect the allowed\nprotocols for redirects.\n\nSigned-off-by: Brandon Williams \n---\n http.c | 5 ++---\n transport.c | 5 -----\n transport.h | 6 ------\n 3 files changed, 2 insertions(+), 14 deletions(-)\n\ndiff --git a/http.c b/http.c\nindex 4c4a812..fee128b 100644\n--- a/http.c\n+++ b/http.c\n@@ -735,9 +735,8 @@ static CURL *get_curl_handle(void)\n \t\tallowed_protocols |= CURLPROTO_FTPS;\n \tcurl_easy_setopt(result, CURLOPT_REDIR_PROTOCOLS, allowed_protocols);\n #else\n-\tif (transport_restrict_protocols())\n-\t\twarning(\"protocol restrictions not applied to curl redirects because\\n\"\n-\t\t\t\"your curl version is too old (>= 7.19.4)\");\n+\twarning(\"protocol restrictions not applied to curl redirects because\\n\"\n+\t\t\"your curl version is too old (>= 7.19.4)\");\n #endif\n \tif (getenv(\"GIT_CURL_VERBOSE\"))\n \t\tcurl_easy_setopt(result, CURLOPT_VERBOSE, 1L);\ndiff --git a/transport.c b/transport.c\nindex 2c0ec76..186de9a 100644\n--- a/transport.c\n+++ b/transport.c\n@@ -747,11 +747,6 @@ void transport_check_allowed(const char *type)\n \t\tdie(\"transport '%s' not allowed\", type);\n }\n \n-int transport_restrict_protocols(void)\n-{\n-\treturn !!protocol_whitelist();\n-}\n-\n struct transport *transport_get(struct remote *remote, const char *url)\n {\n \tconst char *helper;\ndiff --git a/transport.h b/transport.h\nindex b8e4ee8..f4998bc 100644\n--- a/transport.h\n+++ b/transport.h\n@@ -164,12 +164,6 @@ int is_transport_allowed(const char *type);\n */\n void transport_check_allowed(const char *type);\n \n-/*\n- * Returns true if the user has attempted to turn on protocol\n- * restrictions at all.\n- */\n-int transport_restrict_protocols(void);\n-\n /* Transport options which apply to git:// and scp-style URLs */\n \n /* The program to use on the remote side to send a pack */\n-- \n2.8.0.rc3.226.g39d4020\n\n"},{"id":"306723","messageId":"1480621700-53222-1-git-send-email-bmwill@google.com","threadId":"44417","inReplyTo":"1480621447-52399-5-git-send-email-bmwill@google.com","subject":"[PATCH v6 4/4] transport: check if protocol can be used on a redirect","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-12-01T19:48:20Z","receivedAt":"2016-12-01T19:48:40Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"Add a the 'redirect' parameter to 'is_transport_allowed' which allows\ncallers to query if a transport protocol can be used on a redirect.\n\nSigned-off-by: Brandon Williams \n---\n http.c | 8 ++++----\n transport.c | 6 +++---\n transport.h | 7 ++++---\n 3 files changed, 11 insertions(+), 10 deletions(-)\n\ndiff --git a/http.c b/http.c\nindex fee128b..d31ded8 100644\n--- a/http.c\n+++ b/http.c\n@@ -725,13 +725,13 @@ static CURL *get_curl_handle(void)\n \tcurl_easy_setopt(result, CURLOPT_POST301, 1);\n #endif\n #if LIBCURL_VERSION_NUM >= 0x071304\n-\tif (is_transport_allowed(\"http\"))\n+\tif (is_transport_allowed(\"http\", 1))\n \t\tallowed_protocols |= CURLPROTO_HTTP;\n-\tif (is_transport_allowed(\"https\"))\n+\tif (is_transport_allowed(\"https\", 1))\n \t\tallowed_protocols |= CURLPROTO_HTTPS;\n-\tif (is_transport_allowed(\"ftp\"))\n+\tif (is_transport_allowed(\"ftp\", 1))\n \t\tallowed_protocols |= CURLPROTO_FTP;\n-\tif (is_transport_allowed(\"ftps\"))\n+\tif (is_transport_allowed(\"ftps\", 1))\n \t\tallowed_protocols |= CURLPROTO_FTPS;\n \tcurl_easy_setopt(result, CURLOPT_REDIR_PROTOCOLS, allowed_protocols);\n #else\ndiff --git a/transport.c b/transport.c\nindex 186de9a..7c4a757 100644\n--- a/transport.c\n+++ b/transport.c\n@@ -723,7 +723,7 @@ static enum protocol_allow_config get_protocol_config(const char *type)\n \treturn PROTOCOL_ALLOW_USER_ONLY;\n }\n \n-int is_transport_allowed(const char *type)\n+int is_transport_allowed(const char *type, int redirect)\n {\n \tconst struct string_list *whitelist = protocol_whitelist();\n \tif (whitelist)\n@@ -735,7 +735,7 @@ int is_transport_allowed(const char *type)\n \tcase PROTOCOL_ALLOW_NEVER:\n \t\treturn 0;\n \tcase PROTOCOL_ALLOW_USER_ONLY:\n-\t\treturn git_env_bool(\"GIT_PROTOCOL_FROM_USER\", 1);\n+\t\treturn redirect ? 0 : git_env_bool(\"GIT_PROTOCOL_FROM_USER\", 1);\n \t}\n \n \tdie(\"BUG: invalid protocol_allow_config type\");\n@@ -743,7 +743,7 @@ int is_transport_allowed(const char *type)\n \n void transport_check_allowed(const char *type)\n {\n-\tif (!is_transport_allowed(type))\n+\tif (!is_transport_allowed(type, 0))\n \t\tdie(\"transport '%s' not allowed\", type);\n }\n \ndiff --git a/transport.h b/transport.h\nindex f4998bc..4bcf5d3 100644\n--- a/transport.h\n+++ b/transport.h\n@@ -153,10 +153,11 @@ extern int transport_summary_width(const struct ref *refs);\n struct transport *transport_get(struct remote *, const char *);\n \n /*\n- * Check whether a transport is allowed by the environment. Type should\n- * generally be the URL scheme, as described in Documentation/git.txt\n+ * Check whether a transport is allowed by the environment. Setting 'redirect'\n+ * can be used to query if the transport can be used in a redirect. Type\n+ * should generally be the URL scheme, as described in Documentation/git.txt\n */\n-int is_transport_allowed(const char *type);\n+int is_transport_allowed(const char *type, int redirect);\n \n /*\n * Check whether a transport is allowed by the environment,\n-- \n2.8.0.rc3.226.g39d4020\n\n"},{"id":"306724","messageId":"20161201194928.GD54082@google.com","threadId":"44417","inReplyTo":"1480621700-53222-1-git-send-email-bmwill@google.com","subject":"Re: [PATCH v6 4/4] transport: check if protocol can be used on a redirect","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-12-01T19:49:28Z","receivedAt":"2016-12-01T19:49:35Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 12/01, Brandon Williams wrote:\n> Add a the 'redirect' parameter to 'is_transport_allowed' which allows\n> callers to query if a transport protocol can be used on a redirect.\n> \n> Signed-off-by: Brandon Williams \n> ---\n> http.c | 8 ++++----\n> transport.c | 6 +++---\n> transport.h | 7 ++++---\n> 3 files changed, 11 insertions(+), 10 deletions(-)\n> \n\nAccidentally sent out an old version of just this patch. Here is the\nupdated one.\n\n-- \nBrandon Williams\n"},{"id":"306725","messageId":"20161201195031.fd4uwmvkyhk4so7i@sigill.intra.peff.net","threadId":"44417","inReplyTo":"1480621447-52399-5-git-send-email-bmwill@google.com","subject":"Re: [PATCH v6 4/4] transport: check if protocol can be used on a redirect","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-12-01T19:50:31Z","receivedAt":"2016-12-01T19:50:38Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Dec 01, 2016 at 11:44:07AM -0800, Brandon Williams wrote:\n\n> Add a the 'redirect' parameter to 'is_transport_allowed' which allows\n> callers to query if a transport protocol can be used on a redirect.\n\ns/a the/a/\n\n> -int is_transport_allowed(const char *type)\n> +int is_transport_allowed(const char *type, int redirect)\n> {\n> \tconst struct string_list *whitelist = protocol_whitelist();\n> \tif (whitelist)\n> @@ -735,7 +735,7 @@ int is_transport_allowed(const char *type)\n> \tcase PROTOCOL_ALLOW_NEVER:\n> \t\treturn 0;\n> \tcase PROTOCOL_ALLOW_USER_ONLY:\n> -\t\treturn git_env_bool(\"GIT_PROTOCOL_FROM_USER\", 1);\n> +\t\treturn git_env_bool(\"GIT_PROTOCOL_FROM_USER\", !redirect);\n> \t}\n\nThis has the older logic still.\n\nI'm not sure if we should call this \"redirect\" here. That's how it's\nused by the curl code, but I think from the perspective of the transport\nwhitelist, it is really \"are you overriding the from_user environment\".\n\nCalling it \"from_user\" may be confusing though, as the default value\nwould become \"1\", even though it means only \"as far as I know this is\nfrom the user, but maybe the environment says otherwise\". So bizarrely,\nI think calling it \"not_from_user\" is the clearest value.\n\n-Peff\n"},{"id":"306727","messageId":"xmqqbmwva0im.fsf@gitster.mtv.corp.google.com","threadId":"44417","inReplyTo":"20161201195031.fd4uwmvkyhk4so7i@sigill.intra.peff.net","subject":"Re: [PATCH v6 4/4] transport: check if protocol can be used on a redirect","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-12-01T19:54:09Z","receivedAt":"2016-12-01T19:54:18Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King writes:\n\n> On Thu, Dec 01, 2016 at 11:44:07AM -0800, Brandon Williams wrote:\n>\n>> Add a the 'redirect' parameter to 'is_transport_allowed' which allows\n>> callers to query if a transport protocol can be used on a redirect.\n>\n> s/a the/a/\n>\n>> -int is_transport_allowed(const char *type)\n>> +int is_transport_allowed(const char *type, int redirect)\n>> {\n>> \tconst struct string_list *whitelist = protocol_whitelist();\n>> \tif (whitelist)\n>> @@ -735,7 +735,7 @@ int is_transport_allowed(const char *type)\n>> \tcase PROTOCOL_ALLOW_NEVER:\n>> \t\treturn 0;\n>> \tcase PROTOCOL_ALLOW_USER_ONLY:\n>> -\t\treturn git_env_bool(\"GIT_PROTOCOL_FROM_USER\", 1);\n>> +\t\treturn git_env_bool(\"GIT_PROTOCOL_FROM_USER\", !redirect);\n>> \t}\n>\n> This has the older logic still.\n>\n> I'm not sure if we should call this \"redirect\" here. That's how it's\n> used by the curl code, but I think from the perspective of the transport\n> whitelist, it is really \"are you overriding the from_user environment\".\n>\n> Calling it \"from_user\" may be confusing though, as the default value\n> would become \"1\", even though it means only \"as far as I know this is\n> from the user, but maybe the environment says otherwise\". So bizarrely,\n> I think calling it \"not_from_user\" is the clearest value.\n\nBikeshedding: perhaps call it \"unsafe\" (in the sense that it is \"not\nknown to be safe\")?\n\n"},{"id":"306728","messageId":"20161201195902.td4zfolqpc3uwfgq@sigill.intra.peff.net","threadId":"44417","inReplyTo":"xmqqbmwva0im.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH v6 4/4] transport: check if protocol can be used on a redirect","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-12-01T19:59:02Z","receivedAt":"2016-12-01T19:59:09Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Dec 01, 2016 at 11:54:09AM -0800, Junio C Hamano wrote:\n\n> > I'm not sure if we should call this \"redirect\" here. That's how it's\n> > used by the curl code, but I think from the perspective of the transport\n> > whitelist, it is really \"are you overriding the from_user environment\".\n> >\n> > Calling it \"from_user\" may be confusing though, as the default value\n> > would become \"1\", even though it means only \"as far as I know this is\n> > from the user, but maybe the environment says otherwise\". So bizarrely,\n> > I think calling it \"not_from_user\" is the clearest value.\n> \n> Bikeshedding: perhaps call it \"unsafe\" (in the sense that it is \"not\n> known to be safe\")?\n\nThat is definitely what we are going for, but it is vague about how it\nis unsafe. :)\n\nI think I may have converted Brandon in the other thread to my way of\nthinking of it as a tristate[1]. That lets us call it \"from_user\", and\njust do:\n\n case PROTOCOL_ALLOW_FROM_USER:\n\tif (from_user < 0)\n\t\tfrom_user = git_env_bool(\"GIT_PROTOCOL_FROM_USER\", 1);\n\treturn from_user;\n\nwhich is pretty clear. Nobody would ever pass \"1\" as from_user to the\nfunction, but it does the sensible thing if they do.\n\n-Peff\n\n[1] The original I posted calling it \"redirect\" was totally bogus\n because the logic between the two names is inverted.\n"},{"id":"306729","messageId":"20161201200154.GF54082@google.com","threadId":"44417","inReplyTo":"20161201195902.td4zfolqpc3uwfgq@sigill.intra.peff.net","subject":"Re: [PATCH v6 4/4] transport: check if protocol can be used on a redirect","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-12-01T20:01:54Z","receivedAt":"2016-12-01T20:02:02Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 12/01, Jeff King wrote:\n> On Thu, Dec 01, 2016 at 11:54:09AM -0800, Junio C Hamano wrote:\n> \n> > > I'm not sure if we should call this \"redirect\" here. That's how it's\n> > > used by the curl code, but I think from the perspective of the transport\n> > > whitelist, it is really \"are you overriding the from_user environment\".\n> > >\n> > > Calling it \"from_user\" may be confusing though, as the default value\n> > > would become \"1\", even though it means only \"as far as I know this is\n> > > from the user, but maybe the environment says otherwise\". So bizarrely,\n> > > I think calling it \"not_from_user\" is the clearest value.\n> > \n> > Bikeshedding: perhaps call it \"unsafe\" (in the sense that it is \"not\n> > known to be safe\")?\n> \n> That is definitely what we are going for, but it is vague about how it\n> is unsafe. :)\n> \n> I think I may have converted Brandon in the other thread to my way of\n\nYep, I've been converted :D\n\nIf we agree on that then I can make the change and resend the patch.\n\n> thinking of it as a tristate[1]. That lets us call it \"from_user\", and\n> just do:\n> \n> case PROTOCOL_ALLOW_FROM_USER:\n> \tif (from_user < 0)\n> \t\tfrom_user = git_env_bool(\"GIT_PROTOCOL_FROM_USER\", 1);\n> \treturn from_user;\n> \n> which is pretty clear. Nobody would ever pass \"1\" as from_user to the\n> function, but it does the sensible thing if they do.\n> \n> -Peff\n> \n> [1] The original I posted calling it \"redirect\" was totally bogus\n> because the logic between the two names is inverted.\n\n-- \nBrandon Williams\n"},{"id":"306743","messageId":"1480623959-126129-1-git-send-email-bmwill@google.com","threadId":"44417","inReplyTo":"1480621447-52399-1-git-send-email-bmwill@google.com","subject":"[PATCH v7 0/4] transport protocol policy configuration","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-12-01T20:25:55Z","receivedAt":"2016-12-01T20:26:36Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"Changed the last patch in the series to use the parameter 'from_user' instead\nof 'redirect'. This allows us to use the same logic polarity and maintain use\nof the same vocabulary.\n\nBrandon Williams (4):\n lib-proto-disable: variable name fix\n transport: add protocol policy config option\n http: always warn if libcurl version is too old\n transport: add from_user parameter to is_transport_allowed\n\n Documentation/config.txt | 46 +++++++++++++\n Documentation/git.txt | 38 ++++-------\n git-submodule.sh | 12 ++--\n http.c | 13 ++--\n t/lib-proto-disable.sh | 142 ++++++++++++++++++++++++++++++++++++---\n t/t5509-fetch-push-namespaces.sh | 1 +\n t/t5802-connect-helper.sh | 1 +\n transport.c | 84 ++++++++++++++++++++---\n transport.h | 19 +++---\n 9 files changed, 289 insertions(+), 67 deletions(-)\n\n-- \n2.8.0.rc3.226.g39d4020\n\n"},{"id":"306744","messageId":"1480623959-126129-4-git-send-email-bmwill@google.com","threadId":"44417","inReplyTo":"1480623959-126129-1-git-send-email-bmwill@google.com","subject":"[PATCH v7 3/4] http: always warn if libcurl version is too old","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-12-01T20:25:58Z","receivedAt":"2016-12-01T20:26:39Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"Now that there are default \"known-good\" and \"known-bad\" protocols which\nare allowed/disallowed by 'is_transport_allowed' we should always warn\nthe user that older versions of libcurl can't respect the allowed\nprotocols for redirects.\n\nSigned-off-by: Brandon Williams \n---\n http.c | 5 ++---\n transport.c | 5 -----\n transport.h | 6 ------\n 3 files changed, 2 insertions(+), 14 deletions(-)\n\ndiff --git a/http.c b/http.c\nindex 4c4a812..fee128b 100644\n--- a/http.c\n+++ b/http.c\n@@ -735,9 +735,8 @@ static CURL *get_curl_handle(void)\n \t\tallowed_protocols |= CURLPROTO_FTPS;\n \tcurl_easy_setopt(result, CURLOPT_REDIR_PROTOCOLS, allowed_protocols);\n #else\n-\tif (transport_restrict_protocols())\n-\t\twarning(\"protocol restrictions not applied to curl redirects because\\n\"\n-\t\t\t\"your curl version is too old (>= 7.19.4)\");\n+\twarning(\"protocol restrictions not applied to curl redirects because\\n\"\n+\t\t\"your curl version is too old (>= 7.19.4)\");\n #endif\n \tif (getenv(\"GIT_CURL_VERBOSE\"))\n \t\tcurl_easy_setopt(result, CURLOPT_VERBOSE, 1L);\ndiff --git a/transport.c b/transport.c\nindex 2c0ec76..186de9a 100644\n--- a/transport.c\n+++ b/transport.c\n@@ -747,11 +747,6 @@ void transport_check_allowed(const char *type)\n \t\tdie(\"transport '%s' not allowed\", type);\n }\n \n-int transport_restrict_protocols(void)\n-{\n-\treturn !!protocol_whitelist();\n-}\n-\n struct transport *transport_get(struct remote *remote, const char *url)\n {\n \tconst char *helper;\ndiff --git a/transport.h b/transport.h\nindex b8e4ee8..f4998bc 100644\n--- a/transport.h\n+++ b/transport.h\n@@ -164,12 +164,6 @@ int is_transport_allowed(const char *type);\n */\n void transport_check_allowed(const char *type);\n \n-/*\n- * Returns true if the user has attempted to turn on protocol\n- * restrictions at all.\n- */\n-int transport_restrict_protocols(void);\n-\n /* Transport options which apply to git:// and scp-style URLs */\n \n /* The program to use on the remote side to send a pack */\n-- \n2.8.0.rc3.226.g39d4020\n\n"},{"id":"306745","messageId":"1480623959-126129-2-git-send-email-bmwill@google.com","threadId":"44417","inReplyTo":"1480623959-126129-1-git-send-email-bmwill@google.com","subject":"[PATCH v7 1/4] lib-proto-disable: variable name fix","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-12-01T20:25:56Z","receivedAt":"2016-12-01T20:26:40Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"The test_proto function assigns the positional parameters to named\nvariables, but then still refers to \"$desc\" as \"$1\". Using $desc is\nmore readable and less error-prone.\n\nSigned-off-by: Brandon Williams \n---\n t/lib-proto-disable.sh | 12 ++++++------\n 1 file changed, 6 insertions(+), 6 deletions(-)\n\ndiff --git a/t/lib-proto-disable.sh b/t/lib-proto-disable.sh\nindex b0917d9..be88e9a 100644\n--- a/t/lib-proto-disable.sh\n+++ b/t/lib-proto-disable.sh\n@@ -9,7 +9,7 @@ test_proto () {\n \tproto=$2\n \turl=$3\n \n-\ttest_expect_success \"clone $1 (enabled)\" '\n+\ttest_expect_success \"clone $desc (enabled)\" '\n \t\trm -rf tmp.git &&\n \t\t(\n \t\t\tGIT_ALLOW_PROTOCOL=$proto &&\n@@ -18,7 +18,7 @@ test_proto () {\n \t\t)\n \t'\n \n-\ttest_expect_success \"fetch $1 (enabled)\" '\n+\ttest_expect_success \"fetch $desc (enabled)\" '\n \t\t(\n \t\t\tcd tmp.git &&\n \t\t\tGIT_ALLOW_PROTOCOL=$proto &&\n@@ -27,7 +27,7 @@ test_proto () {\n \t\t)\n \t'\n \n-\ttest_expect_success \"push $1 (enabled)\" '\n+\ttest_expect_success \"push $desc (enabled)\" '\n \t\t(\n \t\t\tcd tmp.git &&\n \t\t\tGIT_ALLOW_PROTOCOL=$proto &&\n@@ -36,7 +36,7 @@ test_proto () {\n \t\t)\n \t'\n \n-\ttest_expect_success \"push $1 (disabled)\" '\n+\ttest_expect_success \"push $desc (disabled)\" '\n \t\t(\n \t\t\tcd tmp.git &&\n \t\t\tGIT_ALLOW_PROTOCOL=none &&\n@@ -45,7 +45,7 @@ test_proto () {\n \t\t)\n \t'\n \n-\ttest_expect_success \"fetch $1 (disabled)\" '\n+\ttest_expect_success \"fetch $desc (disabled)\" '\n \t\t(\n \t\t\tcd tmp.git &&\n \t\t\tGIT_ALLOW_PROTOCOL=none &&\n@@ -54,7 +54,7 @@ test_proto () {\n \t\t)\n \t'\n \n-\ttest_expect_success \"clone $1 (disabled)\" '\n+\ttest_expect_success \"clone $desc (disabled)\" '\n \t\trm -rf tmp.git &&\n \t\t(\n \t\t\tGIT_ALLOW_PROTOCOL=none &&\n-- \n2.8.0.rc3.226.g39d4020\n\n"},{"id":"306746","messageId":"1480623959-126129-3-git-send-email-bmwill@google.com","threadId":"44417","inReplyTo":"1480623959-126129-1-git-send-email-bmwill@google.com","subject":"[PATCH v7 2/4] transport: add protocol policy config option","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-12-01T20:25:57Z","receivedAt":"2016-12-01T20:26:44Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"Previously the `GIT_ALLOW_PROTOCOL` environment variable was used to\nspecify a whitelist of protocols to be used in clone/fetch/push\ncommands. This patch introduces new configuration options for more\nfine-grained control for allowing/disallowing protocols. This also has\nthe added benefit of allowing easier construction of a protocol\nwhitelist on systems where setting an environment variable is\nnon-trivial.\n\nNow users can specify a policy to be used for each type of protocol via\nthe 'protocol..allow' config option. A default policy for all\nunconfigured protocols can be set with the 'protocol.allow' config\noption. If no user configured default is made git will allow known-safe\nprotocols (http, https, git, ssh, file), disallow known-dangerous\nprotocols (ext), and have a default policy of `user` for all other\nprotocols.\n\nThe supported policies are `always`, `never`, and `user`. The `user`\npolicy can be used to configure a protocol to be usable when explicitly\nused by a user, while disallowing it for commands which run\nclone/fetch/push commands without direct user intervention (e.g.\nrecursive initialization of submodules). Commands which can potentially\nclone/fetch/push from untrusted repositories without user intervention\ncan export `GIT_PROTOCOL_FROM_USER` with a value of '0' to prevent\nprotocols configured to the `user` policy from being used.\n\nFix remote-ext tests to use the new config to allow the ext\nprotocol to be tested.\n\nBased on a patch by Jeff King \n\nSigned-off-by: Brandon Williams \n---\n Documentation/config.txt | 46 ++++++++++++++\n Documentation/git.txt | 38 +++++-------\n git-submodule.sh | 12 ++--\n t/lib-proto-disable.sh | 130 +++++++++++++++++++++++++++++++++++++--\n t/t5509-fetch-push-namespaces.sh | 1 +\n t/t5802-connect-helper.sh | 1 +\n transport.c | 75 +++++++++++++++++++++-\n 7 files changed, 264 insertions(+), 39 deletions(-)\n\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex 27069ac..5fe50bc 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -2308,6 +2308,52 @@ pretty.::\n \tNote that an alias with the same name as a built-in format\n \twill be silently ignored.\n \n+protocol.allow::\n+\tIf set, provide a user defined default policy for all protocols which\n+\tdon't explicitly have a policy (`protocol..allow`). By default,\n+\tif unset, known-safe protocols (http, https, git, ssh, file) have a\n+\tdefault policy of `always`, known-dangerous protocols (ext) have a\n+\tdefault policy of `never`, and all other protocols have a default\n+\tpolicy of `user`. Supported policies:\n++\n+--\n+\n+* `always` - protocol is always able to be used.\n+\n+* `never` - protocol is never able to be used.\n+\n+* `user` - protocol is only able to be used when `GIT_PROTOCOL_FROM_USER` is\n+ either unset or has a value of 1. This policy should be used when you want a\n+ protocol to be directly usable by the user but don't want it used by commands which\n+ execute clone/fetch/push commands without user input, e.g. recursive\n+ submodule initialization.\n+\n+--\n+\n+protocol..allow::\n+\tSet a policy to be used by protocol `` with clone/fetch/push\n+\tcommands. See `protocol.allow` above for the available policies.\n++\n+The protocol names currently used by git are:\n++\n+--\n+ - `file`: any local file-based path (including `file://` URLs,\n+ or local paths)\n+\n+ - `git`: the anonymous git protocol over a direct TCP\n+ connection (or proxy, if configured)\n+\n+ - `ssh`: git over ssh (including `host:path` syntax,\n+ `ssh://`, etc).\n+\n+ - `http`: git over http, both \"smart http\" and \"dumb http\".\n+ Note that this does _not_ include `https`; if you want to configure\n+ both, you must do so individually.\n+\n+ - any external helpers are named by their protocol (e.g., use\n+ `hg` to allow the `git-remote-hg` helper)\n+--\n+\n pull.ff::\n \tBy default, Git does not create an extra merge commit when merging\n \ta commit that is a descendant of the current commit. Instead, the\ndiff --git a/Documentation/git.txt b/Documentation/git.txt\nindex ab7215e..c52cec8 100644\n--- a/Documentation/git.txt\n+++ b/Documentation/git.txt\n@@ -1150,30 +1150,20 @@ of clones and fetches.\n \tcloning a repository to make a backup).\n \n `GIT_ALLOW_PROTOCOL`::\n-\tIf set, provide a colon-separated list of protocols which are\n-\tallowed to be used with fetch/push/clone. This is useful to\n-\trestrict recursive submodule initialization from an untrusted\n-\trepository. Any protocol not mentioned will be disallowed (i.e.,\n-\tthis is a whitelist, not a blacklist). If the variable is not\n-\tset at all, all protocols are enabled. The protocol names\n-\tcurrently used by git are:\n-\n-\t - `file`: any local file-based path (including `file://` URLs,\n-\t or local paths)\n-\n-\t - `git`: the anonymous git protocol over a direct TCP\n-\t connection (or proxy, if configured)\n-\n-\t - `ssh`: git over ssh (including `host:path` syntax,\n-\t `ssh://`, etc).\n-\n-\t - `http`: git over http, both \"smart http\" and \"dumb http\".\n-\t Note that this does _not_ include `https`; if you want both,\n-\t you should specify both as `http:https`.\n-\n-\t - any external helpers are named by their protocol (e.g., use\n-\t `hg` to allow the `git-remote-hg` helper)\n-\n+\tIf set to a colon-separated list of protocols, behave as if\n+\t`protocol.allow` is set to `never`, and each of the listed\n+\tprotocols has `protocol..allow` set to `always`\n+\t(overriding any existing configuration). In other words, any\n+\tprotocol not mentioned will be disallowed (i.e., this is a\n+\twhitelist, not a blacklist). See the description of\n+\t`protocol.allow` in linkgit:git-config[1] for more details.\n+\n+`GIT_PROTOCOL_FROM_USER`::\n+\tSet to 0 to prevent protocols used by fetch/push/clone which are\n+\tconfigured to the `user` state. This is useful to restrict recursive\n+\tsubmodule initialization from an untrusted repository or for programs\n+\twhich feed potentially-untrusted URLS to git commands. See\n+\tlinkgit:git-config[1] for more details.\n \n Discussion[[Discussion]]\n ------------------------\ndiff --git a/git-submodule.sh b/git-submodule.sh\nindex a024a13..0a477b4 100755\n--- a/git-submodule.sh\n+++ b/git-submodule.sh\n@@ -21,14 +21,10 @@ require_work_tree\n wt_prefix=$(git rev-parse --show-prefix)\n cd_to_toplevel\n \n-# Restrict ourselves to a vanilla subset of protocols; the URLs\n-# we get are under control of a remote repository, and we do not\n-# want them kicking off arbitrary git-remote-* programs.\n-#\n-# If the user has already specified a set of allowed protocols,\n-# we assume they know what they're doing and use that instead.\n-: ${GIT_ALLOW_PROTOCOL=file:git:http:https:ssh}\n-export GIT_ALLOW_PROTOCOL\n+# Tell the rest of git that any URLs we get don't come\n+# directly from the user, so it can apply policy as appropriate.\n+GIT_PROTOCOL_FROM_USER=0\n+export GIT_PROTOCOL_FROM_USER\n \n command=\n branch=\ndiff --git a/t/lib-proto-disable.sh b/t/lib-proto-disable.sh\nindex be88e9a..02f49cb 100644\n--- a/t/lib-proto-disable.sh\n+++ b/t/lib-proto-disable.sh\n@@ -1,10 +1,7 @@\n # Test routines for checking protocol disabling.\n \n-# test cloning a particular protocol\n-# $1 - description of the protocol\n-# $2 - machine-readable name of the protocol\n-# $3 - the URL to try cloning\n-test_proto () {\n+# Test clone/fetch/push with GIT_ALLOW_PROTOCOL whitelist\n+test_whitelist () {\n \tdesc=$1\n \tproto=$2\n \turl=$3\n@@ -62,6 +59,129 @@ test_proto () {\n \t\t\ttest_must_fail git clone --bare \"$url\" tmp.git\n \t\t)\n \t'\n+\n+\ttest_expect_success \"clone $desc (env var has precedence)\" '\n+\t\trm -rf tmp.git &&\n+\t\t(\n+\t\t\tGIT_ALLOW_PROTOCOL=none &&\n+\t\t\texport GIT_ALLOW_PROTOCOL &&\n+\t\t\ttest_must_fail git -c protocol.allow=always clone --bare \"$url\" tmp.git &&\n+\t\t\ttest_must_fail git -c protocol.$proto.allow=always clone --bare \"$url\" tmp.git\n+\t\t)\n+\t'\n+}\n+\n+test_config () {\n+\tdesc=$1\n+\tproto=$2\n+\turl=$3\n+\n+\t# Test clone/fetch/push with protocol..allow config\n+\ttest_expect_success \"clone $desc (enabled with config)\" '\n+\t\trm -rf tmp.git &&\n+\t\tgit -c protocol.$proto.allow=always clone --bare \"$url\" tmp.git\n+\t'\n+\n+\ttest_expect_success \"fetch $desc (enabled)\" '\n+\t\tgit -C tmp.git -c protocol.$proto.allow=always fetch\n+\t'\n+\n+\ttest_expect_success \"push $desc (enabled)\" '\n+\t\tgit -C tmp.git -c protocol.$proto.allow=always push origin HEAD:pushed\n+\t'\n+\n+\ttest_expect_success \"push $desc (disabled)\" '\n+\t\ttest_must_fail git -C tmp.git -c protocol.$proto.allow=never push origin HEAD:pushed\n+\t'\n+\n+\ttest_expect_success \"fetch $desc (disabled)\" '\n+\t\ttest_must_fail git -C tmp.git -c protocol.$proto.allow=never fetch\n+\t'\n+\n+\ttest_expect_success \"clone $desc (disabled)\" '\n+\t\trm -rf tmp.git &&\n+\t\ttest_must_fail git -c protocol.$proto.allow=never clone --bare \"$url\" tmp.git\n+\t'\n+\n+\t# Test clone/fetch/push with protocol.user.allow and its env var\n+\ttest_expect_success \"clone $desc (enabled)\" '\n+\t\trm -rf tmp.git &&\n+\t\tgit -c protocol.$proto.allow=user clone --bare \"$url\" tmp.git\n+\t'\n+\n+\ttest_expect_success \"fetch $desc (enabled)\" '\n+\t\tgit -C tmp.git -c protocol.$proto.allow=user fetch\n+\t'\n+\n+\ttest_expect_success \"push $desc (enabled)\" '\n+\t\tgit -C tmp.git -c protocol.$proto.allow=user push origin HEAD:pushed\n+\t'\n+\n+\ttest_expect_success \"push $desc (disabled)\" '\n+\t\t(\n+\t\t\tcd tmp.git &&\n+\t\t\tGIT_PROTOCOL_FROM_USER=0 &&\n+\t\t\texport GIT_PROTOCOL_FROM_USER &&\n+\t\t\ttest_must_fail git -c protocol.$proto.allow=user push origin HEAD:pushed\n+\t\t)\n+\t'\n+\n+\ttest_expect_success \"fetch $desc (disabled)\" '\n+\t\t(\n+\t\t\tcd tmp.git &&\n+\t\t\tGIT_PROTOCOL_FROM_USER=0 &&\n+\t\t\texport GIT_PROTOCOL_FROM_USER &&\n+\t\t\ttest_must_fail git -c protocol.$proto.allow=user fetch\n+\t\t)\n+\t'\n+\n+\ttest_expect_success \"clone $desc (disabled)\" '\n+\t\trm -rf tmp.git &&\n+\t\t(\n+\t\t\tGIT_PROTOCOL_FROM_USER=0 &&\n+\t\t\texport GIT_PROTOCOL_FROM_USER &&\n+\t\t\ttest_must_fail git -c protocol.$proto.allow=user clone --bare \"$url\" tmp.git\n+\t\t)\n+\t'\n+\n+\t# Test clone/fetch/push with protocol.allow user defined default\n+\ttest_expect_success \"clone $desc (enabled)\" '\n+\t\trm -rf tmp.git &&\n+\t\tgit config --global protocol.allow always &&\n+\t\tgit clone --bare \"$url\" tmp.git\n+\t'\n+\n+\ttest_expect_success \"fetch $desc (enabled)\" '\n+\t\tgit -C tmp.git fetch\n+\t'\n+\n+\ttest_expect_success \"push $desc (enabled)\" '\n+\t\tgit -C tmp.git push origin HEAD:pushed\n+\t'\n+\n+\ttest_expect_success \"push $desc (disabled)\" '\n+\t\tgit config --global protocol.allow never &&\n+\t\ttest_must_fail git -C tmp.git push origin HEAD:pushed\n+\t'\n+\n+\ttest_expect_success \"fetch $desc (disabled)\" '\n+\t\ttest_must_fail git -C tmp.git fetch\n+\t'\n+\n+\ttest_expect_success \"clone $desc (disabled)\" '\n+\t\trm -rf tmp.git &&\n+\t\ttest_must_fail git clone --bare \"$url\" tmp.git\n+\t'\n+}\n+\n+# test cloning a particular protocol\n+# $1 - description of the protocol\n+# $2 - machine-readable name of the protocol\n+# $3 - the URL to try cloning\n+test_proto () {\n+\ttest_whitelist \"$@\"\n+\n+\ttest_config \"$@\"\n }\n \n # set up an ssh wrapper that will access $host/$repo in the\ndiff --git a/t/t5509-fetch-push-namespaces.sh b/t/t5509-fetch-push-namespaces.sh\nindex bc44ac3..75c570a 100755\n--- a/t/t5509-fetch-push-namespaces.sh\n+++ b/t/t5509-fetch-push-namespaces.sh\n@@ -4,6 +4,7 @@ test_description='fetch/push involving ref namespaces'\n . ./test-lib.sh\n \n test_expect_success setup '\n+\tgit config --global protocol.ext.allow user &&\n \ttest_tick &&\n \tgit init original &&\n \t(\ndiff --git a/t/t5802-connect-helper.sh b/t/t5802-connect-helper.sh\nindex b7a7f9d..c6c2661 100755\n--- a/t/t5802-connect-helper.sh\n+++ b/t/t5802-connect-helper.sh\n@@ -4,6 +4,7 @@ test_description='ext::cmd remote \"connect\" helper'\n . ./test-lib.sh\n \n test_expect_success setup '\n+\tgit config --global protocol.ext.allow user &&\n \ttest_tick &&\n \tgit commit --allow-empty -m initial &&\n \ttest_tick &&\ndiff --git a/transport.c b/transport.c\nindex d57e8de..2c0ec76 100644\n--- a/transport.c\n+++ b/transport.c\n@@ -664,10 +664,81 @@ static const struct string_list *protocol_whitelist(void)\n \treturn enabled ? &allowed : NULL;\n }\n \n+enum protocol_allow_config {\n+\tPROTOCOL_ALLOW_NEVER = 0,\n+\tPROTOCOL_ALLOW_USER_ONLY,\n+\tPROTOCOL_ALLOW_ALWAYS\n+};\n+\n+static enum protocol_allow_config parse_protocol_config(const char *key,\n+\t\t\t\t\t\t\tconst char *value)\n+{\n+\tif (!strcasecmp(value, \"always\"))\n+\t\treturn PROTOCOL_ALLOW_ALWAYS;\n+\telse if (!strcasecmp(value, \"never\"))\n+\t\treturn PROTOCOL_ALLOW_NEVER;\n+\telse if (!strcasecmp(value, \"user\"))\n+\t\treturn PROTOCOL_ALLOW_USER_ONLY;\n+\n+\tdie(\"unknown value for config '%s': %s\", key, value);\n+}\n+\n+static enum protocol_allow_config get_protocol_config(const char *type)\n+{\n+\tchar *key = xstrfmt(\"protocol.%s.allow\", type);\n+\tchar *value;\n+\n+\t/* first check the per-protocol config */\n+\tif (!git_config_get_string(key, &value)) {\n+\t\tenum protocol_allow_config ret =\n+\t\t\tparse_protocol_config(key, value);\n+\t\tfree(key);\n+\t\tfree(value);\n+\t\treturn ret;\n+\t}\n+\tfree(key);\n+\n+\t/* if defined, fallback to user-defined default for unknown protocols */\n+\tif (!git_config_get_string(\"protocol.allow\", &value)) {\n+\t\tenum protocol_allow_config ret =\n+\t\t\tparse_protocol_config(\"protocol.allow\", value);\n+\t\tfree(value);\n+\t\treturn ret;\n+\t}\n+\n+\t/* fallback to built-in defaults */\n+\t/* known safe */\n+\tif (!strcmp(type, \"http\") ||\n+\t !strcmp(type, \"https\") ||\n+\t !strcmp(type, \"git\") ||\n+\t !strcmp(type, \"ssh\") ||\n+\t !strcmp(type, \"file\"))\n+\t\treturn PROTOCOL_ALLOW_ALWAYS;\n+\n+\t/* known scary; err on the side of caution */\n+\tif (!strcmp(type, \"ext\"))\n+\t\treturn PROTOCOL_ALLOW_NEVER;\n+\n+\t/* unknown; by default let them be used only directly by the user */\n+\treturn PROTOCOL_ALLOW_USER_ONLY;\n+}\n+\n int is_transport_allowed(const char *type)\n {\n-\tconst struct string_list *allowed = protocol_whitelist();\n-\treturn !allowed || string_list_has_string(allowed, type);\n+\tconst struct string_list *whitelist = protocol_whitelist();\n+\tif (whitelist)\n+\t\treturn string_list_has_string(whitelist, type);\n+\n+\tswitch (get_protocol_config(type)) {\n+\tcase PROTOCOL_ALLOW_ALWAYS:\n+\t\treturn 1;\n+\tcase PROTOCOL_ALLOW_NEVER:\n+\t\treturn 0;\n+\tcase PROTOCOL_ALLOW_USER_ONLY:\n+\t\treturn git_env_bool(\"GIT_PROTOCOL_FROM_USER\", 1);\n+\t}\n+\n+\tdie(\"BUG: invalid protocol_allow_config type\");\n }\n \n void transport_check_allowed(const char *type)\n-- \n2.8.0.rc3.226.g39d4020\n\n"},{"id":"306747","messageId":"1480623959-126129-5-git-send-email-bmwill@google.com","threadId":"44417","inReplyTo":"1480623959-126129-1-git-send-email-bmwill@google.com","subject":"[PATCH v7 4/4] transport: add from_user parameter to is_transport_allowed","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-12-01T20:25:59Z","receivedAt":"2016-12-01T20:26:47Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"Add the from_user parameter to the 'is_transport_allowed' function.\nThis allows callers to query if a transport protocol is allowed, given\nthat the caller knows that the protocol is coming from the user (1) or\nnot from the user (0), such as redirects in libcurl. If unknown, a -1\nshould be provided which falls back to reading `GIT_PROTOCOL_FROM_USER`\nto determine if the protocol came from the user.\n\nSigned-off-by: Brandon Williams \n---\n http.c | 8 ++++----\n transport.c | 8 +++++---\n transport.h | 13 ++++++++++---\n 3 files changed, 19 insertions(+), 10 deletions(-)\n\ndiff --git a/http.c b/http.c\nindex fee128b..e74c0f0 100644\n--- a/http.c\n+++ b/http.c\n@@ -725,13 +725,13 @@ static CURL *get_curl_handle(void)\n \tcurl_easy_setopt(result, CURLOPT_POST301, 1);\n #endif\n #if LIBCURL_VERSION_NUM >= 0x071304\n-\tif (is_transport_allowed(\"http\"))\n+\tif (is_transport_allowed(\"http\", 0))\n \t\tallowed_protocols |= CURLPROTO_HTTP;\n-\tif (is_transport_allowed(\"https\"))\n+\tif (is_transport_allowed(\"https\", 0))\n \t\tallowed_protocols |= CURLPROTO_HTTPS;\n-\tif (is_transport_allowed(\"ftp\"))\n+\tif (is_transport_allowed(\"ftp\", 0))\n \t\tallowed_protocols |= CURLPROTO_FTP;\n-\tif (is_transport_allowed(\"ftps\"))\n+\tif (is_transport_allowed(\"ftps\", 0))\n \t\tallowed_protocols |= CURLPROTO_FTPS;\n \tcurl_easy_setopt(result, CURLOPT_REDIR_PROTOCOLS, allowed_protocols);\n #else\ndiff --git a/transport.c b/transport.c\nindex 186de9a..8a3597b 100644\n--- a/transport.c\n+++ b/transport.c\n@@ -723,7 +723,7 @@ static enum protocol_allow_config get_protocol_config(const char *type)\n \treturn PROTOCOL_ALLOW_USER_ONLY;\n }\n \n-int is_transport_allowed(const char *type)\n+int is_transport_allowed(const char *type, int from_user)\n {\n \tconst struct string_list *whitelist = protocol_whitelist();\n \tif (whitelist)\n@@ -735,7 +735,9 @@ int is_transport_allowed(const char *type)\n \tcase PROTOCOL_ALLOW_NEVER:\n \t\treturn 0;\n \tcase PROTOCOL_ALLOW_USER_ONLY:\n-\t\treturn git_env_bool(\"GIT_PROTOCOL_FROM_USER\", 1);\n+\t\tif (from_user < 0)\n+\t\t\tfrom_user = git_env_bool(\"GIT_PROTOCOL_FROM_USER\", 1);\n+\t\treturn from_user;\n \t}\n \n \tdie(\"BUG: invalid protocol_allow_config type\");\n@@ -743,7 +745,7 @@ int is_transport_allowed(const char *type)\n \n void transport_check_allowed(const char *type)\n {\n-\tif (!is_transport_allowed(type))\n+\tif (!is_transport_allowed(type, -1))\n \t\tdie(\"transport '%s' not allowed\", type);\n }\n \ndiff --git a/transport.h b/transport.h\nindex f4998bc..9820f10 100644\n--- a/transport.h\n+++ b/transport.h\n@@ -153,10 +153,17 @@ extern int transport_summary_width(const struct ref *refs);\n struct transport *transport_get(struct remote *, const char *);\n \n /*\n- * Check whether a transport is allowed by the environment. Type should\n- * generally be the URL scheme, as described in Documentation/git.txt\n+ * Check whether a transport is allowed by the environment.\n+ *\n+ * Type should generally be the URL scheme, as described in\n+ * Documentation/git.txt\n+ *\n+ * from_user specifies if the transport was given by the user. If unknown pass\n+ * a -1 to read from the environment to determine if the transport was given by\n+ * the user.\n+ *\n */\n-int is_transport_allowed(const char *type);\n+int is_transport_allowed(const char *type, int from_user);\n \n /*\n * Check whether a transport is allowed by the environment,\n-- \n2.8.0.rc3.226.g39d4020\n\n"},{"id":"306758","messageId":"20161201214004.3qujo5sfdn3y6c5u@sigill.intra.peff.net","threadId":"44417","inReplyTo":"1480623959-126129-5-git-send-email-bmwill@google.com","subject":"Re: [PATCH v7 4/4] transport: add from_user parameter to is_transport_allowed","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-12-01T21:40:04Z","receivedAt":"2016-12-01T21:40:12Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Dec 01, 2016 at 12:25:59PM -0800, Brandon Williams wrote:\n\n> Add the from_user parameter to the 'is_transport_allowed' function.\n> This allows callers to query if a transport protocol is allowed, given\n> that the caller knows that the protocol is coming from the user (1) or\n> not from the user (0), such as redirects in libcurl. If unknown, a -1\n> should be provided which falls back to reading `GIT_PROTOCOL_FROM_USER`\n> to determine if the protocol came from the user.\n\nPatches 3 and 4 look good to me (1 and 2 are unchanged, right? They are\nalready in 'next' anyway, though I guess we are due for a post-release\nreset of 'next').\n\n> diff --git a/http.c b/http.c\n> index fee128b..e74c0f0 100644\n> --- a/http.c\n> +++ b/http.c\n> @@ -725,13 +725,13 @@ static CURL *get_curl_handle(void)\n> \tcurl_easy_setopt(result, CURLOPT_POST301, 1);\n> #endif\n> #if LIBCURL_VERSION_NUM >= 0x071304\n> -\tif (is_transport_allowed(\"http\"))\n> +\tif (is_transport_allowed(\"http\", 0))\n> \t\tallowed_protocols |= CURLPROTO_HTTP;\n> -\tif (is_transport_allowed(\"https\"))\n> +\tif (is_transport_allowed(\"https\", 0))\n> \t\tallowed_protocols |= CURLPROTO_HTTPS;\n> -\tif (is_transport_allowed(\"ftp\"))\n> +\tif (is_transport_allowed(\"ftp\", 0))\n> \t\tallowed_protocols |= CURLPROTO_FTP;\n> -\tif (is_transport_allowed(\"ftps\"))\n> +\tif (is_transport_allowed(\"ftps\", 0))\n> \t\tallowed_protocols |= CURLPROTO_FTPS;\n> \tcurl_easy_setopt(result, CURLOPT_REDIR_PROTOCOLS, allowed_protocols);\n> #else\n\nThis is better, but I think we still need to deal with http-alternates\non top.\n\nI think we'd need to move this allowed_protocols setup into a function\nlike:\n\n int generate_allowed_protocols(int from_user)\n {\n\tint ret;\n\tif (is_transport_allowed(\"http\", from_user))\n\t\tret |= CURLPROTO_HTTP;\n\t... etc ...\n\treturn ret;\n }\n\nand then create a protocol list for each situation:\n\n allowed_protocols = generate_allowed_protocols(-1);\n allowed_redir_protocols = generate_allowed_protocols(0);\n\nand then we know we can always set up the redir protocols:\n\n curl_easy_setopt(result, CURLOPT_REDIR_PROTOCOLS, allowed_redir_protocols);\n\nand which we feed for CURLOPT_PROTOCOLS depends on whether we are\nfollowing an http-alternates redirect or not. But I suspect it will be a\nnasty change to plumb through the idea of \"this request is on behalf of\nan http-alternates redirect\".\n\nGiven how few people probably care, I'm tempted to document it as a\nquirk and direct people to the upcoming http.followRedirects. The newly\nproposed default value of that disables http-alternates entirely anyway.\n\n-Peff\n"},{"id":"306766","messageId":"xmqqpolb70d2.fsf@gitster.mtv.corp.google.com","threadId":"44417","inReplyTo":"20161201214004.3qujo5sfdn3y6c5u@sigill.intra.peff.net","subject":"Re: [PATCH v7 4/4] transport: add from_user parameter to is_transport_allowed","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-12-01T22:25:45Z","receivedAt":"2016-12-01T22:26:45Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King writes:\n\n> On Thu, Dec 01, 2016 at 12:25:59PM -0800, Brandon Williams wrote:\n>\n>> Add the from_user parameter to the 'is_transport_allowed' function.\n>> This allows callers to query if a transport protocol is allowed, given\n>> that the caller knows that the protocol is coming from the user (1) or\n>> not from the user (0), such as redirects in libcurl. If unknown, a -1\n>> should be provided which falls back to reading `GIT_PROTOCOL_FROM_USER`\n>> to determine if the protocol came from the user.\n>\n> Patches 3 and 4 look good to me (1 and 2 are unchanged, right? They are\n> already in 'next' anyway, though I guess we are due for a post-release\n> reset of 'next').\n\nYes. I am planning to take a day off tomorrow, and probably will\nrewind 'next' sometime this the weekend.\n\nI agree with the comments you made in the remainder of the message I\nam responding to, so I'll snip it.\n\n>> diff --git a/http.c b/http.c\n>> index fee128b..e74c0f0 100644\n>> --- a/http.c\n>> +++ b/http.c\n>> ...\n>\n> This is better, but I think we still need to deal with http-alternates\n> on top.\n> ...\n"},{"id":"306772","messageId":"20161201230738.GJ54082@google.com","threadId":"44417","inReplyTo":"20161201214004.3qujo5sfdn3y6c5u@sigill.intra.peff.net","subject":"Re: [PATCH v7 4/4] transport: add from_user parameter to is_transport_allowed","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-12-01T23:07:38Z","receivedAt":"2016-12-01T23:07:58Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 12/01, Jeff King wrote:\n> On Thu, Dec 01, 2016 at 12:25:59PM -0800, Brandon Williams wrote:\n> \n> > Add the from_user parameter to the 'is_transport_allowed' function.\n> > This allows callers to query if a transport protocol is allowed, given\n> > that the caller knows that the protocol is coming from the user (1) or\n> > not from the user (0), such as redirects in libcurl. If unknown, a -1\n> > should be provided which falls back to reading `GIT_PROTOCOL_FROM_USER`\n> > to determine if the protocol came from the user.\n> \n> Patches 3 and 4 look good to me (1 and 2 are unchanged, right? They are\n> already in 'next' anyway, though I guess we are due for a post-release\n> reset of 'next').\n> \n> > diff --git a/http.c b/http.c\n> > index fee128b..e74c0f0 100644\n> > --- a/http.c\n> > +++ b/http.c\n> > @@ -725,13 +725,13 @@ static CURL *get_curl_handle(void)\n> > \tcurl_easy_setopt(result, CURLOPT_POST301, 1);\n> > #endif\n> > #if LIBCURL_VERSION_NUM >= 0x071304\n> > -\tif (is_transport_allowed(\"http\"))\n> > +\tif (is_transport_allowed(\"http\", 0))\n> > \t\tallowed_protocols |= CURLPROTO_HTTP;\n> > -\tif (is_transport_allowed(\"https\"))\n> > +\tif (is_transport_allowed(\"https\", 0))\n> > \t\tallowed_protocols |= CURLPROTO_HTTPS;\n> > -\tif (is_transport_allowed(\"ftp\"))\n> > +\tif (is_transport_allowed(\"ftp\", 0))\n> > \t\tallowed_protocols |= CURLPROTO_FTP;\n> > -\tif (is_transport_allowed(\"ftps\"))\n> > +\tif (is_transport_allowed(\"ftps\", 0))\n> > \t\tallowed_protocols |= CURLPROTO_FTPS;\n> > \tcurl_easy_setopt(result, CURLOPT_REDIR_PROTOCOLS, allowed_protocols);\n> > #else\n> \n> This is better, but I think we still need to deal with http-alternates\n> on top.\n> \n> I think we'd need to move this allowed_protocols setup into a function\n> like:\n> \n> int generate_allowed_protocols(int from_user)\n> {\n> \tint ret;\n> \tif (is_transport_allowed(\"http\", from_user))\n> \t\tret |= CURLPROTO_HTTP;\n> \t... etc ...\n> \treturn ret;\n> }\n> \n> and then create a protocol list for each situation:\n> \n> allowed_protocols = generate_allowed_protocols(-1);\n> allowed_redir_protocols = generate_allowed_protocols(0);\n> \n> and then we know we can always set up the redir protocols:\n> \n> curl_easy_setopt(result, CURLOPT_REDIR_PROTOCOLS, allowed_redir_protocols);\n> \n> and which we feed for CURLOPT_PROTOCOLS depends on whether we are\n> following an http-alternates redirect or not. But I suspect it will be a\n> nasty change to plumb through the idea of \"this request is on behalf of\n> an http-alternates redirect\".\n> \n> Given how few people probably care, I'm tempted to document it as a\n> quirk and direct people to the upcoming http.followRedirects. The newly\n> proposed default value of that disables http-alternates entirely anyway.\n> \n> -Peff\n\nI started taking a look at your http redirect series (I really should\nhave taking a look at it sooner) and I see exactly what you're talking\nabout. We can easily move this logic into a function to make it easier\nto generate the two whitelists.\n\n-- \nBrandon Williams\n"},{"id":"306774","messageId":"20161201232656.GK54082@google.com","threadId":"44417","inReplyTo":"20161201230738.GJ54082@google.com","subject":"Re: [PATCH v7 4/4] transport: add from_user parameter to is_transport_allowed","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-12-01T23:26:56Z","receivedAt":"2016-12-01T23:27:05Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 12/01, Brandon Williams wrote:\n> On 12/01, Jeff King wrote:\n> > On Thu, Dec 01, 2016 at 12:25:59PM -0800, Brandon Williams wrote:\n> > \n> > > Add the from_user parameter to the 'is_transport_allowed' function.\n> > > This allows callers to query if a transport protocol is allowed, given\n> > > that the caller knows that the protocol is coming from the user (1) or\n> > > not from the user (0), such as redirects in libcurl. If unknown, a -1\n> > > should be provided which falls back to reading `GIT_PROTOCOL_FROM_USER`\n> > > to determine if the protocol came from the user.\n> > \n> > Patches 3 and 4 look good to me (1 and 2 are unchanged, right? They are\n> > already in 'next' anyway, though I guess we are due for a post-release\n> > reset of 'next').\n> > \n> > > diff --git a/http.c b/http.c\n> > > index fee128b..e74c0f0 100644\n> > > --- a/http.c\n> > > +++ b/http.c\n> > > @@ -725,13 +725,13 @@ static CURL *get_curl_handle(void)\n> > > \tcurl_easy_setopt(result, CURLOPT_POST301, 1);\n> > > #endif\n> > > #if LIBCURL_VERSION_NUM >= 0x071304\n> > > -\tif (is_transport_allowed(\"http\"))\n> > > +\tif (is_transport_allowed(\"http\", 0))\n> > > \t\tallowed_protocols |= CURLPROTO_HTTP;\n> > > -\tif (is_transport_allowed(\"https\"))\n> > > +\tif (is_transport_allowed(\"https\", 0))\n> > > \t\tallowed_protocols |= CURLPROTO_HTTPS;\n> > > -\tif (is_transport_allowed(\"ftp\"))\n> > > +\tif (is_transport_allowed(\"ftp\", 0))\n> > > \t\tallowed_protocols |= CURLPROTO_FTP;\n> > > -\tif (is_transport_allowed(\"ftps\"))\n> > > +\tif (is_transport_allowed(\"ftps\", 0))\n> > > \t\tallowed_protocols |= CURLPROTO_FTPS;\n> > > \tcurl_easy_setopt(result, CURLOPT_REDIR_PROTOCOLS, allowed_protocols);\n> > > #else\n> > \n> > This is better, but I think we still need to deal with http-alternates\n> > on top.\n> > \n> > I think we'd need to move this allowed_protocols setup into a function\n> > like:\n> > \n> > int generate_allowed_protocols(int from_user)\n> > {\n> > \tint ret;\n> > \tif (is_transport_allowed(\"http\", from_user))\n> > \t\tret |= CURLPROTO_HTTP;\n> > \t... etc ...\n> > \treturn ret;\n> > }\n> > \n> > and then create a protocol list for each situation:\n> > \n> > allowed_protocols = generate_allowed_protocols(-1);\n> > allowed_redir_protocols = generate_allowed_protocols(0);\n> > \n> > and then we know we can always set up the redir protocols:\n> > \n> > curl_easy_setopt(result, CURLOPT_REDIR_PROTOCOLS, allowed_redir_protocols);\n> > \n> > and which we feed for CURLOPT_PROTOCOLS depends on whether we are\n> > following an http-alternates redirect or not. But I suspect it will be a\n> > nasty change to plumb through the idea of \"this request is on behalf of\n> > an http-alternates redirect\".\n> > \n> > Given how few people probably care, I'm tempted to document it as a\n> > quirk and direct people to the upcoming http.followRedirects. The newly\n> > proposed default value of that disables http-alternates entirely anyway.\n> > \n> > -Peff\n> \n> I started taking a look at your http redirect series (I really should\n> have taking a look at it sooner) and I see exactly what you're talking\n> about. We can easily move this logic into a function to make it easier\n> to generate the two whitelists.\n\nThinking about this some more...I was told that having http redirect to\nfile:// could be scary. The way the new protocol configuration is setup\nwe have file:// as a default known-safe protocol. Do we need to worry\nabout this or can we leave this be since this can be overridden by the\nuser?\n\n-- \nBrandon Williams\n"},{"id":"306776","messageId":"xmqqh96n6x63.fsf@gitster.mtv.corp.google.com","threadId":"44417","inReplyTo":"20161201230738.GJ54082@google.com","subject":"Re: [PATCH v7 4/4] transport: add from_user parameter to is_transport_allowed","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-12-01T23:34:44Z","receivedAt":"2016-12-01T23:35:12Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Brandon Williams writes:\n\n> I started taking a look at your http redirect series (I really should\n> have taking a look at it sooner) and I see exactly what you're talking\n> about. We can easily move this logic into a function to make it easier\n> to generate the two whitelists.\n\nOK. With both of them queued, t5812 seems to barf, just FYI; I'll\nexpect that a future reroll would make things better.\n\nThanks.\n\n"},{"id":"306779","messageId":"20161201235856.GL54082@google.com","threadId":"44417","inReplyTo":"xmqqh96n6x63.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH v7 4/4] transport: add from_user parameter to is_transport_allowed","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-12-01T23:58:56Z","receivedAt":"2016-12-01T23:59:05Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 12/01, Junio C Hamano wrote:\n> Brandon Williams writes:\n> \n> > I started taking a look at your http redirect series (I really should\n> > have taking a look at it sooner) and I see exactly what you're talking\n> > about. We can easily move this logic into a function to make it easier\n> > to generate the two whitelists.\n> \n> OK. With both of them queued, t5812 seems to barf, just FYI; I'll\n> expect that a future reroll would make things better.\n\nYeah I expected we would see an issue since both these series collide in\nhttp.c\n\nI'm sending out another reroll of this series so that in Jeff's he can\njust call 'get_curl_allowed_protocols(-1)' for the non-redirection curl\noption, which should make this test stop barfing.\n\n-- \nBrandon Williams\n"},{"id":"306780","messageId":"1480636862-40489-1-git-send-email-bmwill@google.com","threadId":"44417","inReplyTo":"1480623959-126129-1-git-send-email-bmwill@google.com","subject":"[PATCH v8 0/5] transport protocol policy configuration","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-12-02T00:00:57Z","receivedAt":"2016-12-02T00:01:22Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"v8 of this series moves the creation of an allowed protocol whitelist for\nCURLOPT_REDIR_PROTOCOLS to a helper function. This is to help out another\nseries which depends on the creation of a whitelist for CURLOPT_PROTOCOLS.\n\nBrandon Williams (5):\n lib-proto-disable: variable name fix\n transport: add protocol policy config option\n http: always warn if libcurl version is too old\n http: create function to get curl allowed protocols\n transport: add from_user parameter to is_transport_allowed\n\n Documentation/config.txt | 46 +++++++++++++\n Documentation/git.txt | 38 ++++-------\n git-submodule.sh | 12 ++--\n http.c | 32 +++++----\n t/lib-proto-disable.sh | 142 ++++++++++++++++++++++++++++++++++++---\n t/t5509-fetch-push-namespaces.sh | 1 +\n t/t5802-connect-helper.sh | 1 +\n transport.c | 84 ++++++++++++++++++++---\n transport.h | 19 +++---\n 9 files changed, 302 insertions(+), 73 deletions(-)\n\n-- \n2.8.0.rc3.226.g39d4020\n\n"},{"id":"306781","messageId":"1480636862-40489-2-git-send-email-bmwill@google.com","threadId":"44417","inReplyTo":"1480636862-40489-1-git-send-email-bmwill@google.com","subject":"[PATCH v8 1/5] lib-proto-disable: variable name fix","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-12-02T00:00:58Z","receivedAt":"2016-12-02T00:01:25Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"The test_proto function assigns the positional parameters to named\nvariables, but then still refers to \"$desc\" as \"$1\". Using $desc is\nmore readable and less error-prone.\n\nSigned-off-by: Brandon Williams \n---\n t/lib-proto-disable.sh | 12 ++++++------\n 1 file changed, 6 insertions(+), 6 deletions(-)\n\ndiff --git a/t/lib-proto-disable.sh b/t/lib-proto-disable.sh\nindex b0917d9..be88e9a 100644\n--- a/t/lib-proto-disable.sh\n+++ b/t/lib-proto-disable.sh\n@@ -9,7 +9,7 @@ test_proto () {\n \tproto=$2\n \turl=$3\n \n-\ttest_expect_success \"clone $1 (enabled)\" '\n+\ttest_expect_success \"clone $desc (enabled)\" '\n \t\trm -rf tmp.git &&\n \t\t(\n \t\t\tGIT_ALLOW_PROTOCOL=$proto &&\n@@ -18,7 +18,7 @@ test_proto () {\n \t\t)\n \t'\n \n-\ttest_expect_success \"fetch $1 (enabled)\" '\n+\ttest_expect_success \"fetch $desc (enabled)\" '\n \t\t(\n \t\t\tcd tmp.git &&\n \t\t\tGIT_ALLOW_PROTOCOL=$proto &&\n@@ -27,7 +27,7 @@ test_proto () {\n \t\t)\n \t'\n \n-\ttest_expect_success \"push $1 (enabled)\" '\n+\ttest_expect_success \"push $desc (enabled)\" '\n \t\t(\n \t\t\tcd tmp.git &&\n \t\t\tGIT_ALLOW_PROTOCOL=$proto &&\n@@ -36,7 +36,7 @@ test_proto () {\n \t\t)\n \t'\n \n-\ttest_expect_success \"push $1 (disabled)\" '\n+\ttest_expect_success \"push $desc (disabled)\" '\n \t\t(\n \t\t\tcd tmp.git &&\n \t\t\tGIT_ALLOW_PROTOCOL=none &&\n@@ -45,7 +45,7 @@ test_proto () {\n \t\t)\n \t'\n \n-\ttest_expect_success \"fetch $1 (disabled)\" '\n+\ttest_expect_success \"fetch $desc (disabled)\" '\n \t\t(\n \t\t\tcd tmp.git &&\n \t\t\tGIT_ALLOW_PROTOCOL=none &&\n@@ -54,7 +54,7 @@ test_proto () {\n \t\t)\n \t'\n \n-\ttest_expect_success \"clone $1 (disabled)\" '\n+\ttest_expect_success \"clone $desc (disabled)\" '\n \t\trm -rf tmp.git &&\n \t\t(\n \t\t\tGIT_ALLOW_PROTOCOL=none &&\n-- \n2.8.0.rc3.226.g39d4020\n\n"},{"id":"306782","messageId":"1480636862-40489-4-git-send-email-bmwill@google.com","threadId":"44417","inReplyTo":"1480636862-40489-1-git-send-email-bmwill@google.com","subject":"[PATCH v8 3/5] http: always warn if libcurl version is too old","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-12-02T00:01:00Z","receivedAt":"2016-12-02T00:01:28Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"Now that there are default \"known-good\" and \"known-bad\" protocols which\nare allowed/disallowed by 'is_transport_allowed' we should always warn\nthe user that older versions of libcurl can't respect the allowed\nprotocols for redirects.\n\nSigned-off-by: Brandon Williams \n---\n http.c | 5 ++---\n transport.c | 5 -----\n transport.h | 6 ------\n 3 files changed, 2 insertions(+), 14 deletions(-)\n\ndiff --git a/http.c b/http.c\nindex 4c4a812..fee128b 100644\n--- a/http.c\n+++ b/http.c\n@@ -735,9 +735,8 @@ static CURL *get_curl_handle(void)\n \t\tallowed_protocols |= CURLPROTO_FTPS;\n \tcurl_easy_setopt(result, CURLOPT_REDIR_PROTOCOLS, allowed_protocols);\n #else\n-\tif (transport_restrict_protocols())\n-\t\twarning(\"protocol restrictions not applied to curl redirects because\\n\"\n-\t\t\t\"your curl version is too old (>= 7.19.4)\");\n+\twarning(\"protocol restrictions not applied to curl redirects because\\n\"\n+\t\t\"your curl version is too old (>= 7.19.4)\");\n #endif\n \tif (getenv(\"GIT_CURL_VERBOSE\"))\n \t\tcurl_easy_setopt(result, CURLOPT_VERBOSE, 1L);\ndiff --git a/transport.c b/transport.c\nindex 2c0ec76..186de9a 100644\n--- a/transport.c\n+++ b/transport.c\n@@ -747,11 +747,6 @@ void transport_check_allowed(const char *type)\n \t\tdie(\"transport '%s' not allowed\", type);\n }\n \n-int transport_restrict_protocols(void)\n-{\n-\treturn !!protocol_whitelist();\n-}\n-\n struct transport *transport_get(struct remote *remote, const char *url)\n {\n \tconst char *helper;\ndiff --git a/transport.h b/transport.h\nindex b8e4ee8..f4998bc 100644\n--- a/transport.h\n+++ b/transport.h\n@@ -164,12 +164,6 @@ int is_transport_allowed(const char *type);\n */\n void transport_check_allowed(const char *type);\n \n-/*\n- * Returns true if the user has attempted to turn on protocol\n- * restrictions at all.\n- */\n-int transport_restrict_protocols(void);\n-\n /* Transport options which apply to git:// and scp-style URLs */\n \n /* The program to use on the remote side to send a pack */\n-- \n2.8.0.rc3.226.g39d4020\n\n"},{"id":"306783","messageId":"1480636862-40489-6-git-send-email-bmwill@google.com","threadId":"44417","inReplyTo":"1480636862-40489-1-git-send-email-bmwill@google.com","subject":"[PATCH v8 5/5] transport: add from_user parameter to is_transport_allowed","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-12-02T00:01:02Z","receivedAt":"2016-12-02T00:01:31Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"Add the from_user parameter to the 'is_transport_allowed' function.\nThis allows callers to query if a transport protocol is allowed, given\nthat the caller knows that the protocol is coming from the user (1) or\nnot from the user (0) such as redirects in libcurl. If unknown a -1\nshould be provided which falls back to reading `GIT_PROTOCOL_FROM_USER`\nto determine if the protocol came from the user.\n\nSigned-off-by: Brandon Williams \n---\n http.c | 12 ++++++------\n transport.c | 8 +++++---\n transport.h | 13 ++++++++++---\n 3 files changed, 21 insertions(+), 12 deletions(-)\n\ndiff --git a/http.c b/http.c\nindex a1c3a0e..2a02941 100644\n--- a/http.c\n+++ b/http.c\n@@ -624,17 +624,17 @@ void setup_curl_trace(CURL *handle)\n \tcurl_easy_setopt(handle, CURLOPT_DEBUGDATA, NULL);\n }\n \n-static long get_curl_allowed_protocols(void)\n+static long get_curl_allowed_protocols(int from_user)\n {\n \tlong allowed_protocols = 0;\n \n-\tif (is_transport_allowed(\"http\"))\n+\tif (is_transport_allowed(\"http\", from_user))\n \t\tallowed_protocols |= CURLPROTO_HTTP;\n-\tif (is_transport_allowed(\"https\"))\n+\tif (is_transport_allowed(\"https\", from_user))\n \t\tallowed_protocols |= CURLPROTO_HTTPS;\n-\tif (is_transport_allowed(\"ftp\"))\n+\tif (is_transport_allowed(\"ftp\", from_user))\n \t\tallowed_protocols |= CURLPROTO_FTP;\n-\tif (is_transport_allowed(\"ftps\"))\n+\tif (is_transport_allowed(\"ftps\", from_user))\n \t\tallowed_protocols |= CURLPROTO_FTPS;\n \n \treturn allowed_protocols;\n@@ -740,7 +740,7 @@ static CURL *get_curl_handle(void)\n #endif\n #if LIBCURL_VERSION_NUM >= 0x071304\n \tcurl_easy_setopt(result, CURLOPT_REDIR_PROTOCOLS,\n-\t\t\t get_curl_allowed_protocols());\n+\t\t\t get_curl_allowed_protocols(0));\n #else\n \twarning(\"protocol restrictions not applied to curl redirects because\\n\"\n \t\t\"your curl version is too old (>= 7.19.4)\");\ndiff --git a/transport.c b/transport.c\nindex 186de9a..8a3597b 100644\n--- a/transport.c\n+++ b/transport.c\n@@ -723,7 +723,7 @@ static enum protocol_allow_config get_protocol_config(const char *type)\n \treturn PROTOCOL_ALLOW_USER_ONLY;\n }\n \n-int is_transport_allowed(const char *type)\n+int is_transport_allowed(const char *type, int from_user)\n {\n \tconst struct string_list *whitelist = protocol_whitelist();\n \tif (whitelist)\n@@ -735,7 +735,9 @@ int is_transport_allowed(const char *type)\n \tcase PROTOCOL_ALLOW_NEVER:\n \t\treturn 0;\n \tcase PROTOCOL_ALLOW_USER_ONLY:\n-\t\treturn git_env_bool(\"GIT_PROTOCOL_FROM_USER\", 1);\n+\t\tif (from_user < 0)\n+\t\t\tfrom_user = git_env_bool(\"GIT_PROTOCOL_FROM_USER\", 1);\n+\t\treturn from_user;\n \t}\n \n \tdie(\"BUG: invalid protocol_allow_config type\");\n@@ -743,7 +745,7 @@ int is_transport_allowed(const char *type)\n \n void transport_check_allowed(const char *type)\n {\n-\tif (!is_transport_allowed(type))\n+\tif (!is_transport_allowed(type, -1))\n \t\tdie(\"transport '%s' not allowed\", type);\n }\n \ndiff --git a/transport.h b/transport.h\nindex f4998bc..9820f10 100644\n--- a/transport.h\n+++ b/transport.h\n@@ -153,10 +153,17 @@ extern int transport_summary_width(const struct ref *refs);\n struct transport *transport_get(struct remote *, const char *);\n \n /*\n- * Check whether a transport is allowed by the environment. Type should\n- * generally be the URL scheme, as described in Documentation/git.txt\n+ * Check whether a transport is allowed by the environment.\n+ *\n+ * Type should generally be the URL scheme, as described in\n+ * Documentation/git.txt\n+ *\n+ * from_user specifies if the transport was given by the user. If unknown pass\n+ * a -1 to read from the environment to determine if the transport was given by\n+ * the user.\n+ *\n */\n-int is_transport_allowed(const char *type);\n+int is_transport_allowed(const char *type, int from_user);\n \n /*\n * Check whether a transport is allowed by the environment,\n-- \n2.8.0.rc3.226.g39d4020\n\n"},{"id":"306784","messageId":"1480636862-40489-3-git-send-email-bmwill@google.com","threadId":"44417","inReplyTo":"1480636862-40489-1-git-send-email-bmwill@google.com","subject":"[PATCH v8 2/5] transport: add protocol policy config option","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-12-02T00:00:59Z","receivedAt":"2016-12-02T00:01:39Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"Previously the `GIT_ALLOW_PROTOCOL` environment variable was used to\nspecify a whitelist of protocols to be used in clone/fetch/push\ncommands. This patch introduces new configuration options for more\nfine-grained control for allowing/disallowing protocols. This also has\nthe added benefit of allowing easier construction of a protocol\nwhitelist on systems where setting an environment variable is\nnon-trivial.\n\nNow users can specify a policy to be used for each type of protocol via\nthe 'protocol..allow' config option. A default policy for all\nunconfigured protocols can be set with the 'protocol.allow' config\noption. If no user configured default is made git will allow known-safe\nprotocols (http, https, git, ssh, file), disallow known-dangerous\nprotocols (ext), and have a default policy of `user` for all other\nprotocols.\n\nThe supported policies are `always`, `never`, and `user`. The `user`\npolicy can be used to configure a protocol to be usable when explicitly\nused by a user, while disallowing it for commands which run\nclone/fetch/push commands without direct user intervention (e.g.\nrecursive initialization of submodules). Commands which can potentially\nclone/fetch/push from untrusted repositories without user intervention\ncan export `GIT_PROTOCOL_FROM_USER` with a value of '0' to prevent\nprotocols configured to the `user` policy from being used.\n\nFix remote-ext tests to use the new config to allow the ext\nprotocol to be tested.\n\nBased on a patch by Jeff King \n\nSigned-off-by: Brandon Williams \n---\n Documentation/config.txt | 46 ++++++++++++++\n Documentation/git.txt | 38 +++++-------\n git-submodule.sh | 12 ++--\n t/lib-proto-disable.sh | 130 +++++++++++++++++++++++++++++++++++++--\n t/t5509-fetch-push-namespaces.sh | 1 +\n t/t5802-connect-helper.sh | 1 +\n transport.c | 75 +++++++++++++++++++++-\n 7 files changed, 264 insertions(+), 39 deletions(-)\n\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex 27069ac..5fe50bc 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -2308,6 +2308,52 @@ pretty.::\n \tNote that an alias with the same name as a built-in format\n \twill be silently ignored.\n \n+protocol.allow::\n+\tIf set, provide a user defined default policy for all protocols which\n+\tdon't explicitly have a policy (`protocol..allow`). By default,\n+\tif unset, known-safe protocols (http, https, git, ssh, file) have a\n+\tdefault policy of `always`, known-dangerous protocols (ext) have a\n+\tdefault policy of `never`, and all other protocols have a default\n+\tpolicy of `user`. Supported policies:\n++\n+--\n+\n+* `always` - protocol is always able to be used.\n+\n+* `never` - protocol is never able to be used.\n+\n+* `user` - protocol is only able to be used when `GIT_PROTOCOL_FROM_USER` is\n+ either unset or has a value of 1. This policy should be used when you want a\n+ protocol to be directly usable by the user but don't want it used by commands which\n+ execute clone/fetch/push commands without user input, e.g. recursive\n+ submodule initialization.\n+\n+--\n+\n+protocol..allow::\n+\tSet a policy to be used by protocol `` with clone/fetch/push\n+\tcommands. See `protocol.allow` above for the available policies.\n++\n+The protocol names currently used by git are:\n++\n+--\n+ - `file`: any local file-based path (including `file://` URLs,\n+ or local paths)\n+\n+ - `git`: the anonymous git protocol over a direct TCP\n+ connection (or proxy, if configured)\n+\n+ - `ssh`: git over ssh (including `host:path` syntax,\n+ `ssh://`, etc).\n+\n+ - `http`: git over http, both \"smart http\" and \"dumb http\".\n+ Note that this does _not_ include `https`; if you want to configure\n+ both, you must do so individually.\n+\n+ - any external helpers are named by their protocol (e.g., use\n+ `hg` to allow the `git-remote-hg` helper)\n+--\n+\n pull.ff::\n \tBy default, Git does not create an extra merge commit when merging\n \ta commit that is a descendant of the current commit. Instead, the\ndiff --git a/Documentation/git.txt b/Documentation/git.txt\nindex ab7215e..c52cec8 100644\n--- a/Documentation/git.txt\n+++ b/Documentation/git.txt\n@@ -1150,30 +1150,20 @@ of clones and fetches.\n \tcloning a repository to make a backup).\n \n `GIT_ALLOW_PROTOCOL`::\n-\tIf set, provide a colon-separated list of protocols which are\n-\tallowed to be used with fetch/push/clone. This is useful to\n-\trestrict recursive submodule initialization from an untrusted\n-\trepository. Any protocol not mentioned will be disallowed (i.e.,\n-\tthis is a whitelist, not a blacklist). If the variable is not\n-\tset at all, all protocols are enabled. The protocol names\n-\tcurrently used by git are:\n-\n-\t - `file`: any local file-based path (including `file://` URLs,\n-\t or local paths)\n-\n-\t - `git`: the anonymous git protocol over a direct TCP\n-\t connection (or proxy, if configured)\n-\n-\t - `ssh`: git over ssh (including `host:path` syntax,\n-\t `ssh://`, etc).\n-\n-\t - `http`: git over http, both \"smart http\" and \"dumb http\".\n-\t Note that this does _not_ include `https`; if you want both,\n-\t you should specify both as `http:https`.\n-\n-\t - any external helpers are named by their protocol (e.g., use\n-\t `hg` to allow the `git-remote-hg` helper)\n-\n+\tIf set to a colon-separated list of protocols, behave as if\n+\t`protocol.allow` is set to `never`, and each of the listed\n+\tprotocols has `protocol..allow` set to `always`\n+\t(overriding any existing configuration). In other words, any\n+\tprotocol not mentioned will be disallowed (i.e., this is a\n+\twhitelist, not a blacklist). See the description of\n+\t`protocol.allow` in linkgit:git-config[1] for more details.\n+\n+`GIT_PROTOCOL_FROM_USER`::\n+\tSet to 0 to prevent protocols used by fetch/push/clone which are\n+\tconfigured to the `user` state. This is useful to restrict recursive\n+\tsubmodule initialization from an untrusted repository or for programs\n+\twhich feed potentially-untrusted URLS to git commands. See\n+\tlinkgit:git-config[1] for more details.\n \n Discussion[[Discussion]]\n ------------------------\ndiff --git a/git-submodule.sh b/git-submodule.sh\nindex a024a13..0a477b4 100755\n--- a/git-submodule.sh\n+++ b/git-submodule.sh\n@@ -21,14 +21,10 @@ require_work_tree\n wt_prefix=$(git rev-parse --show-prefix)\n cd_to_toplevel\n \n-# Restrict ourselves to a vanilla subset of protocols; the URLs\n-# we get are under control of a remote repository, and we do not\n-# want them kicking off arbitrary git-remote-* programs.\n-#\n-# If the user has already specified a set of allowed protocols,\n-# we assume they know what they're doing and use that instead.\n-: ${GIT_ALLOW_PROTOCOL=file:git:http:https:ssh}\n-export GIT_ALLOW_PROTOCOL\n+# Tell the rest of git that any URLs we get don't come\n+# directly from the user, so it can apply policy as appropriate.\n+GIT_PROTOCOL_FROM_USER=0\n+export GIT_PROTOCOL_FROM_USER\n \n command=\n branch=\ndiff --git a/t/lib-proto-disable.sh b/t/lib-proto-disable.sh\nindex be88e9a..02f49cb 100644\n--- a/t/lib-proto-disable.sh\n+++ b/t/lib-proto-disable.sh\n@@ -1,10 +1,7 @@\n # Test routines for checking protocol disabling.\n \n-# test cloning a particular protocol\n-# $1 - description of the protocol\n-# $2 - machine-readable name of the protocol\n-# $3 - the URL to try cloning\n-test_proto () {\n+# Test clone/fetch/push with GIT_ALLOW_PROTOCOL whitelist\n+test_whitelist () {\n \tdesc=$1\n \tproto=$2\n \turl=$3\n@@ -62,6 +59,129 @@ test_proto () {\n \t\t\ttest_must_fail git clone --bare \"$url\" tmp.git\n \t\t)\n \t'\n+\n+\ttest_expect_success \"clone $desc (env var has precedence)\" '\n+\t\trm -rf tmp.git &&\n+\t\t(\n+\t\t\tGIT_ALLOW_PROTOCOL=none &&\n+\t\t\texport GIT_ALLOW_PROTOCOL &&\n+\t\t\ttest_must_fail git -c protocol.allow=always clone --bare \"$url\" tmp.git &&\n+\t\t\ttest_must_fail git -c protocol.$proto.allow=always clone --bare \"$url\" tmp.git\n+\t\t)\n+\t'\n+}\n+\n+test_config () {\n+\tdesc=$1\n+\tproto=$2\n+\turl=$3\n+\n+\t# Test clone/fetch/push with protocol..allow config\n+\ttest_expect_success \"clone $desc (enabled with config)\" '\n+\t\trm -rf tmp.git &&\n+\t\tgit -c protocol.$proto.allow=always clone --bare \"$url\" tmp.git\n+\t'\n+\n+\ttest_expect_success \"fetch $desc (enabled)\" '\n+\t\tgit -C tmp.git -c protocol.$proto.allow=always fetch\n+\t'\n+\n+\ttest_expect_success \"push $desc (enabled)\" '\n+\t\tgit -C tmp.git -c protocol.$proto.allow=always push origin HEAD:pushed\n+\t'\n+\n+\ttest_expect_success \"push $desc (disabled)\" '\n+\t\ttest_must_fail git -C tmp.git -c protocol.$proto.allow=never push origin HEAD:pushed\n+\t'\n+\n+\ttest_expect_success \"fetch $desc (disabled)\" '\n+\t\ttest_must_fail git -C tmp.git -c protocol.$proto.allow=never fetch\n+\t'\n+\n+\ttest_expect_success \"clone $desc (disabled)\" '\n+\t\trm -rf tmp.git &&\n+\t\ttest_must_fail git -c protocol.$proto.allow=never clone --bare \"$url\" tmp.git\n+\t'\n+\n+\t# Test clone/fetch/push with protocol.user.allow and its env var\n+\ttest_expect_success \"clone $desc (enabled)\" '\n+\t\trm -rf tmp.git &&\n+\t\tgit -c protocol.$proto.allow=user clone --bare \"$url\" tmp.git\n+\t'\n+\n+\ttest_expect_success \"fetch $desc (enabled)\" '\n+\t\tgit -C tmp.git -c protocol.$proto.allow=user fetch\n+\t'\n+\n+\ttest_expect_success \"push $desc (enabled)\" '\n+\t\tgit -C tmp.git -c protocol.$proto.allow=user push origin HEAD:pushed\n+\t'\n+\n+\ttest_expect_success \"push $desc (disabled)\" '\n+\t\t(\n+\t\t\tcd tmp.git &&\n+\t\t\tGIT_PROTOCOL_FROM_USER=0 &&\n+\t\t\texport GIT_PROTOCOL_FROM_USER &&\n+\t\t\ttest_must_fail git -c protocol.$proto.allow=user push origin HEAD:pushed\n+\t\t)\n+\t'\n+\n+\ttest_expect_success \"fetch $desc (disabled)\" '\n+\t\t(\n+\t\t\tcd tmp.git &&\n+\t\t\tGIT_PROTOCOL_FROM_USER=0 &&\n+\t\t\texport GIT_PROTOCOL_FROM_USER &&\n+\t\t\ttest_must_fail git -c protocol.$proto.allow=user fetch\n+\t\t)\n+\t'\n+\n+\ttest_expect_success \"clone $desc (disabled)\" '\n+\t\trm -rf tmp.git &&\n+\t\t(\n+\t\t\tGIT_PROTOCOL_FROM_USER=0 &&\n+\t\t\texport GIT_PROTOCOL_FROM_USER &&\n+\t\t\ttest_must_fail git -c protocol.$proto.allow=user clone --bare \"$url\" tmp.git\n+\t\t)\n+\t'\n+\n+\t# Test clone/fetch/push with protocol.allow user defined default\n+\ttest_expect_success \"clone $desc (enabled)\" '\n+\t\trm -rf tmp.git &&\n+\t\tgit config --global protocol.allow always &&\n+\t\tgit clone --bare \"$url\" tmp.git\n+\t'\n+\n+\ttest_expect_success \"fetch $desc (enabled)\" '\n+\t\tgit -C tmp.git fetch\n+\t'\n+\n+\ttest_expect_success \"push $desc (enabled)\" '\n+\t\tgit -C tmp.git push origin HEAD:pushed\n+\t'\n+\n+\ttest_expect_success \"push $desc (disabled)\" '\n+\t\tgit config --global protocol.allow never &&\n+\t\ttest_must_fail git -C tmp.git push origin HEAD:pushed\n+\t'\n+\n+\ttest_expect_success \"fetch $desc (disabled)\" '\n+\t\ttest_must_fail git -C tmp.git fetch\n+\t'\n+\n+\ttest_expect_success \"clone $desc (disabled)\" '\n+\t\trm -rf tmp.git &&\n+\t\ttest_must_fail git clone --bare \"$url\" tmp.git\n+\t'\n+}\n+\n+# test cloning a particular protocol\n+# $1 - description of the protocol\n+# $2 - machine-readable name of the protocol\n+# $3 - the URL to try cloning\n+test_proto () {\n+\ttest_whitelist \"$@\"\n+\n+\ttest_config \"$@\"\n }\n \n # set up an ssh wrapper that will access $host/$repo in the\ndiff --git a/t/t5509-fetch-push-namespaces.sh b/t/t5509-fetch-push-namespaces.sh\nindex bc44ac3..75c570a 100755\n--- a/t/t5509-fetch-push-namespaces.sh\n+++ b/t/t5509-fetch-push-namespaces.sh\n@@ -4,6 +4,7 @@ test_description='fetch/push involving ref namespaces'\n . ./test-lib.sh\n \n test_expect_success setup '\n+\tgit config --global protocol.ext.allow user &&\n \ttest_tick &&\n \tgit init original &&\n \t(\ndiff --git a/t/t5802-connect-helper.sh b/t/t5802-connect-helper.sh\nindex b7a7f9d..c6c2661 100755\n--- a/t/t5802-connect-helper.sh\n+++ b/t/t5802-connect-helper.sh\n@@ -4,6 +4,7 @@ test_description='ext::cmd remote \"connect\" helper'\n . ./test-lib.sh\n \n test_expect_success setup '\n+\tgit config --global protocol.ext.allow user &&\n \ttest_tick &&\n \tgit commit --allow-empty -m initial &&\n \ttest_tick &&\ndiff --git a/transport.c b/transport.c\nindex d57e8de..2c0ec76 100644\n--- a/transport.c\n+++ b/transport.c\n@@ -664,10 +664,81 @@ static const struct string_list *protocol_whitelist(void)\n \treturn enabled ? &allowed : NULL;\n }\n \n+enum protocol_allow_config {\n+\tPROTOCOL_ALLOW_NEVER = 0,\n+\tPROTOCOL_ALLOW_USER_ONLY,\n+\tPROTOCOL_ALLOW_ALWAYS\n+};\n+\n+static enum protocol_allow_config parse_protocol_config(const char *key,\n+\t\t\t\t\t\t\tconst char *value)\n+{\n+\tif (!strcasecmp(value, \"always\"))\n+\t\treturn PROTOCOL_ALLOW_ALWAYS;\n+\telse if (!strcasecmp(value, \"never\"))\n+\t\treturn PROTOCOL_ALLOW_NEVER;\n+\telse if (!strcasecmp(value, \"user\"))\n+\t\treturn PROTOCOL_ALLOW_USER_ONLY;\n+\n+\tdie(\"unknown value for config '%s': %s\", key, value);\n+}\n+\n+static enum protocol_allow_config get_protocol_config(const char *type)\n+{\n+\tchar *key = xstrfmt(\"protocol.%s.allow\", type);\n+\tchar *value;\n+\n+\t/* first check the per-protocol config */\n+\tif (!git_config_get_string(key, &value)) {\n+\t\tenum protocol_allow_config ret =\n+\t\t\tparse_protocol_config(key, value);\n+\t\tfree(key);\n+\t\tfree(value);\n+\t\treturn ret;\n+\t}\n+\tfree(key);\n+\n+\t/* if defined, fallback to user-defined default for unknown protocols */\n+\tif (!git_config_get_string(\"protocol.allow\", &value)) {\n+\t\tenum protocol_allow_config ret =\n+\t\t\tparse_protocol_config(\"protocol.allow\", value);\n+\t\tfree(value);\n+\t\treturn ret;\n+\t}\n+\n+\t/* fallback to built-in defaults */\n+\t/* known safe */\n+\tif (!strcmp(type, \"http\") ||\n+\t !strcmp(type, \"https\") ||\n+\t !strcmp(type, \"git\") ||\n+\t !strcmp(type, \"ssh\") ||\n+\t !strcmp(type, \"file\"))\n+\t\treturn PROTOCOL_ALLOW_ALWAYS;\n+\n+\t/* known scary; err on the side of caution */\n+\tif (!strcmp(type, \"ext\"))\n+\t\treturn PROTOCOL_ALLOW_NEVER;\n+\n+\t/* unknown; by default let them be used only directly by the user */\n+\treturn PROTOCOL_ALLOW_USER_ONLY;\n+}\n+\n int is_transport_allowed(const char *type)\n {\n-\tconst struct string_list *allowed = protocol_whitelist();\n-\treturn !allowed || string_list_has_string(allowed, type);\n+\tconst struct string_list *whitelist = protocol_whitelist();\n+\tif (whitelist)\n+\t\treturn string_list_has_string(whitelist, type);\n+\n+\tswitch (get_protocol_config(type)) {\n+\tcase PROTOCOL_ALLOW_ALWAYS:\n+\t\treturn 1;\n+\tcase PROTOCOL_ALLOW_NEVER:\n+\t\treturn 0;\n+\tcase PROTOCOL_ALLOW_USER_ONLY:\n+\t\treturn git_env_bool(\"GIT_PROTOCOL_FROM_USER\", 1);\n+\t}\n+\n+\tdie(\"BUG: invalid protocol_allow_config type\");\n }\n \n void transport_check_allowed(const char *type)\n-- \n2.8.0.rc3.226.g39d4020\n\n"},{"id":"306785","messageId":"1480636862-40489-5-git-send-email-bmwill@google.com","threadId":"44417","inReplyTo":"1480636862-40489-1-git-send-email-bmwill@google.com","subject":"[PATCH v8 4/5] http: create function to get curl allowed protocols","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-12-02T00:01:01Z","receivedAt":"2016-12-02T00:01:42Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"Move the creation of an allowed protocols whitelist to a helper\nfunction.\n\nSigned-off-by: Brandon Williams \n---\n http.c | 27 +++++++++++++++++----------\n 1 file changed, 17 insertions(+), 10 deletions(-)\n\ndiff --git a/http.c b/http.c\nindex fee128b..a1c3a0e 100644\n--- a/http.c\n+++ b/http.c\n@@ -624,11 +624,25 @@ void setup_curl_trace(CURL *handle)\n \tcurl_easy_setopt(handle, CURLOPT_DEBUGDATA, NULL);\n }\n \n+static long get_curl_allowed_protocols(void)\n+{\n+\tlong allowed_protocols = 0;\n+\n+\tif (is_transport_allowed(\"http\"))\n+\t\tallowed_protocols |= CURLPROTO_HTTP;\n+\tif (is_transport_allowed(\"https\"))\n+\t\tallowed_protocols |= CURLPROTO_HTTPS;\n+\tif (is_transport_allowed(\"ftp\"))\n+\t\tallowed_protocols |= CURLPROTO_FTP;\n+\tif (is_transport_allowed(\"ftps\"))\n+\t\tallowed_protocols |= CURLPROTO_FTPS;\n+\n+\treturn allowed_protocols;\n+}\n \n static CURL *get_curl_handle(void)\n {\n \tCURL *result = curl_easy_init();\n-\tlong allowed_protocols = 0;\n \n \tif (!result)\n \t\tdie(\"curl_easy_init failed\");\n@@ -725,15 +739,8 @@ static CURL *get_curl_handle(void)\n \tcurl_easy_setopt(result, CURLOPT_POST301, 1);\n #endif\n #if LIBCURL_VERSION_NUM >= 0x071304\n-\tif (is_transport_allowed(\"http\"))\n-\t\tallowed_protocols |= CURLPROTO_HTTP;\n-\tif (is_transport_allowed(\"https\"))\n-\t\tallowed_protocols |= CURLPROTO_HTTPS;\n-\tif (is_transport_allowed(\"ftp\"))\n-\t\tallowed_protocols |= CURLPROTO_FTP;\n-\tif (is_transport_allowed(\"ftps\"))\n-\t\tallowed_protocols |= CURLPROTO_FTPS;\n-\tcurl_easy_setopt(result, CURLOPT_REDIR_PROTOCOLS, allowed_protocols);\n+\tcurl_easy_setopt(result, CURLOPT_REDIR_PROTOCOLS,\n+\t\t\t get_curl_allowed_protocols());\n #else\n \twarning(\"protocol restrictions not applied to curl redirects because\\n\"\n \t\t\"your curl version is too old (>= 7.19.4)\");\n-- \n2.8.0.rc3.226.g39d4020\n\n"},{"id":"306789","messageId":"20161202001353.jiw4hjqg75dr6psw@sigill.intra.peff.net","threadId":"44417","inReplyTo":"20161201232656.GK54082@google.com","subject":"Re: [PATCH v7 4/4] transport: add from_user parameter to is_transport_allowed","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-12-02T00:13:53Z","receivedAt":"2016-12-02T00:21:00Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Dec 01, 2016 at 03:26:56PM -0800, Brandon Williams wrote:\n\n> > I started taking a look at your http redirect series (I really should\n> > have taking a look at it sooner) and I see exactly what you're talking\n> > about. We can easily move this logic into a function to make it easier\n> > to generate the two whitelists.\n> \n> Thinking about this some more...I was told that having http redirect to\n> file:// could be scary. The way the new protocol configuration is setup\n> we have file:// as a default known-safe protocol. Do we need to worry\n> about this or can we leave this be since this can be overridden by the\n> user?\n\nHmm. I'm not sure if file:// should actually be USER_ONLY. The submodule\ncode allows it, and it's certainly a convenience, but I guess you could\ndo tricky things by probing somebody's filesystem with submodules URLs.\nOn the other hand, if you are recursively cloning untrusted repos and\nhave sensitive contents on disk, you really _should_ be setting up a\nprotocol whitelist.\n\nFor HTTP redirects within curl, I think it's a non-issue; curl\nautomatically disallows file:// for redirects, even without us telling\nit so.\n\nFor redirects via http-alternates, it's a bit more tricky, as we feed\nthe URL to curl ourselves, so it can't tell the difference between\ntrusted and untrusted input. The main protection provided by my series\nis \"don't follow http-alternates at all\". But assuming you did want to\nuse them (by setting http.followRedirects to \"true\", at least for the\nserver in question), we could then feed file:// directly to curl. But I\nthink we are still OK, because the restricted CURLOPT_PROTOCOL setting\nwould prevent that from working. I.e., git _never_ wants curl to handle\nfile://, because it handles it without calling into remote-curl.c at\nall.\n\nSo arguably file:// should be USER_ONLY, but I'm not sure how much it\nmatters in practice.\n\n-Peff\n"},{"id":"306796","messageId":"20161202173349.GA117792@google.com","threadId":"44417","inReplyTo":"20161202001353.jiw4hjqg75dr6psw@sigill.intra.peff.net","subject":"Re: [PATCH v7 4/4] transport: add from_user parameter to is_transport_allowed","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-12-02T17:33:49Z","receivedAt":"2016-12-02T17:34:00Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 12/01, Jeff King wrote:\n> On Thu, Dec 01, 2016 at 03:26:56PM -0800, Brandon Williams wrote:\n> \n> > > I started taking a look at your http redirect series (I really should\n> > > have taking a look at it sooner) and I see exactly what you're talking\n> > > about. We can easily move this logic into a function to make it easier\n> > > to generate the two whitelists.\n> > \n> > Thinking about this some more...I was told that having http redirect to\n> > file:// could be scary. The way the new protocol configuration is setup\n> > we have file:// as a default known-safe protocol. Do we need to worry\n> > about this or can we leave this be since this can be overridden by the\n> > user?\n> \n> Hmm. I'm not sure if file:// should actually be USER_ONLY. The submodule\n> code allows it, and it's certainly a convenience, but I guess you could\n> do tricky things by probing somebody's filesystem with submodules URLs.\n> On the other hand, if you are recursively cloning untrusted repos and\n> have sensitive contents on disk, you really _should_ be setting up a\n> protocol whitelist.\n> \n> For HTTP redirects within curl, I think it's a non-issue; curl\n> automatically disallows file:// for redirects, even without us telling\n> it so.\n> \n> For redirects via http-alternates, it's a bit more tricky, as we feed\n> the URL to curl ourselves, so it can't tell the difference between\n> trusted and untrusted input. The main protection provided by my series\n> is \"don't follow http-alternates at all\". But assuming you did want to\n> use them (by setting http.followRedirects to \"true\", at least for the\n> server in question), we could then feed file:// directly to curl. But I\n> think we are still OK, because the restricted CURLOPT_PROTOCOL setting\n> would prevent that from working. I.e., git _never_ wants curl to handle\n> file://, because it handles it without calling into remote-curl.c at\n> all.\n> \n> So arguably file:// should be USER_ONLY, but I'm not sure how much it\n> matters in practice.\n\nAh ok thanks for the good explanation. I was mostly interested in the\nhttp redirect case which, as you said, becomes a non-issue due to how we\nconfigure curl.\n\nThanks!\n-- \nBrandon Williams\n"},{"id":"306952","messageId":"xmqqr35m3zx7.fsf@gitster.mtv.corp.google.com","threadId":"44417","inReplyTo":"20161201235856.GL54082@google.com","subject":"Re: [PATCH v7 4/4] transport: add from_user parameter to is_transport_allowed","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-12-05T20:04:52Z","receivedAt":"2016-12-05T20:05:23Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Brandon Williams writes:\n\n> On 12/01, Junio C Hamano wrote:\n>> Brandon Williams writes:\n>> \n>> > I started taking a look at your http redirect series (I really should\n>> > have taking a look at it sooner) and I see exactly what you're talking\n>> > about. We can easily move this logic into a function to make it easier\n>> > to generate the two whitelists.\n>> \n>> OK. With both of them queued, t5812 seems to barf, just FYI; I'll\n>> expect that a future reroll would make things better.\n>\n> Yeah I expected we would see an issue since both these series collide in\n> http.c\n>\n> I'm sending out another reroll of this series so that in Jeff's he can\n> just call 'get_curl_allowed_protocols(-1)' for the non-redirection curl\n> option, which should make this test stop barfing.\n\nI was hoping to eventually merge Peff's series to older maintenance\ntracks. How bad would it be if we rebased the v8 of this series\ntogether with Peff's series to say v2.9 (or even older if it does\nnot look too bad)?\n\n"},{"id":"306970","messageId":"20161205222211.GF68588@google.com","threadId":"44417","inReplyTo":"xmqqr35m3zx7.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH v7 4/4] transport: add from_user parameter to is_transport_allowed","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-12-05T22:22:11Z","receivedAt":"2016-12-05T22:22:19Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 12/05, Junio C Hamano wrote:\n> Brandon Williams writes:\n> \n> > On 12/01, Junio C Hamano wrote:\n> >> Brandon Williams writes:\n> >> \n> >> > I started taking a look at your http redirect series (I really should\n> >> > have taking a look at it sooner) and I see exactly what you're talking\n> >> > about. We can easily move this logic into a function to make it easier\n> >> > to generate the two whitelists.\n> >> \n> >> OK. With both of them queued, t5812 seems to barf, just FYI; I'll\n> >> expect that a future reroll would make things better.\n> >\n> > Yeah I expected we would see an issue since both these series collide in\n> > http.c\n> >\n> > I'm sending out another reroll of this series so that in Jeff's he can\n> > just call 'get_curl_allowed_protocols(-1)' for the non-redirection curl\n> > option, which should make this test stop barfing.\n> \n> I was hoping to eventually merge Peff's series to older maintenance\n> tracks. How bad would it be if we rebased the v8 of this series\n> together with Peff's series to say v2.9 (or even older if it does\n> not look too bad)?\n\nI just took Jeff's series and applied it on top of mine (and fixed the\nsmall problem causing t5812 to fail) and then rebased it to v2.9.0.\nThere were a few issues that needed to be resolved but nothing too\nhairy. So it would definitely be doable to backport it to the\nmaintenance tracks.\n\n-- \nBrandon Williams\n"},{"id":"306979","messageId":"xmqqa8ca2cc7.fsf@gitster.mtv.corp.google.com","threadId":"44417","inReplyTo":"20161205222211.GF68588@google.com","subject":"Re: [PATCH v7 4/4] transport: add from_user parameter to is_transport_allowed","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-12-05T23:19:36Z","receivedAt":"2016-12-05T23:19:53Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Brandon Williams writes:\n\n> I just took Jeff's series and applied it on top of mine (and fixed the\n> small problem causing t5812 to fail) and then rebased it to v2.9.0.\n> There were a few issues that needed to be resolved but nothing too\n> hairy. So it would definitely be doable to backport it to the\n> maintenance tracks.\n\nThanks for trying. I'd definitely prefer a series that is based on\nan older codebase that is merged to pu->next->master->maint (in\nother words, avoid \"backport\" and rather have \"forward merge\").\n\n"},{"id":"306980","messageId":"20161205232250.GA173397@google.com","threadId":"44417","inReplyTo":"xmqqa8ca2cc7.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH v7 4/4] transport: add from_user parameter to is_transport_allowed","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-12-05T23:22:50Z","receivedAt":"2016-12-05T23:23:00Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 12/05, Junio C Hamano wrote:\n> Brandon Williams writes:\n> \n> > I just took Jeff's series and applied it on top of mine (and fixed the\n> > small problem causing t5812 to fail) and then rebased it to v2.9.0.\n> > There were a few issues that needed to be resolved but nothing too\n> > hairy. So it would definitely be doable to backport it to the\n> > maintenance tracks.\n> \n> Thanks for trying. I'd definitely prefer a series that is based on\n> an older codebase that is merged to pu->next->master->maint (in\n> other words, avoid \"backport\" and rather have \"forward merge\").\n\nAh ok. Do you want me to send out the combined patch series based on\n2.9.0 (or some other point in time) then?\n\n-- \nBrandon Williams\n"},{"id":"307005","messageId":"20161206135113.i7nlr45vg7uzgfcn@sigill.intra.peff.net","threadId":"44417","inReplyTo":"xmqqr35m3zx7.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH v7 4/4] transport: add from_user parameter to is_transport_allowed","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-12-06T13:51:13Z","receivedAt":"2016-12-06T13:51:19Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Dec 05, 2016 at 12:04:52PM -0800, Junio C Hamano wrote:\n\n> > I'm sending out another reroll of this series so that in Jeff's he can\n> > just call 'get_curl_allowed_protocols(-1)' for the non-redirection curl\n> > option, which should make this test stop barfing.\n> \n> I was hoping to eventually merge Peff's series to older maintenance\n> tracks. How bad would it be if we rebased the v8 of this series\n> together with Peff's series to say v2.9 (or even older if it does\n> not look too bad)?\n\nMy series actually fixes existing security problems, so I'd consider it\na bug-fix. I _think_ Brandon's series is purely about allowing more\nexpressiveness in the whitelist policy, and so could be considered more\nof a feature.\n\nSo one option is to apply my series for older 'maint', and then just\nrebase Brandon's on top of that for 'master'.\n\nI don't know if that makes things any easier. I feel funny saying \"no,\nno, mine preempts yours because it is more maint-worthy\", but I think\nthat order does make sense.\n\nI think it would be OK to put Brandon's on maint, too, though. It is a\nrefactor of an existing security feature to make it more featureful, but\nthe way it is implemented could not cause security regressions unless\nyou use the new feature (IOW, we still respect the whitelist environment\nexactly as before).\n\n-Peff\n"},{"id":"307020","messageId":"xmqq60mx2bbi.fsf@gitster.mtv.corp.google.com","threadId":"44417","inReplyTo":"20161206135113.i7nlr45vg7uzgfcn@sigill.intra.peff.net","subject":"Re: [PATCH v7 4/4] transport: add from_user parameter to is_transport_allowed","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-12-06T17:53:53Z","receivedAt":"2016-12-06T17:54:02Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King writes:\n\n> I don't know if that makes things any easier. I feel funny saying \"no,\n> no, mine preempts yours because it is more maint-worthy\", but I think\n> that order does make sense.\n>\n> I think it would be OK to put Brandon's on maint, too, though. It is a\n> refactor of an existing security feature to make it more featureful, but\n> the way it is implemented could not cause security regressions unless\n> you use the new feature (IOW, we still respect the whitelist environment\n> exactly as before).\n\nI think I merged yours and then Brandon's on jch/pu branches in that\norder, and the conflict resolution should look OK.\n\nI however forked yours on v2.11.0-rc1, which would need to be\nrebased to one of the earlier maintenance tracks, before we can\nmerge it to 'next'.\n\n\n"},{"id":"307022","messageId":"20161206181008.yaz2md3343pukaov@sigill.intra.peff.net","threadId":"44417","inReplyTo":"xmqq60mx2bbi.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH v7 4/4] transport: add from_user parameter to is_transport_allowed","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-12-06T18:10:09Z","receivedAt":"2016-12-06T18:10:17Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Dec 06, 2016 at 09:53:53AM -0800, Junio C Hamano wrote:\n\n> Jeff King writes:\n> \n> > I don't know if that makes things any easier. I feel funny saying \"no,\n> > no, mine preempts yours because it is more maint-worthy\", but I think\n> > that order does make sense.\n> >\n> > I think it would be OK to put Brandon's on maint, too, though. It is a\n> > refactor of an existing security feature to make it more featureful, but\n> > the way it is implemented could not cause security regressions unless\n> > you use the new feature (IOW, we still respect the whitelist environment\n> > exactly as before).\n> \n> I think I merged yours and then Brandon's on jch/pu branches in that\n> order, and the conflict resolution should look OK.\n> \n> I however forked yours on v2.11.0-rc1, which would need to be\n> rebased to one of the earlier maintenance tracks, before we can\n> merge it to 'next'.\n\nYeah, I built it on top of master.\n\nIt does depend on some of the http-walker changes Eric made a few months\nago. In particular, 17966c0a6 (http: avoid disconnecting on 404s for\nloose objects, 2016-07-11) added some checks against the HTTP status\ncode, and my series modifies the checks (mostly so that \">= 400\" becomes\n\">= 300\").\n\nRebasing on maint-2.9 means omitting those changes. That preserves the\nsecurity properties, but means that the error handling is worse when we\nsee an illegal redirect. That may be OK, though.\n\nSince the resolution is to omit the changes entirely from my series,\nmerging up to v2.11 wouldn't produce any conflicts. We'd need to have a\nseparate set of patches adding those changes back in.\n\n-Peff\n"},{"id":"307074","messageId":"20161206222416.GB103573@google.com","threadId":"44417","inReplyTo":"20161206135113.i7nlr45vg7uzgfcn@sigill.intra.peff.net","subject":"Re: [PATCH v7 4/4] transport: add from_user parameter to is_transport_allowed","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-12-06T22:24:16Z","receivedAt":"2016-12-06T22:24:25Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 12/06, Jeff King wrote:\n> On Mon, Dec 05, 2016 at 12:04:52PM -0800, Junio C Hamano wrote:\n> \n> > > I'm sending out another reroll of this series so that in Jeff's he can\n> > > just call 'get_curl_allowed_protocols(-1)' for the non-redirection curl\n> > > option, which should make this test stop barfing.\n> > \n> > I was hoping to eventually merge Peff's series to older maintenance\n> > tracks. How bad would it be if we rebased the v8 of this series\n> > together with Peff's series to say v2.9 (or even older if it does\n> > not look too bad)?\n> \n> My series actually fixes existing security problems, so I'd consider it\n> a bug-fix. I _think_ Brandon's series is purely about allowing more\n> expressiveness in the whitelist policy, and so could be considered more\n> of a feature.\n\nYes this was really the main intent on my series.\n\n> So one option is to apply my series for older 'maint', and then just\n> rebase Brandon's on top of that for 'master'.\n> \n> I don't know if that makes things any easier. I feel funny saying \"no,\n> no, mine preempts yours because it is more maint-worthy\", but I think\n> that order does make sense.\n> \n> I think it would be OK to put Brandon's on maint, too, though. It is a\n> refactor of an existing security feature to make it more featureful, but\n> the way it is implemented could not cause security regressions unless\n> you use the new feature (IOW, we still respect the whitelist environment\n> exactly as before).\n\nEither way let me know if there is something I need to do.\n\n-- \nBrandon Williams\n"},{"id":"307729","messageId":"1481679637-133137-2-git-send-email-bmwill@google.com","threadId":"44417","inReplyTo":"1481679637-133137-1-git-send-email-bmwill@google.com","subject":"[PATCH v9 1/5] lib-proto-disable: variable name fix","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-12-14T01:40:33Z","receivedAt":"2016-12-14T01:42:26Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"The test_proto function assigns the positional parameters to named\nvariables, but then still refers to \"$desc\" as \"$1\". Using $desc is\nmore readable and less error-prone.\n\nSigned-off-by: Brandon Williams \n---\n t/lib-proto-disable.sh | 12 ++++++------\n 1 file changed, 6 insertions(+), 6 deletions(-)\n\ndiff --git a/t/lib-proto-disable.sh b/t/lib-proto-disable.sh\nindex b0917d9..be88e9a 100644\n--- a/t/lib-proto-disable.sh\n+++ b/t/lib-proto-disable.sh\n@@ -9,7 +9,7 @@ test_proto () {\n \tproto=$2\n \turl=$3\n \n-\ttest_expect_success \"clone $1 (enabled)\" '\n+\ttest_expect_success \"clone $desc (enabled)\" '\n \t\trm -rf tmp.git &&\n \t\t(\n \t\t\tGIT_ALLOW_PROTOCOL=$proto &&\n@@ -18,7 +18,7 @@ test_proto () {\n \t\t)\n \t'\n \n-\ttest_expect_success \"fetch $1 (enabled)\" '\n+\ttest_expect_success \"fetch $desc (enabled)\" '\n \t\t(\n \t\t\tcd tmp.git &&\n \t\t\tGIT_ALLOW_PROTOCOL=$proto &&\n@@ -27,7 +27,7 @@ test_proto () {\n \t\t)\n \t'\n \n-\ttest_expect_success \"push $1 (enabled)\" '\n+\ttest_expect_success \"push $desc (enabled)\" '\n \t\t(\n \t\t\tcd tmp.git &&\n \t\t\tGIT_ALLOW_PROTOCOL=$proto &&\n@@ -36,7 +36,7 @@ test_proto () {\n \t\t)\n \t'\n \n-\ttest_expect_success \"push $1 (disabled)\" '\n+\ttest_expect_success \"push $desc (disabled)\" '\n \t\t(\n \t\t\tcd tmp.git &&\n \t\t\tGIT_ALLOW_PROTOCOL=none &&\n@@ -45,7 +45,7 @@ test_proto () {\n \t\t)\n \t'\n \n-\ttest_expect_success \"fetch $1 (disabled)\" '\n+\ttest_expect_success \"fetch $desc (disabled)\" '\n \t\t(\n \t\t\tcd tmp.git &&\n \t\t\tGIT_ALLOW_PROTOCOL=none &&\n@@ -54,7 +54,7 @@ test_proto () {\n \t\t)\n \t'\n \n-\ttest_expect_success \"clone $1 (disabled)\" '\n+\ttest_expect_success \"clone $desc (disabled)\" '\n \t\trm -rf tmp.git &&\n \t\t(\n \t\t\tGIT_ALLOW_PROTOCOL=none &&\n-- \n2.8.0.rc3.226.g39d4020\n\n"},{"id":"307730","messageId":"1481679637-133137-1-git-send-email-bmwill@google.com","threadId":"44417","inReplyTo":"1480636862-40489-1-git-send-email-bmwill@google.com","subject":"[PATCH v9 0/5] transport protocol policy configuration","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-12-14T01:40:32Z","receivedAt":"2016-12-14T01:42:30Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"Only difference between v8 and v9 is that v9 has been rebased ontop of Jeff's\nhttp-walker-limit-redirect series 'jk/http-walker-limit-redirect'.\n\nBrandon Williams (5):\n lib-proto-disable: variable name fix\n transport: add protocol policy config option\n http: always warn if libcurl version is too old\n http: create function to get curl allowed protocols\n transport: add from_user parameter to is_transport_allowed\n\n Documentation/config.txt | 46 +++++++++++++\n Documentation/git.txt | 38 ++++-------\n git-submodule.sh | 12 ++--\n http.c | 36 ++++++----\n t/lib-proto-disable.sh | 142 ++++++++++++++++++++++++++++++++++++---\n t/t5509-fetch-push-namespaces.sh | 1 +\n t/t5802-connect-helper.sh | 1 +\n transport.c | 84 ++++++++++++++++++++---\n transport.h | 19 +++---\n 9 files changed, 305 insertions(+), 74 deletions(-)\n\n-- \n2.8.0.rc3.226.g39d4020\n\n"},{"id":"307731","messageId":"1481679637-133137-3-git-send-email-bmwill@google.com","threadId":"44417","inReplyTo":"1481679637-133137-1-git-send-email-bmwill@google.com","subject":"[PATCH v9 2/5] transport: add protocol policy config option","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-12-14T01:40:34Z","receivedAt":"2016-12-14T01:42:33Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"Previously the `GIT_ALLOW_PROTOCOL` environment variable was used to\nspecify a whitelist of protocols to be used in clone/fetch/push\ncommands. This patch introduces new configuration options for more\nfine-grained control for allowing/disallowing protocols. This also has\nthe added benefit of allowing easier construction of a protocol\nwhitelist on systems where setting an environment variable is\nnon-trivial.\n\nNow users can specify a policy to be used for each type of protocol via\nthe 'protocol..allow' config option. A default policy for all\nunconfigured protocols can be set with the 'protocol.allow' config\noption. If no user configured default is made git will allow known-safe\nprotocols (http, https, git, ssh, file), disallow known-dangerous\nprotocols (ext), and have a default policy of `user` for all other\nprotocols.\n\nThe supported policies are `always`, `never`, and `user`. The `user`\npolicy can be used to configure a protocol to be usable when explicitly\nused by a user, while disallowing it for commands which run\nclone/fetch/push commands without direct user intervention (e.g.\nrecursive initialization of submodules). Commands which can potentially\nclone/fetch/push from untrusted repositories without user intervention\ncan export `GIT_PROTOCOL_FROM_USER` with a value of '0' to prevent\nprotocols configured to the `user` policy from being used.\n\nFix remote-ext tests to use the new config to allow the ext\nprotocol to be tested.\n\nBased on a patch by Jeff King \n\nSigned-off-by: Brandon Williams \n---\n Documentation/config.txt | 46 ++++++++++++++\n Documentation/git.txt | 38 +++++-------\n git-submodule.sh | 12 ++--\n t/lib-proto-disable.sh | 130 +++++++++++++++++++++++++++++++++++++--\n t/t5509-fetch-push-namespaces.sh | 1 +\n t/t5802-connect-helper.sh | 1 +\n transport.c | 75 +++++++++++++++++++++-\n 7 files changed, 264 insertions(+), 39 deletions(-)\n\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex 8153336..50d3d06 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -2260,6 +2260,52 @@ pretty.::\n \tNote that an alias with the same name as a built-in format\n \twill be silently ignored.\n \n+protocol.allow::\n+\tIf set, provide a user defined default policy for all protocols which\n+\tdon't explicitly have a policy (`protocol..allow`). By default,\n+\tif unset, known-safe protocols (http, https, git, ssh, file) have a\n+\tdefault policy of `always`, known-dangerous protocols (ext) have a\n+\tdefault policy of `never`, and all other protocols have a default\n+\tpolicy of `user`. Supported policies:\n++\n+--\n+\n+* `always` - protocol is always able to be used.\n+\n+* `never` - protocol is never able to be used.\n+\n+* `user` - protocol is only able to be used when `GIT_PROTOCOL_FROM_USER` is\n+ either unset or has a value of 1. This policy should be used when you want a\n+ protocol to be directly usable by the user but don't want it used by commands which\n+ execute clone/fetch/push commands without user input, e.g. recursive\n+ submodule initialization.\n+\n+--\n+\n+protocol..allow::\n+\tSet a policy to be used by protocol `` with clone/fetch/push\n+\tcommands. See `protocol.allow` above for the available policies.\n++\n+The protocol names currently used by git are:\n++\n+--\n+ - `file`: any local file-based path (including `file://` URLs,\n+ or local paths)\n+\n+ - `git`: the anonymous git protocol over a direct TCP\n+ connection (or proxy, if configured)\n+\n+ - `ssh`: git over ssh (including `host:path` syntax,\n+ `ssh://`, etc).\n+\n+ - `http`: git over http, both \"smart http\" and \"dumb http\".\n+ Note that this does _not_ include `https`; if you want to configure\n+ both, you must do so individually.\n+\n+ - any external helpers are named by their protocol (e.g., use\n+ `hg` to allow the `git-remote-hg` helper)\n+--\n+\n pull.ff::\n \tBy default, Git does not create an extra merge commit when merging\n \ta commit that is a descendant of the current commit. Instead, the\ndiff --git a/Documentation/git.txt b/Documentation/git.txt\nindex 923aa49..d9fb937 100644\n--- a/Documentation/git.txt\n+++ b/Documentation/git.txt\n@@ -1129,30 +1129,20 @@ of clones and fetches.\n \tcloning a repository to make a backup).\n \n `GIT_ALLOW_PROTOCOL`::\n-\tIf set, provide a colon-separated list of protocols which are\n-\tallowed to be used with fetch/push/clone. This is useful to\n-\trestrict recursive submodule initialization from an untrusted\n-\trepository. Any protocol not mentioned will be disallowed (i.e.,\n-\tthis is a whitelist, not a blacklist). If the variable is not\n-\tset at all, all protocols are enabled. The protocol names\n-\tcurrently used by git are:\n-\n-\t - `file`: any local file-based path (including `file://` URLs,\n-\t or local paths)\n-\n-\t - `git`: the anonymous git protocol over a direct TCP\n-\t connection (or proxy, if configured)\n-\n-\t - `ssh`: git over ssh (including `host:path` syntax,\n-\t `ssh://`, etc).\n-\n-\t - `http`: git over http, both \"smart http\" and \"dumb http\".\n-\t Note that this does _not_ include `https`; if you want both,\n-\t you should specify both as `http:https`.\n-\n-\t - any external helpers are named by their protocol (e.g., use\n-\t `hg` to allow the `git-remote-hg` helper)\n-\n+\tIf set to a colon-separated list of protocols, behave as if\n+\t`protocol.allow` is set to `never`, and each of the listed\n+\tprotocols has `protocol..allow` set to `always`\n+\t(overriding any existing configuration). In other words, any\n+\tprotocol not mentioned will be disallowed (i.e., this is a\n+\twhitelist, not a blacklist). See the description of\n+\t`protocol.allow` in linkgit:git-config[1] for more details.\n+\n+`GIT_PROTOCOL_FROM_USER`::\n+\tSet to 0 to prevent protocols used by fetch/push/clone which are\n+\tconfigured to the `user` state. This is useful to restrict recursive\n+\tsubmodule initialization from an untrusted repository or for programs\n+\twhich feed potentially-untrusted URLS to git commands. See\n+\tlinkgit:git-config[1] for more details.\n \n Discussion[[Discussion]]\n ------------------------\ndiff --git a/git-submodule.sh b/git-submodule.sh\nindex 78fdac9..fc44076 100755\n--- a/git-submodule.sh\n+++ b/git-submodule.sh\n@@ -22,14 +22,10 @@ require_work_tree\n wt_prefix=$(git rev-parse --show-prefix)\n cd_to_toplevel\n \n-# Restrict ourselves to a vanilla subset of protocols; the URLs\n-# we get are under control of a remote repository, and we do not\n-# want them kicking off arbitrary git-remote-* programs.\n-#\n-# If the user has already specified a set of allowed protocols,\n-# we assume they know what they're doing and use that instead.\n-: ${GIT_ALLOW_PROTOCOL=file:git:http:https:ssh}\n-export GIT_ALLOW_PROTOCOL\n+# Tell the rest of git that any URLs we get don't come\n+# directly from the user, so it can apply policy as appropriate.\n+GIT_PROTOCOL_FROM_USER=0\n+export GIT_PROTOCOL_FROM_USER\n \n command=\n branch=\ndiff --git a/t/lib-proto-disable.sh b/t/lib-proto-disable.sh\nindex be88e9a..02f49cb 100644\n--- a/t/lib-proto-disable.sh\n+++ b/t/lib-proto-disable.sh\n@@ -1,10 +1,7 @@\n # Test routines for checking protocol disabling.\n \n-# test cloning a particular protocol\n-# $1 - description of the protocol\n-# $2 - machine-readable name of the protocol\n-# $3 - the URL to try cloning\n-test_proto () {\n+# Test clone/fetch/push with GIT_ALLOW_PROTOCOL whitelist\n+test_whitelist () {\n \tdesc=$1\n \tproto=$2\n \turl=$3\n@@ -62,6 +59,129 @@ test_proto () {\n \t\t\ttest_must_fail git clone --bare \"$url\" tmp.git\n \t\t)\n \t'\n+\n+\ttest_expect_success \"clone $desc (env var has precedence)\" '\n+\t\trm -rf tmp.git &&\n+\t\t(\n+\t\t\tGIT_ALLOW_PROTOCOL=none &&\n+\t\t\texport GIT_ALLOW_PROTOCOL &&\n+\t\t\ttest_must_fail git -c protocol.allow=always clone --bare \"$url\" tmp.git &&\n+\t\t\ttest_must_fail git -c protocol.$proto.allow=always clone --bare \"$url\" tmp.git\n+\t\t)\n+\t'\n+}\n+\n+test_config () {\n+\tdesc=$1\n+\tproto=$2\n+\turl=$3\n+\n+\t# Test clone/fetch/push with protocol..allow config\n+\ttest_expect_success \"clone $desc (enabled with config)\" '\n+\t\trm -rf tmp.git &&\n+\t\tgit -c protocol.$proto.allow=always clone --bare \"$url\" tmp.git\n+\t'\n+\n+\ttest_expect_success \"fetch $desc (enabled)\" '\n+\t\tgit -C tmp.git -c protocol.$proto.allow=always fetch\n+\t'\n+\n+\ttest_expect_success \"push $desc (enabled)\" '\n+\t\tgit -C tmp.git -c protocol.$proto.allow=always push origin HEAD:pushed\n+\t'\n+\n+\ttest_expect_success \"push $desc (disabled)\" '\n+\t\ttest_must_fail git -C tmp.git -c protocol.$proto.allow=never push origin HEAD:pushed\n+\t'\n+\n+\ttest_expect_success \"fetch $desc (disabled)\" '\n+\t\ttest_must_fail git -C tmp.git -c protocol.$proto.allow=never fetch\n+\t'\n+\n+\ttest_expect_success \"clone $desc (disabled)\" '\n+\t\trm -rf tmp.git &&\n+\t\ttest_must_fail git -c protocol.$proto.allow=never clone --bare \"$url\" tmp.git\n+\t'\n+\n+\t# Test clone/fetch/push with protocol.user.allow and its env var\n+\ttest_expect_success \"clone $desc (enabled)\" '\n+\t\trm -rf tmp.git &&\n+\t\tgit -c protocol.$proto.allow=user clone --bare \"$url\" tmp.git\n+\t'\n+\n+\ttest_expect_success \"fetch $desc (enabled)\" '\n+\t\tgit -C tmp.git -c protocol.$proto.allow=user fetch\n+\t'\n+\n+\ttest_expect_success \"push $desc (enabled)\" '\n+\t\tgit -C tmp.git -c protocol.$proto.allow=user push origin HEAD:pushed\n+\t'\n+\n+\ttest_expect_success \"push $desc (disabled)\" '\n+\t\t(\n+\t\t\tcd tmp.git &&\n+\t\t\tGIT_PROTOCOL_FROM_USER=0 &&\n+\t\t\texport GIT_PROTOCOL_FROM_USER &&\n+\t\t\ttest_must_fail git -c protocol.$proto.allow=user push origin HEAD:pushed\n+\t\t)\n+\t'\n+\n+\ttest_expect_success \"fetch $desc (disabled)\" '\n+\t\t(\n+\t\t\tcd tmp.git &&\n+\t\t\tGIT_PROTOCOL_FROM_USER=0 &&\n+\t\t\texport GIT_PROTOCOL_FROM_USER &&\n+\t\t\ttest_must_fail git -c protocol.$proto.allow=user fetch\n+\t\t)\n+\t'\n+\n+\ttest_expect_success \"clone $desc (disabled)\" '\n+\t\trm -rf tmp.git &&\n+\t\t(\n+\t\t\tGIT_PROTOCOL_FROM_USER=0 &&\n+\t\t\texport GIT_PROTOCOL_FROM_USER &&\n+\t\t\ttest_must_fail git -c protocol.$proto.allow=user clone --bare \"$url\" tmp.git\n+\t\t)\n+\t'\n+\n+\t# Test clone/fetch/push with protocol.allow user defined default\n+\ttest_expect_success \"clone $desc (enabled)\" '\n+\t\trm -rf tmp.git &&\n+\t\tgit config --global protocol.allow always &&\n+\t\tgit clone --bare \"$url\" tmp.git\n+\t'\n+\n+\ttest_expect_success \"fetch $desc (enabled)\" '\n+\t\tgit -C tmp.git fetch\n+\t'\n+\n+\ttest_expect_success \"push $desc (enabled)\" '\n+\t\tgit -C tmp.git push origin HEAD:pushed\n+\t'\n+\n+\ttest_expect_success \"push $desc (disabled)\" '\n+\t\tgit config --global protocol.allow never &&\n+\t\ttest_must_fail git -C tmp.git push origin HEAD:pushed\n+\t'\n+\n+\ttest_expect_success \"fetch $desc (disabled)\" '\n+\t\ttest_must_fail git -C tmp.git fetch\n+\t'\n+\n+\ttest_expect_success \"clone $desc (disabled)\" '\n+\t\trm -rf tmp.git &&\n+\t\ttest_must_fail git clone --bare \"$url\" tmp.git\n+\t'\n+}\n+\n+# test cloning a particular protocol\n+# $1 - description of the protocol\n+# $2 - machine-readable name of the protocol\n+# $3 - the URL to try cloning\n+test_proto () {\n+\ttest_whitelist \"$@\"\n+\n+\ttest_config \"$@\"\n }\n \n # set up an ssh wrapper that will access $host/$repo in the\ndiff --git a/t/t5509-fetch-push-namespaces.sh b/t/t5509-fetch-push-namespaces.sh\nindex bc44ac3..75c570a 100755\n--- a/t/t5509-fetch-push-namespaces.sh\n+++ b/t/t5509-fetch-push-namespaces.sh\n@@ -4,6 +4,7 @@ test_description='fetch/push involving ref namespaces'\n . ./test-lib.sh\n \n test_expect_success setup '\n+\tgit config --global protocol.ext.allow user &&\n \ttest_tick &&\n \tgit init original &&\n \t(\ndiff --git a/t/t5802-connect-helper.sh b/t/t5802-connect-helper.sh\nindex b7a7f9d..c6c2661 100755\n--- a/t/t5802-connect-helper.sh\n+++ b/t/t5802-connect-helper.sh\n@@ -4,6 +4,7 @@ test_description='ext::cmd remote \"connect\" helper'\n . ./test-lib.sh\n \n test_expect_success setup '\n+\tgit config --global protocol.ext.allow user &&\n \ttest_tick &&\n \tgit commit --allow-empty -m initial &&\n \ttest_tick &&\ndiff --git a/transport.c b/transport.c\nindex 41eb82c..e1ba78b 100644\n--- a/transport.c\n+++ b/transport.c\n@@ -617,10 +617,81 @@ static const struct string_list *protocol_whitelist(void)\n \treturn enabled ? &allowed : NULL;\n }\n \n+enum protocol_allow_config {\n+\tPROTOCOL_ALLOW_NEVER = 0,\n+\tPROTOCOL_ALLOW_USER_ONLY,\n+\tPROTOCOL_ALLOW_ALWAYS\n+};\n+\n+static enum protocol_allow_config parse_protocol_config(const char *key,\n+\t\t\t\t\t\t\tconst char *value)\n+{\n+\tif (!strcasecmp(value, \"always\"))\n+\t\treturn PROTOCOL_ALLOW_ALWAYS;\n+\telse if (!strcasecmp(value, \"never\"))\n+\t\treturn PROTOCOL_ALLOW_NEVER;\n+\telse if (!strcasecmp(value, \"user\"))\n+\t\treturn PROTOCOL_ALLOW_USER_ONLY;\n+\n+\tdie(\"unknown value for config '%s': %s\", key, value);\n+}\n+\n+static enum protocol_allow_config get_protocol_config(const char *type)\n+{\n+\tchar *key = xstrfmt(\"protocol.%s.allow\", type);\n+\tchar *value;\n+\n+\t/* first check the per-protocol config */\n+\tif (!git_config_get_string(key, &value)) {\n+\t\tenum protocol_allow_config ret =\n+\t\t\tparse_protocol_config(key, value);\n+\t\tfree(key);\n+\t\tfree(value);\n+\t\treturn ret;\n+\t}\n+\tfree(key);\n+\n+\t/* if defined, fallback to user-defined default for unknown protocols */\n+\tif (!git_config_get_string(\"protocol.allow\", &value)) {\n+\t\tenum protocol_allow_config ret =\n+\t\t\tparse_protocol_config(\"protocol.allow\", value);\n+\t\tfree(value);\n+\t\treturn ret;\n+\t}\n+\n+\t/* fallback to built-in defaults */\n+\t/* known safe */\n+\tif (!strcmp(type, \"http\") ||\n+\t !strcmp(type, \"https\") ||\n+\t !strcmp(type, \"git\") ||\n+\t !strcmp(type, \"ssh\") ||\n+\t !strcmp(type, \"file\"))\n+\t\treturn PROTOCOL_ALLOW_ALWAYS;\n+\n+\t/* known scary; err on the side of caution */\n+\tif (!strcmp(type, \"ext\"))\n+\t\treturn PROTOCOL_ALLOW_NEVER;\n+\n+\t/* unknown; by default let them be used only directly by the user */\n+\treturn PROTOCOL_ALLOW_USER_ONLY;\n+}\n+\n int is_transport_allowed(const char *type)\n {\n-\tconst struct string_list *allowed = protocol_whitelist();\n-\treturn !allowed || string_list_has_string(allowed, type);\n+\tconst struct string_list *whitelist = protocol_whitelist();\n+\tif (whitelist)\n+\t\treturn string_list_has_string(whitelist, type);\n+\n+\tswitch (get_protocol_config(type)) {\n+\tcase PROTOCOL_ALLOW_ALWAYS:\n+\t\treturn 1;\n+\tcase PROTOCOL_ALLOW_NEVER:\n+\t\treturn 0;\n+\tcase PROTOCOL_ALLOW_USER_ONLY:\n+\t\treturn git_env_bool(\"GIT_PROTOCOL_FROM_USER\", 1);\n+\t}\n+\n+\tdie(\"BUG: invalid protocol_allow_config type\");\n }\n \n void transport_check_allowed(const char *type)\n-- \n2.8.0.rc3.226.g39d4020\n\n"},{"id":"307732","messageId":"1481679637-133137-4-git-send-email-bmwill@google.com","threadId":"44417","inReplyTo":"1481679637-133137-1-git-send-email-bmwill@google.com","subject":"[PATCH v9 3/5] http: always warn if libcurl version is too old","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-12-14T01:40:35Z","receivedAt":"2016-12-14T01:50:51Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"Now that there are default \"known-good\" and \"known-bad\" protocols which\nare allowed/disallowed by 'is_transport_allowed' we should always warn\nthe user that older versions of libcurl can't respect the allowed\nprotocols for redirects.\n\nSigned-off-by: Brandon Williams \n---\n http.c | 5 ++---\n transport.c | 5 -----\n transport.h | 6 ------\n 3 files changed, 2 insertions(+), 14 deletions(-)\n\ndiff --git a/http.c b/http.c\nindex 5cd3ffd..034426e 100644\n--- a/http.c\n+++ b/http.c\n@@ -583,9 +583,8 @@ static CURL *get_curl_handle(void)\n \tcurl_easy_setopt(result, CURLOPT_REDIR_PROTOCOLS, allowed_protocols);\n \tcurl_easy_setopt(result, CURLOPT_PROTOCOLS, allowed_protocols);\n #else\n-\tif (transport_restrict_protocols())\n-\t\twarning(\"protocol restrictions not applied to curl redirects because\\n\"\n-\t\t\t\"your curl version is too old (>= 7.19.4)\");\n+\twarning(\"protocol restrictions not applied to curl redirects because\\n\"\n+\t\t\"your curl version is too old (>= 7.19.4)\");\n #endif\n \n \tif (getenv(\"GIT_CURL_VERBOSE\"))\ndiff --git a/transport.c b/transport.c\nindex e1ba78b..fbd799d 100644\n--- a/transport.c\n+++ b/transport.c\n@@ -700,11 +700,6 @@ void transport_check_allowed(const char *type)\n \t\tdie(\"transport '%s' not allowed\", type);\n }\n \n-int transport_restrict_protocols(void)\n-{\n-\treturn !!protocol_whitelist();\n-}\n-\n struct transport *transport_get(struct remote *remote, const char *url)\n {\n \tconst char *helper;\ndiff --git a/transport.h b/transport.h\nindex c681408..3396e1d 100644\n--- a/transport.h\n+++ b/transport.h\n@@ -153,12 +153,6 @@ int is_transport_allowed(const char *type);\n */\n void transport_check_allowed(const char *type);\n \n-/*\n- * Returns true if the user has attempted to turn on protocol\n- * restrictions at all.\n- */\n-int transport_restrict_protocols(void);\n-\n /* Transport options which apply to git:// and scp-style URLs */\n \n /* The program to use on the remote side to send a pack */\n-- \n2.8.0.rc3.226.g39d4020\n\n"},{"id":"307733","messageId":"1481679637-133137-5-git-send-email-bmwill@google.com","threadId":"44417","inReplyTo":"1481679637-133137-1-git-send-email-bmwill@google.com","subject":"[PATCH v9 4/5] http: create function to get curl allowed protocols","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-12-14T01:40:36Z","receivedAt":"2016-12-14T01:50:54Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"Move the creation of an allowed protocols whitelist to a helper\nfunction.\n\nSigned-off-by: Brandon Williams \n---\n http.c | 31 ++++++++++++++++++++-----------\n 1 file changed, 20 insertions(+), 11 deletions(-)\n\ndiff --git a/http.c b/http.c\nindex 034426e..f7c488a 100644\n--- a/http.c\n+++ b/http.c\n@@ -489,10 +489,25 @@ static void set_curl_keepalive(CURL *c)\n }\n #endif\n \n+static long get_curl_allowed_protocols(void)\n+{\n+\tlong allowed_protocols = 0;\n+\n+\tif (is_transport_allowed(\"http\"))\n+\t\tallowed_protocols |= CURLPROTO_HTTP;\n+\tif (is_transport_allowed(\"https\"))\n+\t\tallowed_protocols |= CURLPROTO_HTTPS;\n+\tif (is_transport_allowed(\"ftp\"))\n+\t\tallowed_protocols |= CURLPROTO_FTP;\n+\tif (is_transport_allowed(\"ftps\"))\n+\t\tallowed_protocols |= CURLPROTO_FTPS;\n+\n+\treturn allowed_protocols;\n+}\n+\n static CURL *get_curl_handle(void)\n {\n \tCURL *result = curl_easy_init();\n-\tlong allowed_protocols = 0;\n \n \tif (!result)\n \t\tdie(\"curl_easy_init failed\");\n@@ -572,16 +587,10 @@ static CURL *get_curl_handle(void)\n \tcurl_easy_setopt(result, CURLOPT_POST301, 1);\n #endif\n #if LIBCURL_VERSION_NUM >= 0x071304\n-\tif (is_transport_allowed(\"http\"))\n-\t\tallowed_protocols |= CURLPROTO_HTTP;\n-\tif (is_transport_allowed(\"https\"))\n-\t\tallowed_protocols |= CURLPROTO_HTTPS;\n-\tif (is_transport_allowed(\"ftp\"))\n-\t\tallowed_protocols |= CURLPROTO_FTP;\n-\tif (is_transport_allowed(\"ftps\"))\n-\t\tallowed_protocols |= CURLPROTO_FTPS;\n-\tcurl_easy_setopt(result, CURLOPT_REDIR_PROTOCOLS, allowed_protocols);\n-\tcurl_easy_setopt(result, CURLOPT_PROTOCOLS, allowed_protocols);\n+\tcurl_easy_setopt(result, CURLOPT_REDIR_PROTOCOLS,\n+\t\t\t get_curl_allowed_protocols());\n+\tcurl_easy_setopt(result, CURLOPT_PROTOCOLS,\n+\t\t\t get_curl_allowed_protocols());\n #else\n \twarning(\"protocol restrictions not applied to curl redirects because\\n\"\n \t\t\"your curl version is too old (>= 7.19.4)\");\n-- \n2.8.0.rc3.226.g39d4020\n\n"},{"id":"307734","messageId":"1481679637-133137-6-git-send-email-bmwill@google.com","threadId":"44417","inReplyTo":"1481679637-133137-1-git-send-email-bmwill@google.com","subject":"[PATCH v9 5/5] transport: add from_user parameter to is_transport_allowed","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-12-14T01:40:37Z","receivedAt":"2016-12-14T01:51:37Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"Add the from_user parameter to the 'is_transport_allowed' function.\nThis allows callers to query if a transport protocol is allowed, given\nthat the caller knows that the protocol is coming from the user (1) or\nnot from the user (0) such as redirects in libcurl. If unknown a -1\nshould be provided which falls back to reading `GIT_PROTOCOL_FROM_USER`\nto determine if the protocol came from the user.\n\nSigned-off-by: Brandon Williams \n---\n http.c | 14 +++++++-------\n transport.c | 8 +++++---\n transport.h | 13 ++++++++++---\n 3 files changed, 22 insertions(+), 13 deletions(-)\n\ndiff --git a/http.c b/http.c\nindex f7c488a..2208269 100644\n--- a/http.c\n+++ b/http.c\n@@ -489,17 +489,17 @@ static void set_curl_keepalive(CURL *c)\n }\n #endif\n \n-static long get_curl_allowed_protocols(void)\n+static long get_curl_allowed_protocols(int from_user)\n {\n \tlong allowed_protocols = 0;\n \n-\tif (is_transport_allowed(\"http\"))\n+\tif (is_transport_allowed(\"http\", from_user))\n \t\tallowed_protocols |= CURLPROTO_HTTP;\n-\tif (is_transport_allowed(\"https\"))\n+\tif (is_transport_allowed(\"https\", from_user))\n \t\tallowed_protocols |= CURLPROTO_HTTPS;\n-\tif (is_transport_allowed(\"ftp\"))\n+\tif (is_transport_allowed(\"ftp\", from_user))\n \t\tallowed_protocols |= CURLPROTO_FTP;\n-\tif (is_transport_allowed(\"ftps\"))\n+\tif (is_transport_allowed(\"ftps\", from_user))\n \t\tallowed_protocols |= CURLPROTO_FTPS;\n \n \treturn allowed_protocols;\n@@ -588,9 +588,9 @@ static CURL *get_curl_handle(void)\n #endif\n #if LIBCURL_VERSION_NUM >= 0x071304\n \tcurl_easy_setopt(result, CURLOPT_REDIR_PROTOCOLS,\n-\t\t\t get_curl_allowed_protocols());\n+\t\t\t get_curl_allowed_protocols(0));\n \tcurl_easy_setopt(result, CURLOPT_PROTOCOLS,\n-\t\t\t get_curl_allowed_protocols());\n+\t\t\t get_curl_allowed_protocols(-1));\n #else\n \twarning(\"protocol restrictions not applied to curl redirects because\\n\"\n \t\t\"your curl version is too old (>= 7.19.4)\");\ndiff --git a/transport.c b/transport.c\nindex fbd799d..f50c31a 100644\n--- a/transport.c\n+++ b/transport.c\n@@ -676,7 +676,7 @@ static enum protocol_allow_config get_protocol_config(const char *type)\n \treturn PROTOCOL_ALLOW_USER_ONLY;\n }\n \n-int is_transport_allowed(const char *type)\n+int is_transport_allowed(const char *type, int from_user)\n {\n \tconst struct string_list *whitelist = protocol_whitelist();\n \tif (whitelist)\n@@ -688,7 +688,9 @@ int is_transport_allowed(const char *type)\n \tcase PROTOCOL_ALLOW_NEVER:\n \t\treturn 0;\n \tcase PROTOCOL_ALLOW_USER_ONLY:\n-\t\treturn git_env_bool(\"GIT_PROTOCOL_FROM_USER\", 1);\n+\t\tif (from_user < 0)\n+\t\t\tfrom_user = git_env_bool(\"GIT_PROTOCOL_FROM_USER\", 1);\n+\t\treturn from_user;\n \t}\n \n \tdie(\"BUG: invalid protocol_allow_config type\");\n@@ -696,7 +698,7 @@ int is_transport_allowed(const char *type)\n \n void transport_check_allowed(const char *type)\n {\n-\tif (!is_transport_allowed(type))\n+\tif (!is_transport_allowed(type, -1))\n \t\tdie(\"transport '%s' not allowed\", type);\n }\n \ndiff --git a/transport.h b/transport.h\nindex 3396e1d..4f1c801 100644\n--- a/transport.h\n+++ b/transport.h\n@@ -142,10 +142,17 @@ struct transport {\n struct transport *transport_get(struct remote *, const char *);\n \n /*\n- * Check whether a transport is allowed by the environment. Type should\n- * generally be the URL scheme, as described in Documentation/git.txt\n+ * Check whether a transport is allowed by the environment.\n+ *\n+ * Type should generally be the URL scheme, as described in\n+ * Documentation/git.txt\n+ *\n+ * from_user specifies if the transport was given by the user. If unknown pass\n+ * a -1 to read from the environment to determine if the transport was given by\n+ * the user.\n+ *\n */\n-int is_transport_allowed(const char *type);\n+int is_transport_allowed(const char *type, int from_user);\n \n /*\n * Check whether a transport is allowed by the environment,\n-- \n2.8.0.rc3.226.g39d4020\n\n"},{"id":"307783","messageId":"20161214160123.a6a7fve5qz5xgg7n@sigill.intra.peff.net","threadId":"44417","inReplyTo":"1481679637-133137-4-git-send-email-bmwill@google.com","subject":"Re: [PATCH v9 3/5] http: always warn if libcurl version is too old","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-12-14T16:01:24Z","receivedAt":"2016-12-14T16:01:36Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Dec 13, 2016 at 05:40:35PM -0800, Brandon Williams wrote:\n\n> diff --git a/transport.c b/transport.c\n> index e1ba78b..fbd799d 100644\n> --- a/transport.c\n> +++ b/transport.c\n> @@ -700,11 +700,6 @@ void transport_check_allowed(const char *type)\n> \t\tdie(\"transport '%s' not allowed\", type);\n> }\n> \n> -int transport_restrict_protocols(void)\n> -{\n> -\treturn !!protocol_whitelist();\n> -}\n> -\n\nThis function was subtly broken as of patch 2 of the series. It's\nprobably not a big deal in the long run, but should the series be\nre-ordered to put this one first?\n\nI think the commit message would need adjusted, but it probably should\nmention the reasons this is a good idea even _without_ the new config\nsystem. Namely that even when there's no protocol whitelist, newer\nversions of curl have all of the other non-http protocols disabled.\n\nI wonder if anybody is actually using a version of curl old enough to\ntrigger this. If so, they're going to get the warning every time they\nfetch via http. We might need to stick it behind an \"advice.*\" config\noption, though I'm inclined to leave it as-is and see if anybody\nactually complains.\n\n-Peff\n"},{"id":"307784","messageId":"20161214160330.iqvwxshsgk4n2gm7@sigill.intra.peff.net","threadId":"44417","inReplyTo":"1481679637-133137-5-git-send-email-bmwill@google.com","subject":"Re: [PATCH v9 4/5] http: create function to get curl allowed protocols","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-12-14T16:03:30Z","receivedAt":"2016-12-14T16:03:40Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Dec 13, 2016 at 05:40:36PM -0800, Brandon Williams wrote:\n\n> Move the creation of an allowed protocols whitelist to a helper\n> function.\n\nThis is \"what\" but not \"why\". You can figure it out if you see the next\npatch, but it's often nice to make a brief mention, like:\n\n This will be useful when we need to compute the set of allowed\n protocols differently for normal and redirect cases.\n\n-Peff\n"},{"id":"307785","messageId":"20161214164050.uxk434kzhw6au4c2@sigill.intra.peff.net","threadId":"44417","inReplyTo":"1481679637-133137-6-git-send-email-bmwill@google.com","subject":"Re: [PATCH v9 5/5] transport: add from_user parameter to is_transport_allowed","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-12-14T16:40:50Z","receivedAt":"2016-12-14T16:41:00Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Dec 13, 2016 at 05:40:37PM -0800, Brandon Williams wrote:\n\n> Add the from_user parameter to the 'is_transport_allowed' function.\n> This allows callers to query if a transport protocol is allowed, given\n> that the caller knows that the protocol is coming from the user (1) or\n> not from the user (0) such as redirects in libcurl. If unknown a -1\n> should be provided which falls back to reading `GIT_PROTOCOL_FROM_USER`\n> to determine if the protocol came from the user.\n\nI think your commit message is upside-down with respect to the purpose\nof the patch. The end goal we want is for http to distinguish between\nprotocol restrictions for redirects versus initial requests. The rest is\nan implementation detail. It's definitely still worth discussing that\nimplementation detail (though I think your in-code comments may be\nsufficient), but I don't see the rationale discussed here at all.\n\n> Signed-off-by: Brandon Williams \n> ---\n> http.c | 14 +++++++-------\n> transport.c | 8 +++++---\n> transport.h | 13 ++++++++++---\n> 3 files changed, 22 insertions(+), 13 deletions(-)\n\nI'm trying to think of a way to test this. I guess the case we are\ncovering here is when a server redirects, but the protocol is only\nallowed from the user. So:\n\ndiff --git a/t/t5812-proto-disable-http.sh b/t/t5812-proto-disable-http.sh\nindex 044cc152f..d911afd24 100755\n--- a/t/t5812-proto-disable-http.sh\n+++ b/t/t5812-proto-disable-http.sh\n@@ -30,5 +30,12 @@ test_expect_success 'curl limits redirects' '\n \ttest_must_fail git clone \"$HTTPD_URL/loop-redir/smart/repo.git\"\n '\n \n+test_expect_success 'http can be limited to from-user' '\n+\tgit -c protocol.http.allow=user \\\n+\t\tclone \"$HTTPD_URL/smart/repo.git\" plain.git &&\n+\ttest_must_fail git -c protocol.http.allow=user \\\n+\t\tclone \"$HTTPD_URL/smart-redir-perm/repo.git\" redir.git\n+'\n+\n stop_httpd\n test_done\n\nIt's an oddball configuration, and you'd probably just set\nhttp.followRedirects=false in practice, but it does correctly check this\ncase.\n\n> @@ -588,9 +588,9 @@ static CURL *get_curl_handle(void)\n> #endif\n> #if LIBCURL_VERSION_NUM >= 0x071304\n> \tcurl_easy_setopt(result, CURLOPT_REDIR_PROTOCOLS,\n> -\t\t\t get_curl_allowed_protocols());\n> +\t\t\t get_curl_allowed_protocols(0));\n> \tcurl_easy_setopt(result, CURLOPT_PROTOCOLS,\n> -\t\t\t get_curl_allowed_protocols());\n> +\t\t\t get_curl_allowed_protocols(-1));\n\nThis covers internal redirects done by libcurl, but not the dumb-walker\nhttp-alternates nonsense. We have to feed the URL from http-alternates\nback to curl ourselves, so it uses CURLOPT_PROTOCOLS even though it\nshould count as \"not from the user\".\n\nTo fix that, I think we'd need something like:\n\n - get_curl_handle() stops setting these options, as it is done only\n once when the curl handle is initialized. Instead, the protocol\n restrictions should go into get_active_slot(), which is called for\n each request. The values set would remain the same, and be the\n baseline.\n\n - the http-walker.c code would need to know when it's requesting from\n the base URL, and when it's an alternate. I think this would depend\n on the position of the \"alt\" in in the linked list it keeps.\n\n - when requesting from an alternate, http-walker would set\n CURLOPT_PROTOCOLS with get_curl_allowed_protocols(0)\n\nI have to admit that it sounds like a fair bit of work for a pretty\nobscure case. You'd have to:\n\n 1. Turn http.allowRedirects to \"true\", to allow redirects even for\n non-initial contact.\n\n 2. Turn one of protocol.{http,https,ftp,ftps}.allow to \"user\" to\n restrict it from being used in a redirect.\n\nI'm tempted to punt on it and just do:\n\n if (http_follow_config == HTTP_FOLLOW_ALWAYS &&\n get_curl_allowed_protocols(0) != get_curl_allowed_protocols(-1))\n\tdie(\"user-only protocol restrictions not implemented for http-alternates\");\n\nwhich errs on the safe side. We could even shove that down into the case\nwhere we actually see some alternates, like:\n\ndiff --git a/http-walker.c b/http-walker.c\nindex c2f81cd6a..5bcc850b1 100644\n--- a/http-walker.c\n+++ b/http-walker.c\n@@ -160,6 +160,12 @@ static void prefetch(struct walker *walker, unsigned char *sha1)\n #endif\n }\n \n+static void check_alternates_protocol_restrictions(void)\n+{\n+\tif (get_curl_allowed_protocols(0) != get_curl_allowed_protocol(-1))\n+\t\tdie(\"user-only protocol restrictions not implemented for http alternates\");\n+}\n+\n static void process_alternates_response(void *callback_data)\n {\n \tstruct alternates_request *alt_req =\n@@ -272,6 +278,7 @@ static void process_alternates_response(void *callback_data)\n \t\t\t/* skip \"objects\\n\" at end */\n \t\t\tif (okay) {\n \t\t\t\tstruct strbuf target = STRBUF_INIT;\n+\t\t\t\tcheck_alternates_protocol_restrictions();\n \t\t\t\tstrbuf_add(&target, base, serverlen);\n \t\t\t\tstrbuf_add(&target, data + i, posn - i - 7);\n \t\t\t\twarning(\"adding alternate object store: %s\",\n\nI find it unlikely that anybody would ever care, but at least we'd do\nthe safe thing. I dunno. Maybe I am just being lazy.\n\n-Peff\n"},{"id":"307792","messageId":"20161214175635.GA20063@google.com","threadId":"44417","inReplyTo":"20161214160123.a6a7fve5qz5xgg7n@sigill.intra.peff.net","subject":"Re: [PATCH v9 3/5] http: always warn if libcurl version is too old","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-12-14T17:56:35Z","receivedAt":"2016-12-14T17:56:43Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 12/14, Jeff King wrote:\n> On Tue, Dec 13, 2016 at 05:40:35PM -0800, Brandon Williams wrote:\n> \n> > diff --git a/transport.c b/transport.c\n> > index e1ba78b..fbd799d 100644\n> > --- a/transport.c\n> > +++ b/transport.c\n> > @@ -700,11 +700,6 @@ void transport_check_allowed(const char *type)\n> > \t\tdie(\"transport '%s' not allowed\", type);\n> > }\n> > \n> > -int transport_restrict_protocols(void)\n> > -{\n> > -\treturn !!protocol_whitelist();\n> > -}\n> > -\n> \n> This function was subtly broken as of patch 2 of the series. It's\n> probably not a big deal in the long run, but should the series be\n> re-ordered to put this one first?\n> \n> I think the commit message would need adjusted, but it probably should\n> mention the reasons this is a good idea even _without_ the new config\n> system. Namely that even when there's no protocol whitelist, newer\n> versions of curl have all of the other non-http protocols disabled.\n> \n> I wonder if anybody is actually using a version of curl old enough to\n> trigger this. If so, they're going to get the warning every time they\n> fetch via http. We might need to stick it behind an \"advice.*\" config\n> option, though I'm inclined to leave it as-is and see if anybody\n> actually complains.\n> \n> -Peff\n\nYeah you're right, transport_restrict_protocols() is definitely broken\nafter patch 2. Since I'm probably going to need to do a reroll based on\nsome of your comments in the other patches in the series we might as\nwell reorder patch 2 and 3 so this isn't broken between patches.\n\n-- \nBrandon Williams\n"},{"id":"307794","messageId":"20161214180056.GB20063@google.com","threadId":"44417","inReplyTo":"20161214160330.iqvwxshsgk4n2gm7@sigill.intra.peff.net","subject":"Re: [PATCH v9 4/5] http: create function to get curl allowed protocols","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-12-14T18:00:56Z","receivedAt":"2016-12-14T18:01:03Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 12/14, Jeff King wrote:\n> On Tue, Dec 13, 2016 at 05:40:36PM -0800, Brandon Williams wrote:\n> \n> > Move the creation of an allowed protocols whitelist to a helper\n> > function.\n> \n> This is \"what\" but not \"why\". You can figure it out if you see the next\n> patch, but it's often nice to make a brief mention, like:\n> \n> This will be useful when we need to compute the set of allowed\n> protocols differently for normal and redirect cases.\n\nCommit message writing is hard (at least for me :). I'll update the\nmessage to indicate the why.\n\n-- \nBrandon Williams\n"},{"id":"307804","messageId":"20161214201323.GC20063@google.com","threadId":"44417","inReplyTo":"20161214164050.uxk434kzhw6au4c2@sigill.intra.peff.net","subject":"Re: [PATCH v9 5/5] transport: add from_user parameter to is_transport_allowed","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-12-14T20:13:23Z","receivedAt":"2016-12-14T20:13:39Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 12/14, Jeff King wrote:\n> On Tue, Dec 13, 2016 at 05:40:37PM -0800, Brandon Williams wrote:\n> \n> > Add the from_user parameter to the 'is_transport_allowed' function.\n> > This allows callers to query if a transport protocol is allowed, given\n> > that the caller knows that the protocol is coming from the user (1) or\n> > not from the user (0) such as redirects in libcurl. If unknown a -1\n> > should be provided which falls back to reading `GIT_PROTOCOL_FROM_USER`\n> > to determine if the protocol came from the user.\n> \n> I think your commit message is upside-down with respect to the purpose\n> of the patch. The end goal we want is for http to distinguish between\n> protocol restrictions for redirects versus initial requests. The rest is\n> an implementation detail. It's definitely still worth discussing that\n> implementation detail (though I think your in-code comments may be\n> sufficient), but I don't see the rationale discussed here at all.\n\nI'll fix the commit message to better discuss the reasoning behind the\nchange.\n\n> > Signed-off-by: Brandon Williams \n> > ---\n> > http.c | 14 +++++++-------\n> > transport.c | 8 +++++---\n> > transport.h | 13 ++++++++++---\n> > 3 files changed, 22 insertions(+), 13 deletions(-)\n> \n> I'm trying to think of a way to test this. I guess the case we are\n> covering here is when a server redirects, but the protocol is only\n> allowed from the user. So:\n> \n> diff --git a/t/t5812-proto-disable-http.sh b/t/t5812-proto-disable-http.sh\n> index 044cc152f..d911afd24 100755\n> --- a/t/t5812-proto-disable-http.sh\n> +++ b/t/t5812-proto-disable-http.sh\n> @@ -30,5 +30,12 @@ test_expect_success 'curl limits redirects' '\n> \ttest_must_fail git clone \"$HTTPD_URL/loop-redir/smart/repo.git\"\n> '\n> \n> +test_expect_success 'http can be limited to from-user' '\n> +\tgit -c protocol.http.allow=user \\\n> +\t\tclone \"$HTTPD_URL/smart/repo.git\" plain.git &&\n> +\ttest_must_fail git -c protocol.http.allow=user \\\n> +\t\tclone \"$HTTPD_URL/smart-redir-perm/repo.git\" redir.git\n> +'\n> +\n> stop_httpd\n> test_done\n> \n> It's an oddball configuration, and you'd probably just set\n> http.followRedirects=false in practice, but it does correctly check this\n> case.\n\nK I'll add this in as a test.\n\n> > @@ -588,9 +588,9 @@ static CURL *get_curl_handle(void)\n> > #endif\n> > #if LIBCURL_VERSION_NUM >= 0x071304\n> > \tcurl_easy_setopt(result, CURLOPT_REDIR_PROTOCOLS,\n> > -\t\t\t get_curl_allowed_protocols());\n> > +\t\t\t get_curl_allowed_protocols(0));\n> > \tcurl_easy_setopt(result, CURLOPT_PROTOCOLS,\n> > -\t\t\t get_curl_allowed_protocols());\n> > +\t\t\t get_curl_allowed_protocols(-1));\n> \n> This covers internal redirects done by libcurl, but not the dumb-walker\n> http-alternates nonsense. We have to feed the URL from http-alternates\n> back to curl ourselves, so it uses CURLOPT_PROTOCOLS even though it\n> should count as \"not from the user\".\n> \n> To fix that, I think we'd need something like:\n> \n> - get_curl_handle() stops setting these options, as it is done only\n> once when the curl handle is initialized. Instead, the protocol\n> restrictions should go into get_active_slot(), which is called for\n> each request. The values set would remain the same, and be the\n> baseline.\n> \n> - the http-walker.c code would need to know when it's requesting from\n> the base URL, and when it's an alternate. I think this would depend\n> on the position of the \"alt\" in in the linked list it keeps.\n> \n> - when requesting from an alternate, http-walker would set\n> CURLOPT_PROTOCOLS with get_curl_allowed_protocols(0)\n> \n> I have to admit that it sounds like a fair bit of work for a pretty\n> obscure case. You'd have to:\n> \n> 1. Turn http.allowRedirects to \"true\", to allow redirects even for\n> non-initial contact.\n> \n> 2. Turn one of protocol.{http,https,ftp,ftps}.allow to \"user\" to\n> restrict it from being used in a redirect.\n> \n> I'm tempted to punt on it and just do:\n> \n> if (http_follow_config == HTTP_FOLLOW_ALWAYS &&\n> get_curl_allowed_protocols(0) != get_curl_allowed_protocols(-1))\n> \tdie(\"user-only protocol restrictions not implemented for http-alternates\");\n> \n> which errs on the safe side. We could even shove that down into the case\n> where we actually see some alternates, like:\n> \n> diff --git a/http-walker.c b/http-walker.c\n> index c2f81cd6a..5bcc850b1 100644\n> --- a/http-walker.c\n> +++ b/http-walker.c\n> @@ -160,6 +160,12 @@ static void prefetch(struct walker *walker, unsigned char *sha1)\n> #endif\n> }\n> \n> +static void check_alternates_protocol_restrictions(void)\n> +{\n> +\tif (get_curl_allowed_protocols(0) != get_curl_allowed_protocol(-1))\n> +\t\tdie(\"user-only protocol restrictions not implemented for http alternates\");\n> +}\n> +\n> static void process_alternates_response(void *callback_data)\n> {\n> \tstruct alternates_request *alt_req =\n> @@ -272,6 +278,7 @@ static void process_alternates_response(void *callback_data)\n> \t\t\t/* skip \"objects\\n\" at end */\n> \t\t\tif (okay) {\n> \t\t\t\tstruct strbuf target = STRBUF_INIT;\n> +\t\t\t\tcheck_alternates_protocol_restrictions();\n> \t\t\t\tstrbuf_add(&target, base, serverlen);\n> \t\t\t\tstrbuf_add(&target, data + i, posn - i - 7);\n> \t\t\t\twarning(\"adding alternate object store: %s\",\n> \n> I find it unlikely that anybody would ever care, but at least we'd do\n> the safe thing. I dunno. Maybe I am just being lazy.\n\nWell, that's unfortunate! It does sound like a more full-proof solution\nto these dumb http alternates could be involved. I don't think your\nsimple \"lazy\" solution may be enough to not just die by default.\n\nBy default ftp/ftps will have a policy of \"user only\" which means they\nwill be set by the call to get_curl_allowed_protocol(-1) but not set by\nget_curl_allowed_protocol(0). This would result in the call to\ncheck_alternates_protocol_restrictions failing all the time unless the\nuser explicitly sets ftp/ftps to \"always\" or \"never\". If that is the\ndesired behavior then your proposed solution would be fine, otherwise we\nmay have to do the more involved approach.\n\n-- \nBrandon Williams\n"},{"id":"307805","messageId":"20161214203349.6ym3v2ny2gonovx2@sigill.intra.peff.net","threadId":"44417","inReplyTo":"20161214201323.GC20063@google.com","subject":"Re: [PATCH v9 5/5] transport: add from_user parameter to is_transport_allowed","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-12-14T20:33:49Z","receivedAt":"2016-12-14T20:35:25Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Dec 14, 2016 at 12:13:23PM -0800, Brandon Williams wrote:\n\n> > I find it unlikely that anybody would ever care, but at least we'd do\n> > the safe thing. I dunno. Maybe I am just being lazy.\n> \n> Well, that's unfortunate! It does sound like a more full-proof solution\n> to these dumb http alternates could be involved. I don't think your\n> simple \"lazy\" solution may be enough to not just die by default.\n> \n> By default ftp/ftps will have a policy of \"user only\" which means they\n> will be set by the call to get_curl_allowed_protocol(-1) but not set by\n> get_curl_allowed_protocol(0). This would result in the call to\n> check_alternates_protocol_restrictions failing all the time unless the\n> user explicitly sets ftp/ftps to \"always\" or \"never\". If that is the\n> desired behavior then your proposed solution would be fine, otherwise we\n> may have to do the more involved approach.\n\nOh, hrm, you're right. I was definitely meaning for this to kick in only\nwhen you had explicitly configured a protocol to \"user\" (they'd still\nneed to enable redirects, but that's much more likely).\n\nArguably ftp should be on the \"safe\" list, since it's implemented via\nthe exact same code as https. That list comes originally from 33cfccbbf\n(submodule: allow only certain protocols for submodule fetches,\n2015-09-16), but that was just based on what I thought was reasonable at\nthe time.\n\nI find it hard to believe anybody actually uses git-over-ftp these days,\nlet alone as a protocol redirect via http-alternates. But AFAIK it does\nwork.\n\nOne other \"simple\" fix is at the moment we parse http-alternates, to\nparse the URL ourself and check the policy. Like:\n\n const char *protocols[] = {\n\t\"http\", \"https\", \"ftp\", \"ftps\"\n };\n\n for (i = 0; i < ARRAY_SIZE(protocols); i++) {\n\tconst char *end;\n\tif (skip_prefix(url, protocols[i], end) && starts_with(end, \"://\"))\n\t\tbreak;\n }\n\n if (i >= ARRAY_SIZE(protocols))\n\twarning(\"ignoring alternate with unknown protocol: %s\", url);\n else if (!is_transport_allowed(protocols[i], 0))\n\twarning(\"ignoring alternate with restricted protocol: %s\", url);\n else {\n\t/* actually set up the alt struct */\n }\n\nThat keeps the logic nicely contained. The downside is that if our\ninterpretation of the URL is ever different than curl's, it could create\na security problem. The bit above seems fairly foolproof, though,\nbecause it functions as a whitelist. So it doesn't matter if you can\ntrigger an http request via curl with exotic syntax; it matters whether\ncurl will trigger anything _besides_ an http syntax if the string starts\nwith \"http://\". Which seems unlikely.\n\nI may have convinced myself this is a reasonable approach.\n\n-Peff\n"},{"id":"307806","messageId":"20161214203752.GD20063@google.com","threadId":"44417","inReplyTo":"20161214201323.GC20063@google.com","subject":"Re: [PATCH v9 5/5] transport: add from_user parameter to is_transport_allowed","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-12-14T20:37:52Z","receivedAt":"2016-12-14T20:38:00Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 12/14, Brandon Williams wrote:\n> On 12/14, Jeff King wrote:\n> > On Tue, Dec 13, 2016 at 05:40:37PM -0800, Brandon Williams wrote:\n> > \n> > > Add the from_user parameter to the 'is_transport_allowed' function.\n> > > This allows callers to query if a transport protocol is allowed, given\n> > > that the caller knows that the protocol is coming from the user (1) or\n> > > not from the user (0) such as redirects in libcurl. If unknown a -1\n> > > should be provided which falls back to reading `GIT_PROTOCOL_FROM_USER`\n> > > to determine if the protocol came from the user.\n> > \n> > I think your commit message is upside-down with respect to the purpose\n> > of the patch. The end goal we want is for http to distinguish between\n> > protocol restrictions for redirects versus initial requests. The rest is\n> > an implementation detail. It's definitely still worth discussing that\n> > implementation detail (though I think your in-code comments may be\n> > sufficient), but I don't see the rationale discussed here at all.\n> \n> I'll fix the commit message to better discuss the reasoning behind the\n> change.\n> \n> > > Signed-off-by: Brandon Williams \n> > > ---\n> > > http.c | 14 +++++++-------\n> > > transport.c | 8 +++++---\n> > > transport.h | 13 ++++++++++---\n> > > 3 files changed, 22 insertions(+), 13 deletions(-)\n> > \n> > I'm trying to think of a way to test this. I guess the case we are\n> > covering here is when a server redirects, but the protocol is only\n> > allowed from the user. So:\n> > \n> > diff --git a/t/t5812-proto-disable-http.sh b/t/t5812-proto-disable-http.sh\n> > index 044cc152f..d911afd24 100755\n> > --- a/t/t5812-proto-disable-http.sh\n> > +++ b/t/t5812-proto-disable-http.sh\n> > @@ -30,5 +30,12 @@ test_expect_success 'curl limits redirects' '\n> > \ttest_must_fail git clone \"$HTTPD_URL/loop-redir/smart/repo.git\"\n> > '\n> > \n> > +test_expect_success 'http can be limited to from-user' '\n> > +\tgit -c protocol.http.allow=user \\\n> > +\t\tclone \"$HTTPD_URL/smart/repo.git\" plain.git &&\n> > +\ttest_must_fail git -c protocol.http.allow=user \\\n> > +\t\tclone \"$HTTPD_URL/smart-redir-perm/repo.git\" redir.git\n> > +'\n> > +\n> > stop_httpd\n> > test_done\n> > \n> > It's an oddball configuration, and you'd probably just set\n> > http.followRedirects=false in practice, but it does correctly check this\n> > case.\n> \n> K I'll add this in as a test.\n> \n> > > @@ -588,9 +588,9 @@ static CURL *get_curl_handle(void)\n> > > #endif\n> > > #if LIBCURL_VERSION_NUM >= 0x071304\n> > > \tcurl_easy_setopt(result, CURLOPT_REDIR_PROTOCOLS,\n> > > -\t\t\t get_curl_allowed_protocols());\n> > > +\t\t\t get_curl_allowed_protocols(0));\n> > > \tcurl_easy_setopt(result, CURLOPT_PROTOCOLS,\n> > > -\t\t\t get_curl_allowed_protocols());\n> > > +\t\t\t get_curl_allowed_protocols(-1));\n> > \n> > This covers internal redirects done by libcurl, but not the dumb-walker\n> > http-alternates nonsense. We have to feed the URL from http-alternates\n> > back to curl ourselves, so it uses CURLOPT_PROTOCOLS even though it\n> > should count as \"not from the user\".\n> > \n> > To fix that, I think we'd need something like:\n> > \n> > - get_curl_handle() stops setting these options, as it is done only\n> > once when the curl handle is initialized. Instead, the protocol\n> > restrictions should go into get_active_slot(), which is called for\n> > each request. The values set would remain the same, and be the\n> > baseline.\n> > \n> > - the http-walker.c code would need to know when it's requesting from\n> > the base URL, and when it's an alternate. I think this would depend\n> > on the position of the \"alt\" in in the linked list it keeps.\n> > \n> > - when requesting from an alternate, http-walker would set\n> > CURLOPT_PROTOCOLS with get_curl_allowed_protocols(0)\n> > \n> > I have to admit that it sounds like a fair bit of work for a pretty\n> > obscure case. You'd have to:\n> > \n> > 1. Turn http.allowRedirects to \"true\", to allow redirects even for\n> > non-initial contact.\n> > \n> > 2. Turn one of protocol.{http,https,ftp,ftps}.allow to \"user\" to\n> > restrict it from being used in a redirect.\n> > \n> > I'm tempted to punt on it and just do:\n> > \n> > if (http_follow_config == HTTP_FOLLOW_ALWAYS &&\n> > get_curl_allowed_protocols(0) != get_curl_allowed_protocols(-1))\n> > \tdie(\"user-only protocol restrictions not implemented for http-alternates\");\n> > \n> > which errs on the safe side. We could even shove that down into the case\n> > where we actually see some alternates, like:\n> > \n> > diff --git a/http-walker.c b/http-walker.c\n> > index c2f81cd6a..5bcc850b1 100644\n> > --- a/http-walker.c\n> > +++ b/http-walker.c\n> > @@ -160,6 +160,12 @@ static void prefetch(struct walker *walker, unsigned char *sha1)\n> > #endif\n> > }\n> > \n> > +static void check_alternates_protocol_restrictions(void)\n> > +{\n> > +\tif (get_curl_allowed_protocols(0) != get_curl_allowed_protocol(-1))\n> > +\t\tdie(\"user-only protocol restrictions not implemented for http alternates\");\n> > +}\n> > +\n> > static void process_alternates_response(void *callback_data)\n> > {\n> > \tstruct alternates_request *alt_req =\n> > @@ -272,6 +278,7 @@ static void process_alternates_response(void *callback_data)\n> > \t\t\t/* skip \"objects\\n\" at end */\n> > \t\t\tif (okay) {\n> > \t\t\t\tstruct strbuf target = STRBUF_INIT;\n> > +\t\t\t\tcheck_alternates_protocol_restrictions();\n> > \t\t\t\tstrbuf_add(&target, base, serverlen);\n> > \t\t\t\tstrbuf_add(&target, data + i, posn - i - 7);\n> > \t\t\t\twarning(\"adding alternate object store: %s\",\n> > \n> > I find it unlikely that anybody would ever care, but at least we'd do\n> > the safe thing. I dunno. Maybe I am just being lazy.\n> \n> Well, that's unfortunate! It does sound like a more full-proof solution\n> to these dumb http alternates could be involved. I don't think your\n> simple \"lazy\" solution may be enough to not just die by default.\n> \n> By default ftp/ftps will have a policy of \"user only\" which means they\n> will be set by the call to get_curl_allowed_protocol(-1) but not set by\n> get_curl_allowed_protocol(0). This would result in the call to\n> check_alternates_protocol_restrictions failing all the time unless the\n> user explicitly sets ftp/ftps to \"always\" or \"never\". If that is the\n> desired behavior then your proposed solution would be fine, otherwise we\n> may have to do the more involved approach.\n\nNaively looking at the code (and your longer suggestion), is there a\nreason why we couldn't simply have http-walker set CURLOPT_PROTOCOLS\nwith get_curl_allowed_protocols(0) in the fetch_alternates() function?\nThat way we just override the CURLOPT_PROTOCOLS value when alternates\nare involved.\nLike so:\n\ndiff --git a/http-walker.c b/http-walker.c\nindex c2f81cd..b284cec 100644\n--- a/http-walker.c\n+++ b/http-walker.c\n@@ -339,6 +339,8 @@ static void fetch_alternates(struct walker *walker, const char *base)\n \tcurl_easy_setopt(slot->curl, CURLOPT_FILE, &buffer);\n \tcurl_easy_setopt(slot->curl, CURLOPT_WRITEFUNCTION, fwrite_buffer);\n \tcurl_easy_setopt(slot->curl, CURLOPT_URL, url.buf);\n+\tcurl_easy_setopt(slot->curl, CURLOPT_PROTOCOLS,\n+\t\t\t get_curl_allowed_protocols(0));\n \n \talt_req.base = base;\n \talt_req.url = &url;\n\n-- \nBrandon Williams\n"},{"id":"307807","messageId":"20161214204102.hwjb3i4aaxf3oigq@sigill.intra.peff.net","threadId":"44417","inReplyTo":"20161214203752.GD20063@google.com","subject":"Re: [PATCH v9 5/5] transport: add from_user parameter to is_transport_allowed","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-12-14T20:41:02Z","receivedAt":"2016-12-14T20:47:50Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Dec 14, 2016 at 12:37:52PM -0800, Brandon Williams wrote:\n\n> Naively looking at the code (and your longer suggestion), is there a\n> reason why we couldn't simply have http-walker set CURLOPT_PROTOCOLS\n> with get_curl_allowed_protocols(0) in the fetch_alternates() function?\n> That way we just override the CURLOPT_PROTOCOLS value when alternates\n> are involved.\n\nNo, because we may have many curl handles (especially for the\nhttp-walker, which wants to fetch several objects simultaneously), and\nthey get recycled as needed for many requests.\n\nSo setting a restriction there on slot->curl will only cover the one\nhandle, and miss other ones which may be used later (and likewise, that\none handle with the restriction may get recycled and used for a\nnon-alternate fetch, and would be unnecessarily restrictive).\n\nThat's why any curl-level settings have to happen when we call\nget_active_slot(), since that's when we know what we're actually using\nthe handle for.\n\n-Peff\n"},{"id":"307808","messageId":"20161214205059.GE20063@google.com","threadId":"44417","inReplyTo":"20161214204102.hwjb3i4aaxf3oigq@sigill.intra.peff.net","subject":"Re: [PATCH v9 5/5] transport: add from_user parameter to is_transport_allowed","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-12-14T20:50:59Z","receivedAt":"2016-12-14T20:52:38Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 12/14, Jeff King wrote:\n> On Wed, Dec 14, 2016 at 12:37:52PM -0800, Brandon Williams wrote:\n> \n> > Naively looking at the code (and your longer suggestion), is there a\n> > reason why we couldn't simply have http-walker set CURLOPT_PROTOCOLS\n> > with get_curl_allowed_protocols(0) in the fetch_alternates() function?\n> > That way we just override the CURLOPT_PROTOCOLS value when alternates\n> > are involved.\n> \n> No, because we may have many curl handles (especially for the\n> http-walker, which wants to fetch several objects simultaneously), and\n> they get recycled as needed for many requests.\n> \n> So setting a restriction there on slot->curl will only cover the one\n> handle, and miss other ones which may be used later (and likewise, that\n> one handle with the restriction may get recycled and used for a\n> non-alternate fetch, and would be unnecessarily restrictive).\n> \n> That's why any curl-level settings have to happen when we call\n> get_active_slot(), since that's when we know what we're actually using\n> the handle for.\n\nFair enough, I figured there may be some reuse happening with the curl\nhandles but didn't do enough digging to discover that myself. Thanks :)\n\n-- \nBrandon Williams\n"},{"id":"307809","messageId":"20161214211229.mpgu3hahlfbdxys6@sigill.intra.peff.net","threadId":"44417","inReplyTo":"20161214203349.6ym3v2ny2gonovx2@sigill.intra.peff.net","subject":"Re: [PATCH v9 5/5] transport: add from_user parameter to is_transport_allowed","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-12-14T21:12:29Z","receivedAt":"2016-12-14T21:19:17Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Dec 14, 2016 at 03:33:49PM -0500, Jeff King wrote:\n\n> One other \"simple\" fix is at the moment we parse http-alternates, to\n> parse the URL ourself and check the policy. Like:\n> [...]\n> I may have convinced myself this is a reasonable approach.\n\nSo here it is in patch form, with a test.\n\nI also took a look at how bad it would be to plumb through the \"this is\nan alternate\" flag. And it's actually a little nasty, because the\nhttp-walker isn't calling get_active_slot() itself. It's relying on\nhttp_get_file() and other wrappers. The recent http_get_options makes\nthis slightly less terrible, but I'd still rather avoid infecting the\ngeneral http code that is used for the smart-http code paths.\n\nSo I think this patch on top of your series, plus the other minor fixes\nwe've discussed, the topic should be ready for 'next'.\n\n-- >8 --\nSubject: http: respect protocol.*.allow=user for http-alternates\n\nThe http-walker may fetch the http-alternates (or\nalternates) file from a remote in order to find more\nobjects. This should count as a \"not from the user\" use of\nthe protocol. But because we implement the redirection\nourselves and feed the new URL to curl, it will use the\nCURLOPT_PROTOCOLS rules, not the more restrictive\nCURLOPT_REDIR_PROTOCOLS.\n\nThe ideal solution would be for each curl request we make to\nknow whether or not is directly from the user or part of an\nalternates redirect, and then set CURLOPT_PROTOCOLS as\nappropriate. However, that would require plumbing that\ninformation through all of the various layers of the http\ncode.\n\nInstead, let's check the protocol at the source: when we are\nparsing the remote http-alternates file. The only downside\nis that if there's any mismatch between what protocol we\nthink it is versus what curl thinks it is, it could violate\nthe policy.\n\nTo address this, we'll make the parsing err on the picky\nside, and only allow protocols that it can parse\ndefinitively. So for example, you can't elude the \"http\"\npolicy by asking for \"HTTP://\", even though curl might\nhandle it; we would reject it as unknown. The only unsafe\ncase would be if you have a URL that starts with \"http://\"\nbut curl interprets as another protocol. That seems like an\nunlikely failure mode (and we are still protected by our\nbase CURLOPT_PROTOCOL setting, so the worst you could do is\ntrigger one of https, ftp, or ftps).\n\nSigned-off-by: Jeff King \n---\nI actually do reject \"HTTP://\" here, though I suspect curl does take it.\nI notice that you cannot ask to clone \"HTTP://...\" in the first place,\nas our remote-helper interface is case sensitive (or maybe it would even\nrespect \"HTTP://\" on a case-insensitive file system!).\n\nPossibly we should be more consistent about down-casing protocols when\ncomparing them, but I'm not sure if anybody actually cares in practice.\n\n http-walker.c | 52 ++++++++++++++++++++++++++++++++++++----------\n t/t5550-http-fetch-dumb.sh | 10 +++++++++\n 2 files changed, 51 insertions(+), 11 deletions(-)\n\ndiff --git a/http-walker.c b/http-walker.c\nindex c2f81cd6a..b34b6ace7 100644\n--- a/http-walker.c\n+++ b/http-walker.c\n@@ -3,6 +3,7 @@\n #include \"walker.h\"\n #include \"http.h\"\n #include \"list.h\"\n+#include \"transport.h\"\n \n struct alt_base {\n \tchar *base;\n@@ -160,6 +161,32 @@ static void prefetch(struct walker *walker, unsigned char *sha1)\n #endif\n }\n \n+static int is_alternate_allowed(const char *url)\n+{\n+\tconst char *protocols[] = {\n+\t\t\"http\", \"https\", \"ftp\", \"ftps\"\n+\t};\n+\tint i;\n+\n+\tfor (i = 0; i < ARRAY_SIZE(protocols); i++) {\n+\t\tconst char *end;\n+\t\tif (skip_prefix(url, protocols[i], &end) &&\n+\t\t starts_with(end, \"://\"))\n+\t\t\tbreak;\n+\t}\n+\n+\tif (i >= ARRAY_SIZE(protocols)) {\n+\t\twarning(\"ignoring alternate with unknown protocol: %s\", url);\n+\t\treturn 0;\n+\t}\n+\tif (!is_transport_allowed(protocols[i], 0)) {\n+\t\twarning(\"ignoring alternate with restricted protocol: %s\", url);\n+\t\treturn 0;\n+\t}\n+\n+\treturn 1;\n+}\n+\n static void process_alternates_response(void *callback_data)\n {\n \tstruct alternates_request *alt_req =\n@@ -274,17 +301,20 @@ static void process_alternates_response(void *callback_data)\n \t\t\t\tstruct strbuf target = STRBUF_INIT;\n \t\t\t\tstrbuf_add(&target, base, serverlen);\n \t\t\t\tstrbuf_add(&target, data + i, posn - i - 7);\n-\t\t\t\twarning(\"adding alternate object store: %s\",\n-\t\t\t\t\ttarget.buf);\n-\t\t\t\tnewalt = xmalloc(sizeof(*newalt));\n-\t\t\t\tnewalt->next = NULL;\n-\t\t\t\tnewalt->base = strbuf_detach(&target, NULL);\n-\t\t\t\tnewalt->got_indices = 0;\n-\t\t\t\tnewalt->packs = NULL;\n-\n-\t\t\t\twhile (tail->next != NULL)\n-\t\t\t\t\ttail = tail->next;\n-\t\t\t\ttail->next = newalt;\n+\n+\t\t\t\tif (is_alternate_allowed(target.buf)) {\n+\t\t\t\t\twarning(\"adding alternate object store: %s\",\n+\t\t\t\t\t\ttarget.buf);\n+\t\t\t\t\tnewalt = xmalloc(sizeof(*newalt));\n+\t\t\t\t\tnewalt->next = NULL;\n+\t\t\t\t\tnewalt->base = strbuf_detach(&target, NULL);\n+\t\t\t\t\tnewalt->got_indices = 0;\n+\t\t\t\t\tnewalt->packs = NULL;\n+\n+\t\t\t\t\twhile (tail->next != NULL)\n+\t\t\t\t\t\ttail = tail->next;\n+\t\t\t\t\ttail->next = newalt;\n+\t\t\t\t}\n \t\t\t}\n \t\t}\n \t\ti = posn + 1;\ndiff --git a/t/t5550-http-fetch-dumb.sh b/t/t5550-http-fetch-dumb.sh\nindex 22011f0b6..c0ee29c65 100755\n--- a/t/t5550-http-fetch-dumb.sh\n+++ b/t/t5550-http-fetch-dumb.sh\n@@ -360,5 +360,15 @@ test_expect_success 'http-alternates cannot point at funny protocols' '\n \t\tclone \"$HTTPD_URL/dumb/evil.git\" evil-file\n '\n \n+test_expect_success 'http-alternates triggers not-from-user protocol check' '\n+\techo \"$HTTPD_URL/dumb/victim.git/objects\" \\\n+\t\t>\"$evil/objects/info/http-alternates\" &&\n+\ttest_config_global http.followRedirects true &&\n+\ttest_must_fail git -c protocol.http.allow=user \\\n+\t\tclone $HTTPD_URL/dumb/evil.git evil-user &&\n+\tgit -c protocol.http.allow=always \\\n+\t\tclone $HTTPD_URL/dumb/evil.git evil-user\n+'\n+\n stop_httpd\n test_done\n-- \n2.11.0.341.g202cd3142\n\n"},{"id":"307810","messageId":"20161214215832.GF20063@google.com","threadId":"44417","inReplyTo":"20161214211229.mpgu3hahlfbdxys6@sigill.intra.peff.net","subject":"Re: [PATCH v9 5/5] transport: add from_user parameter to is_transport_allowed","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-12-14T21:58:32Z","receivedAt":"2016-12-14T22:06:46Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 12/14, Jeff King wrote:\n> On Wed, Dec 14, 2016 at 03:33:49PM -0500, Jeff King wrote:\n> \n> > One other \"simple\" fix is at the moment we parse http-alternates, to\n> > parse the URL ourself and check the policy. Like:\n> > [...]\n> > I may have convinced myself this is a reasonable approach.\n> \n> So here it is in patch form, with a test.\n> \n> I also took a look at how bad it would be to plumb through the \"this is\n> an alternate\" flag. And it's actually a little nasty, because the\n> http-walker isn't calling get_active_slot() itself. It's relying on\n> http_get_file() and other wrappers. The recent http_get_options makes\n> this slightly less terrible, but I'd still rather avoid infecting the\n> general http code that is used for the smart-http code paths.\n> \n> So I think this patch on top of your series, plus the other minor fixes\n> we've discussed, the topic should be ready for 'next'.\n\nSounds good, I'll make those few changes, place this patch ontop of\nthe series and send out v10 of the series in just a bit.\n\n-- \nBrandon Williams\n"},{"id":"307811","messageId":"1481755195-174539-3-git-send-email-bmwill@google.com","threadId":"44417","inReplyTo":"1481755195-174539-1-git-send-email-bmwill@google.com","subject":"[PATCH v10 2/6] http: always warn if libcurl version is too old","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-12-14T22:39:51Z","receivedAt":"2016-12-14T22:40:23Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"Always warn if libcurl version is too old because:\n\n1. Even without a protocol whitelist, newer versions of curl have all\n non-http protocols disabled by default.\n2. A future patch will introduce default \"known-good\" and \"known-bad\"\n protocols which are allowed/disallowed by 'is_transport_allowed'\n which older version of libcurl can't respect.\n\nSigned-off-by: Brandon Williams \n---\n http.c | 5 ++---\n transport.c | 5 -----\n transport.h | 6 ------\n 3 files changed, 2 insertions(+), 14 deletions(-)\n\ndiff --git a/http.c b/http.c\nindex 5cd3ffd..034426e 100644\n--- a/http.c\n+++ b/http.c\n@@ -583,9 +583,8 @@ static CURL *get_curl_handle(void)\n \tcurl_easy_setopt(result, CURLOPT_REDIR_PROTOCOLS, allowed_protocols);\n \tcurl_easy_setopt(result, CURLOPT_PROTOCOLS, allowed_protocols);\n #else\n-\tif (transport_restrict_protocols())\n-\t\twarning(\"protocol restrictions not applied to curl redirects because\\n\"\n-\t\t\t\"your curl version is too old (>= 7.19.4)\");\n+\twarning(\"protocol restrictions not applied to curl redirects because\\n\"\n+\t\t\"your curl version is too old (>= 7.19.4)\");\n #endif\n \n \tif (getenv(\"GIT_CURL_VERBOSE\"))\ndiff --git a/transport.c b/transport.c\nindex 41eb82c..dff929e 100644\n--- a/transport.c\n+++ b/transport.c\n@@ -629,11 +629,6 @@ void transport_check_allowed(const char *type)\n \t\tdie(\"transport '%s' not allowed\", type);\n }\n \n-int transport_restrict_protocols(void)\n-{\n-\treturn !!protocol_whitelist();\n-}\n-\n struct transport *transport_get(struct remote *remote, const char *url)\n {\n \tconst char *helper;\ndiff --git a/transport.h b/transport.h\nindex c681408..3396e1d 100644\n--- a/transport.h\n+++ b/transport.h\n@@ -153,12 +153,6 @@ int is_transport_allowed(const char *type);\n */\n void transport_check_allowed(const char *type);\n \n-/*\n- * Returns true if the user has attempted to turn on protocol\n- * restrictions at all.\n- */\n-int transport_restrict_protocols(void);\n-\n /* Transport options which apply to git:// and scp-style URLs */\n \n /* The program to use on the remote side to send a pack */\n-- \n2.8.0.rc3.226.g39d4020\n\n"},{"id":"307812","messageId":"1481755195-174539-4-git-send-email-bmwill@google.com","threadId":"44417","inReplyTo":"1481755195-174539-1-git-send-email-bmwill@google.com","subject":"[PATCH v10 3/6] transport: add protocol policy config option","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-12-14T22:39:52Z","receivedAt":"2016-12-14T22:40:33Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"Previously the `GIT_ALLOW_PROTOCOL` environment variable was used to\nspecify a whitelist of protocols to be used in clone/fetch/push\ncommands. This patch introduces new configuration options for more\nfine-grained control for allowing/disallowing protocols. This also has\nthe added benefit of allowing easier construction of a protocol\nwhitelist on systems where setting an environment variable is\nnon-trivial.\n\nNow users can specify a policy to be used for each type of protocol via\nthe 'protocol..allow' config option. A default policy for all\nunconfigured protocols can be set with the 'protocol.allow' config\noption. If no user configured default is made git will allow known-safe\nprotocols (http, https, git, ssh, file), disallow known-dangerous\nprotocols (ext), and have a default policy of `user` for all other\nprotocols.\n\nThe supported policies are `always`, `never`, and `user`. The `user`\npolicy can be used to configure a protocol to be usable when explicitly\nused by a user, while disallowing it for commands which run\nclone/fetch/push commands without direct user intervention (e.g.\nrecursive initialization of submodules). Commands which can potentially\nclone/fetch/push from untrusted repositories without user intervention\ncan export `GIT_PROTOCOL_FROM_USER` with a value of '0' to prevent\nprotocols configured to the `user` policy from being used.\n\nFix remote-ext tests to use the new config to allow the ext\nprotocol to be tested.\n\nBased on a patch by Jeff King \n\nSigned-off-by: Brandon Williams \n---\n Documentation/config.txt | 46 ++++++++++++++\n Documentation/git.txt | 38 +++++-------\n git-submodule.sh | 12 ++--\n t/lib-proto-disable.sh | 130 +++++++++++++++++++++++++++++++++++++--\n t/t5509-fetch-push-namespaces.sh | 1 +\n t/t5802-connect-helper.sh | 1 +\n transport.c | 75 +++++++++++++++++++++-\n 7 files changed, 264 insertions(+), 39 deletions(-)\n\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex 8153336..50d3d06 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -2260,6 +2260,52 @@ pretty.::\n \tNote that an alias with the same name as a built-in format\n \twill be silently ignored.\n \n+protocol.allow::\n+\tIf set, provide a user defined default policy for all protocols which\n+\tdon't explicitly have a policy (`protocol..allow`). By default,\n+\tif unset, known-safe protocols (http, https, git, ssh, file) have a\n+\tdefault policy of `always`, known-dangerous protocols (ext) have a\n+\tdefault policy of `never`, and all other protocols have a default\n+\tpolicy of `user`. Supported policies:\n++\n+--\n+\n+* `always` - protocol is always able to be used.\n+\n+* `never` - protocol is never able to be used.\n+\n+* `user` - protocol is only able to be used when `GIT_PROTOCOL_FROM_USER` is\n+ either unset or has a value of 1. This policy should be used when you want a\n+ protocol to be directly usable by the user but don't want it used by commands which\n+ execute clone/fetch/push commands without user input, e.g. recursive\n+ submodule initialization.\n+\n+--\n+\n+protocol..allow::\n+\tSet a policy to be used by protocol `` with clone/fetch/push\n+\tcommands. See `protocol.allow` above for the available policies.\n++\n+The protocol names currently used by git are:\n++\n+--\n+ - `file`: any local file-based path (including `file://` URLs,\n+ or local paths)\n+\n+ - `git`: the anonymous git protocol over a direct TCP\n+ connection (or proxy, if configured)\n+\n+ - `ssh`: git over ssh (including `host:path` syntax,\n+ `ssh://`, etc).\n+\n+ - `http`: git over http, both \"smart http\" and \"dumb http\".\n+ Note that this does _not_ include `https`; if you want to configure\n+ both, you must do so individually.\n+\n+ - any external helpers are named by their protocol (e.g., use\n+ `hg` to allow the `git-remote-hg` helper)\n+--\n+\n pull.ff::\n \tBy default, Git does not create an extra merge commit when merging\n \ta commit that is a descendant of the current commit. Instead, the\ndiff --git a/Documentation/git.txt b/Documentation/git.txt\nindex 923aa49..d9fb937 100644\n--- a/Documentation/git.txt\n+++ b/Documentation/git.txt\n@@ -1129,30 +1129,20 @@ of clones and fetches.\n \tcloning a repository to make a backup).\n \n `GIT_ALLOW_PROTOCOL`::\n-\tIf set, provide a colon-separated list of protocols which are\n-\tallowed to be used with fetch/push/clone. This is useful to\n-\trestrict recursive submodule initialization from an untrusted\n-\trepository. Any protocol not mentioned will be disallowed (i.e.,\n-\tthis is a whitelist, not a blacklist). If the variable is not\n-\tset at all, all protocols are enabled. The protocol names\n-\tcurrently used by git are:\n-\n-\t - `file`: any local file-based path (including `file://` URLs,\n-\t or local paths)\n-\n-\t - `git`: the anonymous git protocol over a direct TCP\n-\t connection (or proxy, if configured)\n-\n-\t - `ssh`: git over ssh (including `host:path` syntax,\n-\t `ssh://`, etc).\n-\n-\t - `http`: git over http, both \"smart http\" and \"dumb http\".\n-\t Note that this does _not_ include `https`; if you want both,\n-\t you should specify both as `http:https`.\n-\n-\t - any external helpers are named by their protocol (e.g., use\n-\t `hg` to allow the `git-remote-hg` helper)\n-\n+\tIf set to a colon-separated list of protocols, behave as if\n+\t`protocol.allow` is set to `never`, and each of the listed\n+\tprotocols has `protocol..allow` set to `always`\n+\t(overriding any existing configuration). In other words, any\n+\tprotocol not mentioned will be disallowed (i.e., this is a\n+\twhitelist, not a blacklist). See the description of\n+\t`protocol.allow` in linkgit:git-config[1] for more details.\n+\n+`GIT_PROTOCOL_FROM_USER`::\n+\tSet to 0 to prevent protocols used by fetch/push/clone which are\n+\tconfigured to the `user` state. This is useful to restrict recursive\n+\tsubmodule initialization from an untrusted repository or for programs\n+\twhich feed potentially-untrusted URLS to git commands. See\n+\tlinkgit:git-config[1] for more details.\n \n Discussion[[Discussion]]\n ------------------------\ndiff --git a/git-submodule.sh b/git-submodule.sh\nindex 78fdac9..fc44076 100755\n--- a/git-submodule.sh\n+++ b/git-submodule.sh\n@@ -22,14 +22,10 @@ require_work_tree\n wt_prefix=$(git rev-parse --show-prefix)\n cd_to_toplevel\n \n-# Restrict ourselves to a vanilla subset of protocols; the URLs\n-# we get are under control of a remote repository, and we do not\n-# want them kicking off arbitrary git-remote-* programs.\n-#\n-# If the user has already specified a set of allowed protocols,\n-# we assume they know what they're doing and use that instead.\n-: ${GIT_ALLOW_PROTOCOL=file:git:http:https:ssh}\n-export GIT_ALLOW_PROTOCOL\n+# Tell the rest of git that any URLs we get don't come\n+# directly from the user, so it can apply policy as appropriate.\n+GIT_PROTOCOL_FROM_USER=0\n+export GIT_PROTOCOL_FROM_USER\n \n command=\n branch=\ndiff --git a/t/lib-proto-disable.sh b/t/lib-proto-disable.sh\nindex be88e9a..02f49cb 100644\n--- a/t/lib-proto-disable.sh\n+++ b/t/lib-proto-disable.sh\n@@ -1,10 +1,7 @@\n # Test routines for checking protocol disabling.\n \n-# test cloning a particular protocol\n-# $1 - description of the protocol\n-# $2 - machine-readable name of the protocol\n-# $3 - the URL to try cloning\n-test_proto () {\n+# Test clone/fetch/push with GIT_ALLOW_PROTOCOL whitelist\n+test_whitelist () {\n \tdesc=$1\n \tproto=$2\n \turl=$3\n@@ -62,6 +59,129 @@ test_proto () {\n \t\t\ttest_must_fail git clone --bare \"$url\" tmp.git\n \t\t)\n \t'\n+\n+\ttest_expect_success \"clone $desc (env var has precedence)\" '\n+\t\trm -rf tmp.git &&\n+\t\t(\n+\t\t\tGIT_ALLOW_PROTOCOL=none &&\n+\t\t\texport GIT_ALLOW_PROTOCOL &&\n+\t\t\ttest_must_fail git -c protocol.allow=always clone --bare \"$url\" tmp.git &&\n+\t\t\ttest_must_fail git -c protocol.$proto.allow=always clone --bare \"$url\" tmp.git\n+\t\t)\n+\t'\n+}\n+\n+test_config () {\n+\tdesc=$1\n+\tproto=$2\n+\turl=$3\n+\n+\t# Test clone/fetch/push with protocol..allow config\n+\ttest_expect_success \"clone $desc (enabled with config)\" '\n+\t\trm -rf tmp.git &&\n+\t\tgit -c protocol.$proto.allow=always clone --bare \"$url\" tmp.git\n+\t'\n+\n+\ttest_expect_success \"fetch $desc (enabled)\" '\n+\t\tgit -C tmp.git -c protocol.$proto.allow=always fetch\n+\t'\n+\n+\ttest_expect_success \"push $desc (enabled)\" '\n+\t\tgit -C tmp.git -c protocol.$proto.allow=always push origin HEAD:pushed\n+\t'\n+\n+\ttest_expect_success \"push $desc (disabled)\" '\n+\t\ttest_must_fail git -C tmp.git -c protocol.$proto.allow=never push origin HEAD:pushed\n+\t'\n+\n+\ttest_expect_success \"fetch $desc (disabled)\" '\n+\t\ttest_must_fail git -C tmp.git -c protocol.$proto.allow=never fetch\n+\t'\n+\n+\ttest_expect_success \"clone $desc (disabled)\" '\n+\t\trm -rf tmp.git &&\n+\t\ttest_must_fail git -c protocol.$proto.allow=never clone --bare \"$url\" tmp.git\n+\t'\n+\n+\t# Test clone/fetch/push with protocol.user.allow and its env var\n+\ttest_expect_success \"clone $desc (enabled)\" '\n+\t\trm -rf tmp.git &&\n+\t\tgit -c protocol.$proto.allow=user clone --bare \"$url\" tmp.git\n+\t'\n+\n+\ttest_expect_success \"fetch $desc (enabled)\" '\n+\t\tgit -C tmp.git -c protocol.$proto.allow=user fetch\n+\t'\n+\n+\ttest_expect_success \"push $desc (enabled)\" '\n+\t\tgit -C tmp.git -c protocol.$proto.allow=user push origin HEAD:pushed\n+\t'\n+\n+\ttest_expect_success \"push $desc (disabled)\" '\n+\t\t(\n+\t\t\tcd tmp.git &&\n+\t\t\tGIT_PROTOCOL_FROM_USER=0 &&\n+\t\t\texport GIT_PROTOCOL_FROM_USER &&\n+\t\t\ttest_must_fail git -c protocol.$proto.allow=user push origin HEAD:pushed\n+\t\t)\n+\t'\n+\n+\ttest_expect_success \"fetch $desc (disabled)\" '\n+\t\t(\n+\t\t\tcd tmp.git &&\n+\t\t\tGIT_PROTOCOL_FROM_USER=0 &&\n+\t\t\texport GIT_PROTOCOL_FROM_USER &&\n+\t\t\ttest_must_fail git -c protocol.$proto.allow=user fetch\n+\t\t)\n+\t'\n+\n+\ttest_expect_success \"clone $desc (disabled)\" '\n+\t\trm -rf tmp.git &&\n+\t\t(\n+\t\t\tGIT_PROTOCOL_FROM_USER=0 &&\n+\t\t\texport GIT_PROTOCOL_FROM_USER &&\n+\t\t\ttest_must_fail git -c protocol.$proto.allow=user clone --bare \"$url\" tmp.git\n+\t\t)\n+\t'\n+\n+\t# Test clone/fetch/push with protocol.allow user defined default\n+\ttest_expect_success \"clone $desc (enabled)\" '\n+\t\trm -rf tmp.git &&\n+\t\tgit config --global protocol.allow always &&\n+\t\tgit clone --bare \"$url\" tmp.git\n+\t'\n+\n+\ttest_expect_success \"fetch $desc (enabled)\" '\n+\t\tgit -C tmp.git fetch\n+\t'\n+\n+\ttest_expect_success \"push $desc (enabled)\" '\n+\t\tgit -C tmp.git push origin HEAD:pushed\n+\t'\n+\n+\ttest_expect_success \"push $desc (disabled)\" '\n+\t\tgit config --global protocol.allow never &&\n+\t\ttest_must_fail git -C tmp.git push origin HEAD:pushed\n+\t'\n+\n+\ttest_expect_success \"fetch $desc (disabled)\" '\n+\t\ttest_must_fail git -C tmp.git fetch\n+\t'\n+\n+\ttest_expect_success \"clone $desc (disabled)\" '\n+\t\trm -rf tmp.git &&\n+\t\ttest_must_fail git clone --bare \"$url\" tmp.git\n+\t'\n+}\n+\n+# test cloning a particular protocol\n+# $1 - description of the protocol\n+# $2 - machine-readable name of the protocol\n+# $3 - the URL to try cloning\n+test_proto () {\n+\ttest_whitelist \"$@\"\n+\n+\ttest_config \"$@\"\n }\n \n # set up an ssh wrapper that will access $host/$repo in the\ndiff --git a/t/t5509-fetch-push-namespaces.sh b/t/t5509-fetch-push-namespaces.sh\nindex bc44ac3..75c570a 100755\n--- a/t/t5509-fetch-push-namespaces.sh\n+++ b/t/t5509-fetch-push-namespaces.sh\n@@ -4,6 +4,7 @@ test_description='fetch/push involving ref namespaces'\n . ./test-lib.sh\n \n test_expect_success setup '\n+\tgit config --global protocol.ext.allow user &&\n \ttest_tick &&\n \tgit init original &&\n \t(\ndiff --git a/t/t5802-connect-helper.sh b/t/t5802-connect-helper.sh\nindex b7a7f9d..c6c2661 100755\n--- a/t/t5802-connect-helper.sh\n+++ b/t/t5802-connect-helper.sh\n@@ -4,6 +4,7 @@ test_description='ext::cmd remote \"connect\" helper'\n . ./test-lib.sh\n \n test_expect_success setup '\n+\tgit config --global protocol.ext.allow user &&\n \ttest_tick &&\n \tgit commit --allow-empty -m initial &&\n \ttest_tick &&\ndiff --git a/transport.c b/transport.c\nindex dff929e..fbd799d 100644\n--- a/transport.c\n+++ b/transport.c\n@@ -617,10 +617,81 @@ static const struct string_list *protocol_whitelist(void)\n \treturn enabled ? &allowed : NULL;\n }\n \n+enum protocol_allow_config {\n+\tPROTOCOL_ALLOW_NEVER = 0,\n+\tPROTOCOL_ALLOW_USER_ONLY,\n+\tPROTOCOL_ALLOW_ALWAYS\n+};\n+\n+static enum protocol_allow_config parse_protocol_config(const char *key,\n+\t\t\t\t\t\t\tconst char *value)\n+{\n+\tif (!strcasecmp(value, \"always\"))\n+\t\treturn PROTOCOL_ALLOW_ALWAYS;\n+\telse if (!strcasecmp(value, \"never\"))\n+\t\treturn PROTOCOL_ALLOW_NEVER;\n+\telse if (!strcasecmp(value, \"user\"))\n+\t\treturn PROTOCOL_ALLOW_USER_ONLY;\n+\n+\tdie(\"unknown value for config '%s': %s\", key, value);\n+}\n+\n+static enum protocol_allow_config get_protocol_config(const char *type)\n+{\n+\tchar *key = xstrfmt(\"protocol.%s.allow\", type);\n+\tchar *value;\n+\n+\t/* first check the per-protocol config */\n+\tif (!git_config_get_string(key, &value)) {\n+\t\tenum protocol_allow_config ret =\n+\t\t\tparse_protocol_config(key, value);\n+\t\tfree(key);\n+\t\tfree(value);\n+\t\treturn ret;\n+\t}\n+\tfree(key);\n+\n+\t/* if defined, fallback to user-defined default for unknown protocols */\n+\tif (!git_config_get_string(\"protocol.allow\", &value)) {\n+\t\tenum protocol_allow_config ret =\n+\t\t\tparse_protocol_config(\"protocol.allow\", value);\n+\t\tfree(value);\n+\t\treturn ret;\n+\t}\n+\n+\t/* fallback to built-in defaults */\n+\t/* known safe */\n+\tif (!strcmp(type, \"http\") ||\n+\t !strcmp(type, \"https\") ||\n+\t !strcmp(type, \"git\") ||\n+\t !strcmp(type, \"ssh\") ||\n+\t !strcmp(type, \"file\"))\n+\t\treturn PROTOCOL_ALLOW_ALWAYS;\n+\n+\t/* known scary; err on the side of caution */\n+\tif (!strcmp(type, \"ext\"))\n+\t\treturn PROTOCOL_ALLOW_NEVER;\n+\n+\t/* unknown; by default let them be used only directly by the user */\n+\treturn PROTOCOL_ALLOW_USER_ONLY;\n+}\n+\n int is_transport_allowed(const char *type)\n {\n-\tconst struct string_list *allowed = protocol_whitelist();\n-\treturn !allowed || string_list_has_string(allowed, type);\n+\tconst struct string_list *whitelist = protocol_whitelist();\n+\tif (whitelist)\n+\t\treturn string_list_has_string(whitelist, type);\n+\n+\tswitch (get_protocol_config(type)) {\n+\tcase PROTOCOL_ALLOW_ALWAYS:\n+\t\treturn 1;\n+\tcase PROTOCOL_ALLOW_NEVER:\n+\t\treturn 0;\n+\tcase PROTOCOL_ALLOW_USER_ONLY:\n+\t\treturn git_env_bool(\"GIT_PROTOCOL_FROM_USER\", 1);\n+\t}\n+\n+\tdie(\"BUG: invalid protocol_allow_config type\");\n }\n \n void transport_check_allowed(const char *type)\n-- \n2.8.0.rc3.226.g39d4020\n\n"},{"id":"307813","messageId":"1481755195-174539-1-git-send-email-bmwill@google.com","threadId":"44417","inReplyTo":"1481679637-133137-1-git-send-email-bmwill@google.com","subject":"[PATCH v10 0/6] transport protocol policy configuration","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-12-14T22:39:49Z","receivedAt":"2016-12-14T22:40:38Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"v10 of this series fixes the following:\n* A few updates to the commit messages in order to better convey the reasoning\n behind the a few of the patches.\n* Additional test to verify that curl redirects respect configured protocol\n policies.\n* Patch added by Jeff King to make http alternates respect configured\n protocol policies.\n\nBrandon Williams (5):\n lib-proto-disable: variable name fix\n http: always warn if libcurl version is too old\n transport: add protocol policy config option\n http: create function to get curl allowed protocols\n transport: add from_user parameter to is_transport_allowed\n\nJeff King (1):\n http: respect protocol.*.allow=user for http-alternates\n\n Documentation/config.txt | 46 +++++++++++++\n Documentation/git.txt | 38 ++++-------\n git-submodule.sh | 12 ++--\n http-walker.c | 52 +++++++++++---\n http.c | 36 ++++++----\n t/lib-proto-disable.sh | 142 ++++++++++++++++++++++++++++++++++++---\n t/t5509-fetch-push-namespaces.sh | 1 +\n t/t5550-http-fetch-dumb.sh | 10 +++\n t/t5802-connect-helper.sh | 1 +\n t/t5812-proto-disable-http.sh | 7 ++\n transport.c | 84 ++++++++++++++++++++---\n transport.h | 19 +++---\n 12 files changed, 363 insertions(+), 85 deletions(-)\n\n-- \n2.8.0.rc3.226.g39d4020\n\n"},{"id":"307814","messageId":"1481755195-174539-6-git-send-email-bmwill@google.com","threadId":"44417","inReplyTo":"1481755195-174539-1-git-send-email-bmwill@google.com","subject":"[PATCH v10 5/6] transport: add from_user parameter to is_transport_allowed","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-12-14T22:39:54Z","receivedAt":"2016-12-14T22:40:40Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"Add a from_user parameter to is_transport_allowed() to allow http to be\nable to distinguish between protocol restrictions for redirects versus\ninitial requests. CURLOPT_REDIR_PROTOCOLS can now be set differently\nfrom CURLOPT_PROTOCOLS to disallow use of protocols with the \"user\"\npolicy in redirects.\n\nThis change allows callers to query if a transport protocol is allowed,\ngiven that the caller knows that the protocol is coming from the user\n(1) or not from the user (0) such as redirects in libcurl. If unknown a\n-1 should be provided which falls back to reading\n`GIT_PROTOCOL_FROM_USER` to determine if the protocol came from the\nuser.\n\nSigned-off-by: Brandon Williams \n---\n http.c | 14 +++++++-------\n t/t5812-proto-disable-http.sh | 7 +++++++\n transport.c | 8 +++++---\n transport.h | 13 ++++++++++---\n 4 files changed, 29 insertions(+), 13 deletions(-)\n\ndiff --git a/http.c b/http.c\nindex f7c488a..2208269 100644\n--- a/http.c\n+++ b/http.c\n@@ -489,17 +489,17 @@ static void set_curl_keepalive(CURL *c)\n }\n #endif\n \n-static long get_curl_allowed_protocols(void)\n+static long get_curl_allowed_protocols(int from_user)\n {\n \tlong allowed_protocols = 0;\n \n-\tif (is_transport_allowed(\"http\"))\n+\tif (is_transport_allowed(\"http\", from_user))\n \t\tallowed_protocols |= CURLPROTO_HTTP;\n-\tif (is_transport_allowed(\"https\"))\n+\tif (is_transport_allowed(\"https\", from_user))\n \t\tallowed_protocols |= CURLPROTO_HTTPS;\n-\tif (is_transport_allowed(\"ftp\"))\n+\tif (is_transport_allowed(\"ftp\", from_user))\n \t\tallowed_protocols |= CURLPROTO_FTP;\n-\tif (is_transport_allowed(\"ftps\"))\n+\tif (is_transport_allowed(\"ftps\", from_user))\n \t\tallowed_protocols |= CURLPROTO_FTPS;\n \n \treturn allowed_protocols;\n@@ -588,9 +588,9 @@ static CURL *get_curl_handle(void)\n #endif\n #if LIBCURL_VERSION_NUM >= 0x071304\n \tcurl_easy_setopt(result, CURLOPT_REDIR_PROTOCOLS,\n-\t\t\t get_curl_allowed_protocols());\n+\t\t\t get_curl_allowed_protocols(0));\n \tcurl_easy_setopt(result, CURLOPT_PROTOCOLS,\n-\t\t\t get_curl_allowed_protocols());\n+\t\t\t get_curl_allowed_protocols(-1));\n #else\n \twarning(\"protocol restrictions not applied to curl redirects because\\n\"\n \t\t\"your curl version is too old (>= 7.19.4)\");\ndiff --git a/t/t5812-proto-disable-http.sh b/t/t5812-proto-disable-http.sh\nindex 044cc15..d911afd 100755\n--- a/t/t5812-proto-disable-http.sh\n+++ b/t/t5812-proto-disable-http.sh\n@@ -30,5 +30,12 @@ test_expect_success 'curl limits redirects' '\n \ttest_must_fail git clone \"$HTTPD_URL/loop-redir/smart/repo.git\"\n '\n \n+test_expect_success 'http can be limited to from-user' '\n+\tgit -c protocol.http.allow=user \\\n+\t\tclone \"$HTTPD_URL/smart/repo.git\" plain.git &&\n+\ttest_must_fail git -c protocol.http.allow=user \\\n+\t\tclone \"$HTTPD_URL/smart-redir-perm/repo.git\" redir.git\n+'\n+\n stop_httpd\n test_done\ndiff --git a/transport.c b/transport.c\nindex fbd799d..f50c31a 100644\n--- a/transport.c\n+++ b/transport.c\n@@ -676,7 +676,7 @@ static enum protocol_allow_config get_protocol_config(const char *type)\n \treturn PROTOCOL_ALLOW_USER_ONLY;\n }\n \n-int is_transport_allowed(const char *type)\n+int is_transport_allowed(const char *type, int from_user)\n {\n \tconst struct string_list *whitelist = protocol_whitelist();\n \tif (whitelist)\n@@ -688,7 +688,9 @@ int is_transport_allowed(const char *type)\n \tcase PROTOCOL_ALLOW_NEVER:\n \t\treturn 0;\n \tcase PROTOCOL_ALLOW_USER_ONLY:\n-\t\treturn git_env_bool(\"GIT_PROTOCOL_FROM_USER\", 1);\n+\t\tif (from_user < 0)\n+\t\t\tfrom_user = git_env_bool(\"GIT_PROTOCOL_FROM_USER\", 1);\n+\t\treturn from_user;\n \t}\n \n \tdie(\"BUG: invalid protocol_allow_config type\");\n@@ -696,7 +698,7 @@ int is_transport_allowed(const char *type)\n \n void transport_check_allowed(const char *type)\n {\n-\tif (!is_transport_allowed(type))\n+\tif (!is_transport_allowed(type, -1))\n \t\tdie(\"transport '%s' not allowed\", type);\n }\n \ndiff --git a/transport.h b/transport.h\nindex 3396e1d..4f1c801 100644\n--- a/transport.h\n+++ b/transport.h\n@@ -142,10 +142,17 @@ struct transport {\n struct transport *transport_get(struct remote *, const char *);\n \n /*\n- * Check whether a transport is allowed by the environment. Type should\n- * generally be the URL scheme, as described in Documentation/git.txt\n+ * Check whether a transport is allowed by the environment.\n+ *\n+ * Type should generally be the URL scheme, as described in\n+ * Documentation/git.txt\n+ *\n+ * from_user specifies if the transport was given by the user. If unknown pass\n+ * a -1 to read from the environment to determine if the transport was given by\n+ * the user.\n+ *\n */\n-int is_transport_allowed(const char *type);\n+int is_transport_allowed(const char *type, int from_user);\n \n /*\n * Check whether a transport is allowed by the environment,\n-- \n2.8.0.rc3.226.g39d4020\n\n"},{"id":"307815","messageId":"1481755195-174539-7-git-send-email-bmwill@google.com","threadId":"44417","inReplyTo":"1481755195-174539-1-git-send-email-bmwill@google.com","subject":"[PATCH v10 6/6] http: respect protocol.*.allow=user for http-alternates","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-12-14T22:39:55Z","receivedAt":"2016-12-14T22:40:41Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"From: Jeff King \n\nThe http-walker may fetch the http-alternates (or\nalternates) file from a remote in order to find more\nobjects. This should count as a \"not from the user\" use of\nthe protocol. But because we implement the redirection\nourselves and feed the new URL to curl, it will use the\nCURLOPT_PROTOCOLS rules, not the more restrictive\nCURLOPT_REDIR_PROTOCOLS.\n\nThe ideal solution would be for each curl request we make to\nknow whether or not is directly from the user or part of an\nalternates redirect, and then set CURLOPT_PROTOCOLS as\nappropriate. However, that would require plumbing that\ninformation through all of the various layers of the http\ncode.\n\nInstead, let's check the protocol at the source: when we are\nparsing the remote http-alternates file. The only downside\nis that if there's any mismatch between what protocol we\nthink it is versus what curl thinks it is, it could violate\nthe policy.\n\nTo address this, we'll make the parsing err on the picky\nside, and only allow protocols that it can parse\ndefinitively. So for example, you can't elude the \"http\"\npolicy by asking for \"HTTP://\", even though curl might\nhandle it; we would reject it as unknown. The only unsafe\ncase would be if you have a URL that starts with \"http://\"\nbut curl interprets as another protocol. That seems like an\nunlikely failure mode (and we are still protected by our\nbase CURLOPT_PROTOCOL setting, so the worst you could do is\ntrigger one of https, ftp, or ftps).\n\nSigned-off-by: Jeff King \n---\n http-walker.c | 52 ++++++++++++++++++++++++++++++++++++----------\n t/t5550-http-fetch-dumb.sh | 10 +++++++++\n 2 files changed, 51 insertions(+), 11 deletions(-)\n\ndiff --git a/http-walker.c b/http-walker.c\nindex c2f81cd..b34b6ac 100644\n--- a/http-walker.c\n+++ b/http-walker.c\n@@ -3,6 +3,7 @@\n #include \"walker.h\"\n #include \"http.h\"\n #include \"list.h\"\n+#include \"transport.h\"\n \n struct alt_base {\n \tchar *base;\n@@ -160,6 +161,32 @@ static void prefetch(struct walker *walker, unsigned char *sha1)\n #endif\n }\n \n+static int is_alternate_allowed(const char *url)\n+{\n+\tconst char *protocols[] = {\n+\t\t\"http\", \"https\", \"ftp\", \"ftps\"\n+\t};\n+\tint i;\n+\n+\tfor (i = 0; i < ARRAY_SIZE(protocols); i++) {\n+\t\tconst char *end;\n+\t\tif (skip_prefix(url, protocols[i], &end) &&\n+\t\t starts_with(end, \"://\"))\n+\t\t\tbreak;\n+\t}\n+\n+\tif (i >= ARRAY_SIZE(protocols)) {\n+\t\twarning(\"ignoring alternate with unknown protocol: %s\", url);\n+\t\treturn 0;\n+\t}\n+\tif (!is_transport_allowed(protocols[i], 0)) {\n+\t\twarning(\"ignoring alternate with restricted protocol: %s\", url);\n+\t\treturn 0;\n+\t}\n+\n+\treturn 1;\n+}\n+\n static void process_alternates_response(void *callback_data)\n {\n \tstruct alternates_request *alt_req =\n@@ -274,17 +301,20 @@ static void process_alternates_response(void *callback_data)\n \t\t\t\tstruct strbuf target = STRBUF_INIT;\n \t\t\t\tstrbuf_add(&target, base, serverlen);\n \t\t\t\tstrbuf_add(&target, data + i, posn - i - 7);\n-\t\t\t\twarning(\"adding alternate object store: %s\",\n-\t\t\t\t\ttarget.buf);\n-\t\t\t\tnewalt = xmalloc(sizeof(*newalt));\n-\t\t\t\tnewalt->next = NULL;\n-\t\t\t\tnewalt->base = strbuf_detach(&target, NULL);\n-\t\t\t\tnewalt->got_indices = 0;\n-\t\t\t\tnewalt->packs = NULL;\n-\n-\t\t\t\twhile (tail->next != NULL)\n-\t\t\t\t\ttail = tail->next;\n-\t\t\t\ttail->next = newalt;\n+\n+\t\t\t\tif (is_alternate_allowed(target.buf)) {\n+\t\t\t\t\twarning(\"adding alternate object store: %s\",\n+\t\t\t\t\t\ttarget.buf);\n+\t\t\t\t\tnewalt = xmalloc(sizeof(*newalt));\n+\t\t\t\t\tnewalt->next = NULL;\n+\t\t\t\t\tnewalt->base = strbuf_detach(&target, NULL);\n+\t\t\t\t\tnewalt->got_indices = 0;\n+\t\t\t\t\tnewalt->packs = NULL;\n+\n+\t\t\t\t\twhile (tail->next != NULL)\n+\t\t\t\t\t\ttail = tail->next;\n+\t\t\t\t\ttail->next = newalt;\n+\t\t\t\t}\n \t\t\t}\n \t\t}\n \t\ti = posn + 1;\ndiff --git a/t/t5550-http-fetch-dumb.sh b/t/t5550-http-fetch-dumb.sh\nindex 22011f0..c0ee29c 100755\n--- a/t/t5550-http-fetch-dumb.sh\n+++ b/t/t5550-http-fetch-dumb.sh\n@@ -360,5 +360,15 @@ test_expect_success 'http-alternates cannot point at funny protocols' '\n \t\tclone \"$HTTPD_URL/dumb/evil.git\" evil-file\n '\n \n+test_expect_success 'http-alternates triggers not-from-user protocol check' '\n+\techo \"$HTTPD_URL/dumb/victim.git/objects\" \\\n+\t\t>\"$evil/objects/info/http-alternates\" &&\n+\ttest_config_global http.followRedirects true &&\n+\ttest_must_fail git -c protocol.http.allow=user \\\n+\t\tclone $HTTPD_URL/dumb/evil.git evil-user &&\n+\tgit -c protocol.http.allow=always \\\n+\t\tclone $HTTPD_URL/dumb/evil.git evil-user\n+'\n+\n stop_httpd\n test_done\n-- \n2.8.0.rc3.226.g39d4020\n\n"},{"id":"307816","messageId":"1481755195-174539-5-git-send-email-bmwill@google.com","threadId":"44417","inReplyTo":"1481755195-174539-1-git-send-email-bmwill@google.com","subject":"[PATCH v10 4/6] http: create function to get curl allowed protocols","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-12-14T22:39:53Z","receivedAt":"2016-12-14T22:41:17Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"Move the creation of an allowed protocols whitelist to a helper\nfunction. This will be useful when we need to compute the set of\nallowed protocols differently for normal and redirect cases.\n\nSigned-off-by: Brandon Williams \n---\n http.c | 31 ++++++++++++++++++++-----------\n 1 file changed, 20 insertions(+), 11 deletions(-)\n\ndiff --git a/http.c b/http.c\nindex 034426e..f7c488a 100644\n--- a/http.c\n+++ b/http.c\n@@ -489,10 +489,25 @@ static void set_curl_keepalive(CURL *c)\n }\n #endif\n \n+static long get_curl_allowed_protocols(void)\n+{\n+\tlong allowed_protocols = 0;\n+\n+\tif (is_transport_allowed(\"http\"))\n+\t\tallowed_protocols |= CURLPROTO_HTTP;\n+\tif (is_transport_allowed(\"https\"))\n+\t\tallowed_protocols |= CURLPROTO_HTTPS;\n+\tif (is_transport_allowed(\"ftp\"))\n+\t\tallowed_protocols |= CURLPROTO_FTP;\n+\tif (is_transport_allowed(\"ftps\"))\n+\t\tallowed_protocols |= CURLPROTO_FTPS;\n+\n+\treturn allowed_protocols;\n+}\n+\n static CURL *get_curl_handle(void)\n {\n \tCURL *result = curl_easy_init();\n-\tlong allowed_protocols = 0;\n \n \tif (!result)\n \t\tdie(\"curl_easy_init failed\");\n@@ -572,16 +587,10 @@ static CURL *get_curl_handle(void)\n \tcurl_easy_setopt(result, CURLOPT_POST301, 1);\n #endif\n #if LIBCURL_VERSION_NUM >= 0x071304\n-\tif (is_transport_allowed(\"http\"))\n-\t\tallowed_protocols |= CURLPROTO_HTTP;\n-\tif (is_transport_allowed(\"https\"))\n-\t\tallowed_protocols |= CURLPROTO_HTTPS;\n-\tif (is_transport_allowed(\"ftp\"))\n-\t\tallowed_protocols |= CURLPROTO_FTP;\n-\tif (is_transport_allowed(\"ftps\"))\n-\t\tallowed_protocols |= CURLPROTO_FTPS;\n-\tcurl_easy_setopt(result, CURLOPT_REDIR_PROTOCOLS, allowed_protocols);\n-\tcurl_easy_setopt(result, CURLOPT_PROTOCOLS, allowed_protocols);\n+\tcurl_easy_setopt(result, CURLOPT_REDIR_PROTOCOLS,\n+\t\t\t get_curl_allowed_protocols());\n+\tcurl_easy_setopt(result, CURLOPT_PROTOCOLS,\n+\t\t\t get_curl_allowed_protocols());\n #else\n \twarning(\"protocol restrictions not applied to curl redirects because\\n\"\n \t\t\"your curl version is too old (>= 7.19.4)\");\n-- \n2.8.0.rc3.226.g39d4020\n\n"},{"id":"307817","messageId":"1481755195-174539-2-git-send-email-bmwill@google.com","threadId":"44417","inReplyTo":"1481755195-174539-1-git-send-email-bmwill@google.com","subject":"[PATCH v10 1/6] lib-proto-disable: variable name fix","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-12-14T22:39:50Z","receivedAt":"2016-12-14T22:41:20Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"The test_proto function assigns the positional parameters to named\nvariables, but then still refers to \"$desc\" as \"$1\". Using $desc is\nmore readable and less error-prone.\n\nSigned-off-by: Brandon Williams \n---\n t/lib-proto-disable.sh | 12 ++++++------\n 1 file changed, 6 insertions(+), 6 deletions(-)\n\ndiff --git a/t/lib-proto-disable.sh b/t/lib-proto-disable.sh\nindex b0917d9..be88e9a 100644\n--- a/t/lib-proto-disable.sh\n+++ b/t/lib-proto-disable.sh\n@@ -9,7 +9,7 @@ test_proto () {\n \tproto=$2\n \turl=$3\n \n-\ttest_expect_success \"clone $1 (enabled)\" '\n+\ttest_expect_success \"clone $desc (enabled)\" '\n \t\trm -rf tmp.git &&\n \t\t(\n \t\t\tGIT_ALLOW_PROTOCOL=$proto &&\n@@ -18,7 +18,7 @@ test_proto () {\n \t\t)\n \t'\n \n-\ttest_expect_success \"fetch $1 (enabled)\" '\n+\ttest_expect_success \"fetch $desc (enabled)\" '\n \t\t(\n \t\t\tcd tmp.git &&\n \t\t\tGIT_ALLOW_PROTOCOL=$proto &&\n@@ -27,7 +27,7 @@ test_proto () {\n \t\t)\n \t'\n \n-\ttest_expect_success \"push $1 (enabled)\" '\n+\ttest_expect_success \"push $desc (enabled)\" '\n \t\t(\n \t\t\tcd tmp.git &&\n \t\t\tGIT_ALLOW_PROTOCOL=$proto &&\n@@ -36,7 +36,7 @@ test_proto () {\n \t\t)\n \t'\n \n-\ttest_expect_success \"push $1 (disabled)\" '\n+\ttest_expect_success \"push $desc (disabled)\" '\n \t\t(\n \t\t\tcd tmp.git &&\n \t\t\tGIT_ALLOW_PROTOCOL=none &&\n@@ -45,7 +45,7 @@ test_proto () {\n \t\t)\n \t'\n \n-\ttest_expect_success \"fetch $1 (disabled)\" '\n+\ttest_expect_success \"fetch $desc (disabled)\" '\n \t\t(\n \t\t\tcd tmp.git &&\n \t\t\tGIT_ALLOW_PROTOCOL=none &&\n@@ -54,7 +54,7 @@ test_proto () {\n \t\t)\n \t'\n \n-\ttest_expect_success \"clone $1 (disabled)\" '\n+\ttest_expect_success \"clone $desc (disabled)\" '\n \t\trm -rf tmp.git &&\n \t\t(\n \t\t\tGIT_ALLOW_PROTOCOL=none &&\n-- \n2.8.0.rc3.226.g39d4020\n\n"},{"id":"307823","messageId":"20161214225259.gk4er3v5izxn7kd4@sigill.intra.peff.net","threadId":"44417","inReplyTo":"CAP3OtXhH++szRws20MaHt-ftLBMUJuYiTmfL50mOFP4FA4Mn6Q@mail.gmail.com","subject":"Re: [PATCH v9 5/5] transport: add from_user parameter to is_transport_allowed","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-12-14T22:52:59Z","receivedAt":"2016-12-14T22:53:40Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Dec 14, 2016 at 04:29:52PM -0600, Blake Burkhart wrote:\n\n> You may want to set CURLOPT_DEFAULT_PROTOCOL if we don't already. Apparently\n> the default value of NULL causes it to make a guess based on the host if no\n> protocol is present. But you are discussing a situation where \"http://\" is\n> present, so that doesn't apply.\n\nCute. I agree it doesn't matter here, where we're sure there's a\nprotocol specifier at the beginning. It might matter if you instruct git\nto use a specific remote-helper, like:\n\n $ echo 127.0.0.1 ftp.example.com >>/etc/hosts\n $ git clone http::ftp.example.com\n\nwhich will try to connect via ftp. Of course that's no different than:\n\n $ git clone http::ftp://example.com\n\nwhich you can already do. git-clone sees \"http::\" and hands it off to\ngit-remote-http, which then processes the rest of the arguments as it\nsees fit (in this case, handing it off to curl).\n\nPrior to these more recent patches I suspect you could do:\n\n $ git clone http::file://whatever\n\nbut now we set CURLOPT_PROTOCOL to restrict it to just http/ftp (so you\ncan be confusing by asking for http and ending up in curl to do ftp, but\nin any such case you could also have just asked for ftp in the first\nplace).\n\n> Also, I thought we left out ftp because it was deprecated, but I don't\n> remember exactly.\n\nI couldn't find anything interesting in the archives (neither the public\nlist nor git-security). Given how unlikely it is to be used, it does\nseem like a good idea to keep it in the \"maybe\" category, if only\nbecause it decreases the attack surface (and for those following along,\nwe're just talking about not-from-user uses here, so people sticking a\nfunny ftp URL in .gitmodules, or redirecting http to ftp, etc).\n\n-Peff\n"},{"id":"307825","messageId":"xmqqtwa6w0si.fsf@gitster.mtv.corp.google.com","threadId":"44417","inReplyTo":"1481755195-174539-1-git-send-email-bmwill@google.com","subject":"Re: [PATCH v10 0/6] transport protocol policy configuration","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-12-14T23:25:49Z","receivedAt":"2016-12-14T23:25:59Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Brandon Williams writes:\n\n> v10 of this series fixes the following:\n> * A few updates to the commit messages in order to better convey the reasoning\n> behind the a few of the patches.\n> * Additional test to verify that curl redirects respect configured protocol\n> policies.\n> * Patch added by Jeff King to make http alternates respect configured\n> protocol policies.\n\nThanks. Will replace the previous one.\n"},{"id":"307828","messageId":"20161215002120.xlnlquuqqw25wngc@sigill.intra.peff.net","threadId":"44417","inReplyTo":"1481755195-174539-3-git-send-email-bmwill@google.com","subject":"Re: [PATCH v10 2/6] http: always warn if libcurl version is too old","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-12-15T00:21:20Z","receivedAt":"2016-12-15T00:21:31Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Dec 14, 2016 at 02:39:51PM -0800, Brandon Williams wrote:\n\n> Always warn if libcurl version is too old because:\n> \n> 1. Even without a protocol whitelist, newer versions of curl have all\n> non-http protocols disabled by default.\n\nTechnically, non-http and non-ftp. Maybe just \"non-standard\" would be\nmore accurate.\n\nNot worth a re-roll, but if Junio hasn't applied yet, maybe worth fixing\nup while applying.\n\n-Peff\n"},{"id":"307829","messageId":"20161215002224.k7uatnh6lwdrxrow@sigill.intra.peff.net","threadId":"44417","inReplyTo":"1481755195-174539-1-git-send-email-bmwill@google.com","subject":"Re: [PATCH v10 0/6] transport protocol policy configuration","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-12-15T00:22:24Z","receivedAt":"2016-12-15T00:22:32Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Dec 14, 2016 at 02:39:49PM -0800, Brandon Williams wrote:\n\n> v10 of this series fixes the following:\n> * A few updates to the commit messages in order to better convey the reasoning\n> behind the a few of the patches.\n> * Additional test to verify that curl redirects respect configured protocol\n> policies.\n> * Patch added by Jeff King to make http alternates respect configured\n> protocol policies.\n\nThanks, this one looks fine to me.\n\n-Peff\n"},{"id":"307848","messageId":"xmqqh965w17c.fsf@gitster.mtv.corp.google.com","threadId":"44417","inReplyTo":"20161215002120.xlnlquuqqw25wngc@sigill.intra.peff.net","subject":"Re: [PATCH v10 2/6] http: always warn if libcurl version is too old","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-12-15T17:29:11Z","receivedAt":"2016-12-15T17:29:40Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King writes:\n\n> On Wed, Dec 14, 2016 at 02:39:51PM -0800, Brandon Williams wrote:\n>\n>> Always warn if libcurl version is too old because:\n>> \n>> 1. Even without a protocol whitelist, newer versions of curl have all\n>> non-http protocols disabled by default.\n>\n> Technically, non-http and non-ftp. Maybe just \"non-standard\" would be\n> more accurate.\n>\n> Not worth a re-roll, but if Junio hasn't applied yet, maybe worth fixing\n> up while applying.\n\nI just did a \"rebase -i\"; thanks always for your careful reading.\n"}]}