{"thread":{"id":"47527","subject":"[RFC/PATCH] connect: add GIT_SSH_{SEND,RECEIVE}{,_COMMAND} env variables","startedAt":"2018-01-03T10:28:57Z","lastAt":"2018-01-04T19:06:33Z","messageCount":8,"participants":["Ævar Arnfjörð Bjarmason","Junio C Hamano","Jeff King"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"335711","messageId":"20180103102840.27897-1-avarab@gmail.com","threadId":"47527","inReplyTo":null,"subject":"[RFC/PATCH] connect: add GIT_SSH_{SEND,RECEIVE}{,_COMMAND} env variables","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2018-01-03T10:28:40Z","receivedAt":"2018-01-03T10:28:57Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Amend the long-standing logic for overriding the ssh command with\nGIT_SSH or GIT_SSH_COMMAND to also support\ne.g. GIT_SSH_SEND_COMMAND. The new specific send/receive variables\ntake priority over the old ones, and fall back to the older ones if\nthey exist.\n\nThis is useful for talking to systems such as Github or Gitlab that\nidentify user accounts (or deploy keys) by ssh keys. Normally, ssh\ncould do this itself by supplying multiple keys via -i, but that trick\ndoesn't work on these systems as the connection will have already been\naccepted when the \"wrong\" key gets rejected.\n\nThis new feature is redundant to and less general than setting the\nGIT_SSH_COMMAND to the path of a script that's going to dispatch to\nssh depending on what the second argument to the script is, e.g.:\n\n    $ cat ./git-ssh-command\n    #!/usr/bin/perl\n    if ($ARGV[1] =~ /^git-upload-pack /) {\n       system qw(ssh -i /some/ro/key) => @ARGV;\n    } elsif ($ARGV[1] =~ /^git-receive-pack /) {\n       system qw(ssh -i /some/rw/key) => @ARGV;\n    } else { ... }\n    $ GIT_TRACE=1 GIT_SSH_COMMAND=\"./git-ssh-command\" git fetch\n    10:22:39.415684 git.c:344               trace: built-in: git 'fetch'\n    10:22:39.432192 run-command.c:627       trace: run_command: './git-ssh-command' '-G' 'git@github.com'\n    10:22:39.434156 run-command.c:627       trace: run_command: './git-ssh-command' 'git@github.com' 'git-upload-pack '\\''git/git.git'\\'''\n    Warning: Identity file /some/ro/key not accessible: No such file or directory.\n\nHowever, I feel that this is a common enough case to be worth\nsupporting explicitly, and such a script will also need to deal with\narbitrary arguments fed via git-fetch's --upload-pack=\"...\", and\ngit-push's corresponding --receive-pack argument.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>\n---\n\nI'm not 100% sure about this one myself, but am leaning towards\ninclusion for the reasons explained above, and the patch is trivial\nenough that I think we can discuss whether it's worthwhile without\ntest / documentation.\n\n builtin/fetch-pack.c |  2 +-\n builtin/send-pack.c  |  3 ++-\n connect.c            | 21 ++++++++++++++++++---\n connect.h            |  2 ++\n 4 files changed, 23 insertions(+), 5 deletions(-)\n\ndiff --git a/builtin/fetch-pack.c b/builtin/fetch-pack.c\nindex 366b9d13f9..dae10f8419 100644\n--- a/builtin/fetch-pack.c\n+++ b/builtin/fetch-pack.c\n@@ -189,7 +189,7 @@ int cmd_fetch_pack(int argc, const char **argv, const char *prefix)\n \t\tif (args.diag_url)\n \t\t\tflags |= CONNECT_DIAG_URL;\n \t\tconn = git_connect(fd, dest, args.uploadpack,\n-\t\t\t\t   flags);\n+\t\t\t\t   flags | CONNECT_RECEIVE);\n \t\tif (!conn)\n \t\t\treturn args.diag_url ? 0 : 1;\n \t}\ndiff --git a/builtin/send-pack.c b/builtin/send-pack.c\nindex fc4f0bb5fb..2374d2b29c 100644\n--- a/builtin/send-pack.c\n+++ b/builtin/send-pack.c\n@@ -252,8 +252,9 @@ int cmd_send_pack(int argc, const char **argv, const char *prefix)\n \t\tfd[0] = 0;\n \t\tfd[1] = 1;\n \t} else {\n+\t\tint flags = args.verbose ? CONNECT_VERBOSE : 0;\n \t\tconn = git_connect(fd, dest, receivepack,\n-\t\t\targs.verbose ? CONNECT_VERBOSE : 0);\n+\t\t\tflags | CONNECT_SEND);\n \t}\n \n \tget_remote_heads(fd[0], NULL, 0, &remote_refs, REF_NORMAL,\ndiff --git a/connect.c b/connect.c\nindex c3a014c5ba..2a35924292 100644\n--- a/connect.c\n+++ b/connect.c\n@@ -774,13 +774,23 @@ static enum protocol parse_connect_url(const char *url_orig, char **ret_host,\n \treturn protocol;\n }\n \n-static const char *get_ssh_command(void)\n+static const char *get_ssh_command(int flags)\n {\n \tconst char *ssh;\n \n+\tif (flags & CONNECT_SEND && (ssh = getenv(\"GIT_SSH_SEND_COMMAND\")))\n+\t\treturn ssh;\n+\telse if (flags & CONNECT_RECEIVE && (ssh = getenv(\"GIT_SSH_RECEIVE_COMMAND\")))\n+\t\treturn ssh;\n \tif ((ssh = getenv(\"GIT_SSH_COMMAND\")))\n \t\treturn ssh;\n \n+\tif (flags & CONNECT_SEND &&\n+\t    !git_config_get_string_const(\"core.sshsendcommand\", &ssh))\n+\t\treturn ssh;\n+\telse if (flags & CONNECT_RECEIVE &&\n+\t    !git_config_get_string_const(\"core.sshreceivecommand\", &ssh))\n+\t\treturn ssh;\n \tif (!git_config_get_string_const(\"core.sshcommand\", &ssh))\n \t\treturn ssh;\n \n@@ -997,7 +1007,7 @@ static void fill_ssh_args(struct child_process *conn, const char *ssh_host,\n \tif (looks_like_command_line_option(ssh_host))\n \t\tdie(\"strange hostname '%s' blocked\", ssh_host);\n \n-\tssh = get_ssh_command();\n+\tssh = get_ssh_command(flags);\n \tif (ssh) {\n \t\tvariant = determine_ssh_variant(ssh, 1);\n \t} else {\n@@ -1008,7 +1018,12 @@ static void fill_ssh_args(struct child_process *conn, const char *ssh_host,\n \t\t */\n \t\tconn->use_shell = 0;\n \n-\t\tssh = getenv(\"GIT_SSH\");\n+\t\tif (flags & CONNECT_SEND)\n+\t\t\tssh = getenv(\"GIT_SSH_SEND\");\n+\t\telse if (flags & CONNECT_RECEIVE)\n+\t\t\tssh = getenv(\"GIT_SSH_RECEIVE\");\n+\t\tif (!ssh)\n+\t\t\tssh = getenv(\"GIT_SSH\");\n \t\tif (!ssh)\n \t\t\tssh = \"ssh\";\n \t\tvariant = determine_ssh_variant(ssh, 0);\ndiff --git a/connect.h b/connect.h\nindex 01f14cdf3f..e3ff0d5838 100644\n--- a/connect.h\n+++ b/connect.h\n@@ -5,6 +5,8 @@\n #define CONNECT_DIAG_URL      (1u << 1)\n #define CONNECT_IPV4          (1u << 2)\n #define CONNECT_IPV6          (1u << 3)\n+#define CONNECT_SEND          (1u << 4)\n+#define CONNECT_RECEIVE       (1u << 5)\n extern struct child_process *git_connect(int fd[2], const char *url, const char *prog, int flags);\n extern int finish_connect(struct child_process *conn);\n extern int git_connection_is_socket(struct child_process *conn);\n-- \n2.15.1.424.g9478a66081\n\n"},{"id":"335802","messageId":"xmqq373mh62r.fsf@gitster.mtv.corp.google.com","threadId":"47527","inReplyTo":"20180103102840.27897-1-avarab@gmail.com","subject":"Re: [RFC/PATCH] connect: add GIT_SSH_{SEND,RECEIVE}{,_COMMAND} env variables","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2018-01-03T23:32:12Z","receivedAt":"2018-01-03T23:32:21Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:\n\n> This is useful for talking to systems such as Github or Gitlab that\n> identify user accounts (or deploy keys) by ssh keys. Normally, ssh\n> could do this itself by supplying multiple keys via -i, but that trick\n> doesn't work on these systems as the connection will have already been\n> accepted when the \"wrong\" key gets rejected.\n\nYou need to explain this a lot better than the above.  \n\nI am sure systems such as Github have more than dozens of users who\npush over ssh and these users identify themselves by which key to\nuse when establishing connection just fine (presumably by using a\n\"Host\" entry for the github URL in ~/.ssh/config), and presumably we\nare not sending \"wrong\" keys over there.  So there needs to be a lot\nmore clear description of the problem you are trying to solve in the\nfirst place.\n"},{"id":"335806","messageId":"87bmiacwoz.fsf@evledraar.gmail.com","threadId":"47527","inReplyTo":"xmqq373mh62r.fsf@gitster.mtv.corp.google.com","subject":"Re: [RFC/PATCH] connect: add GIT_SSH_{SEND,RECEIVE}{,_COMMAND} env variables","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2018-01-04T00:08:28Z","receivedAt":"2018-01-04T00:08:39Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Wed, Jan 03 2018, Junio C. Hamano jotted:\n\n> Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:\n>\n>> This is useful for talking to systems such as Github or Gitlab that\n>> identify user accounts (or deploy keys) by ssh keys. Normally, ssh\n>> could do this itself by supplying multiple keys via -i, but that trick\n>> doesn't work on these systems as the connection will have already been\n>> accepted when the \"wrong\" key gets rejected.\n>\n> You need to explain this a lot better than the above.\n>\n> I am sure systems such as Github have more than dozens of users who\n> push over ssh and these users identify themselves by which key to\n> use when establishing connection just fine (presumably by using a\n> \"Host\" entry for the github URL in ~/.ssh/config), and presumably we\n> are not sending \"wrong\" keys over there.  So there needs to be a lot\n> more clear description of the problem you are trying to solve in the\n> first place.\n\nHopefully this is clearer, and depending on how the rest of the\ndiscussion goes I'll submit v2 with something like this in the commit\nmessage:\n\nSSH keys A and B are known to the remote service, and used to identify\ntwo different users.\n\nA can only push to repository X, and B can only fetch from repository Y.\n\nThus, if you have a script that does:\n\n    GIT_SSH_COMMAND=\"ssh -i A -i B\" git ...\n\nIt'll always fail for pulling from X, and pushing to Y. Supply:\n\n    GIT_SSH_COMMAND=\"ssh -i B -i A\" git ...\n\nAnd now pulling will work, but pushing won't.\n\nIf you were to do, where C is a completly unknown key:\n\n    GIT_SSH_COMMAND=\"ssh -i C -i A\" git push X ...\n\nIt would work, since ssh wouldn't get far enough in the key negotiation\nto drop you into a shell. This is the case you had in mind, but is\nunrelated to the problem I'm trying to address.\n\nI tested this on a Gitlab instance, but as far as I know this property\nis going to be intrinsic to anything that uses ssh in this way,\ni.e. once you get past the step where the server says \"this key is OK\"\nand drops you into a shell, it's not going to retry the whole\nnegotiation with another key just because the command you ran exited\nwith non-zero.\n\nSo now I just have a GIT_SSH_COMMAND that dispatches to different keys\ndepending on the operation, as noted in the commit message, and I can\nassure you that without that logic it doesn't work.\n\nI thought that use-case might be useful enough to be natively supported,\nsince right now you either need to hack it up like that, or perform\nsimilar hacks with url/pushurl and ssh host aliases in your config.\n"},{"id":"335816","messageId":"20180104044230.GA12113@sigill.intra.peff.net","threadId":"47527","inReplyTo":"87bmiacwoz.fsf@evledraar.gmail.com","subject":"Re: [RFC/PATCH] connect: add GIT_SSH_{SEND,RECEIVE}{,_COMMAND} env variables","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2018-01-04T04:42:30Z","receivedAt":"2018-01-04T04:42:37Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Jan 04, 2018 at 01:08:28AM +0100, Ævar Arnfjörð Bjarmason wrote:\n\n> Hopefully this is clearer, and depending on how the rest of the\n> discussion goes I'll submit v2 with something like this in the commit\n> message:\n> \n> SSH keys A and B are known to the remote service, and used to identify\n> two different users.\n> \n> A can only push to repository X, and B can only fetch from repository Y.\n> \n> Thus, if you have a script that does:\n> \n>     GIT_SSH_COMMAND=\"ssh -i A -i B\" git ...\n> \n> It'll always fail for pulling from X, and pushing to Y. Supply:\n> \n>     GIT_SSH_COMMAND=\"ssh -i B -i A\" git ...\n> \n> And now pulling will work, but pushing won't.\n\nI get that you may have two different keys to go with two different\nidentities on a remote system. But I'm not sure I understand why\n\"sending\" or \"receiving\" is the right way to split those up. Wouldn't\nyou also sometimes want to fetch from repository X? IOW, wouldn't you\nwant to tie identity \"A\" to repository \"X\", and \"B\" to repository \"Y?\n\n> So now I just have a GIT_SSH_COMMAND that dispatches to different keys\n> depending on the operation, as noted in the commit message, and I can\n> assure you that without that logic it doesn't work.\n\nYou mentioned host aliases later, which is the solution I've seen in the\nwild. And then you can map each remote to a different host alias.\n\n-Peff\n"},{"id":"335817","messageId":"87a7xuc4ty.fsf@evledraar.gmail.com","threadId":"47527","inReplyTo":"20180104044230.GA12113@sigill.intra.peff.net","subject":"Re: [RFC/PATCH] connect: add GIT_SSH_{SEND,RECEIVE}{,_COMMAND} env variables","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2018-01-04T10:10:17Z","receivedAt":"2018-01-04T10:10:27Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Thu, Jan 04 2018, Jeff King jotted:\n\n> On Thu, Jan 04, 2018 at 01:08:28AM +0100, Ævar Arnfjörð Bjarmason wrote:\n>\n>> Hopefully this is clearer, and depending on how the rest of the\n>> discussion goes I'll submit v2 with something like this in the commit\n>> message:\n>>\n>> SSH keys A and B are known to the remote service, and used to identify\n>> two different users.\n>>\n>> A can only push to repository X, and B can only fetch from repository Y.\n>>\n>> Thus, if you have a script that does:\n>>\n>>     GIT_SSH_COMMAND=\"ssh -i A -i B\" git ...\n>>\n>> It'll always fail for pulling from X, and pushing to Y. Supply:\n>>\n>>     GIT_SSH_COMMAND=\"ssh -i B -i A\" git ...\n>>\n>> And now pulling will work, but pushing won't.\n>\n> I get that you may have two different keys to go with two different\n> identities on a remote system. But I'm not sure I understand why\n> \"sending\" or \"receiving\" is the right way to split those up. Wouldn't\n> you also sometimes want to fetch from repository X? IOW, wouldn't you\n> want to tie identity \"A\" to repository \"X\", and \"B\" to repository \"Y?\n\nThat's badly explained, sorry, when I say \"push\" I mean \"push and/or\npull\".\n\nI don't know about Github, but on Gitlab when you provision a deploy key\nand associate it with a repo it must be *globally* rw or ro, there's no\nway to on a per-repo basis say it should be rw ro.\n\nI have a job that's fetching a bunch of repos to review code in them\n(for auditing purposes). It then commits the results of that review to\nother git repos.\n\nThus I want to have a ro key to all those reviewed repos, but rw keys to\nthe audit repo itself (and it'll also pull with the rw key).\n\nHence this patch, I thought *maybe* others would be interested in this\nsince it seems to me to be an easy thing to run into with these ssh-key\nbased hosting providers, but maybe not.\n\n>> So now I just have a GIT_SSH_COMMAND that dispatches to different keys\n>> depending on the operation, as noted in the commit message, and I can\n>> assure you that without that logic it doesn't work.\n>\n> You mentioned host aliases later, which is the solution I've seen in the\n> wild. And then you can map each remote to a different host alias.\n"},{"id":"335821","messageId":"20180104155325.GA18376@sigill.intra.peff.net","threadId":"47527","inReplyTo":"87a7xuc4ty.fsf@evledraar.gmail.com","subject":"Re: [RFC/PATCH] connect: add GIT_SSH_{SEND,RECEIVE}{,_COMMAND} env variables","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2018-01-04T15:53:26Z","receivedAt":"2018-01-04T15:53:32Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Jan 04, 2018 at 11:10:17AM +0100, Ævar Arnfjörð Bjarmason wrote:\n\n> That's badly explained, sorry, when I say \"push\" I mean \"push and/or\n> pull\".\n> \n> I don't know about Github, but on Gitlab when you provision a deploy key\n> and associate it with a repo it must be *globally* rw or ro, there's no\n> way to on a per-repo basis say it should be rw ro.\n> \n> I have a job that's fetching a bunch of repos to review code in them\n> (for auditing purposes). It then commits the results of that review to\n> other git repos.\n> \n> Thus I want to have a ro key to all those reviewed repos, but rw keys to\n> the audit repo itself (and it'll also pull with the rw key).\n\nOK, that part makes sense to me.\n\nBut I'm not sure how your patch solves it. When you \"git fetch\" on the\naudit repo, wouldn't your GIT_SSH_RECEIVE_COMMAND kick in and use the\nwrong key? What am I missing?\n\n-Peff\n"},{"id":"335826","messageId":"87608hczht.fsf@evledraar.gmail.com","threadId":"47527","inReplyTo":"20180104155325.GA18376@sigill.intra.peff.net","subject":"Re: [RFC/PATCH] connect: add GIT_SSH_{SEND,RECEIVE}{,_COMMAND} env variables","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2018-01-04T17:20:14Z","receivedAt":"2018-01-04T17:20:24Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Thu, Jan 04 2018, Jeff King jotted:\n\n> On Thu, Jan 04, 2018 at 11:10:17AM +0100, Ævar Arnfjörð Bjarmason wrote:\n>\n>> That's badly explained, sorry, when I say \"push\" I mean \"push and/or\n>> pull\".\n>>\n>> I don't know about Github, but on Gitlab when you provision a deploy key\n>> and associate it with a repo it must be *globally* rw or ro, there's no\n>> way to on a per-repo basis say it should be rw ro.\n>>\n>> I have a job that's fetching a bunch of repos to review code in them\n>> (for auditing purposes). It then commits the results of that review to\n>> other git repos.\n>>\n>> Thus I want to have a ro key to all those reviewed repos, but rw keys to\n>> the audit repo itself (and it'll also pull with the rw key).\n>\n> OK, that part makes sense to me.\n>\n> But I'm not sure how your patch solves it. When you \"git fetch\" on the\n> audit repo, wouldn't your GIT_SSH_RECEIVE_COMMAND kick in and use the\n> wrong key? What am I missing?\n\nI add both the ro and rw key to some projects. Those are a tiny subset\nof the overall number.\n"},{"id":"335836","messageId":"xmqq608hfnpp.fsf@gitster.mtv.corp.google.com","threadId":"47527","inReplyTo":"20180104044230.GA12113@sigill.intra.peff.net","subject":"Re: [RFC/PATCH] connect: add GIT_SSH_{SEND,RECEIVE}{,_COMMAND} env variables","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2018-01-04T19:06:26Z","receivedAt":"2018-01-04T19:06:33Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King <peff@peff.net> writes:\n\n> I get that you may have two different keys to go with two different\n> identities on a remote system. But I'm not sure I understand why\n> \"sending\" or \"receiving\" is the right way to split those up. Wouldn't\n> you also sometimes want to fetch from repository X? IOW, wouldn't you\n> want to tie identity \"A\" to repository \"X\", and \"B\" to repository \"Y?\n>\n>> So now I just have a GIT_SSH_COMMAND that dispatches to different keys\n>> depending on the operation, as noted in the commit message, and I can\n>> assure you that without that logic it doesn't work.\n>\n> You mentioned host aliases later, which is the solution I've seen in the\n> wild. And then you can map each remote to a different host alias.\n\nYup, I do agree that it is exactly the established solution for this\nkind of situation.\n"}]}