{"thread":{"id":"31129","subject":"Enhancements to git-protocoll","startedAt":"2012-07-28T21:41:16Z","lastAt":"2012-07-30T08:31:11Z","messageCount":25,"participants":["Fredrik Gustafsson","Junio C Hamano","Sitaram Chamarty","Shawn Pearce"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"196051","messageId":"20120728214116.GA3591@paksenarrion.iveqy.com","threadId":"31129","inReplyTo":null,"subject":"Enhancements to git-protocoll","fromName":"Fredrik Gustafsson","fromEmail":"iveqy@iveqy.com","sentAt":"2012-07-28T21:41:16Z","receivedAt":"2012-07-28T21:41:16Z","isPatch":false,"sender":{"key":"iveqy@iveqy.com","avatar":"https://avatars.githubusercontent.com/u/761743?v=4"},"body":"Hi,\nsometimes git communicates with something that's not git on the other\nside (gitolite and github for example).\n\nSometimes the server wants to communicate directly to the git user.\n\ngit isn't really designed for this. gitolite solves this by do user\ninteraction on STDERR instead. The bad thing about this is that it can\nonly be one-direction communication, for example error messages.\n\nIf git would allow for the user to interact direct with the server, a\nlot of cool and and userfriendly features could be developed.\n\nFor example:\ngitolite has something called wild repos[1]. The management is\ncumbersome and if you misspell when you clone a repo you might instead\ncreate a new repo.\n\nThis could have been avoided with a simply:\n\"Do you want to create a new repo[Yn]\"\n\nTo fix this, git protocol should have a command for printing input to\nSTDOUT and accepting input on STDIN which should be sent to the server.\nAnd a command to switch back to orginal of course.\n\nThe server could then switch to user interaction, do that and then\nswitch back to normal operation.\n\nBefore eventually starting to implement this, I would know your\nopinions. This feature would be wortless if it's not in the official git.\n\n[1] http://sitaramc.github.com/gitolite/wild.html\n-- \nMed vänliga hälsningar\nFredrik Gustafsson\n\ntel: 0733-608274\ne-post: iveqy@iveqy.com\n"},{"id":"196056","messageId":"7vehnvvyta.fsf@alter.siamese.dyndns.org","threadId":"31129","inReplyTo":"20120728214116.GA3591@paksenarrion.iveqy.com","subject":"Re: Enhancements to git-protocoll","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2012-07-29T06:58:09Z","receivedAt":"2012-07-29T06:58:09Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Fredrik Gustafsson <iveqy@iveqy.com> writes:\n\n> Sometimes the server wants to communicate directly to the git user.\n> ...\n> For example:\n> gitolite has something called wild repos[1]. The management is\n> cumbersome and if you misspell when you clone a repo you might instead\n> create a new repo.\n>\n> This could have been avoided with a simply:\n> \"Do you want to create a new repo[Yn]\"\n\nI do not think the automatic repository creation done by gitolite is\na good use case or example for whatever you seem to be advocating.\n\nIIUC, the auto-creation in gitolite-shell::main() is done way before\ngitolite-shell (which is used as a login shell for incoming ssh\nsessions) creates a new git repository, goes into it and spawns the\ngit-receive-pack command.  It all happens outside Git.\n\n    # auto-create?\n    if ( repo_missing($repo) and access( $repo, $user, '^C', 'any' ) !~ /DENIED/ ) {\n        require Gitolite::Conf::Store;\n        Gitolite::Conf::Store->import;\n        new_wild_repo( $repo, $user, $aa );\n        gl_log( 'create', $repo, $user, $aa );\n    }\n\nThe \"access()\" we see here is not the Perl builtin access(), but is\na function defined in src/lib/Gitolite/Conf/Load.pm; that would be\nthe place to allow the incoming ssh session to talk back to the end\nuser, but at that point there is no Git processing on the server\nend.\n\nWhile I am not fundamentally opposed to adding yet another sideband\nchannel to the git protocol, I do not think adding user interaction\nat random places in the protocol exchange is a viable or useful\napproach to implement an enhanced server that works with both\nenhanced and vanilla clients (and the same is true for enhanced\nclient that works with both enhanced and vanilla server).\n"},{"id":"196058","messageId":"CAMK1S_i7WjrZ=L_LVoTQriW8kKMPd+ABrgowpFCY3ks43E6K7A@mail.gmail.com","threadId":"31129","inReplyTo":"20120728214116.GA3591@paksenarrion.iveqy.com","subject":"Re: Enhancements to git-protocoll","fromName":"Sitaram Chamarty","fromEmail":"sitaramc@gmail.com","sentAt":"2012-07-29T10:37:13Z","receivedAt":"2012-07-29T10:37:13Z","isPatch":false,"sender":{"key":"sitaramc@gmail.com","avatar":"https://avatars.githubusercontent.com/u/43316?v=4"},"body":"On Sun, Jul 29, 2012 at 3:11 AM, Fredrik Gustafsson <iveqy@iveqy.com> wrote:\n> Hi,\n> sometimes git communicates with something that's not git on the other\n> side (gitolite and github for example).\n>\n> Sometimes the server wants to communicate directly to the git user.\n>\n> git isn't really designed for this. gitolite solves this by do user\n> interaction on STDERR instead. The bad thing about this is that it can\n> only be one-direction communication, for example error messages.\n>\n> If git would allow for the user to interact direct with the server, a\n> lot of cool and and userfriendly features could be developed.\n>\n> For example:\n> gitolite has something called wild repos[1]. The management is\n> cumbersome and if you misspell when you clone a repo you might instead\n> create a new repo.\n\nFor the record, although it cannot do the yes/no part, if you want to\ndisable auto-creation on a fetch/clone (read operation) it's trivial\nto add a \"PRE_CREATE\" trigger to do that.\n"},{"id":"196060","messageId":"20120729141354.GA16223@paksenarrion.iveqy.com","threadId":"31129","inReplyTo":"CAMK1S_i7WjrZ=L_LVoTQriW8kKMPd+ABrgowpFCY3ks43E6K7A@mail.gmail.com","subject":"Re: Enhancements to git-protocoll","fromName":"Fredrik Gustafsson","fromEmail":"iveqy@iveqy.com","sentAt":"2012-07-29T14:13:54Z","receivedAt":"2012-07-29T14:13:54Z","isPatch":false,"sender":{"key":"iveqy@iveqy.com","avatar":"https://avatars.githubusercontent.com/u/761743?v=4"},"body":"On Sun, Jul 29, 2012 at 04:07:13PM +0530, Sitaram Chamarty wrote:\n> On Sun, Jul 29, 2012 at 3:11 AM, Fredrik Gustafsson <iveqy@iveqy.com> wrote:\n> > Hi,\n> > sometimes git communicates with something that's not git on the other\n> > side (gitolite and github for example).\n> >\n> > Sometimes the server wants to communicate directly to the git user.\n> >\n> > git isn't really designed for this. gitolite solves this by do user\n> > interaction on STDERR instead. The bad thing about this is that it can\n> > only be one-direction communication, for example error messages.\n> >\n> > If git would allow for the user to interact direct with the server, a\n> > lot of cool and and userfriendly features could be developed.\n> >\n> > For example:\n> > gitolite has something called wild repos[1]. The management is\n> > cumbersome and if you misspell when you clone a repo you might instead\n> > create a new repo.\n> \n> For the record, although it cannot do the yes/no part, if you want to\n> disable auto-creation on a fetch/clone (read operation) it's trivial\n> to add a \"PRE_CREATE\" trigger to do that.\n\nThanks, however I think auto-creation is a great feature for some cases\nand I think there can be even more useable functions if we could get\nuser interaction.\n\n-- \nMed vänliga hälsningar\nFredrik Gustafsson\n\ntel: 0733-608274\ne-post: iveqy@iveqy.com\n"},{"id":"196061","messageId":"20120729142458.GB16223@paksenarrion.iveqy.com","threadId":"31129","inReplyTo":"7vehnvvyta.fsf@alter.siamese.dyndns.org","subject":"Re: Enhancements to git-protocoll","fromName":"Fredrik Gustafsson","fromEmail":"iveqy@iveqy.com","sentAt":"2012-07-29T14:24:58Z","receivedAt":"2012-07-29T14:24:58Z","isPatch":false,"sender":{"key":"iveqy@iveqy.com","avatar":"https://avatars.githubusercontent.com/u/761743?v=4"},"body":"On Sat, Jul 28, 2012 at 11:58:09PM -0700, Junio C Hamano wrote:\n> Fredrik Gustafsson <iveqy@iveqy.com> writes:\n> \n> > Sometimes the server wants to communicate directly to the git user.\n> > ...\n> > For example:\n> > gitolite has something called wild repos[1]. The management is\n> > cumbersome and if you misspell when you clone a repo you might instead\n> > create a new repo.\n> >\n> > This could have been avoided with a simply:\n> > \"Do you want to create a new repo[Yn]\"\n> \n> I do not think the automatic repository creation done by gitolite is\n> a good use case or example for whatever you seem to be advocating.\n> \n> IIUC, the auto-creation in gitolite-shell::main() is done way before\n> gitolite-shell (which is used as a login shell for incoming ssh\n> sessions) creates a new git repository, goes into it and spawns the\n> git-receive-pack command.  It all happens outside Git.\n> \n>     # auto-create?\n>     if ( repo_missing($repo) and access( $repo, $user, '^C', 'any' ) !~ /DENIED/ ) {\n>         require Gitolite::Conf::Store;\n>         Gitolite::Conf::Store->import;\n>         new_wild_repo( $repo, $user, $aa );\n>         gl_log( 'create', $repo, $user, $aa );\n>     }\n> \n> The \"access()\" we see here is not the Perl builtin access(), but is\n> a function defined in src/lib/Gitolite/Conf/Load.pm; that would be\n> the place to allow the incoming ssh session to talk back to the end\n> user, but at that point there is no Git processing on the server\n> end.\n\nThat's a feature. It means that the impact on git would be rather small,\nwe don't have to involve server-side git at all. The problem so solve is\nhow to get client-side git to pass through STDIN and STDOUT (just as is\ndone with STDERR right now). I see this as a gitolite <-> client-git\ninteraction case. No server-git should be involved.\n\nAll the use casese I can imagine will be done before (or after)\nserverside git is executed.\n\n> While I am not fundamentally opposed to adding yet another sideband\n> channel to the git protocol, I do not think adding user interaction\n> at random places in the protocol exchange is a viable or useful\n> approach to implement an enhanced server that works with both\n> enhanced and vanilla clients (and the same is true for enhanced\n> client that works with both enhanced and vanilla server).\n\nDo we mean the same thing with \"git protocol\"? I specify the protocol as\neverything that happens between the server and the client. Are the\nconnection divided into multiple protocoll after eachother? (would it be\npossible to execute git-user-interaction-protocoll first and the\ngit-protocoll and then git-user-interaction-protocoll again?).\n\nThe vanilla case would be easy to solve if the protocol has git version\nin its handshake. The STDERR approach is already used and working. A\nvanilla client would have the same functionality as today and en\nenhanced client will have enhanced functionality.\n\n-- \nMed vänliga hälsningar\nFredrik Gustafsson\n\ntel: 0733-608274\ne-post: iveqy@iveqy.com\n"},{"id":"196062","messageId":"CAMK1S_iFqQvU+USYf+wcsLFS1QtsytLnPfEhjcAVZa9Vwj8d1w@mail.gmail.com","threadId":"31129","inReplyTo":"20120729141354.GA16223@paksenarrion.iveqy.com","subject":"Re: Enhancements to git-protocoll","fromName":"Sitaram Chamarty","fromEmail":"sitaramc@gmail.com","sentAt":"2012-07-29T14:25:36Z","receivedAt":"2012-07-29T14:25:36Z","isPatch":false,"sender":{"key":"sitaramc@gmail.com","avatar":"https://avatars.githubusercontent.com/u/43316?v=4"},"body":"On Sun, Jul 29, 2012 at 7:43 PM, Fredrik Gustafsson <iveqy@iveqy.com> wrote:\n> On Sun, Jul 29, 2012 at 04:07:13PM +0530, Sitaram Chamarty wrote:\n>> On Sun, Jul 29, 2012 at 3:11 AM, Fredrik Gustafsson <iveqy@iveqy.com> wrote:\n>> > Hi,\n>> > sometimes git communicates with something that's not git on the other\n>> > side (gitolite and github for example).\n>> >\n>> > Sometimes the server wants to communicate directly to the git user.\n>> >\n>> > git isn't really designed for this. gitolite solves this by do user\n>> > interaction on STDERR instead. The bad thing about this is that it can\n>> > only be one-direction communication, for example error messages.\n>> >\n>> > If git would allow for the user to interact direct with the server, a\n>> > lot of cool and and userfriendly features could be developed.\n>> >\n>> > For example:\n>> > gitolite has something called wild repos[1]. The management is\n>> > cumbersome and if you misspell when you clone a repo you might instead\n>> > create a new repo.\n>>\n>> For the record, although it cannot do the yes/no part, if you want to\n>> disable auto-creation on a fetch/clone (read operation) it's trivial\n>> to add a \"PRE_CREATE\" trigger to do that.\n>\n> Thanks, however I think auto-creation is a great feature for some cases\n> and I think there can be even more useable functions if we could get\n> user interaction.\n\nFor the record, I don't think I agree.  There's a place to create a\nhuman-conversation, and there's a place not to.\n\nIf you want a dialog with the server, there should be *other* commands\nthat do that, instead of overloading git's own protocol.\n\nSince you mentioned gitolite, consider copying the fork command\n(src/commands/fork) and munging the code into an explicit wild repo\ncreate.\n"},{"id":"196063","messageId":"CAMK1S_iTv3BOV3LEoj83feuO2iQnPqTrOq0wyFC-GrvvJQqWTA@mail.gmail.com","threadId":"31129","inReplyTo":"20120729142458.GB16223@paksenarrion.iveqy.com","subject":"Re: Enhancements to git-protocoll","fromName":"Sitaram Chamarty","fromEmail":"sitaramc@gmail.com","sentAt":"2012-07-29T14:39:40Z","receivedAt":"2012-07-29T14:39:40Z","isPatch":false,"sender":{"key":"sitaramc@gmail.com","avatar":"https://avatars.githubusercontent.com/u/43316?v=4"},"body":"On Sun, Jul 29, 2012 at 7:54 PM, Fredrik Gustafsson <iveqy@iveqy.com> wrote:\n> On Sat, Jul 28, 2012 at 11:58:09PM -0700, Junio C Hamano wrote:\n>> Fredrik Gustafsson <iveqy@iveqy.com> writes:\n>>\n>> > Sometimes the server wants to communicate directly to the git user.\n>> > ...\n>> > For example:\n>> > gitolite has something called wild repos[1]. The management is\n>> > cumbersome and if you misspell when you clone a repo you might instead\n>> > create a new repo.\n>> >\n>> > This could have been avoided with a simply:\n>> > \"Do you want to create a new repo[Yn]\"\n>>\n>> I do not think the automatic repository creation done by gitolite is\n>> a good use case or example for whatever you seem to be advocating.\n>>\n>> IIUC, the auto-creation in gitolite-shell::main() is done way before\n>> gitolite-shell (which is used as a login shell for incoming ssh\n>> sessions) creates a new git repository, goes into it and spawns the\n>> git-receive-pack command.  It all happens outside Git.\n>>\n>>     # auto-create?\n>>     if ( repo_missing($repo) and access( $repo, $user, '^C', 'any' ) !~ /DENIED/ ) {\n>>         require Gitolite::Conf::Store;\n>>         Gitolite::Conf::Store->import;\n>>         new_wild_repo( $repo, $user, $aa );\n>>         gl_log( 'create', $repo, $user, $aa );\n>>     }\n>>\n>> The \"access()\" we see here is not the Perl builtin access(), but is\n>> a function defined in src/lib/Gitolite/Conf/Load.pm; that would be\n>> the place to allow the incoming ssh session to talk back to the end\n>> user, but at that point there is no Git processing on the server\n>> end.\n>\n> That's a feature. It means that the impact on git would be rather small,\n> we don't have to involve server-side git at all. The problem so solve is\n> how to get client-side git to pass through STDIN and STDOUT (just as is\n> done with STDERR right now). I see this as a gitolite <-> client-git\n> interaction case. No server-git should be involved.\n\nUggh, no.  Client-git should only talk to server-git.  It shouldn't be\ntalking first to some *other* program (in this case gitolite), and\nthen to to server-git.  That doesn't sound sane to me.\n\nYou should wrap this whole thing around something else that does it in\n3 steps.  Check, create if needed, then the actual git command you\nintend to run.  All this should be local to your environment, not\nrolled into git; it's far too specific to be rolled into git itself,\nif you ask me.\n\n> All the use casese I can imagine will be done before (or after)\n> serverside git is executed.\n>\n>> While I am not fundamentally opposed to adding yet another sideband\n>> channel to the git protocol, I do not think adding user interaction\n>> at random places in the protocol exchange is a viable or useful\n>> approach to implement an enhanced server that works with both\n>> enhanced and vanilla clients (and the same is true for enhanced\n>> client that works with both enhanced and vanilla server).\n>\n> Do we mean the same thing with \"git protocol\"? I specify the protocol as\n> everything that happens between the server and the client. Are the\n> connection divided into multiple protocoll after eachother? (would it be\n> possible to execute git-user-interaction-protocoll first and the\n> git-protocoll and then git-user-interaction-protocoll again?).\n>\n> The vanilla case would be easy to solve if the protocol has git version\n> in its handshake. The STDERR approach is already used and working. A\n> vanilla client would have the same functionality as today and en\n> enhanced client will have enhanced functionality.\n>\n> --\n> Med vänliga hälsningar\n> Fredrik Gustafsson\n>\n> tel: 0733-608274\n> e-post: iveqy@iveqy.com\n> --\n> To unsubscribe from this list: send the line \"unsubscribe git\" in\n> the body of a message to majordomo@vger.kernel.org\n> More majordomo info at  http://vger.kernel.org/majordomo-info.html\n\n\n\n-- \nSitaram\n"},{"id":"196064","messageId":"20120729150550.GA17679@paksenarrion.iveqy.com","threadId":"31129","inReplyTo":"CAMK1S_iFqQvU+USYf+wcsLFS1QtsytLnPfEhjcAVZa9Vwj8d1w@mail.gmail.com","subject":"Re: Enhancements to git-protocoll","fromName":"Fredrik Gustafsson","fromEmail":"iveqy@iveqy.com","sentAt":"2012-07-29T15:05:50Z","receivedAt":"2012-07-29T15:05:50Z","isPatch":false,"sender":{"key":"iveqy@iveqy.com","avatar":"https://avatars.githubusercontent.com/u/761743?v=4"},"body":"On Sun, Jul 29, 2012 at 07:55:36PM +0530, Sitaram Chamarty wrote:\n> > Thanks, however I think auto-creation is a great feature for some cases\n> > and I think there can be even more useable functions if we could get\n> > user interaction.\n> \n> For the record, I don't think I agree.  There's a place to create a\n> human-conversation, and there's a place not to.\n> \n> If you want a dialog with the server, there should be *other* commands\n> that do that, instead of overloading git's own protocol.\n> \n> Since you mentioned gitolite, consider copying the fork command\n> (src/commands/fork) and munging the code into an explicit wild repo\n> create.\n\nI appriciate that you clearified you oppinion. Please excuse me if it\nsounded as I in any way speaked for gitolite. I use gitolite as an\nexample becuase the target application in this case is unknown to most\npeople (think gitolite with db-backend for user permissions).\n\nIt's a valid design oppinion to not mix git protocoll with anything\nelse. But gitolite already does that. Gitolite already have user\ninteraction mixed with git interaction. Do you say to me that gitolite\nis broken and should not do user interaction over git-commands? Then why\ndoes wild repos exists and why does gitolite error messages exists?\n\nWe're already down that road, why not do it better?\n\n-- \nMed vänliga hälsningar\nFredrik Gustafsson\n\ntel: 0733-608274\ne-post: iveqy@iveqy.com\n"},{"id":"196065","messageId":"CAMK1S_hHBB9BViH=CFuJNgMJzMtzhE0mGc7ryFaDWNxOoH2Pgg@mail.gmail.com","threadId":"31129","inReplyTo":"20120729150550.GA17679@paksenarrion.iveqy.com","subject":"Re: Enhancements to git-protocoll","fromName":"Sitaram Chamarty","fromEmail":"sitaramc@gmail.com","sentAt":"2012-07-29T15:15:39Z","receivedAt":"2012-07-29T15:15:39Z","isPatch":false,"sender":{"key":"sitaramc@gmail.com","avatar":"https://avatars.githubusercontent.com/u/43316?v=4"},"body":"On Sun, Jul 29, 2012 at 8:35 PM, Fredrik Gustafsson <iveqy@iveqy.com> wrote:\n> On Sun, Jul 29, 2012 at 07:55:36PM +0530, Sitaram Chamarty wrote:\n>> > Thanks, however I think auto-creation is a great feature for some cases\n>> > and I think there can be even more useable functions if we could get\n>> > user interaction.\n>>\n>> For the record, I don't think I agree.  There's a place to create a\n>> human-conversation, and there's a place not to.\n>>\n>> If you want a dialog with the server, there should be *other* commands\n>> that do that, instead of overloading git's own protocol.\n>>\n>> Since you mentioned gitolite, consider copying the fork command\n>> (src/commands/fork) and munging the code into an explicit wild repo\n>> create.\n>\n> I appriciate that you clearified you oppinion. Please excuse me if it\n> sounded as I in any way speaked for gitolite. I use gitolite as an\n> example becuase the target application in this case is unknown to most\n> people (think gitolite with db-backend for user permissions).\n>\n> It's a valid design oppinion to not mix git protocoll with anything\n> else. But gitolite already does that. Gitolite already have user\n> interaction mixed with git interaction. Do you say to me that gitolite\n> is broken and should not do user interaction over git-commands? Then why\n> does wild repos exists and why does gitolite error messages exists?\n>\n> We're already down that road, why not do it better?\n\nI think you misunderstood how gitolite works.  Gitolite does not have\n*any* user interaction other than sending some extra messages back via\nSTDERR if you're using a normal git client to do normal git operations\n(clone/fetch/ls-remote).\n\nSuch messages are *no different* from something that an update or\npre-receive hook might send back even on a normal (no gitolite) git\nserver.\n\nThe only time that gitolite might have any user *interaction* is when\nusing \"gitolite commands\".  These do not run git at all (neither on\nthe client nor on the server), and in fact merely provide a convenient\nway to allow users to run a controlled set of specific *shell*\ncommands.\n"},{"id":"196068","messageId":"20120729154124.GA19201@paksenarrion.iveqy.com","threadId":"31129","inReplyTo":"CAMK1S_hHBB9BViH=CFuJNgMJzMtzhE0mGc7ryFaDWNxOoH2Pgg@mail.gmail.com","subject":"Re: Enhancements to git-protocoll","fromName":"Fredrik Gustafsson","fromEmail":"iveqy@iveqy.com","sentAt":"2012-07-29T15:41:24Z","receivedAt":"2012-07-29T15:41:24Z","isPatch":false,"sender":{"key":"iveqy@iveqy.com","avatar":"https://avatars.githubusercontent.com/u/761743?v=4"},"body":"On Sun, Jul 29, 2012 at 08:45:39PM +0530, Sitaram Chamarty wrote:\n> I think you misunderstood how gitolite works.  Gitolite does not have\n> *any* user interaction other than sending some extra messages back via\n> STDERR if you're using a normal git client to do normal git operations\n> (clone/fetch/ls-remote).\n\nAs you say, gitolite has userinteraction, and its not even standard to\ngit error messages. Try cloning a repo that doesn't exists via gitolite\nand a regular ssh connection:\n\n[iveqy@paksenarrion git]$ git clone ssh://gitolite@localhost/testing2\nCloning into testing2...\nFATAL: R any testing2 id_rsa DENIED by fallthru\n(or you mis-spelled the reponame)\nfatal: The remote end hung up unexpectedly\n[iveqy@paksenarrion git]$ git clone ssh://iveqy@localhost/testing2\nCloning into testing...\nfatal: '/testing2' does not appear to be a git repository\nfatal: The remote end hung up unexpectedly\n\n> Such messages are *no different* from something that an update or\n> pre-receive hook might send back even on a normal (no gitolite) git\n> server.\n\nAs I showed above the \"non-existing\" repo is a case when it's different.\nBut you have a good point in hooks, of course a hook also should be able\nto have two way user interaction.\n\n> The only time that gitolite might have any user *interaction* is when\n> using \"gitolite commands\".  These do not run git at all (neither on\n> the client nor on the server), and in fact merely provide a convenient\n> way to allow users to run a controlled set of specific *shell*\n> commands.\n\nI do understand how gitolite works. However this is off-topic to my\noriginal question. I also do not have an oppinion on how gitolite\nshould work, I simply don't care. Gitolite is an widely acceptet git\ntool, I see improvement opportunities in git to allow an other\nprogram to utilize two-way user interaction all the time, this will not\neffect gitolite at all.\n\nSo in my point of view, it's up to Junio if I shall continue explore this\npath and maybe find a way of doing this in git, \"the right way\". Or if\nthis is something unwanted and gitolite and alike programs should\ncontinue with STDERR \"hacks\".\n\n-- \nMed vänliga hälsningar\nFredrik Gustafsson\n\ntel: 0733-608274\ne-post: iveqy@iveqy.com\n"},{"id":"196070","messageId":"CAMK1S_jhxOEd10pMUp9Wt4VmTD9QRVgVvWz4p=k9TxL3QSdzAQ@mail.gmail.com","threadId":"31129","inReplyTo":"20120729154124.GA19201@paksenarrion.iveqy.com","subject":"Re: Enhancements to git-protocoll","fromName":"Sitaram Chamarty","fromEmail":"sitaramc@gmail.com","sentAt":"2012-07-29T18:22:07Z","receivedAt":"2012-07-29T18:22:07Z","isPatch":false,"sender":{"key":"sitaramc@gmail.com","avatar":"https://avatars.githubusercontent.com/u/43316?v=4"},"body":"On Sun, Jul 29, 2012 at 9:11 PM, Fredrik Gustafsson <iveqy@iveqy.com> wrote:\n> On Sun, Jul 29, 2012 at 08:45:39PM +0530, Sitaram Chamarty wrote:\n>> I think you misunderstood how gitolite works.  Gitolite does not have\n>> *any* user interaction other than sending some extra messages back via\n>> STDERR if you're using a normal git client to do normal git operations\n>> (clone/fetch/ls-remote).\n>\n> As you say, gitolite has userinteraction, and its not even standard to\n\nI think we differ on the meaning of the word \"interaction\".\n"},{"id":"196077","messageId":"7vvch6uw89.fsf@alter.siamese.dyndns.org","threadId":"31129","inReplyTo":"CAMK1S_iTv3BOV3LEoj83feuO2iQnPqTrOq0wyFC-GrvvJQqWTA@mail.gmail.com","subject":"Re: Enhancements to git-protocoll","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2012-07-29T20:51:34Z","receivedAt":"2012-07-29T20:51:34Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Sitaram Chamarty <sitaramc@gmail.com> writes:\n\n> Uggh, no.  Client-git should only talk to server-git.  It shouldn't be\n> talking first to some *other* program (in this case gitolite), and\n> then to to server-git.  That doesn't sound sane to me.\n>\n> You should wrap this whole thing around something else that does it in\n> 3 steps.  Check, create if needed, then the actual git command you\n> intend to run.  All this should be local to your environment, not\n> rolled into git; it's far too specific to be rolled into git itself,\n> if you ask me.\n\nThanks for saving me from having to state an obvious sanity ;-)\n"},{"id":"196080","messageId":"20120729212627.GA26430@paksenarrion.iveqy.com","threadId":"31129","inReplyTo":"7vvch6uw89.fsf@alter.siamese.dyndns.org","subject":"Re: Enhancements to git-protocoll","fromName":"Fredrik Gustafsson","fromEmail":"iveqy@iveqy.com","sentAt":"2012-07-29T21:26:27Z","receivedAt":"2012-07-29T21:26:27Z","isPatch":false,"sender":{"key":"iveqy@iveqy.com","avatar":"https://avatars.githubusercontent.com/u/761743?v=4"},"body":"Sorry I missed this thread earlier. I'll drop this if it's not something\nthat's wanted.\n\nOn Sun, Jul 29, 2012 at 01:51:34PM -0700, Junio C Hamano wrote:\n> Sitaram Chamarty <sitaramc@gmail.com> writes:\n> \n> > Uggh, no.  Client-git should only talk to server-git.  It shouldn't be\n> > talking first to some *other* program (in this case gitolite), and\n> > then to to server-git.  That doesn't sound sane to me.\n\nThis is exactly the way gitolite works. It's placed between git-server\nand git-client. Does some checks and approves a connection if some\ncriterias isn't met. See the example when trying to clone an\nnon-existing repo from gitolite. You won't get an git error but a\ngitolite error.\n\nI can understand why my idea is beeing rejected but I can't see why the\ngitolite way should be considered sane. It seems more like an hack to\nme (according to git design principles).\n\nSo from a git point of view, why is it sane for passing through STDERR\nbut not STDIN and STDOUT?\n\n(I realize that this is a closed matter but would appriciate an\nexplanation solely for my own educational purpose).\n\n-- \nMed vänliga hälsningar\nFredrik Gustafsson\n\ntel: 0733-608274\ne-post: iveqy@iveqy.com\n"},{"id":"196081","messageId":"7vr4ruuu2e.fsf@alter.siamese.dyndns.org","threadId":"31129","inReplyTo":"7vvch6uw89.fsf@alter.siamese.dyndns.org","subject":"Re: Enhancements to git-protocoll","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2012-07-29T21:38:17Z","receivedAt":"2012-07-29T21:38:17Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Junio C Hamano <gitster@pobox.com> writes:\n\n> Sitaram Chamarty <sitaramc@gmail.com> writes:\n>\n>> Uggh, no.  Client-git should only talk to server-git.  It shouldn't be\n>> talking first to some *other* program (in this case gitolite), and\n>> then to to server-git.  That doesn't sound sane to me.\n>>\n>> You should wrap this whole thing around something else that does it in\n>> 3 steps.  Check, create if needed, then the actual git command you\n>> intend to run.  All this should be local to your environment, not\n>> rolled into git; it's far too specific to be rolled into git itself,\n>> if you ask me.\n>\n> Thanks for saving me from having to state an obvious sanity ;-)\n\nHaving said all that, I am not fundamentally opposed to new features\nadded to the git protocol.  The responses so far from me in this\nthread were primarily me reacting to (1) auto creation by gitolite\nis a bad example and (2) the proposal sounded, at least to me, that\nit wants to add random and uncontrolled interactions between the\nsides outside the defined protocol exchange (e.g. \"pusher connects,\npushee says what it has and what capabilities it supports, pusher\nsays what it wants to update how and chooses what capabilities it\nwants to use in the exchange, sends the pack data, ...\"), which is\nan unworkable idea.\n\nFor people who do not understand how the git protocol works (the\nproposal that started this thread included), a bit of clarification\non (2) may help.\n\nLong time ago, we did not have \"report-status\" capability (think of\na \"capability\" as a \"protocol extension\").  The \"git push\" protocol\nexchange ended with the pusher sending the pack data and that was\nthe end of conversation.\n\nBut we realized that it would be beneficial for the pushee to be\nable to tell the pusher that the push failed at the protocol level,\nso that the \"git push\" program can exit with an error code.  The\nversion of the \"git push\" program that was current at the time\nexited with success unconditionally, as long as all protocol\nexchange went well and all the pack data was sent successfully.  The\nprogram's flow was not expecting any response once the pack data\nstarted flowing from it to the pushee, so that is the only\nreasonable behaviour.\n\nSo what did we do?  We added \"report-status\" capability to the\ncapability suite, and made new versions of \"receive-pack\" program\nadvertise it.  Existing versions of \"git push\" did not know about\nthis capability, so they did not ask for it and pusher and pushee\nworked exactly as before.\n\nBut that way, the new versions of \"git push\" can learn to ask for\n\"report-status\" capability to updated \"receive-pack\", and the new\nversions of \"receive-pack\", after receiving and processing the pack\ndata stream, can send new messages the original protocol did not\nallow it to send to \"git push\".\n\nAn important point to understand is that there is one more thing\nthat is needed.  The updated \"git push\" that asks for \"report-status\"\nneeded to learn how to interpret the new message and act on it.\n\nAnother example in the same protocol was the addition of \"sideband\"\ncapability.  Before that happened, there was no way to send the\nerror stream from the pushee to the pusher.  Unlike \"report-status\"\nthat happens at the very end, this actually changes the way how the\nremainder of the protocol exchange proceeds once it is activated.\n\nYou could add a new capability that says \"when in effect, both ends\ncan write to and read from new sidebands 4 and 5 to communicate out\nof line\", but that is not all that useful.  You need to make the\nupdated programs to agree on what should happen to the main\ninteraction when a new kind of communication is made out of band.\nFor example, you may ask \"do you really mean it [Y/n]?\" to the\nclient, pause the entire transaction until you hear Yes or No from\nthe client, and you may even choose to do something different from\nthe usual when the client says No, but you also need to update the\nclient to behave differently after that, perhaps by defining a new\nconversation path in the protocol.  At that point, you would need to\nhandle real and concrete definition of the extended protocol and the\ncode on the both sides to support it _anyway_.\n\nFor example, imagine that you want to let a new user interaction to\nverify the repository it is pushing to.  How would we do that with\nyour \"adding random interaction between the server and client\"?\n\nThe server end notices that the user is trying to push into one\nrepository.  throws \"Do you really mean to push into repository\nfoo?\"  Updated client that understands your \"random interaction\"\nextention is capable of show this message, pauses until the user\nsays \"Yes\" or \"No\", and sends it back to the server end.  If the\nanswer is \"No\", the client may want to say \"No, the repository I\nreally wanted to push into was not foo, but bar\".\n\nHow did the client _know_ the \"Do you really mean\" message from the\nserver requires a Yes or No response in the above scenario?\n\nHow did the client know, when saying \"No\", to ask the real URL of\nthe repository to the end user, disconnect the current session with\nthe server, and restart itself using the real URL of the repository?\n\nAll of that need clearly defined new protocol exchange steps, state\ntransitions, and new conditional and new codepaths to implement\nthem.  It cannot be just \"let's allow the server and the client to\nrandomly exchange packets out of band\".\n\nIn short, I am not fundamentally opposed to protocol extension, as I\nsaid in my very first response.\n\nI however do not think allowing the server 'to communicate directly\nto the git user' (which was the phrase used in the original message\nthat expressed the idea) is a good idea, at least for two reasons:\n\n - because we already have a proven-to-work-well way to extend the\n   protocol via \"capability\"; and\n\n - because doing so is not useful for the purpose of extending the\n   feature of the protocol,\n"},{"id":"196091","messageId":"CAMK1S_hAO0LGHa7e3KY6bT9z+f76Jc6_gCpwYbLqV9WPdgX1Cw@mail.gmail.com","threadId":"31129","inReplyTo":"20120729212627.GA26430@paksenarrion.iveqy.com","subject":"Re: Enhancements to git-protocoll","fromName":"Sitaram Chamarty","fromEmail":"sitaramc@gmail.com","sentAt":"2012-07-30T00:37:06Z","receivedAt":"2012-07-30T00:37:06Z","isPatch":false,"sender":{"key":"sitaramc@gmail.com","avatar":"https://avatars.githubusercontent.com/u/43316?v=4"},"body":"On Mon, Jul 30, 2012 at 2:56 AM, Fredrik Gustafsson <iveqy@iveqy.com> wrote:\n> Sorry I missed this thread earlier. I'll drop this if it's not something\n> that's wanted.\n>\n> On Sun, Jul 29, 2012 at 01:51:34PM -0700, Junio C Hamano wrote:\n>> Sitaram Chamarty <sitaramc@gmail.com> writes:\n>>\n>> > Uggh, no.  Client-git should only talk to server-git.  It shouldn't be\n>> > talking first to some *other* program (in this case gitolite), and\n>> > then to to server-git.  That doesn't sound sane to me.\n>\n> This is exactly the way gitolite works. It's placed between git-server\n> and git-client. Does some checks and approves a connection if some\n> criterias isn't met. See the example when trying to clone an\n> non-existing repo from gitolite. You won't get an git error but a\n> gitolite error.\n>\n> I can understand why my idea is beeing rejected but I can't see why the\n> gitolite way should be considered sane. It seems more like an hack to\n> me (according to git design principles).\n>\n> So from a git point of view, why is it sane for passing through STDERR\n> but not STDIN and STDOUT?\n\nThat is precisely the point.  The pack protocol (see\nDocumentation/pack-protocol.txt in the git sources) works with\nSTDIN/STDOUT.  Run with GIT_TRACE_PACKET=1 to see some of that info\nfly past on various git commands.\n\nIt explicitly leaves STDERR for the purpose of providing user's extra\ninformation, including errors.\n\nGitolite is merely using that same mechanism.\n\nAnd I repeat, if you insist on calling what gitolite does an\n\"interaction\" (which to me means two-way communication, not a one-way\nerror/warning/diagnostic stream), we do not have enough in common to\ndiscuss this any more.\n"},{"id":"196095","messageId":"CAMK1S_geKq5DkNAy2wxGt5EmoOesxaSKKuQjVaQVcbRVdR+9gA@mail.gmail.com","threadId":"31129","inReplyTo":"7vr4ruuu2e.fsf@alter.siamese.dyndns.org","subject":"Re: Enhancements to git-protocoll","fromName":"Sitaram Chamarty","fromEmail":"sitaramc@gmail.com","sentAt":"2012-07-30T01:04:43Z","receivedAt":"2012-07-30T01:04:43Z","isPatch":false,"sender":{"key":"sitaramc@gmail.com","avatar":"https://avatars.githubusercontent.com/u/43316?v=4"},"body":"On Mon, Jul 30, 2012 at 3:08 AM, Junio C Hamano <gitster@pobox.com> wrote:\n> Junio C Hamano <gitster@pobox.com> writes:\n>\n>> Sitaram Chamarty <sitaramc@gmail.com> writes:\n>>\n>>> Uggh, no.  Client-git should only talk to server-git.  It shouldn't be\n>>> talking first to some *other* program (in this case gitolite), and\n>>> then to to server-git.  That doesn't sound sane to me.\n>>>\n>>> You should wrap this whole thing around something else that does it in\n>>> 3 steps.  Check, create if needed, then the actual git command you\n>>> intend to run.  All this should be local to your environment, not\n>>> rolled into git; it's far too specific to be rolled into git itself,\n>>> if you ask me.\n>>\n>> Thanks for saving me from having to state an obvious sanity ;-)\n>\n> Having said all that, I am not fundamentally opposed to new features\n> added to the git protocol.  The responses so far from me in this\n> thread were primarily me reacting to (1) auto creation by gitolite\n> is a bad example and (2) the proposal sounded, at least to me, that\n> it wants to add random and uncontrolled interactions between the\n> sides outside the defined protocol exchange (e.g. \"pusher connects,\n> pushee says what it has and what capabilities it supports, pusher\n> says what it wants to update how and chooses what capabilities it\n> wants to use in the exchange, sends the pack data, ...\"), which is\n> an unworkable idea.\n>\n> For people who do not understand how the git protocol works (the\n> proposal that started this thread included), a bit of clarification\n> on (2) may help.\n>\n> Long time ago, we did not have \"report-status\" capability (think of\n> a \"capability\" as a \"protocol extension\").  The \"git push\" protocol\n> exchange ended with the pusher sending the pack data and that was\n> the end of conversation.\n>\n> But we realized that it would be beneficial for the pushee to be\n> able to tell the pusher that the push failed at the protocol level,\n> so that the \"git push\" program can exit with an error code.  The\n> version of the \"git push\" program that was current at the time\n> exited with success unconditionally, as long as all protocol\n> exchange went well and all the pack data was sent successfully.  The\n> program's flow was not expecting any response once the pack data\n> started flowing from it to the pushee, so that is the only\n> reasonable behaviour.\n>\n> So what did we do?  We added \"report-status\" capability to the\n> capability suite, and made new versions of \"receive-pack\" program\n> advertise it.  Existing versions of \"git push\" did not know about\n> this capability, so they did not ask for it and pusher and pushee\n> worked exactly as before.\n>\n> But that way, the new versions of \"git push\" can learn to ask for\n> \"report-status\" capability to updated \"receive-pack\", and the new\n> versions of \"receive-pack\", after receiving and processing the pack\n> data stream, can send new messages the original protocol did not\n> allow it to send to \"git push\".\n>\n> An important point to understand is that there is one more thing\n> that is needed.  The updated \"git push\" that asks for \"report-status\"\n> needed to learn how to interpret the new message and act on it.\n>\n> Another example in the same protocol was the addition of \"sideband\"\n> capability.  Before that happened, there was no way to send the\n> error stream from the pushee to the pusher.  Unlike \"report-status\"\n> that happens at the very end, this actually changes the way how the\n> remainder of the protocol exchange proceeds once it is activated.\n>\n> You could add a new capability that says \"when in effect, both ends\n> can write to and read from new sidebands 4 and 5 to communicate out\n> of line\", but that is not all that useful.  You need to make the\n> updated programs to agree on what should happen to the main\n> interaction when a new kind of communication is made out of band.\n> For example, you may ask \"do you really mean it [Y/n]?\" to the\n> client, pause the entire transaction until you hear Yes or No from\n> the client, and you may even choose to do something different from\n> the usual when the client says No, but you also need to update the\n> client to behave differently after that, perhaps by defining a new\n> conversation path in the protocol.  At that point, you would need to\n> handle real and concrete definition of the extended protocol and the\n> code on the both sides to support it _anyway_.\n>\n> For example, imagine that you want to let a new user interaction to\n> verify the repository it is pushing to.  How would we do that with\n> your \"adding random interaction between the server and client\"?\n>\n> The server end notices that the user is trying to push into one\n> repository.  throws \"Do you really mean to push into repository\n> foo?\"  Updated client that understands your \"random interaction\"\n> extention is capable of show this message, pauses until the user\n> says \"Yes\" or \"No\", and sends it back to the server end.  If the\n> answer is \"No\", the client may want to say \"No, the repository I\n> really wanted to push into was not foo, but bar\".\n>\n> How did the client _know_ the \"Do you really mean\" message from the\n> server requires a Yes or No response in the above scenario?\n>\n> How did the client know, when saying \"No\", to ask the real URL of\n> the repository to the end user, disconnect the current session with\n> the server, and restart itself using the real URL of the repository?\n>\n> All of that need clearly defined new protocol exchange steps, state\n> transitions, and new conditional and new codepaths to implement\n> them.  It cannot be just \"let's allow the server and the client to\n> randomly exchange packets out of band\".\n>\n> In short, I am not fundamentally opposed to protocol extension, as I\n> said in my very first response.\n>\n> I however do not think allowing the server 'to communicate directly\n> to the git user' (which was the phrase used in the original message\n> that expressed the idea) is a good idea, at least for two reasons:\n>\n>  - because we already have a proven-to-work-well way to extend the\n>    protocol via \"capability\"; and\n>\n>  - because doing so is not useful for the purpose of extending the\n>    feature of the protocol,\n>\n\nAs I may have said earlier, this interaction is far too site-specific\nto be rolled into git itself.\n\nHow about a new hook instead?  A pre-pack-protocol hook that acts as\nif it was called by the remote user as a command, and if it exit's\nwith 0, then the real pack protocol starts else it gets aborted.  Let\nhim do whatever he wants in there.  Arguments to the hook will be repo\nname and command (git-upload-pack mainly).\n\n>From the ssh stream point of view, he'd have full STDIN/OUT/ERR\navailable in this hook, but when the hook exits, the normal protocol\ntakes over (or the whole thing aborts).\n\nOf course this will only work with ssh.  None of what Fredrik has so\nfar suggested would possibly work on smart http without even more\nhacks, I think.\n\nAnd even then, all we are doing is rolling into git something that he\ncan very easily do outside right now on his own environment if he\nwishes to, using a shell script on each side.  (And therein lies a\nmore general rant for the future, but not now...!)\n\n-- \nSitaram\n"},{"id":"196097","messageId":"CAJo=hJsgMz-=pdG=HB3KtTE7g6Gvu9mr56dg2+V3WUST+rpAQg@mail.gmail.com","threadId":"31129","inReplyTo":"CAMK1S_geKq5DkNAy2wxGt5EmoOesxaSKKuQjVaQVcbRVdR+9gA@mail.gmail.com","subject":"Re: Enhancements to git-protocoll","fromName":"Shawn Pearce","fromEmail":"spearce@spearce.org","sentAt":"2012-07-30T01:21:20Z","receivedAt":"2012-07-30T01:21:20Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"On Sun, Jul 29, 2012 at 6:04 PM, Sitaram Chamarty <sitaramc@gmail.com> wrote:\n> Of course this will only work with ssh.  None of what Fredrik has so\n> far suggested would possibly work on smart http without even more\n> hacks, I think.\n\nNow that we have smart HTTP, and its somewhat popular for sites to\ndeploy with, we need to carefully consider all future protocol changes\nto make sure they are compatible with the HTTP one. Since the HTTP is\nsingle request/response model, its hard to implement a conversation\nwith the end-user.\n\nOne thing I would like to do with the protocol is add custom site\nspecific extensions to the protocol, where hooks are able to advertise\nin the initial capability list something like a namespace prefix that\n`git push` can use to offer site specific command line flags from:\n\n  ....00000000... HEAD^{} report-status delete-refs side-band-64k\nofs-delta hook=gitolite\n\nand the client seeing this would recognize a push command like:\n\n  git push -Lgitolite,create-repository URL master\n\npassing the string \"gitolite,create-repository\" as data in the header\nof the push request. gitolite would need to scan more than just the\ngit receive-pack command line from SSH to see this data, but you can\nuse it to implement an \"are you sure you want to create this\nrepository\" exchange by failing a push with sideband information\ntelling the user to reinvoke push with the create-repository flag if\nthey really mean to create it.\n\nWe sort of want this in Gerrit Code Review to pass reviewer names on\nthe command line of git push, making it easier for users to upload a\ncode review. The idea is similar to what happens with gcc accepting\nlinker flags that are just passed onto the linker. From what I\nunderstand, Mercurial already has something like this in their push\nsystem for hooks to accept additional data one-way from the client.\n"},{"id":"196098","messageId":"7vr4rut4u9.fsf@alter.siamese.dyndns.org","threadId":"31129","inReplyTo":"CAMK1S_geKq5DkNAy2wxGt5EmoOesxaSKKuQjVaQVcbRVdR+9gA@mail.gmail.com","subject":"Re: Enhancements to git-protocoll","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2012-07-30T01:28:30Z","receivedAt":"2012-07-30T01:28:30Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Sitaram Chamarty <sitaramc@gmail.com> writes:\n\n> As I may have said earlier, this interaction is far too site-specific\n> to be rolled into git itself.\n>\n> How about a new hook instead?  A pre-pack-protocol hook that acts as\n> if it was called by the remote user as a command, and if it exit's\n> with 0, then the real pack protocol starts else it gets aborted.  Let\n> him do whatever he wants in there.  Arguments to the hook will be repo\n> name and command (git-upload-pack mainly).\n\nNot very interested.  If it is _known_ to happen before Git protocol\nproper happens, why not give the user something like the gitolite\ncommand \"D\", that is interpreted by gitolite-shell without bothering\nGit at all?\n\nAfter all, at least you and I share the understanding that once the\nconversation in Git protocol proper starts, there is no place for\nsuch a random hook to affect further behaviour of the protocol, so\nthe approach \"hook\" can only solve narrow \"before gitolite-shell (or\ngit-shell or whatever-shell) decides it is time to call Git\" cases,\nand the only way it can affect the outcome of the main conversation\nis to abort it without graceful degradation, or let the main\nconversation continue as if nothing happened.  I think we agree\nbetween us that \"a new hook\", while it may be _a_ way to do\nsomething new, is _not_ a good way to do so.  Why add such a wart?\n\n> And even then, all we are doing is rolling into git something that he\n> can very easily do outside right now on his own environment ...\n\nYes, that is a repeat of what you told him already, to which I said\n\"thanks for sanity\", I think ;-).\n\nI am not opposed to protocol enhancements, but a new feature that\nhas to change what either end does should be added properly at the\nprotocol level as a protocol capability, not added out of band with\nband-aid, leaving the main conversation oblivious to what is going\non ouside.\n"},{"id":"196099","messageId":"CAMK1S_i_jd_C29bBo79iiPMfBZ4XoXhuiP3f05F9xSpUPX=JBQ@mail.gmail.com","threadId":"31129","inReplyTo":"CAJo=hJsgMz-=pdG=HB3KtTE7g6Gvu9mr56dg2+V3WUST+rpAQg@mail.gmail.com","subject":"Re: Enhancements to git-protocoll","fromName":"Sitaram Chamarty","fromEmail":"sitaramc@gmail.com","sentAt":"2012-07-30T01:33:10Z","receivedAt":"2012-07-30T01:33:10Z","isPatch":false,"sender":{"key":"sitaramc@gmail.com","avatar":"https://avatars.githubusercontent.com/u/43316?v=4"},"body":"On Mon, Jul 30, 2012 at 6:51 AM, Shawn Pearce <spearce@spearce.org> wrote:\n> On Sun, Jul 29, 2012 at 6:04 PM, Sitaram Chamarty <sitaramc@gmail.com> wrote:\n>> Of course this will only work with ssh.  None of what Fredrik has so\n>> far suggested would possibly work on smart http without even more\n>> hacks, I think.\n>\n> Now that we have smart HTTP, and its somewhat popular for sites to\n> deploy with, we need to carefully consider all future protocol changes\n> to make sure they are compatible with the HTTP one. Since the HTTP is\n> single request/response model, its hard to implement a conversation\n> with the end-user.\n>\n> One thing I would like to do with the protocol is add custom site\n> specific extensions to the protocol, where hooks are able to advertise\n> in the initial capability list something like a namespace prefix that\n> `git push` can use to offer site specific command line flags from:\n>\n>   ....00000000... HEAD^{} report-status delete-refs side-band-64k\n> ofs-delta hook=gitolite\n>\n> and the client seeing this would recognize a push command like:\n>\n>   git push -Lgitolite,create-repository URL master\n>\n> passing the string \"gitolite,create-repository\" as data in the header\n> of the push request. gitolite would need to scan more than just the\n> git receive-pack command line from SSH to see this data, but you can\n\nI would avoid anything that requires gitolite to even *know* the pack\nprotocol.  They should be orthogonal.\n\nBut if you mean this extra info will be added to the command line\nitself in some way (or in http terms to the REQUEST_URI, PATH_INFO,\nQUERY_STRING, etc) then that's fine.\n\nPeople who use gitolite already should try \"git ls-remote\ngit@host:testing.git1\" instead of the usual \"...testing\" or\n\"...testing.git\" to see how a trace capability is currently hacked\ninto gitolite.  Also change the 1 to a 2 then a 3 if you wish.\n\nI'd certainly love a cleaner way of doing this, and what you suggest\nseems it will satisfy.\n\n> use it to implement an \"are you sure you want to create this\n> repository\" exchange by failing a push with sideband information\n> telling the user to reinvoke push with the create-repository flag if\n> they really mean to create it.\n>\n> We sort of want this in Gerrit Code Review to pass reviewer names on\n> the command line of git push, making it easier for users to upload a\n> code review. The idea is similar to what happens with gcc accepting\n> linker flags that are just passed onto the linker. From what I\n> understand, Mercurial already has something like this in their push\n> system for hooks to accept additional data one-way from the client.\n\n\n\n-- \nSitaram\n"},{"id":"196100","messageId":"CAMK1S_gQWkvmL59aDYMOL1zUv04Wf7WcV7dEi9Uz6=UP9g7AOA@mail.gmail.com","threadId":"31129","inReplyTo":"7vr4rut4u9.fsf@alter.siamese.dyndns.org","subject":"Re: Enhancements to git-protocoll","fromName":"Sitaram Chamarty","fromEmail":"sitaramc@gmail.com","sentAt":"2012-07-30T01:45:09Z","receivedAt":"2012-07-30T01:45:09Z","isPatch":false,"sender":{"key":"sitaramc@gmail.com","avatar":"https://avatars.githubusercontent.com/u/43316?v=4"},"body":"On Mon, Jul 30, 2012 at 6:58 AM, Junio C Hamano <gitster@pobox.com> wrote:\n> Sitaram Chamarty <sitaramc@gmail.com> writes:\n>\n>> As I may have said earlier, this interaction is far too site-specific\n>> to be rolled into git itself.\n>>\n>> How about a new hook instead?  A pre-pack-protocol hook that acts as\n>> if it was called by the remote user as a command, and if it exit's\n>> with 0, then the real pack protocol starts else it gets aborted.  Let\n>> him do whatever he wants in there.  Arguments to the hook will be repo\n>> name and command (git-upload-pack mainly).\n>\n> Not very interested.  If it is _known_ to happen before Git protocol\n> proper happens, why not give the user something like the gitolite\n> command \"D\", that is interpreted by gitolite-shell without bothering\n> Git at all?\n\nIndeed.  In one of the earlier messages I already told him he should\ncopy and munge the fork command in gitolite to do precisely this.\n"},{"id":"196101","messageId":"7vehnut1kt.fsf@alter.siamese.dyndns.org","threadId":"31129","inReplyTo":"CAJo=hJsgMz-=pdG=HB3KtTE7g6Gvu9mr56dg2+V3WUST+rpAQg@mail.gmail.com","subject":"Re: Enhancements to git-protocoll","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2012-07-30T02:38:58Z","receivedAt":"2012-07-30T02:38:58Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Shawn Pearce <spearce@spearce.org> writes:\n\n> We sort of want this in Gerrit Code Review to pass reviewer names on\n> the command line of git push, making it easier for users to upload a\n> code review. The idea is similar to what happens with gcc accepting\n> linker flags that are just passed onto the linker.\n\nFor reviewer names, authentication cookies and things of that nature\nwhere the extra pieces of information affect the outcome in a way\nthat does not have to change how the underlying protocol exchange\nworks, such an additional one-way channel from the pusher to pushee\nto carry auxiliary information would be sufficient.  The server may\ndecide to accept otherwise forbidden, or reject otherwise permitted,\npush based on the extra information given, for example, and that is\nan example of an enhancement that does not have to change how the\nunderlying protocol exchange works.\n\nThe way to expose the extra information parsed by Git to the server\nside could be made into calling out to hooks, and at that point,\ngitolite would not even have to know about the pack protocol.\nPerhaps the interface to such a hook may be \"hook can tell Git to\nabort the communication by exiting non-zero, after giving a message\nto its standard output\".\n\nIt is a separate matter if it makes sense to add another channel\nthat goes the other way on demand (i.e.  taking the end-user\nresponse from the pusher and giving it to the pushee, and then\nallowing the pushee change its behaviour in a way more than just\nsimply aborting the connection but performing a useful alternative\noperation)---I doubt it is.\n"},{"id":"196104","messageId":"CAJo=hJvjFP0m96YPNBCwvnkRGWhMZQdbL7C3Shsa6HQ62a9FuA@mail.gmail.com","threadId":"31129","inReplyTo":"7vehnut1kt.fsf@alter.siamese.dyndns.org","subject":"Re: Enhancements to git-protocoll","fromName":"Shawn Pearce","fromEmail":"spearce@spearce.org","sentAt":"2012-07-30T05:20:43Z","receivedAt":"2012-07-30T05:20:43Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"On Sun, Jul 29, 2012 at 7:38 PM, Junio C Hamano <gitster@pobox.com> wrote:\n> Shawn Pearce <spearce@spearce.org> writes:\n>\n>> We sort of want this in Gerrit Code Review to pass reviewer names on\n>> the command line of git push, making it easier for users to upload a\n>> code review. The idea is similar to what happens with gcc accepting\n>> linker flags that are just passed onto the linker.\n>\n> For reviewer names, authentication cookies and things of that nature\n> where the extra pieces of information affect the outcome in a way\n> that does not have to change how the underlying protocol exchange\n> works, such an additional one-way channel from the pusher to pushee\n> to carry auxiliary information would be sufficient.\n\nYes, that is what I was trying to argue. :-)\n\nI agree that authentication information is outside of the Git protocol\nitself. We rely on SSH authentication for SSH and HTTP native\nauthentication methods for HTTP transport. But at least in the HTTP\ncase, the Git client has learned how to set up the authentication data\nfor the user to make it easier to use HTTP authentication. We don't\nyet support native OAuth 2.0 (ick!) or HTTP cookies as well as we do\nclient side SSL certificates or basic username/password pair.\n\nIf we want to support additional information from pusher to pushee,\nthis is a \"native feature\" of Git and should be supported on all\nnative push type transports, with roughly the same semantics\neverywhere. I don't want to add additional data into X-Git-Foo HTTP\nheaders in HTTP, and as environment variables in SSH, for example.\nAdditional HTTP headers will *probably* transit an HTTP proxy\ncorrectly (but there are a lot of broken proxy servers so I don't put\nit past someone to strip an X-* header they don't think is \"safe\").\nSSH environment variables are icky to set from the client, and server\nside Git would need to know how it was invoked to decode the correct\ndata and make it available uniformly to repository owner authored\nhooks.\n\n>  The server may\n> decide to accept otherwise forbidden, or reject otherwise permitted,\n> push based on the extra information given, for example, and that is\n> an example of an enhancement that does not have to change how the\n> underlying protocol exchange works.\n\nYes.\n\n> The way to expose the extra information parsed by Git to the server\n> side could be made into calling out to hooks, and at that point,\n> gitolite would not even have to know about the pack protocol.\n\nGood point. The case that spawned this thread however still has a\nproblem with this approach. gitolite would need to create a repository\nto invoke the receive-pack process within, and install that new hook\nscript into... when the hook was trying to prevent the creation of\nthat repository in the first place.\n\nMaybe I am jaded by the way JGit handles the protocol, it is easy for\napplication code to glue into and see things going on in the protocol\nin ways that are hard to do from git-core.\n\n> Perhaps the interface to such a hook may be \"hook can tell Git to\n> abort the communication by exiting non-zero, after giving a message\n> to its standard output\".\n\nPerhaps this new channel data is simply passed as arguments to\nreceive-pack on the remote side?\n\nAn ancient Git would abort hard if passed this flag. An updated Git\ncould set environment variables before calling hooks, making the\narguments visible that way. And gitolite can still scrape what it\nneeds from the command line without having to muck about inside of the\nprotocol, but only if it needs to observe this new data from pusher to\npushee?\n\n`git push -Rfoo=baz host:dest.git master` on the client would turn\ninto `git-receive-pack -Rfoo=baz dest.git` in the SSH and git://\ncommand line, and cause GIT_PUSH_ARG_FOO=baz to appear in the\nenvironment of hooks. Over smart HTTP requests would get an additional\nquery parameter of foo=baz.\n\n\nThe other hacky idea I had was to use a fake reference and have the\nclient push a structured blob to that ref. The server would decode the\nblob, and deny the creation of the fake reference, but be able to get\nadditional data from that blob. Its hacky, and I don't like making a\nnew blob on the server just to transport a few small bits of data from\nthe client.\n"},{"id":"196106","messageId":"7vy5m1sqz3.fsf@alter.siamese.dyndns.org","threadId":"31129","inReplyTo":"CAJo=hJvjFP0m96YPNBCwvnkRGWhMZQdbL7C3Shsa6HQ62a9FuA@mail.gmail.com","subject":"Re: Enhancements to git-protocoll","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2012-07-30T06:28:00Z","receivedAt":"2012-07-30T06:28:00Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Shawn Pearce <spearce@spearce.org> writes:\n\n>> The way to expose the extra information parsed by Git to the server\n>> side could be made into calling out to hooks, and at that point,\n>> gitolite would not even have to know about the pack protocol.\n>\n> Good point. The case that spawned this thread however still has a\n> problem with this approach. gitolite would need to create a repository\n> to invoke the receive-pack process within, and install that new hook\n> script into... when the hook was trying to prevent the creation of\n> that repository in the first place.\n\nHeh.  While I do not particularly consider auto-creation-upon-push a\nuseful thing to begin with (after all, once you created a\nrepository, you would want ways to manage it, setting up ACL for it\nand stuff like that, so adding a \"create\" command to the management\ninterface suite would be a more natural direction to go), as long as\nwe are discussing a hack that involves hooks, I do not think your\nobservation is a show-stopper downside.\n\nThe hook can interact with the end user over the back channel and\ndecide to abort the transaction, while leaving some clue in the\nrepository that is pre-agreed between the gitlite server and the\nhook.  When gitolite culls the process with wait4(2), it could\nnotice that \"clue\", read the wish of the hook that the repository\nneeds to be removed from it, and remove the repository. Up to that\npoint, there is no real data transferred, so there isn't much wasted\ntime or network resource anyway.\n\n> An ancient Git would abort hard if passed this flag. An updated Git\n> could set environment variables before calling hooks, making the\n> arguments visible that way. And gitolite can still scrape what it\n> needs from the command line without having to muck about inside of the\n> protocol, but only if it needs to observe this new data from pusher to\n> pushee?\n\nI do not think the details of how the extra information is passed\nvia the Git at the receiving end to its surrounding matters that\nmuch.  It would even work fine if we let the hook to talk with the\nend user sitting at the \"git push\" end, by using two extra sidebands\nto throw bits between them, while the Git process that spawned the\nhook acts as a relay, to establish a custom bi-di conversation (but\nagain, I do not think it is useful, because such an out of band\nconversation cannot affect the outcome of the main protocol exchange\nin a meaningful way other than aborting).\n\nOr you could export environment variables, which would be far more\nlimiting with respect to the nature of the data (i.e. needs to be\nfree of NUL) and the size of data you can pass.  The limitation may\nactually be a feature to discourage people from doing wacky things,\nthough.\n\n> `git push -Rfoo=baz host:dest.git master` on the client would turn\n> into `git-receive-pack -Rfoo=baz dest.git` in the SSH and git://\n> command line, and cause GIT_PUSH_ARG_FOO=baz to appear in the\n> environment of hooks. Over smart HTTP requests would get an additional\n> query parameter of foo=baz.\n\nI think using the same \"extra args on the command line\" would be a\ngood way to upgrade the protocol version in a way the current\n\"capability\" system does not allow us to (namely, stop the party\nthat accepts the connection from immediately advertising its refs).\n\n> The other hacky idea I had was to use a fake reference and have the\n> client push a structured blob to that ref. The server would decode the\n> blob, and deny the creation of the fake reference, but be able to get\n> additional data from that blob. Its hacky, and I don't like making a\n> new blob on the server just to transport a few small bits of data from\n> the client.\n\nThat way lies madness, and at that point, you are better off doing a\nproper protocol extension by registering a new capability and defining\nthe semantics for it.\n"},{"id":"196108","messageId":"CAMK1S_jGXh408K1pr8K5_U8fBMEWnqS7EJAfgoNz=7vYrEf2tQ@mail.gmail.com","threadId":"31129","inReplyTo":"7vy5m1sqz3.fsf@alter.siamese.dyndns.org","subject":"Re: Enhancements to git-protocoll","fromName":"Sitaram Chamarty","fromEmail":"sitaramc@gmail.com","sentAt":"2012-07-30T08:12:36Z","receivedAt":"2012-07-30T08:12:36Z","isPatch":false,"sender":{"key":"sitaramc@gmail.com","avatar":"https://avatars.githubusercontent.com/u/43316?v=4"},"body":"On Mon, Jul 30, 2012 at 11:58 AM, Junio C Hamano <gitster@pobox.com> wrote:\n\n> Heh.  While I do not particularly consider auto-creation-upon-push a\n> useful thing to begin with (after all, once you created a\n> repository, you would want ways to manage it, setting up ACL for it\n\n[side point] these things are managed with templates of ACL rules that\nmap access to \"roles\", and the owner (the guy who created it) defines\nusers for each role.\n"},{"id":"196111","messageId":"CAMK1S_iRh0qVonQ5aDdrqh4H_E+-2WRYt=LzK4MAnnnuWY+DuQ@mail.gmail.com","threadId":"31129","inReplyTo":"7vy5m1sqz3.fsf@alter.siamese.dyndns.org","subject":"Re: Enhancements to git-protocoll","fromName":"Sitaram Chamarty","fromEmail":"sitaramc@gmail.com","sentAt":"2012-07-30T08:31:11Z","receivedAt":"2012-07-30T08:31:11Z","isPatch":false,"sender":{"key":"sitaramc@gmail.com","avatar":"https://avatars.githubusercontent.com/u/43316?v=4"},"body":"On Mon, Jul 30, 2012 at 11:58 AM, Junio C Hamano <gitster@pobox.com> wrote:\n> Shawn Pearce <spearce@spearce.org> writes:\n>\n>>> The way to expose the extra information parsed by Git to the server\n>>> side could be made into calling out to hooks, and at that point,\n>>> gitolite would not even have to know about the pack protocol.\n>>\n>> Good point. The case that spawned this thread however still has a\n>> problem with this approach. gitolite would need to create a repository\n>> to invoke the receive-pack process within, and install that new hook\n>> script into... when the hook was trying to prevent the creation of\n>> that repository in the first place.\n>\n> Heh.  While I do not particularly consider auto-creation-upon-push a\n> useful thing to begin with (after all, once you created a\n> repository, you would want ways to manage it, setting up ACL for it\n> and stuff like that, so adding a \"create\" command to the management\n> interface suite would be a more natural direction to go), as long as\n> we are discussing a hack that involves hooks, I do not think your\n> observation is a show-stopper downside.\n>\n> The hook can interact with the end user over the back channel and\n> decide to abort the transaction, while leaving some clue in the\n> repository that is pre-agreed between the gitlite server and the\n> hook.  When gitolite culls the process with wait4(2), it could\n> notice that \"clue\", read the wish of the hook that the repository\n> needs to be removed from it, and remove the repository. Up to that\n> point, there is no real data transferred, so there isn't much wasted\n> time or network resource anyway.\n>\n>> An ancient Git would abort hard if passed this flag. An updated Git\n>> could set environment variables before calling hooks, making the\n>> arguments visible that way. And gitolite can still scrape what it\n>> needs from the command line without having to muck about inside of the\n>> protocol, but only if it needs to observe this new data from pusher to\n>> pushee?\n>\n> I do not think the details of how the extra information is passed\n> via the Git at the receiving end to its surrounding matters that\n> much.  It would even work fine if we let the hook to talk with the\n> end user sitting at the \"git push\" end, by using two extra sidebands\n> to throw bits between them, while the Git process that spawned the\n> hook acts as a relay, to establish a custom bi-di conversation (but\n> again, I do not think it is useful, because such an out of band\n> conversation cannot affect the outcome of the main protocol exchange\n> in a meaningful way other than aborting).\n>\n> Or you could export environment variables, which would be far more\n> limiting with respect to the nature of the data (i.e. needs to be\n> free of NUL) and the size of data you can pass.  The limitation may\n> actually be a feature to discourage people from doing wacky things,\n> though.\n>\n>> `git push -Rfoo=baz host:dest.git master` on the client would turn\n>> into `git-receive-pack -Rfoo=baz dest.git` in the SSH and git://\n>> command line, and cause GIT_PUSH_ARG_FOO=baz to appear in the\n>> environment of hooks. Over smart HTTP requests would get an additional\n>> query parameter of foo=baz.\n>\n> I think using the same \"extra args on the command line\" would be a\n> good way to upgrade the protocol version in a way the current\n> \"capability\" system does not allow us to (namely, stop the party\n> that accepts the connection from immediately advertising its refs).\n\nMore importantly from gitolite's point of view, this is the only way\ngitolite can see those variables in some situations, because gitolite\nruns *before* git, (and then again later via the update hook for\npushes).\n\n>> The other hacky idea I had was to use a fake reference and have the\n>> client push a structured blob to that ref. The server would decode the\n>> blob, and deny the creation of the fake reference, but be able to get\n>> additional data from that blob. Its hacky, and I don't like making a\n>> new blob on the server just to transport a few small bits of data from\n>> the client.\n>\n> That way lies madness, and at that point, you are better off doing a\n> proper protocol extension by registering a new capability and defining\n> the semantics for it.\n\n\n\n-- \nSitaram\n"}]}