{"thread":{"id":"2206","subject":"[RFC] GIT paths","startedAt":"2005-10-24T08:50:48Z","lastAt":"2005-10-26T08:56:55Z","messageCount":5,"participants":["Junio C Hamano","Andreas Ericsson","H. Peter Anvin"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"10528","messageId":"7vhdb7qown.fsf@assigned-by-dhcp.cox.net","threadId":"2206","inReplyTo":null,"subject":"[RFC] GIT paths","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2005-10-24T08:50:48Z","receivedAt":"2005-10-24T08:50:48Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Our networking commands can take either URL or non URL to\nspecify remote repository. This note first attempts to clarify\nwhat means in the current implementation, and then\ndiscusses two possible enhancements.\n\nA non URL always refers to the pack protocol going over an SSH\nconnection:\n\n\t ':' \n\n\t. path that starts with a slash '/' is absolute path on\n\t the remote site.\n\n\t. path that does not start with a slash '/' is relative\n\t to the home directory of the incoming user.\n\nHowever, note that the administrator could futz with the login\nshell of the user to give restricted access (chroot to change\nthe former). The latter can be made different from the home\ndirectory, if git-shell is changed to chdir() to somewhere else\nfirst. I am not suggesting this as a best practice -- just\nmentioning the possibility for completeness.\n\nA URL form is:\n\n\t ':' ( ':' ) '/' \n\nand is either 'git', or 'ssh' (also spelled as 'ssh+git'\nor 'git+ssh'). In addition, you can use 'http' or 'rsync', but\nthese transports are not discussed further here. They already\nhave established semantics for = '/' + .\n\nFor connections over plain TCP talking with git-daemon, or over\nSSH in this form, path is always relative to the root directory\non the remote site, because '/' that terminate either or\n starts the = '/' + .\n\nThere are two things I would like to discuss here.\n\n - It might make sense to have SERVER_ROOT (similar to\n DOCUMENT_ROOT in Apache) for git-daemon, so does not\n have to be relative to the true filesystem root. Note that\n this is not a security measure, but meant for administration\n convenience [*1*].\n\n - Over a git-daemon connection, supporting ~user expansion\n makes sense. E.g git://host.xz/~junio/ refers to my home\n directory on that machine. It would make it impossible to\n have a directory literally named '~junio' directly underneath\n the root directory, but that is a good limitation anyway.\n\nThe above enhancements, especially SERVER_ROOT, however make\npaths inconsistent between non URL form and URL form. This\nprobably is OK -- people are used to using different paths when\nuploading to HTTP server and testing a download from it. That\nleaves one issue. Do we want to support ~user expansion, and if\nso how, on non git-daemon connections?\n\nI would propose that\n\n\tgit fetch host.xz:~junio/repo\n\tgit fetch ssh://host.xz/~junio/repo\n\nmean the same thing (i.e. both understand ~user expansion).\nAlso these are equivalent (i.e. no ~user expansion; both mean\nabsolute filesystem path without SERVER_ROOT prefixing):\n\n\tgit fetch host.xz:/frotz/repo\n\tgit fetch ssh://host.xz/frotz/repo\n\nWhile these two might not mean the same thing (the former is\nprefixed with SERVER_ROOT, but not the latter):\n\n\tgit fetch git://host.xz/frotz/repo\n\tgit fetch ssh://host.xz/frotz/repo\n\nThere are small technical issues.\n\n - connect.c should not be affected at all, since it does not\n know how the remote site arranges SERVER_ROOT (if we support\n it) or user home directories.\n\n - ssh://host.xz/path and host.xz:path connections spawn\n upload-pack or receive-pack directly, without being mediated\n by git-daemon. This means that ~user expansion, if we want\n to support it, needs to be done by these programs themselves.\n\n - git-daemon needs to validate the incoming requested path and\n in order to avoid aliasing issues, we should resolve ~user\n expansion and SERVER_ROOT prefixing first, then validate the\n resulting path against white/black list, before calling\n upload-pack or receive-pack. However, after git-daemon\n decides to run these programs, they could find out some\n problems with the specified repository and may need to report\n them. Arguably, this reporting should not reveal the real\n path used to address the repository [*2*].\n\nAlthough we _could_ forget about the \"error reporting exposing\nreal path\" issue for now, I think we should at least have a plan\nto make things consistent and well defined. Here is a strawman:\n\n - Have a common library code that takes user supplied path and\n does SERVER_ROOT prefixing and ~user expansion.\n\n - Have git-daemon use it to canonicalize the requested path\n before validating. Make it invoke the programs with the path\n received from the other end (before SERVER_ROOT prefixing, or\n ~user expansion).\n\n - Give --server-root=/path/to/root flag to programs that can be\n called by git-daemon, and have git-daemon run them with this\n flag. Have them use the same library to canonicalize the\n requested path to the real path. When these programs are run\n via direct SSH connection (i.e. ssh://host/path and\n host:path), this flag is not given so they see filesystem\n path as-is, but make the ~user expansion still available.\n\n\n[Footnote]\n\n*1* You do not want to advertise your repo is at /mnt/disk1/repo\nand find out that you need to move the disks around next day.\nOf course you could plan ahead and have a symlink hanging below\nthe root directory (e.g. '/pub -> /mnt/disk1/git'), but it is so\nmuch more convenient if you can just tell git-daemon that the\nroot level used to be /mnt/disk1/git but it is now somewhere\nelse.\n\n*2* This is theoretical right now, since packed transfer\nprotocols cannot report errors back, but Andreas' patch\naddresses this issue by dying carefully in srvside_chdir(). It\nfalls into security-by-obscurity category, so we may choose not\nto worry about it, though.\n"},{"id":"10575","messageId":"435E259D.3040701@op5.se","threadId":"2206","inReplyTo":"7vhdb7qown.fsf@assigned-by-dhcp.cox.net","subject":"Re: [RFC] GIT paths","fromName":"Andreas Ericsson","fromEmail":"ae@op5.se","sentAt":"2005-10-25T12:31:25Z","receivedAt":"2005-10-25T12:31:25Z","isPatch":false,"sender":{"key":"ae@op5.se","avatar":"https://gravatar.com/avatar/426e89595c75a8f5252dd0c989e5fabe5bcac616e68557427ad9aef6b0ca342a?d=mp&s=160"},"body":"Junio C Hamano wrote:\n> Our networking commands can take either URL or non URL to\n> specify remote repository. This note first attempts to clarify\n> what means in the current implementation, and then\n> discusses two possible enhancements.\n> \n> For connections over plain TCP talking with git-daemon, or over\n> SSH in this form, path is always relative to the root directory\n> on the remote site, because '/' that terminate either or\n> starts the = '/' + .\n> \n> There are two things I would like to discuss here.\n> \n> - It might make sense to have SERVER_ROOT (similar to\n> DOCUMENT_ROOT in Apache) for git-daemon, so does not\n> have to be relative to the true filesystem root. Note that\n> this is not a security measure, but meant for administration\n> convenience [*1*].\n> \n> - Over a git-daemon connection, supporting ~user expansion\n> makes sense. E.g git://host.xz/~junio/ refers to my home\n> directory on that machine. It would make it impossible to\n> have a directory literally named '~junio' directly underneath\n> the root directory, but that is a good limitation anyway.\n> \n\nI like this idea, although I'd extend it with a Userdir-like config \noption in git-daemon (like ~/public_html for apache). This makes it a \nbit easier to see what's published and what isn't.\n\nAbout the literally named /~junio directory, it would be possible with \nthis syntax;\n\n\tgit fetch host.xz:/~junio\n\nThe userdir is (with my previous patch) only expanded if the path starts \nwith a tilde.\n\n> The above enhancements, especially SERVER_ROOT, however make\n> paths inconsistent between non URL form and URL form. This\n> probably is OK -- people are used to using different paths when\n> uploading to HTTP server and testing a download from it. That\n> leaves one issue. Do we want to support ~user expansion, and if\n> so how, on non git-daemon connections?\n> \n> I would propose that\n> \n> \tgit fetch host.xz:~junio/repo\n> \tgit fetch ssh://host.xz/~junio/repo\n> \n> mean the same thing (i.e. both understand ~user expansion).\n> Also these are equivalent (i.e. no ~user expansion; both mean\n> absolute filesystem path without SERVER_ROOT prefixing):\n> \n> \tgit fetch host.xz:/frotz/repo\n> \tgit fetch ssh://host.xz/frotz/repo\n> \n> While these two might not mean the same thing (the former is\n> prefixed with SERVER_ROOT, but not the latter):\n> \n> \tgit fetch git://host.xz/frotz/repo\n> \tgit fetch ssh://host.xz/frotz/repo\n> \n> There are small technical issues.\n> \n> - connect.c should not be affected at all, since it does not\n> know how the remote site arranges SERVER_ROOT (if we support\n> it) or user home directories.\n> \n\nIt must remove the leading slash for this syntax:\n\n\tssh://host.xz/~junio/repo\n\nOtherwise it would be passed as /~junio/repo to the remote end and no \n~user interpolation would be done.\n\n> \n> - git-daemon needs to validate the incoming requested path and\n> in order to avoid aliasing issues, we should resolve ~user\n> expansion and SERVER_ROOT prefixing first, then validate the\n> resulting path against white/black list, before calling\n> upload-pack or receive-pack. However, after git-daemon\n> decides to run these programs, they could find out some\n> problems with the specified repository and may need to report\n> them. Arguably, this reporting should not reveal the real\n> path used to address the repository [*2*].\n> \n\nThis could be done by writing the relative path in the error message;\n\n\t.git/foo/bar: failed to do something nifty\n\nThe user or the admin should know where that path is and will know what \nto do. Messages logged on the server-side should ofcourse hold the full \npath.\n\n> Although we _could_ forget about the \"error reporting exposing\n> real path\" issue for now, I think we should at least have a plan\n> to make things consistent and well defined. Here is a strawman:\n> \n> - Have a common library code that takes user supplied path and\n> does SERVER_ROOT prefixing and ~user expansion.\n> \n> - Have git-daemon use it to canonicalize the requested path\n> before validating. Make it invoke the programs with the path\n> received from the other end (before SERVER_ROOT prefixing, or\n> ~user expansion).\n> \n\nI'd say make it invoke the programs with the canonicalized path. As you \nsay, git-daemon has to verify that it's a proper git repo and in the \nwhitelist anyway so I think it would be silly to add extra complexity to \nupload-pack and receive-pack.\n\ngit-daemon could ofcourse present some uniform error message if \ngit-upload-pack or git-receive-pack fails but this wouldn't really be \nnecessary if they use relative paths as mentioned above (someone who \nmakes one of those two fail while working will already know the path).\n\n> - Give --server-root=/path/to/root flag to programs that can be\n> called by git-daemon, and have git-daemon run them with this\n> flag. Have them use the same library to canonicalize the\n> requested path to the real path. When these programs are run\n> via direct SSH connection (i.e. ssh://host/path and\n> host:path), this flag is not given so they see filesystem\n> path as-is, but make the ~user expansion still available.\n> \n\nIf we stick with canonicalized paths I suppose this can be dropped.\n\n> \n> [Footnote]\n> \n> *2* This is theoretical right now, since packed transfer\n> protocols cannot report errors back, but Andreas' patch\n> addresses this issue by dying carefully in srvside_chdir(). It\n> falls into security-by-obscurity category, so we may choose not\n> to worry about it, though.\n> \n\nKeeping valid usernames hidden is normally considered best practice.\n\n-- \nAndreas Ericsson andreas.ericsson@op5.se\nOP5 AB www.op5.se\nTel: +46 8-230225 Fax: +46 8-230231\n"},{"id":"10589","messageId":"435E62F2.6050400@zytor.com","threadId":"2206","inReplyTo":"435E259D.3040701@op5.se","subject":"Re: [RFC] GIT paths","fromName":"H. Peter Anvin","fromEmail":"hpa@zytor.com","sentAt":"2005-10-25T16:53:06Z","receivedAt":"2005-10-25T16:53:06Z","isPatch":false,"sender":{"key":"hpa@zytor.com","avatar":null},"body":"Andreas Ericsson wrote:\n>>\n>> - Over a git-daemon connection, supporting ~user expansion\n>> makes sense. E.g git://host.xz/~junio/ refers to my home\n>> directory on that machine. It would make it impossible to\n>> have a directory literally named '~junio' directly underneath\n>> the root directory, but that is a good limitation anyway.\n>>\n> \n> I like this idea, although I'd extend it with a Userdir-like config \n> option in git-daemon (like ~/public_html for apache). This makes it a \n> bit easier to see what's published and what isn't.\n> \n\nI've found that whenever one does a network daemon which exports paths, \nsooner or later one wants namespace management. In Linux, of course, \nthere are a lot more tricks one can play to actually create the \nnamespace one wants in the filesystem (although it's complicated by \nneeding to have an exec-worthy environment.)\n\nIt might be worth to consider creating a library to do this.\n\n\t-hpa\n"},{"id":"10640","messageId":"7vmzkwajrv.fsf@assigned-by-dhcp.cox.net","threadId":"2206","inReplyTo":"435E259D.3040701@op5.se","subject":"Re: [RFC] GIT paths","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2005-10-26T06:12:52Z","receivedAt":"2005-10-26T06:12:52Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Andreas Ericsson writes:\n\n> About the literally named /~junio directory, it would be possible with \n> this syntax;\n>\n> \tgit fetch host.xz:/~junio\n>\n> The userdir is (with my previous patch) only expanded if the path starts \n> with a tilde.\n\nI do not necessarily consider that a feature; see next item.\n\n> It must remove the leading slash for this syntax:\n>\n> \tssh://host.xz/~junio/repo\n>\n> Otherwise it would be passed as /~junio/repo to the remote end and no \n> ~user interpolation would be done.\n\nNot necessarily. Having the remote end interpret \"/~user\" and\n\"~user\" the same way might make things more consistent; in other\nwords, \"http://host/~user\" is not spelled \"http://host~user\".\n\n> I'd say make it invoke the programs with the canonicalized path. As you \n> say, git-daemon has to verify that it's a proper git repo and in the \n> whitelist anyway so I think it would be silly to add extra complexity to \n> upload-pack and receive-pack.\n\nYeah, I tend to agree here.\n\n>> - Give --server-root=/path/to/root flag to programs...\n>\n> If we stick with canonicalized paths I suppose this can be dropped.\n\nSounds good.\n"},{"id":"10648","messageId":"435F44D7.2030406@op5.se","threadId":"2206","inReplyTo":"7vmzkwajrv.fsf@assigned-by-dhcp.cox.net","subject":"Re: [RFC] GIT paths","fromName":"Andreas Ericsson","fromEmail":"ae@op5.se","sentAt":"2005-10-26T08:56:55Z","receivedAt":"2005-10-26T08:56:55Z","isPatch":false,"sender":{"key":"ae@op5.se","avatar":"https://gravatar.com/avatar/426e89595c75a8f5252dd0c989e5fabe5bcac616e68557427ad9aef6b0ca342a?d=mp&s=160"},"body":"Junio C Hamano wrote:\n> Andreas Ericsson writes:\n> \n>>\n>>The userdir is (with my previous patch) only expanded if the path starts \n>>with a tilde.\n> \n> \n> I do not necessarily consider that a feature; see next item.\n> \n> \n>>It must remove the leading slash for this syntax:\n>>\n>>\tssh://host.xz/~junio/repo\n>>\n>>Otherwise it would be passed as /~junio/repo to the remote end and no \n>>~user interpolation would be done.\n> \n> \n> Not necessarily. Having the remote end interpret \"/~user\" and\n> \"~user\" the same way might make things more consistent;\n\n\nExcept that the shell interprets ~ and /~ differently, so \"consistent\" \nwould depend on what we're consistent with.\n\nThere's also the fact that shell-scripts won't work on the remote end if \n git_connect() maintains the leading slash. This doesn't matter at \npresent, but I think it'd be better to keep all doors open. Having the \ntrivial addendum on the client side also helps keeping the server-side \nnice and simple.\n\n> in other\n> words, \"http://host/~user\" is not spelled \"http://host~user\".\n> \n\nTrue. I meant for this to be invisible to the users ofcourse, with \ngit_connect() having some snippet such as this.\n\nif(use_ssh || use_git && (*path == '/' && *(path + 1) == '~')\n\t*path++ = '\\0';\nelse\n\tcopy_path();\n\n> \n>>I'd say make it invoke the programs with the canonicalized path. As you \n>>say, git-daemon has to verify that it's a proper git repo and in the \n>>whitelist anyway so I think it would be silly to add extra complexity to \n>>upload-pack and receive-pack.\n> \n> \n> Yeah, I tend to agree here.\n> \n> \n>>> - Give --server-root=/path/to/root flag to programs...\n>>\n>>If we stick with canonicalized paths I suppose this can be dropped.\n> \n> \n> Sounds good.\n> \n> \n\nI'll get busy then.\n\n-- \nAndreas Ericsson andreas.ericsson@op5.se\nOP5 AB www.op5.se\nTel: +46 8-230225 Fax: +46 8-230231\n"}]}