{"thread":{"id":"16718","subject":"[PATCH 2/2] gitweb: support hiding projects from user-visible lists","startedAt":"2008-12-13T21:16:47Z","lastAt":"2009-01-03T18:29:16Z","messageCount":7,"participants":["Matt McCutchen","Jakub Narebski"],"isPatch":true,"patchVersion":1,"patchTotal":2},"messages":[{"id":"97814","messageId":"1229203007.31181.6.camel@mattlaptop2.local","threadId":"16718","inReplyTo":"4ba6b6c3fc183002407f322663d7ab53c1c28a91.1229202740.git.matt@mattmccutchen.net","subject":"[PATCH 2/2] gitweb: support hiding projects from user-visible lists","fromName":"Matt McCutchen","fromEmail":"matt@mattmccutchen.net","sentAt":"2008-12-13T21:16:47Z","receivedAt":"2008-12-13T21:16:47Z","isPatch":true,"sender":{"key":"matt@mattmccutchen.net","avatar":"https://avatars.githubusercontent.com/u/8885753?v=4"},"body":"Signed-off-by: Matt McCutchen <matt@mattmccutchen.net>\n---\n\nMy Web site has a single gitweb installation in which some of the\nrepositories are protected by a basic authentication login.  By virtue\nof my aforementioned setup with gitweb and pulling at the same URL, the\nlogin applies uniformly to both.  I had to include these repositories in\nthe projects_list because I use strict_export, but I want to hide them\nwhen the user views the project list.  This patch implements that\nfeature, and the previous one fixes a bug I noticed along the way.\n\nMatt\n\n gitweb/gitweb.perl |   13 +++++++++----\n 1 files changed, 9 insertions(+), 4 deletions(-)\n\ndiff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl\nindex 5357bcc..085cc60 100755\n--- a/gitweb/gitweb.perl\n+++ b/gitweb/gitweb.perl\n@@ -1144,7 +1144,7 @@ sub untabify {\n \n sub project_in_list {\n \tmy $project = shift;\n-\t# Tell git_get_projects_list to include forks.\n+\t# Tell git_get_projects_list to include forks and hidden repositories.\n \tmy @list = git_get_projects_list(undef, 1);\n \treturn @list && scalar(grep { $_->{'path'} eq $project } @list);\n }\n@@ -2174,15 +2174,18 @@ sub git_get_projects_list {\n \t\t# 'git%2Fgit.git Linus+Torvalds'\n \t\t# 'libs%2Fklibc%2Fklibc.git H.+Peter+Anvin'\n \t\t# 'linux%2Fhotplug%2Fudev.git Greg+Kroah-Hartman'\n+\t\t#\n+\t\t# 1 in the third field hides the project from user-visible lists, e.g.:\n+\t\t# 'linux%2Fembargoed-security-fixes.git John+Doe 1'\n \t\tmy %paths;\n \t\topen my ($fd), $projects_list or return;\n \tPROJECT:\n \t\twhile (my $line = <$fd>) {\n \t\t\tchomp $line;\n-\t\t\tmy ($path, $owner) = split ' ', $line;\n+\t\t\tmy ($path, $owner, $hidden) = split ' ', $line;\n \t\t\t$path = unescape($path);\n \t\t\t$owner = unescape($owner);\n-\t\t\tif (!defined $path) {\n+\t\t\tif (!defined $path || ($hidden && !$for_strict_export)) {\n \t\t\t\tnext;\n \t\t\t}\n \t\t\tif ($filter ne '') {\n@@ -2227,6 +2230,8 @@ sub git_get_projects_list {\n \treturn @list;\n }\n \n+# This is used to look up the owner of a project the user is already allowed to\n+# see, so we shouldn't omit hidden repositories.\n our $gitweb_project_owner = undef;\n sub git_get_project_list_from_file {\n \n@@ -2241,7 +2246,7 @@ sub git_get_project_list_from_file {\n \t\topen (my $fd , $projects_list);\n \t\twhile (my $line = <$fd>) {\n \t\t\tchomp $line;\n-\t\t\tmy ($pr, $ow) = split ' ', $line;\n+\t\t\tmy ($pr, $ow, $hidden) = split ' ', $line;\n \t\t\t$pr = unescape($pr);\n \t\t\t$ow = unescape($ow);\n \t\t\t$gitweb_project_owner->{$pr} = to_utf8($ow);\n-- \n1.6.1.rc2.27.gc7114\n"},{"id":"97820","messageId":"m3ljujg2eh.fsf@localhost.localdomain","threadId":"16718","inReplyTo":"1229203007.31181.6.camel@mattlaptop2.local","subject":"Re: [PATCH 2/2] gitweb: support hiding projects from user-visible lists","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2008-12-13T22:02:01Z","receivedAt":"2008-12-13T22:02:01Z","isPatch":true,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"Matt McCutchen <matt@mattmccutchen.net> writes:\n\nCommit message, please?\n\n> Signed-off-by: Matt McCutchen <matt@mattmccutchen.net>\n> ---\n> \n> My Web site has a single gitweb installation in which some of the\n> repositories are protected by a basic authentication login.  By virtue\n> of my aforementioned setup with gitweb and pulling at the same URL, the\n> login applies uniformly to both.  I had to include these repositories in\n> the projects_list because I use strict_export, but I want to hide them\n> when the user views the project list.  This patch implements that\n> feature, and the previous one fixes a bug I noticed along the way.\n> \n> Matt\n\nCannot you do this with new $export_auth_hook gitweb configuration\nvariable, added by Alexander Gavrilov in \n   dd7f5f1 (gitweb: Add a per-repository authorization hook.)\nIt is used in check_export_ok subroutine, and is is checked also when\ngetting list of project from file\n\n>From gitweb/INSTALL\n\n  - Finally, it is possible to specify an arbitrary perl subroutine that\n    will be called for each project to determine if it can be exported.\n    The subroutine receives an absolute path to the project as its only\n    parameter.\n\n    For example, if you use mod_perl to run the script, and have dumb\n    http protocol authentication configured for your repositories, you\n    can use the following hook to allow access only if the user is\n    authorized to read the files:\n\n      $export_auth_hook = sub {\n          use Apache2::SubRequest ();\n          use Apache2::Const -compile => qw(HTTP_OK);\n          my $path = \"$_[0]/HEAD\";\n          my $r    = Apache2::RequestUtil->request;\n          my $sub  = $r->lookup_file($path);\n          return $sub->filename eq $path\n              && $sub->status == Apache2::Const::HTTP_OK;\n      };\n\n\n>  gitweb/gitweb.perl |   13 +++++++++----\n>  1 files changed, 9 insertions(+), 4 deletions(-)\n\nNo documentation, in gitweb/README or gitweb/INSTALL\n\n> \n> diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl\n> index 5357bcc..085cc60 100755\n> --- a/gitweb/gitweb.perl\n> +++ b/gitweb/gitweb.perl\n> @@ -1144,7 +1144,7 @@ sub untabify {\n>  \n>  sub project_in_list {\n>  \tmy $project = shift;\n> -\t# Tell git_get_projects_list to include forks.\n> +\t# Tell git_get_projects_list to include forks and hidden repositories.\n>  \tmy @list = git_get_projects_list(undef, 1);\n>  \treturn @list && scalar(grep { $_->{'path'} eq $project } @list);\n>  }\n> @@ -2174,15 +2174,18 @@ sub git_get_projects_list {\n>  \t\t# 'git%2Fgit.git Linus+Torvalds'\n>  \t\t# 'libs%2Fklibc%2Fklibc.git H.+Peter+Anvin'\n>  \t\t# 'linux%2Fhotplug%2Fudev.git Greg+Kroah-Hartman'\n> +\t\t#\n> +\t\t# 1 in the third field hides the project from user-visible lists, e.g.:\n> +\t\t# 'linux%2Fembargoed-security-fixes.git John+Doe 1'\n\nI guess I'd rather use _last_ field, in the event we add project\ndescription to project list file format.\n\n>  \t\tmy %paths;\n>  \t\topen my ($fd), $projects_list or return;\n>  \tPROJECT:\n>  \t\twhile (my $line = <$fd>) {\n>  \t\t\tchomp $line;\n> -\t\t\tmy ($path, $owner) = split ' ', $line;\n> +\t\t\tmy ($path, $owner, $hidden) = split ' ', $line;\n>  \t\t\t$path = unescape($path);\n>  \t\t\t$owner = unescape($owner);\n> -\t\t\tif (!defined $path) {\n> +\t\t\tif (!defined $path || ($hidden && !$for_strict_export)) {\n>  \t\t\t\tnext;\n>  \t\t\t}\n>  \t\t\tif ($filter ne '') {\n> @@ -2227,6 +2230,8 @@ sub git_get_projects_list {\n>  \treturn @list;\n>  }\n>  \n> +# This is used to look up the owner of a project the user is already allowed to\n> +# see, so we shouldn't omit hidden repositories.\n>  our $gitweb_project_owner = undef;\n>  sub git_get_project_list_from_file {\n>  \n> @@ -2241,7 +2246,7 @@ sub git_get_project_list_from_file {\n>  \t\topen (my $fd , $projects_list);\n>  \t\twhile (my $line = <$fd>) {\n>  \t\t\tchomp $line;\n> -\t\t\tmy ($pr, $ow) = split ' ', $line;\n> +\t\t\tmy ($pr, $ow, $hidden) = split ' ', $line;\n>  \t\t\t$pr = unescape($pr);\n>  \t\t\t$ow = unescape($ow);\n>  \t\t\t$gitweb_project_owner->{$pr} = to_utf8($ow);\n> -- \n> 1.6.1.rc2.27.gc7114\n> \n\n-- \nJakub Narebski\nPoland\nShadeHawk on #git\n"},{"id":"97821","messageId":"m3hc57g28b.fsf@localhost.localdomain","threadId":"16718","inReplyTo":"m3ljujg2eh.fsf@localhost.localdomain","subject":"Re: [PATCH 2/2] gitweb: support hiding projects from user-visible lists","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2008-12-13T22:05:46Z","receivedAt":"2008-12-13T22:05:46Z","isPatch":true,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"\nBy the way, your message [PATCH 2/2] should be threaded, i.e. be\nresponse to [PATCH 1/2] (or to cover letter [PATCH 0/2]), to not\nmistake it with other [PATCH 2/2] patches.\n\n-- \nJakub Narebski\nPoland\nShadeHawk on #git\n"},{"id":"97840","messageId":"1229222058.2838.22.camel@mattlaptop2.local","threadId":"16718","inReplyTo":"m3hc57g28b.fsf@localhost.localdomain","subject":"Sending a threaded patch series with Evolution","fromName":"Matt McCutchen","fromEmail":"matt@mattmccutchen.net","sentAt":"2008-12-14T02:34:18Z","receivedAt":"2008-12-14T02:34:18Z","isPatch":false,"sender":{"key":"matt@mattmccutchen.net","avatar":"https://avatars.githubusercontent.com/u/8885753?v=4"},"body":"On Sat, 2008-12-13 at 14:05 -0800, Jakub Narebski wrote: \n> By the way, your message [PATCH 2/2] should be threaded, i.e. be\n> response to [PATCH 1/2] (or to cover letter [PATCH 0/2]), to not\n> mistake it with other [PATCH 2/2] patches.\n\nI'm using Evolution 2.24 as my mail client, and I have a wrapper script\n\"git draft-patch\" that runs \"git format-patch\" and loads the patches\ninto my Evolution Drafts folder so I can look them over in the composer\nand add non-commit-message text below the \"---\" if I wish before\nsending.  I did pass --thread, but the composer changed the Message-Id,\nbreaking the threading.\n\nI'll try to get this right in the future.  I can see two approaches to\ndoing so:\n\n1. Check over / edit the patches in a text editor before loading them\ninto Drafts, and then send them by moving them directly to Outbox\n(without using the composer) so that the Message-Id won't change.\n\n2. Send the first patch, then manually edit the Message-Id Evolution\nassigned it into the second patch, etc.\n\nNeither of these approaches is particularly nice.  Does anyone have a\nbetter idea?\n\n-- \nMatt\n"},{"id":"98645","messageId":"1230082831.2971.45.camel@localhost","threadId":"16718","inReplyTo":"m3ljujg2eh.fsf@localhost.localdomain","subject":"Re: [PATCH 2/2] gitweb: support hiding projects from user-visible lists","fromName":"Matt McCutchen","fromEmail":"matt@mattmccutchen.net","sentAt":"2008-12-24T01:40:31Z","receivedAt":"2008-12-24T01:40:31Z","isPatch":true,"sender":{"key":"matt@mattmccutchen.net","avatar":"https://avatars.githubusercontent.com/u/8885753?v=4"},"body":"On Sat, 2008-12-13 at 14:02 -0800, Jakub Narebski wrote:\n> Cannot you do this with new $export_auth_hook gitweb configuration\n> variable, added by Alexander Gavrilov in \n>    dd7f5f1 (gitweb: Add a per-repository authorization hook.)\n> It is used in check_export_ok subroutine, and is is checked also when\n> getting list of project from file\n> \n> >From gitweb/INSTALL\n> \n>   - Finally, it is possible to specify an arbitrary perl subroutine that\n>     will be called for each project to determine if it can be exported.\n>     The subroutine receives an absolute path to the project as its only\n>     parameter.\n> \n>     For example, if you use mod_perl to run the script, and have dumb\n>     http protocol authentication configured for your repositories, you\n>     can use the following hook to allow access only if the user is\n>     authorized to read the files:\n> \n>       $export_auth_hook = sub {\n>           use Apache2::SubRequest ();\n>           use Apache2::Const -compile => qw(HTTP_OK);\n>           my $path = \"$_[0]/HEAD\";\n>           my $r    = Apache2::RequestUtil->request;\n>           my $sub  = $r->lookup_file($path);\n>           return $sub->filename eq $path\n>               && $sub->status == Apache2::Const::HTTP_OK;\n>       };\n\n$export_auth_hook would work, and it would have the nice (but not\nessential) feature of including private projects in the list shown to\nsuitably authenticated users.  The only problem is that my Web host\ndoesn't support mod_perl.  Is there a practical way to accomplish the\nsame thing as the above example in a CGI script?  I would like to avoid\nreimplementing Apache authentication-checking functionality if at all\npossible.\n\n-- \nMatt\n"},{"id":"99195","messageId":"200901022033.18041.jnareb@gmail.com","threadId":"16718","inReplyTo":"1230082831.2971.45.camel@localhost","subject":"Re: [PATCH 2/2] gitweb: support hiding projects from user-visible lists","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2009-01-02T19:33:17Z","receivedAt":"2009-01-02T19:33:17Z","isPatch":true,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"On Wed, 2008-12-24, Matt McCutchen wrote:\n> On Sat, 2008-12-13 at 14:02 -0800, Jakub Narebski wrote:\n> >\n> > Cannot you do this with new $export_auth_hook gitweb configuration\n> > variable, added by Alexander Gavrilov in \n> >    dd7f5f1 (gitweb: Add a per-repository authorization hook.)\n> > It is used in check_export_ok subroutine, and is is checked also when\n> > getting list of project from file\n> > \n> > From gitweb/INSTALL\n[...]\n> >     For example, if you use mod_perl to run the script, and have dumb\n> >     http protocol authentication configured for your repositories, you\n> >     can use the following hook to allow access only if the user is\n> >     authorized to read the files:\n[...]\n \n> $export_auth_hook would work, and it would have the nice (but not\n> essential) feature of including private projects in the list shown to\n> suitably authenticated users.  The only problem is that my Web host\n> doesn't support mod_perl.  Is there a practical way to accomplish the\n> same thing as the above example in a CGI script?  I would like to avoid\n> reimplementing Apache authentication-checking functionality if at all\n> possible.\n\nI know it is written that the example code is for mod_perl, but I\ndon't think it is mod_perl specific; have you checked if it works\nfor you? I assume that you use Apache, and have Apache Perl bindings\ninstalled...\n\n-- \nJakub Narebski\nPoland\n"},{"id":"99263","messageId":"1231007356.3416.21.camel@localhost","threadId":"16718","inReplyTo":"200901022033.18041.jnareb@gmail.com","subject":"gitweb config with some public, some basic-authenticated repos","fromName":"Matt McCutchen","fromEmail":"matt@mattmccutchen.net","sentAt":"2009-01-03T18:29:16Z","receivedAt":"2009-01-03T18:29:16Z","isPatch":false,"sender":{"key":"matt@mattmccutchen.net","avatar":"https://avatars.githubusercontent.com/u/8885753?v=4"},"body":"This thread's topic has moved from a proposed patch to how I should\nconfigure my gitweb, so I'm updating the subject.  As a review: I have\nseveral public repos and several basic-authentication realms, each of\nwhich requires a single user and contains a single repo (some realms\nmight contain multiple repos in the future).  Each request has its\nauthorization checked by the Web server before it reaches gitweb, so my\nmain concern here is to avoid publicly disclosing the private repos'\npaths, authors, and descriptions in the main project list.\n\nOn Fri, 2009-01-02 at 20:33 +0100, Jakub Narebski wrote: \n> On Wed, 2008-12-24, Matt McCutchen wrote:\n> > On Sat, 2008-12-13 at 14:02 -0800, Jakub Narebski wrote:\n> > >\n> > > Cannot you do this with new $export_auth_hook gitweb configuration\n> > > variable, added by Alexander Gavrilov in \n> > >    dd7f5f1 (gitweb: Add a per-repository authorization hook.)\n> > > It is used in check_export_ok subroutine, and is is checked also when\n> > > getting list of project from file\n> > > \n> > > From gitweb/INSTALL\n> [...]\n> > >     For example, if you use mod_perl to run the script, and have dumb\n> > >     http protocol authentication configured for your repositories, you\n> > >     can use the following hook to allow access only if the user is\n> > >     authorized to read the files:\n> [...]\n>  \n> > $export_auth_hook would work, and it would have the nice (but not\n> > essential) feature of including private projects in the list shown to\n> > suitably authenticated users.  The only problem is that my Web host\n> > doesn't support mod_perl.  Is there a practical way to accomplish the\n> > same thing as the above example in a CGI script?  I would like to avoid\n> > reimplementing Apache authentication-checking functionality if at all\n> > possible.\n> \n> I know it is written that the example code is for mod_perl, but I\n> don't think it is mod_perl specific; have you checked if it works\n> for you? I assume that you use Apache, and have Apache Perl bindings\n> installed...\n\nI'm quite sure that the code is mod_perl specific.  CGI scripts do get\nsome information from Apache via the environment, but interaction as\nrich as executing Apache subrequests is only possible when the code is\nrunning inside Apache via mod_perl.  In fact, the Apache2::SubRequest\nand Apache2::RequestUtil modules are part of mod_perl.  To make sure I'm\nnot missing something, I tested the code on an Apache with mod_perl\nenabled but gitweb executing as a CGI, and gitweb failed with the\nfollowing message:\n\n        Can't locate object method \"request\" via package\n        \"Apache2::RequestUtil\" at gitweb_config.perl line 60.\n\nSo this approach won't work for me.\n\nBut even ignoring this problem, I'm now thinking that trying to show\nrepos from *multiple authentication realms* in the main list according\nto the user's credentials was a foolish idea.  I don't want to ask\nanonymous visitors to my main list for multiple logins they probably\ndon't have, yet I think it would be poor practice from a predictability\nstandpoint for the list to behave differently if the user volunteers\nlogin information that hasn't been requested.\n\nInstead, I will use a separate project list file for public repositories\nand for each realm, and no export_auth_hook.  This is simple and\nrequires no change to gitweb; my rewrite rule just has to tell my\ngitweb_config via an environment variable which list to use.  Comments\non this solution?\n\n(Note: I'm no longer advocating the hidden-repos feature at this time,\nbut I think I will still advocate the forks-and-strict-export bug fix\nnow that I have it written.)\n\n-- \nMatt\n"}]}