threads / patch / 16718

patch, 2 partsgitweb: support hiding projects from user-visible lists

Subject: [PATCH 2/2] gitweb: support hiding projects from user-visible lists

## tl;dr

7 messages between Dec 13, 2008 and Jan 3, 2009. Diffs are folded; open one to read it.

replies: 6people: 2as markdown or json

Matt McCutchen· Dec 13, 2008, 21:16 UTC · lore
Signed-off-by: Matt McCutchen <matt@mattmccutchen.net>
---

My Web site has a single gitweb installation in which some of the repositories are protected by a basic authentication login. By virtue of my aforementioned setup with gitweb and pulling at the same URL, the login applies uniformly to both. I had to include these repositories in the projects_list because I use strict_export, but I want to hide them when the user views the project list. This patch implements that feature, and the previous one fixes a bug I noticed along the way.

Matt
 gitweb/gitweb.perl |   13 +++++++++----
 1 files changed, 9 insertions(+), 4 deletions(-)
Show changes to gitweb/gitweb.perl +9 −4
diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
index 5357bcc..085cc60 100755
--- a/gitweb/gitweb.perl
+++ b/gitweb/gitweb.perl
@@ -1144,7 +1144,7 @@ sub untabify {
 
 sub project_in_list {
 	my $project = shift;
-	# Tell git_get_projects_list to include forks.
+	# Tell git_get_projects_list to include forks and hidden repositories.
 	my @list = git_get_projects_list(undef, 1);
 	return @list && scalar(grep { $_->{'path'} eq $project } @list);
 }
@@ -2174,15 +2174,18 @@ sub git_get_projects_list {
 		# 'git%2Fgit.git Linus+Torvalds'
 		# 'libs%2Fklibc%2Fklibc.git H.+Peter+Anvin'
 		# 'linux%2Fhotplug%2Fudev.git Greg+Kroah-Hartman'
+		#
+		# 1 in the third field hides the project from user-visible lists, e.g.:
+		# 'linux%2Fembargoed-security-fixes.git John+Doe 1'
 		my %paths;
 		open my ($fd), $projects_list or return;
 	PROJECT:
 		while (my $line = <$fd>) {
 			chomp $line;
-			my ($path, $owner) = split ' ', $line;
+			my ($path, $owner, $hidden) = split ' ', $line;
 			$path = unescape($path);
 			$owner = unescape($owner);
-			if (!defined $path) {
+			if (!defined $path || ($hidden && !$for_strict_export)) {
 				next;
 			}
 			if ($filter ne '') {
@@ -2227,6 +2230,8 @@ sub git_get_projects_list {
 	return @list;
 }
 
+# This is used to look up the owner of a project the user is already allowed to
+# see, so we shouldn't omit hidden repositories.
 our $gitweb_project_owner = undef;
 sub git_get_project_list_from_file {
 
@@ -2241,7 +2246,7 @@ sub git_get_project_list_from_file {
 		open (my $fd , $projects_list);
 		while (my $line = <$fd>) {
 			chomp $line;
-			my ($pr, $ow) = split ' ', $line;
+			my ($pr, $ow, $hidden) = split ' ', $line;
 			$pr = unescape($pr);
 			$ow = unescape($ow);
 			$gitweb_project_owner->{$pr} = to_utf8($ow);

-- 
1.6.1.rc2.27.gc7114
Jakub Narebski· Dec 13, 2008, 22:02 UTC · re: Matt McCutchen · lore

Re: [PATCH 2/2] gitweb: support hiding projects from user-visible lists

Matt McCutchen <matt@mattmccutchen.net> writes:
Commit message, please?
Show 12 quoted lines
> Signed-off-by: Matt McCutchen <matt@mattmccutchen.net>
> ---
> 
> My Web site has a single gitweb installation in which some of the
> repositories are protected by a basic authentication login.  By virtue
> of my aforementioned setup with gitweb and pulling at the same URL, the
> login applies uniformly to both.  I had to include these repositories in
> the projects_list because I use strict_export, but I want to hide them
> when the user views the project list.  This patch implements that
> feature, and the previous one fixes a bug I noticed along the way.
> 
> Matt
Cannot you do this with new $export_auth_hook gitweb configuration
variable, added by Alexander Gavrilov in 
   dd7f5f1 (gitweb: Add a per-repository authorization hook.)
It is used in check_export_ok subroutine, and is is checked also when
getting list of project from file
>From gitweb/INSTALL
  - Finally, it is possible to specify an arbitrary perl subroutine that
    will be called for each project to determine if it can be exported.
    The subroutine receives an absolute path to the project as its only
    parameter.
    For example, if you use mod_perl to run the script, and have dumb
    http protocol authentication configured for your repositories, you
    can use the following hook to allow access only if the user is
    authorized to read the files:
      $export_auth_hook = sub {
          use Apache2::SubRequest ();
          use Apache2::Const -compile => qw(HTTP_OK);
          my $path = "$_[0]/HEAD";
          my $r    = Apache2::RequestUtil->request;
          my $sub  = $r->lookup_file($path);
          return $sub->filename eq $path
              && $sub->status == Apache2::Const::HTTP_OK;
      };
>  gitweb/gitweb.perl |   13 +++++++++----
>  1 files changed, 9 insertions(+), 4 deletions(-)
No documentation, in gitweb/README or gitweb/INSTALL
Show 21 quoted lines
> 
> diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
> index 5357bcc..085cc60 100755
> --- a/gitweb/gitweb.perl
> +++ b/gitweb/gitweb.perl
> @@ -1144,7 +1144,7 @@ sub untabify {
>  
>  sub project_in_list {
>  	my $project = shift;
> -	# Tell git_get_projects_list to include forks.
> +	# Tell git_get_projects_list to include forks and hidden repositories.
>  	my @list = git_get_projects_list(undef, 1);
>  	return @list && scalar(grep { $_->{'path'} eq $project } @list);
>  }
> @@ -2174,15 +2174,18 @@ sub git_get_projects_list {
>  		# 'git%2Fgit.git Linus+Torvalds'
>  		# 'libs%2Fklibc%2Fklibc.git H.+Peter+Anvin'
>  		# 'linux%2Fhotplug%2Fudev.git Greg+Kroah-Hartman'
> +		#
> +		# 1 in the third field hides the project from user-visible lists, e.g.:
> +		# 'linux%2Fembargoed-security-fixes.git John+Doe 1'

I guess I'd rather use _last_ field, in the event we add project description to project list file format.

Show 35 quoted lines
>  		my %paths;
>  		open my ($fd), $projects_list or return;
>  	PROJECT:
>  		while (my $line = <$fd>) {
>  			chomp $line;
> -			my ($path, $owner) = split ' ', $line;
> +			my ($path, $owner, $hidden) = split ' ', $line;
>  			$path = unescape($path);
>  			$owner = unescape($owner);
> -			if (!defined $path) {
> +			if (!defined $path || ($hidden && !$for_strict_export)) {
>  				next;
>  			}
>  			if ($filter ne '') {
> @@ -2227,6 +2230,8 @@ sub git_get_projects_list {
>  	return @list;
>  }
>  
> +# This is used to look up the owner of a project the user is already allowed to
> +# see, so we shouldn't omit hidden repositories.
>  our $gitweb_project_owner = undef;
>  sub git_get_project_list_from_file {
>  
> @@ -2241,7 +2246,7 @@ sub git_get_project_list_from_file {
>  		open (my $fd , $projects_list);
>  		while (my $line = <$fd>) {
>  			chomp $line;
> -			my ($pr, $ow) = split ' ', $line;
> +			my ($pr, $ow, $hidden) = split ' ', $line;
>  			$pr = unescape($pr);
>  			$ow = unescape($ow);
>  			$gitweb_project_owner->{$pr} = to_utf8($ow);
> -- 
> 1.6.1.rc2.27.gc7114
> 
-- 
Jakub Narebski
Poland
ShadeHawk on #git
Jakub Narebski· Dec 13, 2008, 22:05 UTC · re: Jakub Narebski · lore

Re: [PATCH 2/2] gitweb: support hiding projects from user-visible lists

By the way, your message [PATCH 2/2] should be threaded, i.e. be response to [PATCH 1/2] (or to cover letter [PATCH 0/2]), to not mistake it with other [PATCH 2/2] patches.

-- 
Jakub Narebski
Poland
ShadeHawk on #git
Matt McCutchen· Dec 14, 2008, 02:34 UTC · re: Jakub Narebski · lore

Sending a threaded patch series with Evolution

On Sat, 2008-12-13 at 14:05 -0800, Jakub Narebski wrote: 
> By the way, your message [PATCH 2/2] should be threaded, i.e. be
> response to [PATCH 1/2] (or to cover letter [PATCH 0/2]), to not
> mistake it with other [PATCH 2/2] patches.

I'm using Evolution 2.24 as my mail client, and I have a wrapper script "git draft-patch" that runs "git format-patch" and loads the patches into my Evolution Drafts folder so I can look them over in the composer and add non-commit-message text below the "---" if I wish before sending. I did pass --thread, but the composer changed the Message-Id, breaking the threading.

I'll try to get this right in the future. I can see two approaches to doing so:

1. Check over / edit the patches in a text editor before loading them
into Drafts, and then send them by moving them directly to Outbox
(without using the composer) so that the Message-Id won't change.
2. Send the first patch, then manually edit the Message-Id Evolution
assigned it into the second patch, etc.

Neither of these approaches is particularly nice. Does anyone have a better idea?

-- 
Matt
Matt McCutchen· Dec 24, 2008, 01:40 UTC · re: Jakub Narebski · lore

Re: [PATCH 2/2] gitweb: support hiding projects from user-visible lists

On Sat, 2008-12-13 at 14:02 -0800, Jakub Narebski wrote:
Show 27 quoted lines
> Cannot you do this with new $export_auth_hook gitweb configuration
> variable, added by Alexander Gavrilov in 
>    dd7f5f1 (gitweb: Add a per-repository authorization hook.)
> It is used in check_export_ok subroutine, and is is checked also when
> getting list of project from file
> 
> >From gitweb/INSTALL
> 
>   - Finally, it is possible to specify an arbitrary perl subroutine that
>     will be called for each project to determine if it can be exported.
>     The subroutine receives an absolute path to the project as its only
>     parameter.
> 
>     For example, if you use mod_perl to run the script, and have dumb
>     http protocol authentication configured for your repositories, you
>     can use the following hook to allow access only if the user is
>     authorized to read the files:
> 
>       $export_auth_hook = sub {
>           use Apache2::SubRequest ();
>           use Apache2::Const -compile => qw(HTTP_OK);
>           my $path = "$_[0]/HEAD";
>           my $r    = Apache2::RequestUtil->request;
>           my $sub  = $r->lookup_file($path);
>           return $sub->filename eq $path
>               && $sub->status == Apache2::Const::HTTP_OK;
>       };

$export_auth_hook would work, and it would have the nice (but not essential) feature of including private projects in the list shown to suitably authenticated users. The only problem is that my Web host doesn't support mod_perl. Is there a practical way to accomplish the same thing as the above example in a CGI script? I would like to avoid reimplementing Apache authentication-checking functionality if at all possible.

-- 
Matt
Jakub Narebski· Jan 2, 2009, 19:33 UTC · re: Matt McCutchen · lore

Re: [PATCH 2/2] gitweb: support hiding projects from user-visible lists

On Wed, 2008-12-24, Matt McCutchen wrote:
Show 9 quoted lines
> On Sat, 2008-12-13 at 14:02 -0800, Jakub Narebski wrote:
> >
> > Cannot you do this with new $export_auth_hook gitweb configuration
> > variable, added by Alexander Gavrilov in 
> >    dd7f5f1 (gitweb: Add a per-repository authorization hook.)
> > It is used in check_export_ok subroutine, and is is checked also when
> > getting list of project from file
> > 
> > From gitweb/INSTALL
[...]
> >     For example, if you use mod_perl to run the script, and have dumb
> >     http protocol authentication configured for your repositories, you
> >     can use the following hook to allow access only if the user is
> >     authorized to read the files:
[...]
Show 7 quoted lines
> $export_auth_hook would work, and it would have the nice (but not
> essential) feature of including private projects in the list shown to
> suitably authenticated users.  The only problem is that my Web host
> doesn't support mod_perl.  Is there a practical way to accomplish the
> same thing as the above example in a CGI script?  I would like to avoid
> reimplementing Apache authentication-checking functionality if at all
> possible.

I know it is written that the example code is for mod_perl, but I don't think it is mod_perl specific; have you checked if it works for you? I assume that you use Apache, and have Apache Perl bindings installed...

-- 
Jakub Narebski
Poland
Matt McCutchen· Jan 3, 2009, 18:29 UTC · re: Jakub Narebski · lore

gitweb config with some public, some basic-authenticated repos

This thread's topic has moved from a proposed patch to how I should configure my gitweb, so I'm updating the subject. As a review: I have several public repos and several basic-authentication realms, each of which requires a single user and contains a single repo (some realms might contain multiple repos in the future). Each request has its authorization checked by the Web server before it reaches gitweb, so my main concern here is to avoid publicly disclosing the private repos' paths, authors, and descriptions in the main project list.

On Fri, 2009-01-02 at 20:33 +0100, Jakub Narebski wrote:
Show 29 quoted lines
> On Wed, 2008-12-24, Matt McCutchen wrote:
> > On Sat, 2008-12-13 at 14:02 -0800, Jakub Narebski wrote:
> > >
> > > Cannot you do this with new $export_auth_hook gitweb configuration
> > > variable, added by Alexander Gavrilov in 
> > >    dd7f5f1 (gitweb: Add a per-repository authorization hook.)
> > > It is used in check_export_ok subroutine, and is is checked also when
> > > getting list of project from file
> > > 
> > > From gitweb/INSTALL
> [...]
> > >     For example, if you use mod_perl to run the script, and have dumb
> > >     http protocol authentication configured for your repositories, you
> > >     can use the following hook to allow access only if the user is
> > >     authorized to read the files:
> [...]
>  
> > $export_auth_hook would work, and it would have the nice (but not
> > essential) feature of including private projects in the list shown to
> > suitably authenticated users.  The only problem is that my Web host
> > doesn't support mod_perl.  Is there a practical way to accomplish the
> > same thing as the above example in a CGI script?  I would like to avoid
> > reimplementing Apache authentication-checking functionality if at all
> > possible.
> 
> I know it is written that the example code is for mod_perl, but I
> don't think it is mod_perl specific; have you checked if it works
> for you? I assume that you use Apache, and have Apache Perl bindings
> installed...

I'm quite sure that the code is mod_perl specific. CGI scripts do get some information from Apache via the environment, but interaction as rich as executing Apache subrequests is only possible when the code is running inside Apache via mod_perl. In fact, the Apache2::SubRequest and Apache2::RequestUtil modules are part of mod_perl. To make sure I'm not missing something, I tested the code on an Apache with mod_perl enabled but gitweb executing as a CGI, and gitweb failed with the following message:

        Can't locate object method "request" via package
        "Apache2::RequestUtil" at gitweb_config.perl line 60.
So this approach won't work for me.

But even ignoring this problem, I'm now thinking that trying to show repos from *multiple authentication realms* in the main list according to the user's credentials was a foolish idea. I don't want to ask anonymous visitors to my main list for multiple logins they probably don't have, yet I think it would be poor practice from a predictability standpoint for the list to behave differently if the user volunteers login information that hasn't been requested.

Instead, I will use a separate project list file for public repositories and for each realm, and no export_auth_hook. This is simple and requires no change to gitweb; my rewrite rule just has to tell my gitweb_config via an environment variable which list to use. Comments on this solution?

(Note: I'm no longer advocating the hidden-repos feature at this time, but I think I will still advocate the forks-and-strict-export bug fix now that I have it written.)

-- 
Matt

← back to recent threads