threads / rfc / 25718

RFC patchgitweb: selectable configurations that change with each request

Subject: [PATCH/RFC] gitweb: selectable configurations that change with each request

## tl;dr

7 messages between Nov 11, 2010 and Nov 25, 2010. Diffs are folded; open one to read it.

replies: 6people: 3as markdown or json

Jakub Narebski· Nov 11, 2010, 21:36 UTC · lore

Allow selecting whether configuration file should be (re)parsed on each request (the default, for backward compatibility with configurations that change per session, see commit 7f425db (gitweb: allow configurations that change with each request, 2010-07-30)), or whether should it be parsed only once (for performance speedup for persistent environments, though currently only FastCGI is able to make use of it, when flexibility is not important). You can also have configuration file parsed only once, but have parts of configuration (re)evaluated once per each request.

This is done by introducing $per_request_config variable: if set to code reference, this code would be run once per request, while config file would be parsed only once. For example gitolite's contrib/gitweb/gitweb.conf fragment mentioned in 7f425db could be rewritten as

  our $per_request_config = sub {
  	$ENV{GL_USER} = ($cgi && $cgi->remote_user) || "gitweb";
  };
to make use of this feature.

If $per_request_config is not a code reference, it is taken to be boolean variable, to choose between running config file for each request (flexibility), and running config file only once (performance in persistent environments).

The default value for $per_request_config is 1 (true), which means that old configuration that require to change per session (like gitolite's) will keep working.

Because there is a call to evaluate_gitweb_config() in the run() subroutine (the call in run_config() is not invoked at first request to avoid double running it), therefore evaluate_git_version() could be moved back there, and is now evaluated only once.

Signed-off-by: Jakub Narebski <jnareb@gmail.com>
---
Commit 7f425db914 says: 
    Probably in the end there should be a way to specify in the
    configuration whether a particular installation wants the speedup or
    the flexibility.  But for now it is easier to just undo the relevant
    change.
This is the way... well, at least a proposal.
 gitweb/README      |    7 +++++++
 gitweb/gitweb.perl |   23 +++++++++++++++++++++--
 2 files changed, 28 insertions(+), 2 deletions(-)
Show changes to 2 files +28 −2

gitweb/README, gitweb/gitweb.perl

diff --git a/gitweb/README b/gitweb/README
index bf3664f..6646fda 100644
--- a/gitweb/README
+++ b/gitweb/README
@@ -246,6 +246,13 @@ not include variables usually directly set during build):
    http://www.andre-simon.de due to assumptions about parameters and output).
    Useful if highlight is not installed on your webserver's PATH.
    [Default: highlight]
+ * $per_request_config
+   If set to code reference, it would be run once per each request.  You can
+   set parts of configuration that change per session, e.g. by setting it to
+     sub { $ENV{GL_USER} = $cgi->remote_user || "gitweb"; }
+   Otherwise it is treated as boolean value: if true gitweb would process
+   config file once per request, if false it would process config file only
+   once.  The default is true.
 
 Projects list file format
 ~~~~~~~~~~~~~~~~~~~~~~~~~
diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
index 7dbaf0c..c3cfefb 100755
--- a/gitweb/gitweb.perl
+++ b/gitweb/gitweb.perl
@@ -599,6 +599,14 @@ sub filter_snapshot_fmts {
 		!$known_snapshot_formats{$_}{'disabled'}} @fmts;
 }
 
+# If it is set to code reference, it is code that it is to be run once per
+# request, allowing updating configurations that change with each request,
+# while running other code in config file only once.
+#
+# Otherwise, if it is false then gitweb would process config file only once;
+# if it is true then gitweb config would be run for each request.
+our $per_request_config = 1;
+
 our ($GITWEB_CONFIG, $GITWEB_CONFIG_SYSTEM);
 sub evaluate_gitweb_config {
 	our $GITWEB_CONFIG = $ENV{'GITWEB_CONFIG'} || "++GITWEB_CONFIG++";
@@ -1068,12 +1076,18 @@ sub reset_timer {
 	our $number_of_git_cmds = 0;
 }
 
+our $first_request = 1;
 sub run_request {
 	reset_timer();
 
 	evaluate_uri();
-	evaluate_gitweb_config();
-	evaluate_git_version();
+	if ($per_request_config) {
+		if (ref($per_request_config) eq 'CODE') {
+			$per_request_config->();
+		} elsif (!$first_request) {
+			evaluate_gitweb_config();
+		}
+	}
 	check_loadavg();
 
 	# $projectroot and $projects_list might be set in gitweb config file
@@ -1126,6 +1140,10 @@ sub evaluate_argv {
 
 sub run {
 	evaluate_argv();
+	evaluate_gitweb_config();
+	evaluate_git_version();
+
+	$first_request = 1;
 
 	$pre_listen_hook->()
 		if $pre_listen_hook;
@@ -1139,6 +1157,7 @@ sub run {
 
 		$post_dispatch_hook->()
 			if $post_dispatch_hook;
+		$first_request = 0;
 
 		last REQUEST if ($is_last_request->());
 	}
Jonathan Nieder· Nov 11, 2010, 21:42 UTC · re: Jakub Narebski · lore

Re: [PATCH/RFC] gitweb: selectable configurations that change with each request

Jakub Narebski wrote:
Show 17 quoted lines
> This is done by introducing $per_request_config variable: if set to code
> reference, this code would be run once per request, while config file would
> be parsed only once.  For example gitolite's contrib/gitweb/gitweb.conf
> fragment mentioned in 7f425db could be rewritten as
> 
>   our $per_request_config = sub {
>   	$ENV{GL_USER} = ($cgi && $cgi->remote_user) || "gitweb";
>   };
> 
> to make use of this feature.
> 
> If $per_request_config is not a code reference, it is taken to be boolean
> variable, to choose between running config file for each request
> (flexibility), and running config file only once (performance in
> persistent environments).
> 
> The default value for $per_request_config is 1 (true)
Flexible, fast, backward-compatible.  Nice.
Junio C Hamano· Nov 24, 2010, 17:57 UTC · re: Jakub Narebski · lore

Re: [PATCH/RFC] gitweb: selectable configurations that change with each request

Jakub Narebski <jnareb@gmail.com> writes:
Show 10 quoted lines
> The default value for $per_request_config is 1 (true), which means that
> old configuration that require to change per session (like gitolite's)
> will keep working.
>
> Because there is a call to evaluate_gitweb_config() in the run() subroutine
> (the call in run_config() is not invoked at first request to avoid double
> running it), therefore evaluate_git_version() could be moved back there, and
> is now evaluated only once.
>
> Signed-off-by: Jakub Narebski <jnareb@gmail.com>

I like the "we can keep the old behaviour by default, but you can go fancier if you wanted to" attitude of this patch.

I however am not sure if this particular implementation is regression free. People who wanted the per-request configuration used to be able to access what evaluate_uri() did while running evaluate_gitweb_config(), which in turn means that their custom configuration could take things like my-url and path-info into account, but now for the first invocation they cannot. It probably is just the matter of moving some code around from their old custom gitweb-config to a per_request_config sub they need to write and point $per_request_config variable at, but that makes "setting 1 will keep the old behaviour" a false promise, no?

Show 40 quoted lines
> @@ -1068,12 +1076,18 @@ sub reset_timer {
>  	our $number_of_git_cmds = 0;
>  }
>  
> +our $first_request = 1;
>  sub run_request {
>  	reset_timer();
>  
>  	evaluate_uri();
> -	evaluate_gitweb_config();
> -	evaluate_git_version();
> +	if ($per_request_config) {
> +		if (ref($per_request_config) eq 'CODE') {
> +			$per_request_config->();
> +		} elsif (!$first_request) {
> +			evaluate_gitweb_config();
> +		}
> +	}
>  	check_loadavg();
>  
>  	# $projectroot and $projects_list might be set in gitweb config file
> @@ -1126,6 +1140,10 @@ sub evaluate_argv {
>  
>  sub run {
>  	evaluate_argv();
> +	evaluate_gitweb_config();
> +	evaluate_git_version();
> +
> +	$first_request = 1;
>  
>  	$pre_listen_hook->()
>  		if $pre_listen_hook;
> @@ -1139,6 +1157,7 @@ sub run {
>  
>  		$post_dispatch_hook->()
>  			if $post_dispatch_hook;
> +		$first_request = 0;
>  
>  		last REQUEST if ($is_last_request->());
>  	}
Jakub Narebski· Nov 24, 2010, 21:43 UTC · re: Junio C Hamano · lore

Re: [PATCH/RFC] gitweb: selectable configurations that change with each request

On Wed, 24 Nov 2010, Junio C Hamano wrote:
Show 15 quoted lines
> Jakub Narebski <jnareb@gmail.com> writes:
> 
> > The default value for $per_request_config is 1 (true), which means that
> > old configuration that require to change per session (like gitolite's)
> > will keep working.
> >
> > Because there is a call to evaluate_gitweb_config() in the run() subroutine
> > (the call in run_config() is not invoked at first request to avoid double
> > running it), therefore evaluate_git_version() could be moved back there, and
> > is now evaluated only once.
> >
> > Signed-off-by: Jakub Narebski <jnareb@gmail.com>
> 
> I like the "we can keep the old behaviour by default, but you can go
> fancier if you wanted to" attitude of this patch.
Thanks.
Show 9 quoted lines
> I however am not sure if this particular implementation is regression
> free.  People who wanted the per-request configuration used to be able to
> access what evaluate_uri() did while running evaluate_gitweb_config(),
> which in turn means that their custom configuration could take things like
> my-url and path-info into account, but now for the first invocation they
> cannot.  It probably is just the matter of moving some code around from
> their old custom gitweb-config to a per_request_config sub they need to
> write and point $per_request_config variable at, but that makes "setting 1
> will keep the old behaviour" a false promise, no?

Thanks for catching this; I forgot to take into account the fact that gitweb config changing with request would need request-related variables.

So instead of code flow looking like this
  evaluate_gitweb_config();
  $first_request = 1;
  while (...) {
  	if ($per_request_config) {
		if (ref($per_request_config) eq 'CODE') {
			$per_request_config->();
		} elsif (!$first_request) {
			evaluate_gitweb_config();
		}
	}
	$first_request = 0;
  }
we would move to
  $first_request = 1;
  while (...) {
  	if ($first_request) {
		evaluate_gitweb_config();
	} elsif ($per_request_config) {
		if (ref($per_request_config) eq 'CODE') {
			$per_request_config->();
		} else {
  			evaluate_gitweb_config();
  		}
	}
	$first_request = 0;
  }
I'll send new version soon.
P.S. if we could assume that nobody ever edits gitweb.cgi directly
to set $per_request_config to non-default value, we could probably
get rid of $first_request variable...
 
Show 41 quoted lines
> > @@ -1068,12 +1076,18 @@ sub reset_timer {
> >  	our $number_of_git_cmds = 0;
> >  }
> >  
> > +our $first_request = 1;
> >  sub run_request {
> >  	reset_timer();
> >  
> >  	evaluate_uri();
> > -	evaluate_gitweb_config();
> > -	evaluate_git_version();
> > +	if ($per_request_config) {
> > +		if (ref($per_request_config) eq 'CODE') {
> > +			$per_request_config->();
> > +		} elsif (!$first_request) {
> > +			evaluate_gitweb_config();
> > +		}
> > +	}
> >  	check_loadavg();
> >  
> >  	# $projectroot and $projects_list might be set in gitweb config file
> > @@ -1126,6 +1140,10 @@ sub evaluate_argv {
> >  
> >  sub run {
> >  	evaluate_argv();
> > +	evaluate_gitweb_config();
> > +	evaluate_git_version();
> > +
> > +	$first_request = 1;
> >  
> >  	$pre_listen_hook->()
> >  		if $pre_listen_hook;
> > @@ -1139,6 +1157,7 @@ sub run {
> >  
> >  		$post_dispatch_hook->()
> >  			if $post_dispatch_hook;
> > +		$first_request = 0;
> >  
> >  		last REQUEST if ($is_last_request->());
> >  	}
> 
-- 
Jakub Narebski
Poland
Jakub Narebski· Nov 25, 2010, 12:18 UTC · re: Jakub Narebski · lore

[PATCHv2] gitweb: selectable configurations that change with each request

Allow selecting whether configuration file should be (re)parsed on each request (the default, for backward compatibility with configurations that change per session, see commit 7f425db (gitweb: allow configurations that change with each request, 2010-07-30)), or whether should it be parsed only once (for performance speedup for persistent environments, though currently only FastCGI is able to make use of it, when flexibility is not important).

You can also have configuration file parsed only once, but have parts of configuration (re)evaluated once per each request.

This is done by introducing $per_request_config variable: if set to code reference, this code would be run once per request, while config file would be parsed only once. For example gitolite's contrib/gitweb/gitweb.conf fragment mentioned in 7f425db could be rewritten as

  our $per_request_config = sub {
  	$ENV{GL_USER} = ($cgi && $cgi->remote_user) || "gitweb";
  };
to make use of this feature.

If $per_request_config is not a code reference, it is taken to be boolean variable, to choose between running config file for each request (flexibility), and running config file only once (performance in persistent environments).

The default value for $per_request_config is 1 (true), which means that old configuration that require to change per session (like gitolite's) will keep working.

While at it, make it so evaluate_git_version() is run only once.
Signed-off-by: Jakub Narebski <jnareb@gmail.com>
---
On Wed, 24 Nov 2010, Jakub Narebski wrote:
> I'll send new version soon.
> 
And here it is, rebased on top of current 'master'.
 gitweb/README      |    7 +++++++
 gitweb/gitweb.perl |   23 +++++++++++++++++++++--
 2 files changed, 28 insertions(+), 2 deletions(-)
Show changes to 2 files +28 −2

gitweb/README, gitweb/gitweb.perl

diff --git a/gitweb/README b/gitweb/README
index bf3664f..6646fda 100644
--- a/gitweb/README
+++ b/gitweb/README
@@ -246,6 +246,13 @@ not include variables usually directly set during build):
    http://www.andre-simon.de due to assumptions about parameters and output).
    Useful if highlight is not installed on your webserver's PATH.
    [Default: highlight]
+ * $per_request_config
+   If set to code reference, it would be run once per each request.  You can
+   set parts of configuration that change per session, e.g. by setting it to
+     sub { $ENV{GL_USER} = $cgi->remote_user || "gitweb"; }
+   Otherwise it is treated as boolean value: if true gitweb would process
+   config file once per request, if false it would process config file only
+   once.  The default is true.
 
 Projects list file format
 ~~~~~~~~~~~~~~~~~~~~~~~~~
diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
index 7dbaf0c..ac38a92 100755
--- a/gitweb/gitweb.perl
+++ b/gitweb/gitweb.perl
@@ -599,6 +599,14 @@ sub filter_snapshot_fmts {
 		!$known_snapshot_formats{$_}{'disabled'}} @fmts;
 }
 
+# If it is set to code reference, it is code that it is to be run once per
+# request, allowing updating configurations that change with each request,
+# while running other code in config file only once.
+#
+# Otherwise, if it is false then gitweb would process config file only once;
+# if it is true then gitweb config would be run for each request.
+our $per_request_config = 1;
+
 our ($GITWEB_CONFIG, $GITWEB_CONFIG_SYSTEM);
 sub evaluate_gitweb_config {
 	our $GITWEB_CONFIG = $ENV{'GITWEB_CONFIG'} || "++GITWEB_CONFIG++";
@@ -1068,12 +1076,21 @@ sub reset_timer {
 	our $number_of_git_cmds = 0;
 }
 
+our $first_request = 1;
 sub run_request {
 	reset_timer();
 
 	evaluate_uri();
-	evaluate_gitweb_config();
-	evaluate_git_version();
+	if ($first_request) {
+		evaluate_gitweb_config();
+		evaluate_git_version();
+	} elsif ($per_request_config) {
+		if (ref($per_request_config) eq 'CODE') {
+			$per_request_config->();
+		} else {
+			evaluate_gitweb_config();
+		}
+	}
 	check_loadavg();
 
 	# $projectroot and $projects_list might be set in gitweb config file
@@ -1127,6 +1144,7 @@ sub evaluate_argv {
 sub run {
 	evaluate_argv();
 
+	$first_request = 1;
 	$pre_listen_hook->()
 		if $pre_listen_hook;
 
@@ -1139,6 +1157,7 @@ sub run {
 
 		$post_dispatch_hook->()
 			if $post_dispatch_hook;
+		$first_request = 0;
 
 		last REQUEST if ($is_last_request->());
 	}
-- 
1.7.3
Jonathan Nieder· Nov 25, 2010, 18:23 UTC · re: Jakub Narebski · lore

Re: [PATCHv2] gitweb: selectable configurations that change with each request

Jakub Narebski wrote:
> --- a/gitweb/gitweb.perl
> +++ b/gitweb/gitweb.perl
[...]
Show 21 quoted lines
> @@ -1068,12 +1076,21 @@ sub reset_timer {
>  	our $number_of_git_cmds = 0;
>  }
>  
> +our $first_request = 1;
>  sub run_request {
>  	reset_timer();
>  
>  	evaluate_uri();
> -	evaluate_gitweb_config();
> -	evaluate_git_version();
> +	if ($first_request) {
> +		evaluate_gitweb_config();
> +		evaluate_git_version();
> +	} elsif ($per_request_config) {
> +		if (ref($per_request_config) eq 'CODE') {
> +			$per_request_config->();
> +		} else {
> +			evaluate_gitweb_config();
> +		}
> +	}
Should per_request_config() be run for the first request, too?  Maybe:
	if ($first_request) {
		evaluate_gitweb_config();
		evaluate_git_version();
	}
	if ($per_request_config) {
		if (ref($per_request_config) eq 'CODE') {
			$per_request_config->();
		} elsif (!$first_request) {
			evaluate_gitweb_config();
		}
	}
Jakub Narebski· Nov 25, 2010, 18:43 UTC · re: Jonathan Nieder · lore

[PATCHv3] gitweb: selectable configurations that change with each request

Allow selecting whether configuration file should be (re)parsed on each request (the default, for backward compatibility with configurations that change per session, see commit 7f425db (gitweb: allow configurations that change with each request, 2010-07-30)), or whether should it be parsed only once (for performance speedup for persistent environments, though currently only FastCGI is able to make use of it, when flexibility is not important).

You can also have configuration file parsed only once, but have parts of configuration (re)evaluated once per each request.

This is done by introducing $per_request_config variable: if set to code reference, this code would be run once per request, while config file would be parsed only once. For example gitolite's contrib/gitweb/gitweb.conf fragment mentioned in 7f425db could be rewritten as

  our $per_request_config = sub {
  	$ENV{GL_USER} = ($cgi && $cgi->remote_user) || "gitweb";
  };
to make use of this feature.

If $per_request_config is not a code reference, it is taken to be boolean variable, to choose between running config file for each request (flexibility), and running config file only once (performance in persistent environments).

The default value for $per_request_config is 1 (true), which means that old configuration that require to change per session (like gitolite's) will keep working.

While at it, make it so evaluate_git_version() is run only once.
Signed-off-by: Jakub Narebski <jnareb@gmail.com>
---
On Thu, 25 Nov 2010, Jonathan Nieder wrote:
Show 13 quoted lines
> Should per_request_config() be run for the first request, too?  Maybe:
> 
>         if ($first_request) {
>                 evaluate_gitweb_config();
>                 evaluate_git_version();
>         }
>         if ($per_request_config) {
>                 if (ref($per_request_config) eq 'CODE') {
>                         $per_request_config->();
>                 } elsif (!$first_request) {
>                         evaluate_gitweb_config();
>                 }
>         }
You are right, thanks for pointing that out.  I'm sorry for my mistake.
 gitweb/README      |    7 +++++++
 gitweb/gitweb.perl |   24 ++++++++++++++++++++++--
 2 files changed, 29 insertions(+), 2 deletions(-)
Show changes to 2 files +29 −2

gitweb/README, gitweb/gitweb.perl

diff --git a/gitweb/README b/gitweb/README
index bf3664f..6646fda 100644
--- a/gitweb/README
+++ b/gitweb/README
@@ -246,6 +246,13 @@ not include variables usually directly set during build):
    http://www.andre-simon.de due to assumptions about parameters and output).
    Useful if highlight is not installed on your webserver's PATH.
    [Default: highlight]
+ * $per_request_config
+   If set to code reference, it would be run once per each request.  You can
+   set parts of configuration that change per session, e.g. by setting it to
+     sub { $ENV{GL_USER} = $cgi->remote_user || "gitweb"; }
+   Otherwise it is treated as boolean value: if true gitweb would process
+   config file once per request, if false it would process config file only
+   once.  The default is true.
 
 Projects list file format
 ~~~~~~~~~~~~~~~~~~~~~~~~~
diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
index 7dbaf0c..1d94718 100755
--- a/gitweb/gitweb.perl
+++ b/gitweb/gitweb.perl
@@ -599,6 +599,14 @@ sub filter_snapshot_fmts {
 		!$known_snapshot_formats{$_}{'disabled'}} @fmts;
 }
 
+# If it is set to code reference, it is code that it is to be run once per
+# request, allowing updating configurations that change with each request,
+# while running other code in config file only once.
+#
+# Otherwise, if it is false then gitweb would process config file only once;
+# if it is true then gitweb config would be run for each request.
+our $per_request_config = 1;
+
 our ($GITWEB_CONFIG, $GITWEB_CONFIG_SYSTEM);
 sub evaluate_gitweb_config {
 	our $GITWEB_CONFIG = $ENV{'GITWEB_CONFIG'} || "++GITWEB_CONFIG++";
@@ -1068,12 +1076,22 @@ sub reset_timer {
 	our $number_of_git_cmds = 0;
 }
 
+our $first_request = 1;
 sub run_request {
 	reset_timer();
 
 	evaluate_uri();
-	evaluate_gitweb_config();
-	evaluate_git_version();
+	if ($first_request) {
+		evaluate_gitweb_config();
+		evaluate_git_version();
+	}
+	if ($per_request_config) {
+		if (ref($per_request_config) eq 'CODE') {
+			$per_request_config->();
+		} elsif (!$first_request) {
+			evaluate_gitweb_config();
+		}
+	}
 	check_loadavg();
 
 	# $projectroot and $projects_list might be set in gitweb config file
@@ -1127,6 +1145,7 @@ sub evaluate_argv {
 sub run {
 	evaluate_argv();
 
+	$first_request = 1;
 	$pre_listen_hook->()
 		if $pre_listen_hook;
 
@@ -1139,6 +1158,7 @@ sub run {
 
 		$post_dispatch_hook->()
 			if $post_dispatch_hook;
+		$first_request = 0;
 
 		last REQUEST if ($is_last_request->());
 	}
-- 
1.7.3

← back to recent threads