git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[RFC PATCH 24/24] gitweb: Add beginnings of cache administration page (proof of concept)

From
Jakub Narebski <jnareb@gmail.com>
Date
Dec 6, 2010, 23:11 UTC
Message-ID
<1291677069-6559-25-git-send-email-jnareb@gmail.com>
In-Reply-To
<1291677069-6559-1-git-send-email-jnareb@gmail.com>

Currently cache administration page ('cache' action) shows estimated size of cache, and provides link to clearing cache.

Cache administration page is visible only on local computer; the same is true with respect to ability to clear cache. Those are bare beginnings of autorization framework.

If you can use cache administration page, you will see 'admin' link at the botom of the page to it.

Signed-off-by: Jakub Narebski <jnareb@gmail.com>
---
This is "frontend" (interface) patch for gitweb cache management.

This is very much work in progress, sent to git mailing list as a proof of concept. Would something like that (perhaps with more advanced authorization for admin) be useful?

The 'cache' page looks like this:
  Cache location      Size	 
  ------------------+---------+--------------
  cache/gitweb        14 KiB	[Clear cache]
  ------------------+---------+--------------
where '[Clear cache]' is a submit button.

Compared to previous series there is now 'admin' link that leads to cache administration page, which is present only if you have permissions to access said page.

Also for empty cache we show '-'.

Code was made slightly safer, though still I wouldn't use it in production, as there might be a chance that remote attacker could possibly clear cache (meaning I didn't prove that it couldn't happen).

In the final version this patch should probably be split into a few
smaller commits, for example:
* adding support for $opts{'-title'} to git_header_html
* action_is_cacheable subroutine
* human_readable_size (which could also be used in 'tree' view at request)
* xxx_admin_auth_ok, i.e. authorization and authentication for admin and
  write actions in gitweb
* 'cache' (i.e. cache admin) and 'clear_cache' (worker) methods
 gitweb/gitweb.perl |  138 ++++++++++++++++++++++++++++++++++++++++++++++++++--
 1 files changed, 134 insertions(+), 4 deletions(-)
diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
index 1521bf2..66e240f 100755
--- a/gitweb/gitweb.perl
+++ b/gitweb/gitweb.perl
@@ -25,6 +25,8 @@ use Encode;
 use Fcntl qw(:mode :flock);
 use File::Find qw();
 use File::Basename qw(basename);
+use POSIX; # for POSIX::ceil($x)
+
 binmode STDOUT, ':utf8';
 
 our $t0;
@@ -888,6 +890,10 @@ sub evaluate_actions_info {
 	map { $actions_info{$_}{'output_format'} = undef }
 		qw(blob_plain object);
 	$actions_info{'snapshot'}{'output_format'} = 'binary';
+
+	# specify uncacheable actions
+	map { $actions_info{$_}{'uncacheable'} = 1 }
+		qw(cache clear_cache);
 }
 
 sub action_outputs_html {
@@ -895,6 +901,11 @@ sub action_outputs_html {
 	return $actions_info{$action}{'output_format'} eq 'html';
 }
 
+sub action_is_cacheable {
+	my $action = shift;
+	return !$actions_info{$action}{'uncacheable'};
+}
+
 sub browser_is_robot {
 	return 1 if !exists $ENV{'HTTP_USER_AGENT'}; # gitweb run as script
 	if (eval { require HTTP::BrowserDetect; }) {
@@ -1245,12 +1256,13 @@ sub dispatch {
 	if (!defined($actions{$action})) {
 		die_error(400, "Unknown action");
 	}
-	if ($action !~ m/^(?:opml|project_list|project_index)$/ &&
+	if ($action !~ m/^(?:opml|project_list|project_index|cache|clear_cache)$/ &&
 	    !$project) {
 		die_error(400, "Project needed");
 	}
 
-	if ($caching_enabled) {
+	if ($caching_enabled &&
+	    action_is_cacheable($action)) {
 		# human readable key identifying gitweb output
 		my $output_key = href(-replay => 1, -full => 1, -path_info => 0);
 
@@ -1397,6 +1409,10 @@ sub configure_caching {
 		require GitwebCache::Capture::Simple;
 		$capture = GitwebCache::Capture::Simple->new();
 	}
+
+	# some actions are available only if cache is turned on
+	$actions{'cache'} = \&git_cache_admin;
+	$actions{'clear_cache'} = \&git_cache_clear;
 }
 
 run();
@@ -3760,7 +3776,7 @@ sub git_header_html {
 	my $expires = shift;
 	my %opts = @_;
 
-	my $title = get_page_title();
+	my $title = $opts{'-title'} || get_page_title();
 	my $content_type;
 	# require explicit support from the UA if we are to send the page as
 	# 'application/xhtml+xml', otherwise send it as plain old 'text/html'.
@@ -3936,10 +3952,18 @@ sub git_footer_html {
 
 	} else {
 		print $cgi->a({-href => href(project=>undef, action=>"opml"),
-		              -class => $feed_class}, "OPML") . " ";
+		              -class => $feed_class}, "OPML") . "\n";
 		print $cgi->a({-href => href(project=>undef, action=>"project_index"),
 		              -class => $feed_class}, "TXT") . "\n";
+
 	}
+
+	if ($actions{'cache'} &&
+	    cache_admin_auth_ok()) {
+		print $cgi->a({-href => href(project=>undef, action=>"cache"),
+		              -class => $feed_class}, "<i>admin</i>") . "\n";
+	}
+
 	print "</div>\n"; # class="page_footer"
 
 	# timing info doesn't make much sense with output (response) caching,
@@ -7412,3 +7436,109 @@ XML
 </opml>
 XML
 }
+
+# see Number::Bytes::Human
+sub human_readable_size {
+	my $bytes = shift || return;
+
+	my @units = ('', 'KiB', 'MiB', 'GiB', 'TiB');
+	my $block = 1024;
+
+	my $x = $bytes;
+	my $unit;
+	foreach (@units) {
+		$unit = $_, last if POSIX::ceil($x) < $block;
+		$x /= $block;
+	}
+
+	my $num;
+	if ($x < 10.0) {
+		$num = sprintf("%.1f", POSIX::ceil($x*10)/10); 
+	} else {
+		$num = sprintf("%d", POSIX::ceil($x));
+	}
+
+	return "$num $unit";
+}
+
+sub cache_admin_auth_ok {
+	if (defined $ENV{'REMOTE_ADDR'}) {
+		if (defined $ENV{'SERVER_ADDR'}) {
+			# SERVER_ADDR is not in RFC 3875
+			return $ENV{'SERVER_ADDR'} eq $ENV{'REMOTE_ADDR'};
+		} elsif ($ENV{'REMOTE_ADDR'} =~ m!^(?:127\.0\.0\.1|::1/128)$!) {
+			# localhost in IPv4 or IPv6
+			return 1;
+		}
+	} else {
+		# REMOTE_ADDR not defined, probably calling gitweb as script
+		return 1;
+	}
+
+	# restrict all but specified cases
+	return 0;
+}
+
+sub git_cache_admin {
+	$caching_enabled
+		or die_error(403, "Caching disabled");
+	cache_admin_auth_ok()
+		or die_error(403, "Cache administration not allowed");
+	$cache && ref($cache)
+		or die_error(500, "Cache is not present");
+
+	git_header_html(undef, undef,
+		-title => to_utf8($site_name) . " - Gitweb cache");
+
+	print <<'EOF_HTML';
+<table class="cache_admin">
+<tr><th>Cache location</th><th>Size</th><th>&nbsp;</th></tr>
+EOF_HTML
+	print '<tr class="light">' .
+	      '<td class="path">' . esc_path($cache->path_to_namespace()) . '</td>' .
+	      '<td>';
+	my $size;
+	if ($cache->can('size')) {
+		$size = $cache->size();
+	} elsif ($cache->can('get_size')) {
+		$size = $cache->get_size();
+	}
+	if (defined $size) {
+		print human_readable_size($size);
+	} else {
+		print '-';
+	}
+	print '</td><td>';
+	if ($cache->can('clear')) {
+		print $cgi->start_form({-method => "POST",
+		                        -action => $my_uri,
+		                        -enctype => CGI::URL_ENCODED}) .
+		      $cgi->input({-name=>"a", -value=>"clear_cache", -type=>"hidden"}) .
+		      $cgi->submit({-label => 'Clear cache'}) .
+		      $cgi->end_form();
+	}
+	print <<'EOF_HTML';
+</td></tr>
+</table>
+EOF_HTML
+
+	git_footer_html();
+}
+
+sub git_cache_clear {
+	$caching_enabled
+		or die_error(403, "Caching disabled");
+	cache_admin_auth_ok()
+		or die_error(403, "Clearing cache not allowed");
+	$cache && ref($cache)
+		or die_error(500, "Cache is not present");
+
+	if ($cgi->request_method() eq 'POST') {
+
+		$cache->clear();
+	}
+
+	#print "cleared";
+	print $cgi->redirect(-uri => href(action=>'cache', -full=>1),
+	                     -status => '303 See Other');
+}

-- 
1.7.3
Previous: Jakub Narebski
Message 25 of 25 in “[PATCHv6/RFC 00/24] gitweb: Simple file based output caching”
  1. Jakub NarebskiDec 6, 2010
  2. 01/24 t/test-lib.sh: Export also GIT_BUILD_DIR in test_externalJakub Narebski, Dec 6, 2010
  3. 02/24 gitweb: Prepare for splitting gitwebJakub Narebski, Dec 6, 2010
  4. 03/24 gitweb/lib - Very simple file based cacheJakub Narebski, Dec 6, 2010
  5. 04/24 gitweb/lib - Stat-based cache expirationJakub Narebski, Dec 6, 2010
  6. 05/24 gitweb/lib - Regenerate entry if the cache file has size of 0Jakub Narebski, Dec 6, 2010
  7. 06/24 gitweb/lib - Simple output capture by redirecting STDOUTJakub Narebski, Dec 6, 2010
  8. 07/24 gitweb/lib - Cache captured output (using get/set)Jakub Narebski, Dec 6, 2010
  9. 08/24 gitweb: Add optional output cachingJakub Narebski, Dec 6, 2010
  10. 09/24 gitweb/lib - Adaptive cache expiration timeJakub Narebski, Dec 6, 2010
  11. 10/24 gitweb/lib - Use CHI compatibile (compute method) caching interfaceJakub Narebski, Dec 6, 2010
  12. 11/24 gitweb/lib - capture output directly to cache entry fileJakub Narebski, Dec 6, 2010
  13. 12/24 gitweb/lib - Use locking to avoid 'cache miss stampede' problemJakub Narebski, Dec 6, 2010
  14. 13/24 gitweb/lib - No need for File::Temp when lockingJakub Narebski, Dec 6, 2010
  15. 14/24 gitweb/lib - Serve stale data when waiting for filling cacheJakub Narebski, Dec 6, 2010
  16. 15/24 gitweb/lib - Regenerate (refresh) cache in backgroundJakub Narebski, Dec 6, 2010
  17. 16/24 gitweb: Introduce %actions_info, gathering information about actionsJakub Narebski, Dec 6, 2010
  18. 17/24 gitweb: Show appropriate "Generating..." page when regenerating cacheJakub Narebski, Dec 6, 2010
  19. 18/24 gitweb/lib - Configure running 'generating_info' when generating dataJakub Narebski, Dec 6, 2010
  20. 19/24 gitweb: Add startup delay to activity indicator for cacheJakub Narebski, Dec 6, 2010
  21. 20/24 gitweb/lib - Add support for setting error handler in cacheJakub Narebski, Dec 6, 2010
  22. 21/24 gitweb: Wrap die_error to use as error handler for caching engineJakub Narebski, Dec 6, 2010
  23. 22/24 gitweb: Support legacy options used by kernel.org caching engineJakub Narebski, Dec 6, 2010
  24. 23/24 gitweb/lib - Add clear() and size() methods to caching interfaceJakub Narebski, Dec 6, 2010
  25. 24/24 gitweb: Add beginnings of cache administration page (proof of concept)Jakub Narebski, Dec 6, 2010

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.