git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[RFC PATCHv5 19/17] gitweb: Add beginnings of cache administration page

From
Jakub Narebski <jnareb@gmail.com>
Date
Oct 6, 2010, 22:02 UTC
Message-ID
<1286402526-13143-20-git-send-email-jnareb@gmail.com>
In-Reply-To
<1286402526-13143-1-git-send-email-jnareb@gmail.com>

Currently it shows estimated size of cache, and provides link to clearing cache.

Cache administration page is visible only on local computer; the same is true with respect to ability to clear cache. Those are bare beginnings of autorization framework.

Signed-off-by: Jakub Narebski <jnareb@gmail.com>
---
Differences from v4:
* action_is_cacheable() (is_cacheable() in previous version) now uses
  %actions_info hash.
* Adding actions related to gitweb cache administration is now done in
  configure_caching() subroutine.
* cache_admin_auth_ok() now returns true also for running gitweb as
  script, when REMOTE_ADDR environment variable is not define.  It
  also covers situation where non-standard (not in RFC 3875)
  SERVER_ADDR is not defined.
This is "frontend" (interface) patch for gitweb cache management.

This is very much work in progress, sent to git mailing list as a proof of concept. Would something like that (perhaps with more advanced authorization for admin) be useful?

The 'cache' page looks like this:
  Cache location      Size	 
  ------------------+---------+--------------
  cache/gitweb        14 KiB	[Clear cache]
  ------------------+---------+--------------
where '[Clear cache]' is a submit button.
 gitweb/gitweb.perl |  119 ++++++++++++++++++++++++++++++++++++++++++++++++++--
 1 files changed, 115 insertions(+), 4 deletions(-)
diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
index fee8739..c413797 100755
--- a/gitweb/gitweb.perl
+++ b/gitweb/gitweb.perl
@@ -24,6 +24,8 @@ use Encode;
 use Fcntl qw(:mode :flock);
 use File::Find qw();
 use File::Basename qw(basename);
+use POSIX;
+
 binmode STDOUT, ':utf8';
 
 our $t0;
@@ -851,6 +853,10 @@ sub evaluate_actions_info {
 	map { $actions_info{$_}{'output_format'} = undef }
 		qw(blob_plain object);
 	$actions_info{'snapshot'}{'output_format'} = 'binary';
+
+	# specify uncacheable actions
+	map { $actions_info{$_}{'uncacheable'} = 1 }
+		qw(cache clear_cache);
 }
 
 sub action_outputs_html {
@@ -858,6 +864,11 @@ sub action_outputs_html {
 	return $actions_info{$action}{'output_format'} eq 'html';
 }
 
+sub action_is_cacheable {
+	my $action = shift;
+	return !$actions_info{$action}{'uncacheable'};
+}
+
 # fill %input_params with the CGI parameters. All values except for 'opt'
 # should be single values, but opt can be an array. We should probably
 # build an array of parameters that can be multi-valued, but since for the time
@@ -1178,12 +1189,13 @@ sub dispatch {
 	if (!defined($actions{$action})) {
 		die_error(400, "Unknown action");
 	}
-	if ($action !~ m/^(?:opml|project_list|project_index)$/ &&
+	if ($action !~ m/^(?:opml|project_list|project_index|cache|clear_cache)$/ &&
 	    !$project) {
 		die_error(400, "Project needed");
 	}
 
-	if ($caching_enabled) {
+	if ($caching_enabled &&
+	    action_is_cacheable($action)) {
 		# human readable key identifying gitweb output
 		my $output_key = href(-replay => 1, -full => 1, -path_info => 0);
 
@@ -1204,11 +1216,11 @@ sub run_request {
 
 	evaluate_uri();
 	evaluate_gitweb_config();
-	evaluate_actions_info();
 	evaluate_git_version();
 	check_loadavg();
 	configure_caching()
 		if ($caching_enabled);
+	evaluate_actions_info();
 
 	# $projectroot and $projects_list might be set in gitweb config file
 	$projects_list ||= $projectroot;
@@ -1311,6 +1323,10 @@ sub configure_caching {
 			'depth' => $cache_options{'cache_depth'},
 		});
 	}
+
+	# some actions are available only if cache is turned on
+	$actions{'cache'} = \&git_cache_admin;
+	$actions{'clear_cache'} = \&git_cache_clear;
 }
 
 run();
@@ -3673,7 +3689,7 @@ sub git_header_html {
 	my $expires = shift;
 	my %opts = @_;
 
-	my $title = get_page_title();
+	my $title = $opts{'-title'} || get_page_title();
 	my $content_type;
 	# require explicit support from the UA if we are to send the page as
 	# 'application/xhtml+xml', otherwise send it as plain old 'text/html'.
@@ -7325,3 +7341,98 @@ XML
 </opml>
 XML
 }
+
+# see Number::Bytes::Human
+sub human_readable_size {
+	my $bytes = shift || return;
+
+	my @units = ('', 'KiB', 'MiB', 'GiB', 'TiB');
+	my $block = 1024;
+
+	my $x = $bytes;
+	my $unit;
+	foreach (@units) {
+		$unit = $_, last if POSIX::ceil($x) < $block;
+		$x /= $block;
+	}
+
+	my $num;
+	if ($x < 10.0) {
+		$num = sprintf("%.1f", POSIX::ceil($x*10)/10); 
+	} else {
+		$num = sprintf("%d", POSIX::ceil($x));
+	}
+
+	return "$num $unit";
+}
+
+sub cache_admin_auth_ok {
+	if (defined $ENV{'REMOTE_ADDR'}) {
+		if (defined $ENV{'SERVER_ADDR'}) {
+			# SERVER_ADDR is not in RFC 3875
+			return $ENV{'SERVER_ADDR'} eq $ENV{'REMOTE_ADDR'};
+		} elsif ($ENV{'REMOTE_ADDR'} =~ m!^(?:127\.0\.0\.1|::1/128)$!) {
+			# localhost in IPv4 or IPv6
+			return 1;
+		}
+	} else {
+		# REMOTE_ADDR not defined, probably calling gitweb as script
+		return 1;
+	}
+
+	# restrict all but specified cases
+	return 0;
+}
+
+sub git_cache_admin {
+	$caching_enabled
+		or die_error(403, "Caching disabled");
+	cache_admin_auth_ok()
+		or die_error(403, "Cache administration not allowed");
+	$cache && ref($cache)
+		or die_error(500, "Cache is not present");
+
+	git_header_html(undef, undef,
+		-title => to_utf8($site_name) . " - Gitweb cache");
+
+	print <<'EOF_HTML';
+<table class="cache_admin">
+<tr><th>Cache location</th><th>Size</th><th>&nbsp;</th></tr>
+EOF_HTML
+	print '<tr class="light">' .
+	      '<td class="path">' . esc_path($cache->path_to_namespace()) . '</td>' .
+	      '<td>';
+	my $size;
+	$size = $cache->size()     if (!defined $size && $cache->can('size'));
+	$size = $cache->get_size() if (!defined $size && $cache->can('get_size'));
+	if (defined $size) {
+		print human_readable_size($size);
+	}
+	print '</td><td>';
+	if ($cache->can('clear')) {
+		print $cgi->start_form({-method => "POST",
+		                        -action => $my_uri,
+		                        -enctype => CGI::URL_ENCODED}) .
+		      $cgi->input({-name=>"a", -value=>"clear_cache", -type=>"hidden"}) .
+		      $cgi->submit({-label => 'Clear cache'}) .
+		      $cgi->end_form();
+	}
+	print <<'EOF_HTML';
+</td></tr>
+</table>
+EOF_HTML
+
+	git_footer_html();
+}
+
+sub git_cache_clear {
+	if (cache_admin_auth_ok() && $cache &&
+	    $cgi->request_method() eq 'POST') {
+
+		$cache->clear();
+	}
+
+	#print "cleared";
+	print $cgi->redirect(-uri => href(action=>'cache', -full=>1),
+	                     -status => '303 See Other');
+}

-- 
1.7.3
Previous: Thomas AdamNext: Jakub Narebski
Message 37 of 41 in “[PATCHv5 00/17] gitweb: Simple file based output caching”
  1. Jakub NarebskiOct 6, 2010
  2. 01/17 t/test-lib.sh: Export also GIT_BUILD_DIR in test_externalJakub Narebski, Oct 6, 2010
  3. 02/17 gitweb: Prepare for splitting gitwebJakub Narebski, Oct 6, 2010
  4. 03/17 gitweb/lib - Very simple file based cacheJakub Narebski, Oct 6, 2010
  5. Thomas AdamOct 6, 2010
  6. Ævar Arnfjörð BjarmasonOct 6, 2010
  7. Thomas AdamOct 6, 2010
  8. Ævar Arnfjörð BjarmasonOct 6, 2010
  9. Jakub NarebskiOct 6, 2010
  10. Thomas AdamOct 6, 2010
  11. Jakub NarebskiOct 6, 2010
  12. Ævar Arnfjörð BjarmasonOct 6, 2010
  13. Jakub NarebskiOct 6, 2010
  14. 04/17 gitweb/lib - Stat-based cache expirationJakub Narebski, Oct 6, 2010
  15. 05/17 gitweb/lib - Regenerate entry if the cache file has size of 0Jakub Narebski, Oct 6, 2010
  16. 06/17 gitweb/lib - Simple select(FH) based output captureJakub Narebski, Oct 6, 2010
  17. Thomas AdamOct 6, 2010
  18. Jakub NarebskiOct 6, 2010
  19. Ævar Arnfjörð BjarmasonOct 6, 2010
  20. Jakub NarebskiOct 6, 2010
  21. 07/17 gitweb/lib - Cache captured output (using get/set)Jakub Narebski, Oct 6, 2010
  22. 08/17 gitweb: Add optional output cachingJakub Narebski, Oct 6, 2010
  23. Ævar Arnfjörð BjarmasonOct 6, 2010
  24. Jakub NarebskiOct 6, 2010
  25. Ævar Arnfjörð BjarmasonOct 6, 2010
  26. 09/17 gitweb/lib - Adaptive cache expiration timeJakub Narebski, Oct 6, 2010
  27. 10/21 gitweb/lib - Use CHI compatibile (compute method) caching interfaceJakub Narebski, Oct 6, 2010
  28. 11/17 gitweb/lib - Use locking to avoid 'cache miss stampede' problemJakub Narebski, Oct 6, 2010
  29. 12/17 gitweb/lib - No need for File::Temp when lockingJakub Narebski, Oct 6, 2010
  30. 13/17 gitweb/lib - Serve stale data when waiting for filling cacheJakub Narebski, Oct 6, 2010
  31. 14/17 gitweb/lib - Regenerate (refresh) cache in backgroundJakub Narebski, Oct 6, 2010
  32. 15/17 gitweb: Introduce %actions_info, gathering information about actionsJakub Narebski, Oct 6, 2010
  33. 16/17 gitweb: Show appropriate "Generating..." page when regenerating cacheJakub Narebski, Oct 6, 2010
  34. 17/17 gitweb: Add startup delay to activity indicator for cacheJakub Narebski, Oct 6, 2010
  35. 18/17 gitweb/lib - Add clear() and size() methods to caching interfaceJakub Narebski, Oct 6, 2010
  36. Thomas AdamOct 6, 2010
  37. 19/17 gitweb: Add beginnings of cache administration pageJakub Narebski, Oct 6, 2010
  38. 20/17 gitweb/lib - Benchmarking GitwebCache::SimpleFileCache (in t/9603/)Jakub Narebski, Oct 6, 2010
  39. 21/17 gitweb/lib - Alternate ways of capturing outputJakub Narebski, Oct 6, 2010
  40. [RFD] Possible improvements for output caching in gitwebJakub Narebski, Oct 10, 2010
  41. J.H.Oct 24, 2010

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.