git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[RFC PATCHv4 09/17] gitweb/lib - Cache captured output (using get/set)

From
Jakub Narebski <jnareb@gmail.com>
Date
Jun 14, 2010, 16:08 UTC
Message-ID
<1276531710-22945-10-git-send-email-jnareb@gmail.com>
In-Reply-To
<1276531710-22945-1-git-send-email-jnareb@gmail.com>

Add GitwebCache::CacheOutput package, which contains cache_output subroutine, and (re)exports capture_stop from GitwebCache::Capture. The cache_output gets data from cache and prints it, or captures output of provided subroutine (code reference), saves it to cache and prints it. It currently uses Cache::Cache compatibile (get, set) interface to cache. Capturing is currently (not configurable) done using GitwebCache::Capture::SelectFH introduced in previous commit, but any class derived from GitwebCache::Capture (like provided example GitwebCache::Capture::TiedCapture and GitwebCache::Capture::PerlIO) would work.

Gitweb would use cache_output to get page from cache, or to generate page and save it to cache. The capture_stop would be used in die_error subroutine, because error pages would not be cached.

It is assumed that data is saved to cache _converted_, and should therefore be read from cache and printed to STDOUT in ':raw' (binary) mode.

Add t9503/test_cache_output.pl test, run as external test in t9503-gitweb-caching. It checks that cache_output behaves correctly, namely that it saves and restores action output in cache, and that it prints generated output or cached output.

Signed-off-by: Jakub Narebski <jnareb@gmail.com>
---
 gitweb/lib/GitwebCache/CacheOutput.pm |   62 +++++++++++++++++++++++++++++++
 t/t9503-gitweb-caching.sh             |    3 +
 t/t9503/test_cache_output.pl          |   66 +++++++++++++++++++++++++++++++++
 3 files changed, 131 insertions(+), 0 deletions(-)
 create mode 100644 gitweb/lib/GitwebCache/CacheOutput.pm
 create mode 100755 t/t9503/test_cache_output.pl
diff --git a/gitweb/lib/GitwebCache/CacheOutput.pm b/gitweb/lib/GitwebCache/CacheOutput.pm
new file mode 100644
index 0000000..8195f0b
--- /dev/null
+++ b/gitweb/lib/GitwebCache/CacheOutput.pm
@@ -0,0 +1,62 @@
+# gitweb - simple web interface to track changes in git repositories
+#
+# (C) 2010, Jakub Narebski <jnareb@gmail.com>
+# (C) 2006, John 'Warthog9' Hawley <warthog19@eaglescrag.net>
+#
+# This program is licensed under the GPLv2
+
+#
+# Capturing and caching (gitweb) output
+#
+
+# Capture output, save it in cache and print it, or retrieve it from
+# cache and print it.
+
+package GitwebCache::CacheOutput;
+
+use strict;
+use warnings;
+
+use GitwebCache::SimpleFileCache;
+use GitwebCache::Capture::SelectFH qw(:all);
+
+use Exporter qw(import);
+our @EXPORT      = qw(cache_output capture_stop);
+our %EXPORT_TAGS = (all => [ @EXPORT ]);
+
+# cache_output($cache, $key, $action_code);
+#
+# Attempts to get $key from $cache; if successful, prints the value.
+# Otherwise, calls $action_code, capture its output and use
+# the captured output as the new value for $key in $cache,
+# then print captured output.
+our $CAPTURE_CLASS = 'GitwebCache::Capture::SelectFH';
+
+sub cache_output {
+	my ($cache, $key, $code) = @_;
+
+	my $data = $cache->get($key);
+
+	# capture and cache output, if there was nothing in the cache
+	if (!defined $data) {
+		my $capture = $CAPTURE_CLASS;
+		setup_capture($capture);
+
+		# do not use 'capture_block' prototype
+		$data = &capture_block($code);
+		$cache->set($key, $data) if defined $data;
+	}
+
+	# print cached data
+	if (defined $data) {
+		# select() instead of STDOUT is here for t9503 test:
+		binmode select(), ':raw';
+		print $data;
+	}
+
+	return $data;
+}
+
+1;
+__END__
+# end of package GitwebCache::CacheOutput;
diff --git a/t/t9503-gitweb-caching.sh b/t/t9503-gitweb-caching.sh
index 73b3f5a..0afcc0c 100755
--- a/t/t9503-gitweb-caching.sh
+++ b/t/t9503-gitweb-caching.sh
@@ -32,4 +32,7 @@ test_external 'GitwebCache::SimpleFileCache Perl API (in gitweb/cache.pm)' \
 test_external 'GitwebCache::Capture Perl API (in gitweb/cache.pm)' \
 	"$PERL_PATH" "$TEST_DIRECTORY"/t9503/test_capture_interface.pl
 
+test_external 'GitwebCache::CacheOutput Perl API (in gitweb/cache.pm)' \
+	"$PERL_PATH" "$TEST_DIRECTORY"/t9503/test_cache_output.pl
+
 test_done
diff --git a/t/t9503/test_cache_output.pl b/t/t9503/test_cache_output.pl
new file mode 100755
index 0000000..96eedb5
--- /dev/null
+++ b/t/t9503/test_cache_output.pl
@@ -0,0 +1,66 @@
+#!/usr/bin/perl
+use lib (split(/:/, $ENV{GITPERLLIB}));
+
+use warnings;
+use strict;
+
+use Test::More;
+
+# test source version
+use lib "$ENV{TEST_DIRECTORY}/../gitweb/lib";
+
+# ....................................................................
+
+# prototypes must be known at compile time, otherwise they do not work
+BEGIN { use_ok('GitwebCache::CacheOutput'); }
+
+# Test setting up $cache and $capture
+my $cache   = new_ok('GitwebCache::SimpleFileCache'   => [], 'The $cache  ');
+my $capture = new_ok('GitwebCache::Capture::SelectFH' => [], 'The $capture');
+
+# ......................................................................
+
+# Prepare for testing cache_output
+my $key = 'Key';
+my $action_output = <<'EOF';
+# This is data to be cached and shown
+EOF
+my $cached_output = <<"EOF";
+$action_output# (version recovered from cache)
+EOF
+sub action {
+	print $action_output;
+}
+
+# Catch output printed by cache_fetch
+# (only for 'print <sth>' and 'printf <sth>')
+sub capture_output_of_cache_output {
+	my $test_data = '';
+
+	open my $test_data_fh, '>', \$test_data;
+	my $oldfh = select($test_data_fh);
+
+	cache_output($cache, $key, \&action);
+
+	select($oldfh);
+	close $test_data_fh;
+
+	return $test_data;
+}
+
+# clean state
+$cache->set_expires_in(-1);
+$cache->remove($key);
+my $test_data;
+
+# first time (if there is no cache) generates cache entry
+$test_data = capture_output_of_cache_output();
+is($test_data, $action_output,        'action output is printed (generated)');
+is($cache->get($key), $action_output, 'action output is saved in cache (generated)');
+
+# second time (if cache is set/valid) reads from cache
+$cache->set($key, $cached_output);
+$test_data = capture_output_of_cache_output();
+is($test_data, $cached_output,        'action output is printed (from cache)');
+
+done_testing();

-- 
1.7.0.1
Previous: Jakub NarebskiNext: Jakub Narebski
Message 10 of 21 in “[RFC PATCHv4 00/17] gitweb: Simple file based output caching”
  1. Jakub NarebskiJun 14, 2010
  2. 01/17 gitweb: Return or exit after done serving requestJakub Narebski, Jun 14, 2010
  3. 02/17 gitweb: Fix typo in hash key name in %opts in git_header_htmlJakub Narebski, Jun 14, 2010
  4. 03/17 gitweb: Prepare for splitting gitwebJakub Narebski, Jun 14, 2010
  5. 04/17 gitweb/lib - Very simple file based cacheJakub Narebski, Jun 14, 2010
  6. 05/17 gitweb/lib - Stat-based cache expirationJakub Narebski, Jun 14, 2010
  7. 06/17 gitweb/lib - Benchmarking GitwebCache::SimpleFileCache (in t/9603/)Jakub Narebski, Jun 14, 2010
  8. 07/17 gitweb/lib - Simple select(FH) based output captureJakub Narebski, Jun 14, 2010
  9. 08/17 gitweb/lib - Alternate ways of capturing outputJakub Narebski, Jun 14, 2010
  10. 09/17 gitweb/lib - Cache captured output (using get/set)Jakub Narebski, Jun 14, 2010
  11. 10/17 gitweb: Add optional output caching (from gitweb/cache.pm)Jakub Narebski, Jun 14, 2010
  12. 10/18 gitweb/lib - Cache captured output (using get/set)Jakub Narebski, Jun 14, 2010
  13. 11/17 gitweb/lib - Adaptive cache expiration timeJakub Narebski, Jun 14, 2010
  14. 12/17 gitweb/lib - Use CHI compatibile (compute method) caching interfaceJakub Narebski, Jun 14, 2010
  15. 13/17 gitweb/lib - Use locking to avoid 'cache miss stampede' problemJakub Narebski, Jun 14, 2010
  16. 14/17 gitweb/lib - Serve stale data when waiting for filling cacheJakub Narebski, Jun 14, 2010
  17. 15/17 gitweb/lib - Regenerate (refresh) cache in backgroundJakub Narebski, Jun 14, 2010
  18. 16/17 gitweb: Show appropriate "Generating..." page when regenerating cacheJakub Narebski, Jun 14, 2010
  19. 17/17 gitweb: Add startup delay to activity indicator for cacheJakub Narebski, Jun 14, 2010
  20. 18/17 gitweb/lib - Add clear() and size() methods to caching interfaceJakub Narebski, Jun 24, 2010
  21. 19/17 gitweb: Add beginnings of cache administration page (WIP)Jakub Narebski, Jun 24, 2010

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.