[PATCH 07/24] gitweb/lib - Cache captured output (using get/set)

From: Jakub Narebski <jnareb@gmail.com>
Date: Dec 6, 2010, 23:10 UTC
Message-ID: <1291677069-6559-8-git-send-email-jnareb@gmail.com>
In-Reply-To: <1291677069-6559-1-git-send-email-jnareb@gmail.com>

Add GitwebCache::CacheOutput package, which contains cache_output subroutine. The cache_output gets data from cache and prints it, or captures output of provided subroutine (code reference), saves it to cache and prints it. It currently uses Cache::Cache compatibile (get, set) interface to cache. The capture_stop currently simply runs $capture->capture_stop().

Capturing output is done using GitwebCache::Capture::Simple compatibile capture (->new(), ->capture($code)) passed as one of parameters; the default solution is to use GitwebCache::Capture::Simple.

Gitweb would use cache_output to get page from cache, or to generate page and save it to cache. The capture_stop would be used in die_error subroutine, because error pages would not be cached.

It is assumed that data is saved to cache _converted_, and should therefore be read from cache and printed to STDOUT in ':raw' (binary) mode.

Add t9505/test_cache_output.pl test, run as external test in t9505-gitweb-cache. It checks that cache_output behaves correctly, namely that it saves and restores action output in cache, and that it prints generated output or cached output, depending on whether there exist data in cache. This test requires Capture::Tiny to be installed (because GitwebCache::Capture::Simple dosn't support recursive capturing).

Signed-off-by: Jakub Narebski <jnareb@gmail.com>
---
In this version capture_stop is defined in GitwebCache::CacheOutput,
instead of being re-exported from GitwebCache::Capture (which no
longer exists).  Compared to previous version cache_output now expects
$capture (selecting capturing engine) as second argument, and 
capture_stop expect $cache and $capture, while it was paramless before.
t9505 now requires Capture::Tiny, because it needs reliable nested
capture.

capture_stop is here only in the case in the future we would accept wider range of possible $capture values, e.g. \&Capture::Tiny::capture (reference to capturing subroutine).

cache_output($cache, $capture, $key, $code) is roughly equivalent to cache_fetch($action) from cache.pm file in "Gitweb caching v7" series by J.H.... but nothing in GitwebCache::CacheOutput is gitweb specific. This module is simply about caching captured output, using given caching engine, given capturing engine, given key to cache entry, and given coderef output of which is to be captured and cached, and then printing generated data or data from cache.

Similarly to J.H. patch, and differently from CGI::Cache or Plack::Middleware::Cache accessing cache is/would be quite explicit.

 gitweb/lib/GitwebCache/CacheOutput.pm              |   90 ++++++++++++++++++++
 ...-caching-interface.sh => t9505-gitweb-cache.sh} |   15 ++-
 t/t9505/test_cache_output.pl                       |   86 +++++++++++++++++++
 3 files changed, 186 insertions(+), 5 deletions(-)
 create mode 100644 gitweb/lib/GitwebCache/CacheOutput.pm
 copy t/{t9503-gitweb-caching-interface.sh => t9505-gitweb-cache.sh} (58%)
 create mode 100755 t/t9505/test_cache_output.pl

diff --git a/gitweb/lib/GitwebCache/CacheOutput.pm b/gitweb/lib/GitwebCache/CacheOutput.pm
new file mode 100644
index 0000000..458e314
--- /dev/null
+++ b/gitweb/lib/GitwebCache/CacheOutput.pm
@@ -0,0 +1,90 @@
+# gitweb - simple web interface to track changes in git repositories
+#
+# (C) 2010, Jakub Narebski <jnareb@gmail.com>
+# (C) 2006, John 'Warthog9' Hawley <warthog19@eaglescrag.net>
+#
+# This program is licensed under the GPLv2
+
+#
+# Capturing and caching (gitweb) output
+#
+
+# Capture output, save it in cache and print it, or retrieve it from
+# cache and print it.
+
+package GitwebCache::CacheOutput;
+
+use strict;
+use warnings;
+
+use Exporter qw(import);
+our @EXPORT      = qw(cache_output capture_stop);
+our %EXPORT_TAGS = (all => [ @EXPORT ]);
+
+# cache_output($cache, $capture, $key, $action_code);
+#
+# Attempts to get $key from $cache; if successful, prints the value.
+# Otherwise, calls $action_code, capture its output using $capture,
+# and use the captured output as the new value for $key in $cache,
+# then print captured output.
+#
+# It is assumed that captured data is already converted and it is
+# in ':raw' format (and thus restored in ':raw' from cache)
+
+# default capture class (engine), if none provided
+our $DEFAULT_CAPTURE_CLASS = 'GitwebCache::Capture::Simple';
+sub cache_output {
+	my ($cache, $capture, $key, $code) = @_;
+
+	$capture = setup_capture($capture);
+
+	# check if data is in the cache
+	my $data = $cache->get($key);
+
+	# capture and cache output, if there was nothing in the cache
+	if (!defined $data) {
+		$data = $capture->capture($code);
+		$cache->set($key, $data) if defined $data;
+	}
+
+	# print cached data
+	if (defined $data) {
+		binmode STDOUT, ':raw';
+		print $data;
+	}
+
+	return $data;
+}
+
+# capture_stop($cache, $capture);
+#
+# Stops capturing output; to be used in die_error, so that error pages
+# are not cached (not captured and cached).
+sub capture_stop {
+	my ($cache, $capture) = @_;
+
+	if (defined $capture) {
+		return $capture->capture_stop();
+	}
+	return;
+}
+
+# ......................................................................
+# helper subroutines
+
+# setup capture engine
+sub setup_capture {
+	my $capture = shift;
+
+	$capture ||= $DEFAULT_CAPTURE_CLASS;
+	if (!ref($capture)) {
+		eval "require $capture;" or die $@;
+		$capture = $capture->new();
+	}
+
+	return $capture;
+}
+
+1;
+__END__
+# end of package GitwebCache::CacheOutput
diff --git a/t/t9503-gitweb-caching-interface.sh b/t/t9505-gitweb-cache.sh
similarity index 58%
copy from t/t9503-gitweb-caching-interface.sh
copy to t/t9505-gitweb-cache.sh
index 819da1d..181577f 100755
--- a/t/t9503-gitweb-caching-interface.sh
+++ b/t/t9505-gitweb-cache.sh
@@ -3,10 +3,10 @@
 # Copyright (c) 2010 Jakub Narebski
 #
 
-test_description='gitweb caching interface
+test_description='gitweb cache
 
-This test checks caching interface used in gitweb caching, and caching
-infrastructure (GitwebCache::* modules).'
+This test checks GitwebCache::CacheOutput Perl module that is
+responsible for capturing and caching gitweb output.'
 
 # for now we are running only cache interface tests
 . ./test-lib.sh
@@ -22,13 +22,18 @@ fi
 	test_done
 }
 
+"$PERL_PATH" -MCapture::Tiny -e 0 >/dev/null 2>&1 || {
+	skip_all='perl module Capture::Tiny unavailable, skipping test'
+	test_done
+}
+
 # ----------------------------------------------------------------------
 
 # The external test will outputs its own plan
 test_external_has_tap=1
 
 test_external \
-	'GitwebCache::* Perl API (in gitweb/lib/)' \
-	"$PERL_PATH" "$TEST_DIRECTORY"/t9503/test_cache_interface.pl
+	'GitwebCache::CacheOutput Perl API (in gitweb/lib/)' \
+	"$PERL_PATH" "$TEST_DIRECTORY"/t9505/test_cache_output.pl
 
 test_done
diff --git a/t/t9505/test_cache_output.pl b/t/t9505/test_cache_output.pl
new file mode 100755
index 0000000..167bb36
--- /dev/null
+++ b/t/t9505/test_cache_output.pl
@@ -0,0 +1,86 @@
+#!/usr/bin/perl
+use lib (split(/:/, $ENV{GITPERLLIB}));
+
+use warnings;
+use strict;
+
+use Test::More;
+use Capture::Tiny qw(capture);
+
+# test source version
+use lib $ENV{GITWEBLIBDIR} || "$ENV{GIT_BUILD_DIR}/gitweb/lib";
+
+# ....................................................................
+
+# prototypes must be known at compile time, otherwise they do not work
+BEGIN { use_ok('GitwebCache::CacheOutput'); }
+
+require_ok('GitwebCache::SimpleFileCache');
+require_ok('GitwebCache::Capture::Simple');
+
+diag("Using lib '$INC[0]'");
+diag("Testing '$INC{'GitwebCache/CacheOutput.pm'}'");
+diag("Testing '$INC{'GitwebCache/SimpleFileCache.pm'}'");
+diag("Testing '$INC{'GitwebCache/Capture/Simple.pm'}'");
+
+
+# Test setting up $cache and $capture
+my $cache   = new_ok('GitwebCache::SimpleFileCache' => [], 'The $cache  ');
+my $capture = new_ok('GitwebCache::Capture::Simple' => [], 'The $capture');
+
+# ......................................................................
+
+# Prepare for testing cache_output
+my $key = 'Key';
+my $action_output = <<'EOF';
+# This is data to be cached and shown
+EOF
+my $cached_output = <<"EOF";
+$action_output# (version recovered from cache)
+EOF
+sub action {
+	print $action_output;
+}
+
+my $no_capture_output = <<"EOF";
+$action_output# (no capture)
+EOF
+sub no_capture {
+	capture_stop($cache, $capture);
+	print $no_capture_output;
+}
+
+# Catch output printed by cache_fetch
+# (only for 'print <sth>' and 'printf <sth>')
+sub capture_output_of_cache_output {
+	my $code = shift;
+
+	my ($stdout, $stderr) = capture {
+		cache_output($cache, $capture, $key, $code);
+	};
+	print STDERR $stderr;
+	return $stdout;
+}
+
+# clean state
+$cache->set_expires_in(-1);
+$cache->remove($key);
+my $test_data;
+
+# first time (if there is no cache) generates cache entry
+$test_data = capture_output_of_cache_output(\&action);
+is($test_data, $action_output,        'action output is printed (generated)');
+is($cache->get($key), $action_output, 'action output is saved in cache (generated)');
+
+# second time (if cache is set/valid) reads from cache
+$cache->set($key, $cached_output);
+$test_data = capture_output_of_cache_output(\&action);
+is($test_data, $cached_output,        'action output is printed (from cache)');
+
+# test using capture_stop
+$cache->remove($key);
+$test_data = capture_output_of_cache_output(\&no_capture);
+is($test_data, $no_capture_output,    'no_capture output is printed (generated)');
+ok(! $cache->get($key),               'no_capture output is not captured and not cached');
+
+done_testing();

-- 
1.7.3