git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 04/24] gitweb/lib - Stat-based cache expiration

From
Jakub Narebski <jnareb@gmail.com>
Date
Dec 6, 2010, 23:10 UTC
Message-ID
<1291677069-6559-5-git-send-email-jnareb@gmail.com>
In-Reply-To
<1291677069-6559-1-git-send-email-jnareb@gmail.com>

Add stat-based cache expiration to file-based GitwebCache::SimpleFileCache. Contrary to the way other caching interfaces such as Cache::Cache and CHI do it, the time when cache element expires in is _global_ value associated with cache instance, and is not local property of cache entry. (Currently cache entry does not store any metadata associated with entry... which means that there is no need for serialization / marshalling / freezing and thawing.) Default expire time is -1, which means never expire.

To check if cache entry is expired, GitwebCache::SimpleFileCache compares difference between mtime (last modify time) of a cache file and current time with (global) time to expire. It is done using CHI-compatible is_valid() method.

Includes some tests checking that expiring works correctly (on the level of API) in t9503.

Inspired-by-code-by: John 'Warthog9' Hawley <warthog9@kernel.org>
Signed-off-by: Jakub Narebski <jnareb@gmail.com>
---
Compared to "Gitweb caching v7" series, checking if cache entry is
expired (based on mtime of cache file) is simpler: we reuse stat
structure from test if file even exists (via stat(_)).
 gitweb/lib/GitwebCache/SimpleFileCache.pm |   40 +++++++++++++++++++++++++++-
 t/t9503/test_cache_interface.pl           |   19 +++++++++++++
 2 files changed, 57 insertions(+), 2 deletions(-)
diff --git a/gitweb/lib/GitwebCache/SimpleFileCache.pm b/gitweb/lib/GitwebCache/SimpleFileCache.pm
index 2f26a6c..790383d 100644
--- a/gitweb/lib/GitwebCache/SimpleFileCache.pm
+++ b/gitweb/lib/GitwebCache/SimpleFileCache.pm
@@ -57,6 +57,10 @@ our $DEFAULT_NAMESPACE = '';
 #    The number of subdirectories deep to cache object item.  This should be
 #    large enough that no cache directory has more than a few hundred objects.
 #    Defaults to $DEFAULT_CACHE_DEPTH unless explicitly set.
+#  * 'default_expires_in' (Cache::Cache compatibile),
+#    'expires_in' (CHI compatibile) [seconds]
+#    The expiration time for objects place in the cache.
+#    Defaults to -1 (never expire) if not explicitly set.
 sub new {
 	my $class = shift;
 	my %opts = ref $_[0] ? %{ $_[0] } : @_;
@@ -64,7 +68,7 @@ sub new {
 	my $self = {};
 	$self = bless($self, $class);
 
-	my ($root, $depth, $ns);
+	my ($root, $depth, $ns, $expires_in);
 	if (%opts) {
 		$root =
 			$opts{'cache_root'} ||
@@ -73,14 +77,19 @@ sub new {
 			$opts{'cache_depth'} ||
 			$opts{'depth'};
 		$ns = $opts{'namespace'};
+		$expires_in =
+			$opts{'default_expires_in'} ||
+			$opts{'expires_in'};
 	}
 	$root  = $DEFAULT_CACHE_ROOT  unless defined($root);
 	$depth = $DEFAULT_CACHE_DEPTH unless defined($depth);
 	$ns    = $DEFAULT_NAMESPACE   unless defined($ns);
+	$expires_in = -1 unless defined($expires_in); # <0 means never
 
 	$self->set_root($root);
 	$self->set_depth($depth);
 	$self->set_namespace($ns);
+	$self->set_expires_in($expires_in);
 
 	return $self;
 }
@@ -92,7 +101,7 @@ sub new {
 # http://perldesignpatterns.com/perldesignpatterns.html#AccessorPattern
 
 # creates get_depth() and set_depth($depth) etc. methods
-foreach my $i (qw(depth root namespace)) {
+foreach my $i (qw(depth root namespace expires_in)) {
 	my $field = $i;
 	no strict 'refs';
 	*{"get_$field"} = sub {
@@ -287,6 +296,31 @@ sub remove {
 		or die "Couldn't remove file '$file': $!";
 }
 
+# $cache->is_valid($key)
+#
+# Returns a boolean indicating whether $key exists in the cache
+# and has not expired (global per-cache 'expires_in').
+sub is_valid {
+	my ($self, $key) = @_;
+
+	my $path = $self->path_to_key($key);
+
+	# does file exists in cache?
+	return 0 unless -f $path;
+	# get its modification time
+	my $mtime = (stat(_))[9] # _ to reuse stat structure used in -f test
+		or die "Couldn't stat file '$path': $!";
+
+	# expire time can be set to never
+	my $expires_in = $self->get_expires_in();
+	return 1 unless (defined $expires_in && $expires_in >= 0);
+
+	# is file expired?
+	my $now = time();
+
+	return (($now - $mtime) < $expires_in);
+}
+
 # Getting and setting
 
 # $cache->set($key, $data);
@@ -310,6 +344,8 @@ sub set {
 sub get {
 	my ($self, $key) = @_;
 
+	return unless $self->is_valid($key);
+
 	return $self->fetch($key);;
 }
 
diff --git a/t/t9503/test_cache_interface.pl b/t/t9503/test_cache_interface.pl
index 1a3f374..1517fb6 100755
--- a/t/t9503/test_cache_interface.pl
+++ b/t/t9503/test_cache_interface.pl
@@ -82,4 +82,23 @@ subtest 'CHI interface' => sub {
 	done_testing();
 };
 
+# Test cache expiration
+#
+subtest 'cache expiration' => sub {
+	$cache->set_expires_in(60*60*24); # set expire time to 1 day
+	cmp_ok($cache->get_expires_in(), '>', 0, '"expires in" is greater than 0');
+	is($cache->get($key), $value,            'get returns cached value (not expired in 1d)');
+
+	$cache->set_expires_in(-1); # set expire time to never expire
+	is($cache->get_expires_in(), -1,         '"expires in" is set to never (-1)');
+	is($cache->get($key), $value,            'get returns cached value (not expired)');
+
+	$cache->set_expires_in(0);
+	is($cache->get_expires_in(),  0,         '"expires in" is set to now (0)');
+	$cache->set($key, $value);
+	ok(!defined($cache->get($key)),          'cache is expired');
+
+	done_testing();
+};
+
 done_testing();
-- 
1.7.3
Previous: Jakub NarebskiNext: Jakub Narebski
Message 5 of 25 in “[PATCHv6/RFC 00/24] gitweb: Simple file based output caching”
  1. Jakub NarebskiDec 6, 2010
  2. 01/24 t/test-lib.sh: Export also GIT_BUILD_DIR in test_externalJakub Narebski, Dec 6, 2010
  3. 02/24 gitweb: Prepare for splitting gitwebJakub Narebski, Dec 6, 2010
  4. 03/24 gitweb/lib - Very simple file based cacheJakub Narebski, Dec 6, 2010
  5. 04/24 gitweb/lib - Stat-based cache expirationJakub Narebski, Dec 6, 2010
  6. 05/24 gitweb/lib - Regenerate entry if the cache file has size of 0Jakub Narebski, Dec 6, 2010
  7. 06/24 gitweb/lib - Simple output capture by redirecting STDOUTJakub Narebski, Dec 6, 2010
  8. 07/24 gitweb/lib - Cache captured output (using get/set)Jakub Narebski, Dec 6, 2010
  9. 08/24 gitweb: Add optional output cachingJakub Narebski, Dec 6, 2010
  10. 09/24 gitweb/lib - Adaptive cache expiration timeJakub Narebski, Dec 6, 2010
  11. 10/24 gitweb/lib - Use CHI compatibile (compute method) caching interfaceJakub Narebski, Dec 6, 2010
  12. 11/24 gitweb/lib - capture output directly to cache entry fileJakub Narebski, Dec 6, 2010
  13. 12/24 gitweb/lib - Use locking to avoid 'cache miss stampede' problemJakub Narebski, Dec 6, 2010
  14. 13/24 gitweb/lib - No need for File::Temp when lockingJakub Narebski, Dec 6, 2010
  15. 14/24 gitweb/lib - Serve stale data when waiting for filling cacheJakub Narebski, Dec 6, 2010
  16. 15/24 gitweb/lib - Regenerate (refresh) cache in backgroundJakub Narebski, Dec 6, 2010
  17. 16/24 gitweb: Introduce %actions_info, gathering information about actionsJakub Narebski, Dec 6, 2010
  18. 17/24 gitweb: Show appropriate "Generating..." page when regenerating cacheJakub Narebski, Dec 6, 2010
  19. 18/24 gitweb/lib - Configure running 'generating_info' when generating dataJakub Narebski, Dec 6, 2010
  20. 19/24 gitweb: Add startup delay to activity indicator for cacheJakub Narebski, Dec 6, 2010
  21. 20/24 gitweb/lib - Add support for setting error handler in cacheJakub Narebski, Dec 6, 2010
  22. 21/24 gitweb: Wrap die_error to use as error handler for caching engineJakub Narebski, Dec 6, 2010
  23. 22/24 gitweb: Support legacy options used by kernel.org caching engineJakub Narebski, Dec 6, 2010
  24. 23/24 gitweb/lib - Add clear() and size() methods to caching interfaceJakub Narebski, Dec 6, 2010
  25. 24/24 gitweb: Add beginnings of cache administration page (proof of concept)Jakub Narebski, Dec 6, 2010

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.