git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [RFC/PATCHv5 18/17] gitweb/lib - Add clear() and size() methods to caching interface

From
Thomas Adam <thomas@xteddy.org>
Date
Oct 6, 2010, 22:56 UTC
Message-ID
<AANLkTimpXJT-R5vgdS4SvpuvhVOL-HCJxELMbfRcuqx5@mail.gmail.com>
In-Reply-To
<1286402526-13143-19-git-send-email-jnareb@gmail.com>
On 6 October 2010 23:02, Jakub Narebski <jnareb@gmail.com> wrote:
Show 107 quoted lines
> Add ->size() method, which following Cache::Cache interface returns
> estimated total size of all entries in whole cache (in the namsepace
> assiciated with give cache instance).  Note that ->get_size($key)
> returns size of a single entry!
>
> Add ->clear() method, which removes all entries from the namespace
> associated with given cache instance.  For safety it requires
> namespace to be set to true value, which means that it cannot be
> empty; therefore default namespace is changed to 'gitweb'.
>
> The ->clear() method should be fairly safe, because it first renames
> directory (which should be atomic), and only then removes it
> (following code from CGI::Driver::File).
>
> Signed-off-by: Jakub Narebski <jnareb@gmail.com>
> ---
> The only difference from previous version is that a comment about some
> method was moved to earlier commit that introduced it, where it
> belongs.
>
> This patch (implementing "backend") together with next patch
> (implementing "interface") are meant to provide nice web interface to
> safely cleaning (pruning) cache, as described in comments and
> documentation (how to do this manually).
>
>  gitweb/lib/GitwebCache/SimpleFileCache.pm |   48 ++++++++++++++++++++++++++--
>  t/t9503/test_cache_interface.pl           |    1 -
>  2 files changed, 44 insertions(+), 5 deletions(-)
>
> diff --git a/gitweb/lib/GitwebCache/SimpleFileCache.pm b/gitweb/lib/GitwebCache/SimpleFileCache.pm
> index 2a3d9cf..8f172cb 100644
> --- a/gitweb/lib/GitwebCache/SimpleFileCache.pm
> +++ b/gitweb/lib/GitwebCache/SimpleFileCache.pm
> @@ -20,8 +20,9 @@ package GitwebCache::SimpleFileCache;
>  use strict;
>  use warnings;
>
> -use File::Path qw(mkpath);
> -use File::Temp qw(tempfile);
> +use File::Path qw(mkpath rmtree);
> +use File::Temp qw(tempfile mktemp);
> +use File::Find qw(find);
>  use Digest::MD5 qw(md5_hex);
>
>  # by default, the cache nests all entries on the filesystem single
> @@ -37,7 +38,7 @@ our $DEFAULT_CACHE_ROOT = "cache";
>  # by default we don't use cache namespace (empty namespace);
>  # empty namespace does not allow for simple implementation of clear() method.
>  #
> -our $DEFAULT_NAMESPACE = '';
> +our $DEFAULT_NAMESPACE = "gitweb";
>
>  # ......................................................................
>  # constructor
> @@ -334,7 +335,7 @@ sub get_size {
>  }
>
>  # ......................................................................
> -# interface methods
> +# interface methods dealing with single item
>
>  # Removing and expiring
>
> @@ -427,6 +428,45 @@ sub compute {
>        return $data;
>  }
>
> +# ......................................................................
> +# interface methods dealing with whole namespace
> +
> +# $cache->clear();
> +#
> +# Remove all entries from the namespace.
> +# Namespace must be defined and not empty.
> +sub clear {
> +       my $self = shift;
> +
> +       return unless $self->get_namespace();
> +
> +       my $namespace_dir = $self->path_to_namespace();
> +       return if !-d $namespace_dir;
> +
> +       my $renamed_dir = mktemp($namespace_dir . '.XXXX');
> +       rename($namespace_dir, $renamed_dir);
> +       rmtree($renamed_dir);
> +       die "Couldn't remove '$renamed_dir' directory"
> +               if -d $renamed_dir;
> +}
> +
> +# $size = $cache->size();
> +#
> +# Size of whole names (or whole cache if namespace empty)
> +sub size {
> +       my $self = shift;
> +
> +       my $namespace_dir = $self->path_to_namespace();
> +       return if !-d $namespace_dir;
> +
> +       my $total_size = 0;
> +       my $add_size = sub { $total_size += -s $_ };
> +
> +       File::Find::find({ wanted => $add_size, no_chdir => 1 }, $namespace_dir);
> +
> +       return $total_size;
> +}
> +
>  1;
This looks good to me.
Acked-by:  Thomas Adam <thomas@xteddy.org>
-- Thomas Adam
Previous: Jakub NarebskiNext: Jakub Narebski
Message 36 of 41 in “[PATCHv5 00/17] gitweb: Simple file based output caching”
  1. Jakub NarebskiOct 6, 2010
  2. 01/17 t/test-lib.sh: Export also GIT_BUILD_DIR in test_externalJakub Narebski, Oct 6, 2010
  3. 02/17 gitweb: Prepare for splitting gitwebJakub Narebski, Oct 6, 2010
  4. 03/17 gitweb/lib - Very simple file based cacheJakub Narebski, Oct 6, 2010
  5. Thomas AdamOct 6, 2010
  6. Ævar Arnfjörð BjarmasonOct 6, 2010
  7. Thomas AdamOct 6, 2010
  8. Ævar Arnfjörð BjarmasonOct 6, 2010
  9. Jakub NarebskiOct 6, 2010
  10. Thomas AdamOct 6, 2010
  11. Jakub NarebskiOct 6, 2010
  12. Ævar Arnfjörð BjarmasonOct 6, 2010
  13. Jakub NarebskiOct 6, 2010
  14. 04/17 gitweb/lib - Stat-based cache expirationJakub Narebski, Oct 6, 2010
  15. 05/17 gitweb/lib - Regenerate entry if the cache file has size of 0Jakub Narebski, Oct 6, 2010
  16. 06/17 gitweb/lib - Simple select(FH) based output captureJakub Narebski, Oct 6, 2010
  17. Thomas AdamOct 6, 2010
  18. Jakub NarebskiOct 6, 2010
  19. Ævar Arnfjörð BjarmasonOct 6, 2010
  20. Jakub NarebskiOct 6, 2010
  21. 07/17 gitweb/lib - Cache captured output (using get/set)Jakub Narebski, Oct 6, 2010
  22. 08/17 gitweb: Add optional output cachingJakub Narebski, Oct 6, 2010
  23. Ævar Arnfjörð BjarmasonOct 6, 2010
  24. Jakub NarebskiOct 6, 2010
  25. Ævar Arnfjörð BjarmasonOct 6, 2010
  26. 09/17 gitweb/lib - Adaptive cache expiration timeJakub Narebski, Oct 6, 2010
  27. 10/21 gitweb/lib - Use CHI compatibile (compute method) caching interfaceJakub Narebski, Oct 6, 2010
  28. 11/17 gitweb/lib - Use locking to avoid 'cache miss stampede' problemJakub Narebski, Oct 6, 2010
  29. 12/17 gitweb/lib - No need for File::Temp when lockingJakub Narebski, Oct 6, 2010
  30. 13/17 gitweb/lib - Serve stale data when waiting for filling cacheJakub Narebski, Oct 6, 2010
  31. 14/17 gitweb/lib - Regenerate (refresh) cache in backgroundJakub Narebski, Oct 6, 2010
  32. 15/17 gitweb: Introduce %actions_info, gathering information about actionsJakub Narebski, Oct 6, 2010
  33. 16/17 gitweb: Show appropriate "Generating..." page when regenerating cacheJakub Narebski, Oct 6, 2010
  34. 17/17 gitweb: Add startup delay to activity indicator for cacheJakub Narebski, Oct 6, 2010
  35. 18/17 gitweb/lib - Add clear() and size() methods to caching interfaceJakub Narebski, Oct 6, 2010
  36. Thomas AdamOct 6, 2010
  37. 19/17 gitweb: Add beginnings of cache administration pageJakub Narebski, Oct 6, 2010
  38. 20/17 gitweb/lib - Benchmarking GitwebCache::SimpleFileCache (in t/9603/)Jakub Narebski, Oct 6, 2010
  39. 21/17 gitweb/lib - Alternate ways of capturing outputJakub Narebski, Oct 6, 2010
  40. [RFD] Possible improvements for output caching in gitwebJakub Narebski, Oct 10, 2010
  41. J.H.Oct 24, 2010

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.