[RFC PATCHv2 00/10] gitweb: Simple file based output caching

This 10 patches long patch series is intended as preliminary version for splitting large 'gitweb: File based caching layer (from git.kernel.org)' mega-patch by John 'Warthog9' Hawley aka J.H., by starting small and adding features piece by piece.

This is second version (second release) of this series; previous is
available at http://repo.or.cz/w/git/jnareb-git.git as 'gitweb/cache-kernel'
branch.  It was sent as:
* [RFC PATCH 00/10] gitweb: Simple file based output caching
  Message-Id: <cover.1264198194.git.jnareb@gmail.com>
  http://thread.gmane.org/gmane.comp.version-control.git/136913/focus=136917
(note however that v1 series of emails is lacking one of patches because
it was over VGER anti-spam size limit for messages).

This version tries to do without 
  gitweb: Print to explicit filehandle (preparing for caching)
patch, by capturing output using either PerlIO layers manipulated
using PerlIO::Util if this module is available, or direct manipulation
of *STDOUT if PerlIO::Util isn't available.  One of the goals of this
series is then decide whether it is worth to have explicit filehandle
in print statements in gitweb, or not; if the complexity is worth not
having to deal with straightforward but quite intrusive (and large)
patch.

As the earlier version was inspired by file-based caching in Cache::Cache, this one is inspired by file-based caching in more modern CHI (unified cache interface).

It still lacks POD for gitweb/cache.pm (would it be needed, or would comments be enough), and gitweb/cache.pm still ties rather heavily into gitweb (following still what was in original J.H. (mega)patch).

It *does* have quite detailed commit messages, as opposed to v1 of this series, where some commits were described only in comment section of emails containing them. It is also very configurable (Pasky, this would probably be of interest to you, as you didn't want to have "Generating..." pages enabled), even more than in original patch by J.H.

NOTE: there are quite a bit of _API_ tests, but I have not tested gitweb
output with caching enabled extensively (thats how bug in "Generating..."
slipped through - for details see comments in last patch).  I have tested
that caching works around 4th patch in series, in that it doesn't cause
errors and displays page (here the lack of error handling is decidely
unhelpful), and that it displays the time when page was generated.  As I
have installed PerlIO::Util using local::lib, i.e. locally in ~/perl5,
I think that what I have been testing was the "*STDOUT munging" method
of capturing gitweb output.  (See "Capturing gitweb output" section
in PATCHv2 04/10).

This series is based on commit 8424981934c415bd20643de9cc932bd348dfb115:
(in the 'master' branch of git.git repository)
  Jeff King (1):
        Fix invalid read in quote_c_style_counted

and is available in the git repository at:

  git://repo.or.cz/git/jnareb-git.git gitweb/cache-kernel-v2

Jakub Narebski (10):
  gitweb: href(..., -path_info => 0|1)
  gitweb/cache.pm - Very simple file based caching
  gitweb/cache.pm - Stat-based cache expiration
  gitweb: Use Cache::Cache compatible (get, set) output caching
  gitweb/cache.pm - Adaptive cache expiration time
  gitweb: Use CHI compatible (compute method) caching
  gitweb/cache.pm - Use locking to avoid 'cache miss stampede' problem
  gitweb/cache.pm - Serve stale data when waiting for filling cache
  gitweb/cache.pm - Regenerate (refresh) cache in background
  gitweb: Show appropriate "Generating..." page when regenerating cache

Note that compared to previous version of this series, this version
lacks initial commit.
  gitweb: Print to explicit filehandle (preparing for caching)
This is a bit of an experiment if we can do caching without large patch
to gitweb upfront, and to decide whether tradeoff (more complicated
capturing) is worth it.

Also, one of the commits:
  gitweb/cache.pm - Serve stale data when waiting for filling cache (WIP)
was split into two separate commits:
  gitweb/cache.pm - Serve stale data when waiting for filling cache
  gitweb/cache.pm - Regenerate (refresh) cache in background
one serving stale data (in processes waiting for cache to be filled, 
aka readers), and one adding background cache regeneration. 

After previous series I have sent additional (PATCH 11/10) patch:
  gitweb: Ajax-y "Generating..." page when regenerating cache (WIP)
This patch would require rework to apply to this new series.

Diffstat:
~~~~~~~~~
 gitweb/README                          |   70 +++++
 gitweb/cache.pm                        |  530 ++++++++++++++++++++++++++++++++
 gitweb/gitweb.perl                     |  305 +++++++++++++++++-
 t/gitweb-lib.sh                        |    2 +
 t/t9500-gitweb-standalone-no-errors.sh |   19 ++
 t/t9503-gitweb-caching.sh              |   32 ++
 t/t9503/test_cache_interface.pl        |  380 +++++++++++++++++++++++
 t/test-lib.sh                          |    3 +
 8 files changed, 1325 insertions(+), 16 deletions(-)
 create mode 100644 gitweb/cache.pm
 create mode 100755 t/t9503-gitweb-caching.sh
 create mode 100755 t/t9503/test_cache_interface.pl