Re: [PATCH 3/3] gitweb: use new Git::Repo API, and add optional caching

From: J.H. <warthog19@eaglescrag.net>
Date: Jul 15, 2008, 02:03 UTC
Message-ID: <1216087401.18836.11.camel@localhost.localdomain>
In-Reply-To: <487C0252.4030804@gmail.com>

To continue with that benchmarking it seems if gitweb-caching isn't hot it takes about 3 seconds, if it is I'm about 7x faster than Lea's ;-)

$ time wget -qO/dev/null http://localhost/git/

real 0m2.952s user 0m0.001s sys 0m0.004s $ time wget -qO/dev/null http://localhost/git/

real 0m0.108s user 0m0.001s sys 0m0.001s

But regardless of who's faster - I would still argue there are two reasons to at least have some caching, even if it's crappy:

1) Prevent the thundering heard problem - many requests for the same
thing all generating the same data is bad, and kills I/O

2) Relatively static data can be generated once and stick around for a
bit and serve more requests more efficiently.  Now I agree that
invalidating the cache on a new mtime is better than my current
algorithm (which is purely time based on the cache data vs. the original
data, with some allowances for back-off due to load).

- John 'Warthog9' Hawley

On Tue, 2008-07-15 at 03:50 +0200, Lea Wiemann wrote:

Show 20 quoted lines

> Johannes Schindelin wrote:
> > Wasn't the main page (i.e. the projects list) the reason why kernel.org 
> > has its own little caching mechanism in the first place?
> > 
> > And did Pasky not report recently that repo.or.cz got substantially less 
> > loaded with some caching of its own?
> 
> Yes, you need *some* caching mechanism.  Once you have that, it's fine,
> at least if the page cache is hot.  (The x-thousand stat calls that my
> caching implementation issues don't actually take that much time; I
> suspect the ~1000 calls to memcached are the more expensive [and
> optimizable] part, though I'd have to benchmark that.)
> 
> Mainline vs. my caching implementation (both with hot page cache) on
> odin3.kernel.org:
> 
> $ time wget -qO/dev/null http://localhost/git-lewiemann/vanilla/
> real    0m3.070s
> $ time wget -qO/dev/null http://localhost/git-lewiemann/
> real    0m0.719s