Re: [PATCH 3/3] gitweb: use new Git::Repo API, and add optional caching

From: J.H. <warthog19@eaglescrag.net>
Date: Jul 15, 2008, 01:44 UTC
Message-ID: <1216086284.18836.5.camel@localhost.localdomain>
In-Reply-To: <alpine.LSU.1.00.0807150326050.3486@wbgn129.biozentrum.uni-wuerzburg.de>

On Tue, 2008-07-15 at 03:28 +0200, Johannes Schindelin wrote:

Show 20 quoted lines

> Hi,
> 
> On Tue, 15 Jul 2008, Lea Wiemann wrote:
> 
> > Jakub Narebski wrote:
> > > On Tue, 15 July 2008, Lea Wiemann wrote:
> > >> # Transient cache entries (like get_sha1('HEAD')) are automatically
> > >> # invalidated when an mtime [...] changes.
> > > 
> > > Nice idea... for project pages.  I'm not so sure about projects_list 
> > > page, if wouldn't be better to have expire time for *this* page.  You 
> > > would have/have to stat a lot of files/directories to detect changes.
> > 
> > It doesn't seem to be too much of a performance issue (it takes ~500ms
> > to generate the project list on kernel.org), and project lists aren't
> > requested often enough to be a good optimization target.
> 
> Wasn't the main page (i.e. the projects list) the reason why kernel.org 
> has its own little caching mechanism in the first place?
>

Partially - it was by *far* the biggest problem we were facing, and it likely has gotten a lot better since I went on a rampage and did gitweb-caching.

That said there's a relatively small sample-set of files that gets hit more than others it seems (from stuff I was looking at a while ago) and limiting the general disk i/o on the kernel.org machines is *always* a good thing (the less we go to disk the better) so while it started that way it has been very beneficial, if it does chew up something like 15-20G of disk on those machines.)

- John 'Warthog9' Hawley

Show 7 quoted lines

> And did Pasky not report recently that repo.or.cz got substantially less 
> loaded with some caching of its own?
> 
> warthog?  Pasky?
> 
> Ciao,
> Dscho