Re: Gitweb caching: Google Summer of Code project

From: Jakub Narebski <jnareb@gmail.com>
Date: May 30, 2008, 10:02 UTC
Message-ID: <200805301202.25368.jnareb@gmail.com>
In-Reply-To: <483FABB4.1010309@gmail.com>

On Fri, 30 May 2008, Lea Wiemann wrote:

Show 7 quoted lines

> Jakub Narebski wrote:
> >
> > you cannot assume that memcached API is installed, so
> > you have to provide some kind of fallback.
> 
> That fallback would be to have no caching.  I think that's acceptable 
> -- I'm not too willing to implement caching for two API's.

I hope that you would make a wrapper around memcached (caching engine) API (it's a pity we cannot use CHI unified Perl caching interface), so it would be easy for example to change to filesystem based cache, or size aware filesystem based cache, or mmap, etc... I mean here that even if you don't implement two caching API's at least make it possible to easy change caching backend.

Note also that memcached may not have sense for single machine (single server installation), and does not make sense for memory starved machines... and one can want gitweb caching even in that situation.

> (Incidentally, memcached takes two shell commands to install and get 
> running on my machine; I think that's acceptably easy.)

As John 'Warthog9' said wrt. using additional Perl modules for gitweb caching, most sites that are used as web servers (and gitweb servers) have strict requirements on stability of installed programs, libraries and modules. IIRC the policy usually is that one can install packages from main (base) repository for Linux distribution used on server, also from extras repository; sometimes from trusted contrib package repository. Modules which are only in CPAN, and programs which require compilation are out of the question, unfortunately.

I think there is no problem wrt. memcached itself, I'm not so sure
about Perl APIs: Cache::Memcached and/or Cache::Memcached::Fast (and
optionally appropriate CHI modules/backends).

Show 8 quoted lines

> > What's more, if you want to implement If-Modified-Since and
> > If-None-Match, you would have to implement it by yourself, while
> > for static pages (cahing HTML output) web server would do this
> > for us "for free".
> 
> Are web servers doing anything that we can't easily reimplement in a few 
> lines (and, on top of that, more easily tailored to different actions, 
> projects, etc.)?

Can we reimplement it? I think we can. Easily? I'm not sure. HTTP/1.0 If-Modified-Since should be failry easy; it would be harder to support fully and correctly ETag (weak vs. strong tags), If-None-Match (from web browsers I think), If-Match (from web caches) it would take some work.

> > By the way what do you think about adding (as an option) information
> > about gitweb performance to the [HTML] output,
> 
> Definitely a good idea!

I'd try to add it when I'd have a bot more of free time; unless you would do this first.

-- 
Jakub Narebski
Poland