read-for-fill and caching in gitweb (Re: kernel.org mirroring)

From: Martin Langhoff <martin.langhoff@gmail.com>
Date: Dec 28, 2006, 20:45 UTC
Message-ID: <46a038f90612281245s52bdd868h8c421951c7abeb84@mail.gmail.com>

On 12/9/06, Linus Torvalds <torvalds@osdl.org> wrote:

> Actually, just looking at the examples, it looks like memcached is
> fundamentally flawed, exactly the same way Apache mod_cache is
> fundamentally flawed.

memcached is really fast internally, but can be rather slow from the POV of the client code, as it forces a costly marshalling/unmarshalling of data. For perl-only situations where it is OK to have per-server caches, I have been looking at Cache::FastMmap. I will probably try to implement caching for the projects, summary & log/shortlog pages using Cache::FastMap

And I'll do read-for-fill for it, and see how that goes.

(BTW, in the last week I've had to implement a similar anti-thundering-herds cache in PHP using memcached and/or eaccelerator -- a shmem cache -- and I've done a read-for-fill for both of them that works reasonably well.)

cheers,

martin