From: H. Peter Anvin <hpa@zytor.com>
Date: Fri, 08 Dec 2006 23:49:19 GMT
Subject: Re: kernel.org mirroring (Re: [GIT PULL] MMC update)
Message-ID: <4579F9FF.7050701@zytor.com>
In-Reply-To: <f2b55d220612081546u1ffa98e5q75be55d31da82a2f@mail.gmail.com>

Michael K. Edwards wrote:
> On 12/8/06, Linus Torvalds <torvalds@osdl.org> wrote:
>> You can't have a cache architecture where the client just does a "get",
>> like memcached does. You need to have a "read-for-fill" operation ...
> 
> In Squid 2.6:
>    collapsed_forwarding on
>    refresh_stale_window <seconds>
> (apply the latter only to stanzas where you want "readahead" of
> about-to-expire cache entries)
> 
> Brief design description at 
> http://devel.squid-cache.org/collapsed_forwarding/.
> 
> (I didn't write this code, everything I know about squid leaked
> through the Google-shaped pinhole in my tinfoil hat, etc.  But if you
> go this way I'd like to be in the loop to understand the scalability
> issues around netfilter-assisted transparent proxying.)
> 

There is another thing that probably will be required, and I'm not sure 
if something in front of Apache (like Squid) rather than behind it can 
easily deal with: on timeout, the process needs to continue in order to 
feed the cache.  Otherwise, you're still in a failure scenario as soon 
as timeout happens.