Re: kernel.org mirroring (Re: [GIT PULL] MMC update)

From: Jakub Narebski <jnareb@gmail.com>
Date: Dec 10, 2006, 20:27 UTC
Message-ID: <200612102127.05894.jnareb@gmail.com>
In-Reply-To: <Pine.LNX.4.64.0612101129190.12500@woody.osdl.org>

Linus Torvalds wrote:

Show 19 quoted lines

> On Sun, 10 Dec 2006, Jakub Narebski wrote:
>>>> If-Modified-Since:, If-Match:, If-None-Match: do you?
>> 
>> And in CGI standard there is a way to access additional HTTP headers
>> info from CGI script: the envirionmental variables are HTTP_HEADER,
>> for example if browser sent If-Modified-Since: header it's value
>> can be found in HTTP_IF_MODIFIED_SINCE environmental variable.
> 
> Guys, you're missing something fairly fundamnetal. 
> 
> It helps almost _nothing_ to support client-side caching with all these 
> fancy "If-Modified-Since:" etc crap.
> 
> That's not the _problem_.
> 
> It's usually not one client asking for the gitweb pages: the load comes 
> from just lots of people independently asking for it. So client-side 
> caching may help a tiny tiny bit, but it's not actually fixing the 
> fundamental problem at all.

Well, the idea (perhaps stupid idea: I don't know how caching engines / reverse proxy works) was that there would be caching engine / reverse proxy in the front (Squid for example) would cache results and serve it to rampaging hordes. But this caching engine has to ask gitweb if the cache is valid using "If-Modified-Since:" and "If-None-Match:" headers. If gitweb returns 304 Not Modified then it serves contents from cache.

Show 5 quoted lines

> So forget about "If-Modified-Since:" etc. It may help in benchmarks when 
> you try it yourself, and use "refresh" on the client side. But the basic 
> problem is all about lots of clients that do NOT have things cached, 
> because all teh client caches are all filled up with pr0n, not with gitweb 
> data from yesterday.

What about the other idea, the one with raising expires to infinity for immutable pages like "commit" view for commit given by SHA-1? Even if the clients won't cache it, the proxies and caches between gitweb and client might cache it...

Talking about most accessed gitweb pages, the project list page changes on every push, the project summary page and project main RSS feed (now in both RSS and Atom formats) changes on every push to given project. With a help of hooks they can be static pages, generated by push... ...with the exception that projects list and summary pages have _relative_ dates.

-- 
Jakub Narebski