Re: kernel.org mirroring (Re: [GIT PULL] MMC update)

On Thu, 7 Dec 2006, H. Peter Anvin wrote:

I really don't see what gitweb could do that would be somehow better than apache doing the caching in front of it.. Is there some apache reason why that isn't sufficient (ie limitations on its cache size or timeouts?)

Maybe the cacheability hints from gitweb could be tweaked (a lot of it should be "infinitely cacheable", but the stuff that depends on refs and thus can change, could be set to some fixed host-wide value - preferably some that depends on how old the ref is).

Having gitweb be potentially up to an hour out of date is better than causing mirroring problems due to excessive load.

For example, if the git "refs/heads/" (or tags) directory hasn't changed in the last two months, we should probably set any ref-relative gitweb pages to have a caching timeout of a day or two. In contrast, if it's changed in the last hour, maybe we should only cache it for five minutes.

Jakub: any way to make gitweb set the "expires" fields _much_ more 
aggressively. I think we should at least have the ability to set a basic 
rules like

 - a _minimum_ of five minutes regardless of anything else

   We might even tweak this based on loadaverage, and it might be 
   worthwhile to add a randomization, to make sure that you don't get into 
   situations where everything webpage needs to be recalculated at once.

 - if refs/ directories are old, raise the minimum by the age of the refs

   If it's more than an hour old, raise it to ten minutes. If it's more 
   than a day, raise it to an hour. If it's more than a month old, raise 
   it to a day. And if it's more than half a year, it's some historical 
   archive like linux-history, and should probably default to a week or 
   more.

 - infinite for stuff that isn't ref-related.

Hmm?