Re: kernel.org mirroring (Re: [GIT PULL] MMC update)

From: Jeff Garzik <jeff@garzik.org>
Date: Dec 9, 2006, 14:43 UTC
Message-ID: <457ACBA1.4090007@garzik.org>
In-Reply-To: <200612091437.01183.jnareb@gmail.com>

Jakub Narebski wrote:

> Sending Last-Modified: should be easy; sending ETag needs some consensus
> on the contents: mainly about validation. Responding to If-Modified-Since:
> and If-None-Match: should cut at least _some_ of the page generating time.

Definitely.

Show 5 quoted lines

> As I said, I'm not talking (at least now) about saving generated HTML
> output. This I think is better solved in caching engine like Squid can
> be. Although even here some git specific can be of help: we can invalidate
> cache on push, and we know that some results doesn't ever change (well,
> with exception of changing output of gitweb).

It depends on how creatively you think ;-)

Consider generating static HTML files on each push, via a hook, for many of the toplevel files. The static HTML would then link to the CGI for further dynamic querying of the git database.

> What can be _easily_ done:
>  * Use post 1.4.4 gitweb, which uses git-for-each-ref to generate summary
>    page; this leads to around 3 times faster summary page.

This re-opens the question mentioned earlier, is Kay (or anyone?) still actively maintaining gitweb on k.org?

>  * Perhaps using projects list file (which can be now generated by gitweb)
>    instead of scanning directories and stat()-ing for owner would help
>    with time to generate projects lis page

This could be statically generated by a robot. I think everybody would shrink in horror if a human needed to maintain such a file.

> What can be quite easy incorporated into gitweb:
>  * For immutable pages set Expires: or Cache-Control: max-age (or both)
>    to infinity

nice!

>  * Generate Last-Modified: for those views where it can be calculated,
>    and respond with 304 Not Modified as soon as it can.

agreed

Show 5 quoted lines

> What can be easily done using caching engine:
>  * Select top 10 of common queries, and cache them, invalidating cache on push
>    (depending on query: for example invalidate project list on push to any
>    project, invalidate RSS/Atom feed and summary pages only on push to specific
>    project) - can be done with git hooks.

Or simply generate regular filesystem files into the webspace, as triggered by a hook. Let the standard filesystem mirroring/caching work its magic.

	Jeff