[PATCH 0/3] Git::Repo API and gitweb caching

Hi everyone,

As follow-ups to this message, I'll be sending three patches for

1) adding the Mechanize tests,
2) adding the Git::Repo API, and (the important part:)
3) making gitweb use the Git::Repo API, and adding caching to gitweb.

The patches apply on master or next: they're viewable live in action, with cache statistics temporarily enabled at the bottom of each page, at: http://odin3.kernel.org/git-lewiemann/

Patch (3) basically makes two large changes in one patch, but it was pretty hard to separate them during development. I could try to split them up after the fact, but it would take at least an hour or two, since the changes that introduce caching are spread all over the code. I don't think that having separate commits ([a] use Git::Repo API, [b] add caching) brings enough benefit to justify the effort.

There are some other changes in (3) as well, but they fell out as part of the refactoring, so I didn't separate them either -- same thing.

The Mechanize tests succeed before and after patch (3) is applied, so I'm reasonably confident that my refactoring didn't introduce any (major) bugs.

And since you all are curious about the API thing :-), I've added some notes about why I didn't use Git.pm in the patch message of patch (2).

To all reviewers: Since the patches are quite long, I suggest that for anything but major changes that require either discussion or work on my end, you simply send a patch that applies on top of my patches. (Just sending a patch with a bunch of trivial/small changes without comment should be fine; the reasons for simple improvements are normally obvious.)

On my to-do list:

- Benchmarks.  I'm planning to time a replay of kernel.org's gitweb logs
on the test server, with and without caching.  Nothing fancy.  (The
performance of the test setup on odin3.kernel.org is not representative
of gitweb's actual performance under load.)

- Implement support for Last-Modified or ETags, since those basically
fall out for free with the current implementation.  (This will require
mod_perl, since CGI doesn't allow for accessing arbitrary request
headers AFAIK.)  That will make the site a tad more responsive, I hope,
and it will also hugely reduce the load for RSS/Atom requests, which
currently make up almost half of all requests to kernel.org's gitweb and
get served in full each time (i.e. "200 OK" instead of "304 Not Modified").

- Make gitweb use more parts of the Git::Repo API; in particular, the
commit and tag parsing code should be ripped out, and gitweb should use
the (much prettier) Git::Commit/Git::Tag API instead.  Perhaps some more
functions (like ls_tree) can be generalized into the API as well; I went
the easy route for now and simply replaced most "open '-|'" calls with
$repo->cmd_output calls.

-- Lea