git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: Another bench on gitweb (also on gitweb caching)

From
Jakub Narebski <jnareb@gmail.com>
Date
Feb 13, 2008, 12:17 UTC
Message-ID
<200802131317.48815.jnareb@gmail.com>
In-Reply-To
<1202864493.17207.24.camel@localhost.localdomain>
On Wed, 13 Feb 2008, J.H. "Warthog9" wrote:
> Bruno Cesar Ribas <ribas@c3sl.ufpr.br> writes:
>> On Mon, Feb 11, 2008 at 04:44:23PM -0800, Jakub Narebski wrote:
Show 24 quoted lines
>>> Kernel.org even run modified version of gitweb, with some caching
>>> support; Cgit (git web interface in C) also has caching support.
>> 
>> Is this gitweb version for kernel.org available somewhere?
>> 
> 
> It's available from my git tree on kernel.org
> http://git.kernel.org/?p=git/warthog9/gitweb.git;a=summary
> 
> or
> 
> git://git.kernel.org/pub/scm/git/warthog9/gitweb.git
> 
> Mind you my performance on the non-cache state is not going to be any
> better than normal gitweb, however the performance on a cache-hit is
> orders of magnitude faster - though at a rather expensive cost - disk
> space.  There is currently something like 20G of disk being used on one
> of kernel.org's machines providing the cache (this does get flushed on
> occasion - I think) but that is providing caching for everything that
> kernel.org has in it's git trees (or 255188 unique urls currently).  My
> code base is now, horribly, out of date with respect to mainline but it
> works and it's been solid and reasonably reliable (though I do know of
> two bugs in it right now I need to track down - one with respect to a
> failure of the script - and one that is an array out of bounds error)

BTW. did you consider using cgit (C/Caching git web interface) instead or in addition to gitweb? Freedesktop.org uses it side by side with gitweb. I wonder how it would perform on kernel.org...

(Almost) every optimization should begin with profiling. Could you tell us which gitweb pages are most called and perhaps which pages generate most load for kernel.org? How new projects are added (old projects deleted)? Do you control (can add to or can add multiplexing) to update or post-receive hooks?

Without this data we could concentrate on things which are of no importance. BTW. I wonder if slitting projects_list page would help...

-- 
Jakub Narebski
Poland
Previous: J.H.Next: J.H.
Message 7 of 11 in “Another bench on gitweb”
  1. Bruno Cesar RibasFeb 10, 2008
  2. Jakub NarebskiFeb 12, 2008
  3. Bruno Cesar RibasFeb 13, 2008
  4. Bruno Cesar RibasFeb 13, 2008
  5. J.H.Feb 13, 2008
  6. J.H.Feb 13, 2008
  7. Jakub NarebskiFeb 13, 2008
  8. J.H.Feb 13, 2008
  9. Jakub NarebskiFeb 14, 2008
  10. J.H.Feb 14, 2008
  11. Jakub NarebskiFeb 15, 2008

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.