git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 1/9] gitweb: Load checking

From
John 'Warthog9' Hawley <warthog9@eaglescrag.net>
Date
Jan 14, 2010, 01:22 UTC
Message-ID
<1263432185-21334-2-git-send-email-warthog9@eaglescrag.net>
In-Reply-To
<1263432185-21334-1-git-send-email-warthog9@eaglescrag.net>
From: John 'Warthog9' Hawley <warthog9@kernel.org>

This changes slightly the behavior of gitweb, so that it verifies that the box isn't inundated with before attempting to serve gitweb. If the box is overloaded, it basically returns a 503 Server Unavailable until the load falls below the defined threshold. This helps dramatically if you have a box that's I/O bound, reaches a certain load and you don't want gitweb, the I/O hog that it is, increasing the pain the server is already undergoing.

This behavior is controlled by $maxload configuration variable. Default is a load of 300, which for most cases should never be hit. Unset it (set it to undefined value, i.e. undef) to turn off checking.

Currently it requires that '/proc/loadavg' file exists, otherwise the load check is bypassed (load is taken to be 0). So platforms that do not implement '/proc/loadavg' currently cannot use this feature. (provisions are included for additional checks to be added by others)

Signed-off-by: John 'Warthog9' Hawley <warthog9@kernel.org>
Signed-off-by: Jakub Narebski <jnareb@gmail.com>
---
 gitweb/README      |    7 ++++++-
 gitweb/gitweb.perl |   45 +++++++++++++++++++++++++++++++++++++++++----
 2 files changed, 47 insertions(+), 5 deletions(-)
diff --git a/gitweb/README b/gitweb/README
index e34ee79..6c2c8e1 100644
--- a/gitweb/README
+++ b/gitweb/README
@@ -174,7 +174,7 @@ not include variables usually directly set during build):
    Base URL for relative URLs in pages generated by gitweb,
    (e.g. $logo, $favicon, @stylesheets if they are relative URLs),
    needed and used only for URLs with nonempty PATH_INFO via
-   <base href="$base_url>.  Usually gitweb sets its value correctly,
+   <base href="$base_url">.  Usually gitweb sets its value correctly,
    and there is no need to set this variable, e.g. to $my_uri or "/".
  * $home_link
    Target of the home link on top of all pages (the first part of view
@@ -228,6 +228,11 @@ not include variables usually directly set during build):
    repositories from launching cross-site scripting (XSS) attacks.  Set this
    to true if you don't trust the content of your repositories. The default
    is false.
+ * $maxload
+   Used to set the maximum load that we will still respond to gitweb queries.
+   If server load exceed this value then return "503 Service Unavaliable" error.
+   Server load is taken to be 0 if gitweb cannot determine its value.  Set it to
+   undefined value to turn it off.  The default is 300.
 
 
 Projects list file format
diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
index 7e477af..0a07d3a 100755
--- a/gitweb/gitweb.perl
+++ b/gitweb/gitweb.perl
@@ -221,6 +221,12 @@ our %avatar_size = (
 	'double'  => 32
 );
 
+# Used to set the maximum load that we will still respond to gitweb queries.
+# If server load exceed this value then return "503 server busy" error.
+# If gitweb cannot determined server load, it is taken to be 0.
+# Leave it undefined (or set to 'undef') to turn off load checking.
+our $maxload = 300;
+
 # You define site-wide feature defaults here; override them with
 # $GITWEB_CONFIG as necessary.
 our %feature = (
@@ -551,6 +557,32 @@ if (-e $GITWEB_CONFIG) {
 	do $GITWEB_CONFIG_SYSTEM if -e $GITWEB_CONFIG_SYSTEM;
 }
 
+# Get loadavg of system, to compare against $maxload.
+# Currently it requires '/proc/loadavg' present to get loadavg;
+# if it is not present it returns 0, which means no load checking.
+sub get_loadavg {
+	if( -e '/proc/loadavg' ){
+		open my $fd, '<', '/proc/loadavg'
+			or return 0;
+		my @load = split(/\s+/, scalar <$fd>);
+		close $fd;
+
+		# The first three columns measure CPU and IO utilization of the last one,
+		# five, and 10 minute periods.  The fourth column shows the number of
+		# currently running processes and the total number of processes in the m/n
+		# format.  The last column displays the last process ID used.
+		return $load[0] || 0;
+	}
+	# additional checks for load average should go here for things that don't export
+	# /proc/loadavg
+
+	return 0;
+}
+
+if (defined $maxload && get_loadavg() > $maxload) {
+	die_error(503, "The load average on the server is too high");
+}
+
 # version of the core git binary
 our $git_version = qx("$GIT" --version) =~ m/git version (.*)$/ ? $1 : "unknown";
 $number_of_git_cmds++;
@@ -3354,14 +3386,19 @@ sub git_footer_html {
 # 500: The server isn't configured properly, or
 #      an internal error occurred (e.g. failed assertions caused by bugs), or
 #      an unknown error occurred (e.g. the git binary died unexpectedly).
+# 503: The server is currently unavailable (because it is overloaded,
+#      or down for maintenance).  Generally, this is a temporary state.
 sub die_error {
 	my $status = shift || 500;
 	my $error = shift || "Internal server error";
 
-	my %http_responses = (400 => '400 Bad Request',
-			      403 => '403 Forbidden',
-			      404 => '404 Not Found',
-			      500 => '500 Internal Server Error');
+	my %http_responses = (
+		400 => '400 Bad Request',
+		403 => '403 Forbidden',
+		404 => '404 Not Found',
+		500 => '500 Internal Server Error',
+		503 => '503 Service Unavailable',
+	);
 	git_header_html($http_responses{$status});
 	print <<EOF;
 <div class="page_body">
-- 
1.6.5.2
Previous: John 'Warthog9' HawleyNext: John 'Warthog9' Hawley
Message 2 of 60 in “Gitweb caching v5”
  1. 0/9 Gitweb caching v5John 'Warthog9' Hawley, Jan 14, 2010
  2. 1/9 gitweb: Load checkingJohn 'Warthog9' Hawley, Jan 14, 2010
  3. 2/9 gitweb: change die_error to take "extra" argument for extended die informationJohn 'Warthog9' Hawley, Jan 14, 2010
  4. 3/9 gitweb: Add option to force version matchJohn 'Warthog9' Hawley, Jan 14, 2010
  5. 4/9 gitweb: Makefile improvementsJohn 'Warthog9' Hawley, Jan 14, 2010
  6. 5/9 gitweb: add a get function to compliment print_local_timeJohn 'Warthog9' Hawley, Jan 14, 2010
  7. 6/9 gitweb: add a get function to compliment print_sort_thJohn 'Warthog9' Hawley, Jan 14, 2010
  8. 7/9 gitweb: cleanup error message produced by undefined $site_headerJohn 'Warthog9' Hawley, Jan 14, 2010
  9. 8/9 gitweb: Convert output to using indirect file handleJohn 'Warthog9' Hawley, Jan 14, 2010
  10. 9/9 gitweb: File based caching layer (from git.kernel.org)John 'Warthog9' Hawley, Jan 14, 2010
  11. Jakub NarebskiJan 16, 2010
  12. 00/10 gitweb: Simple file based output cachingJakub Narebski, Jan 23, 2010
  13. J.H.Jan 23, 2010
  14. 11/10 gitweb: Ajax-y "Generating..." page when regenerating cache (WIP)Jakub Narebski, Jan 24, 2010
  15. J.H.Feb 6, 2010
  16. Jakub NarebskiFeb 6, 2010
  17. Jakub NarebskiFeb 7, 2010
  18. 02/10 gitweb: href(..., -path_info => 0|1)Jakub Narebski, Jan 23, 2010
  19. 03/10 gitweb/cache.pm - Very simple file based cachingJakub Narebski, Jan 23, 2010
  20. 04/10 gitweb/cache.pm - Stat-based cache expirationJakub Narebski, Jan 23, 2010
  21. 05/10 gitweb: Use Cache::Cache compatibile (get, set) output caching (WIP)Jakub Narebski, Jan 23, 2010
  22. 06/10 gitweb/cache.pm - Adaptive cache expiration time (WIP)Jakub Narebski, Jan 23, 2010
  23. 07/10 gitweb: Use CHI compatibile (compute method) caching (WIP)Jakub Narebski, Jan 23, 2010
  24. 08/10 gitweb/cache.pm - Use locking to avoid 'stampeding herd' problem (WIP)Jakub Narebski, Jan 23, 2010
  25. 09/10 gitweb/cache.pm - Serve stale data when waiting for filling cache (WIP)Jakub Narebski, Jan 23, 2010
  26. 10/10 gitweb: Show appropriate "Generating..." page when regenerating cache (WIP)Jakub Narebski, Jan 23, 2010
  27. Petr BaudisJan 24, 2010
  28. Jakub NarebskiJan 25, 2010
  29. Jakub NarebskiJan 25, 2010
  30. Jakub NarebskiJan 25, 2010
  31. Petr BaudisJan 25, 2010
  32. Jakub NarebskiJan 25, 2010
  33. Petr BaudisJan 25, 2010
  34. J.H.Jan 25, 2010
  35. Jakub NarebskiJan 26, 2010
  36. Petr BaudisJan 28, 2010
  37. Jakub NarebskiJan 31, 2010
  38. Jakub NarebskiJan 25, 2010
  39. J.H.Jan 25, 2010
  40. Jakub NarebskiJan 26, 2010
  41. Jakub NarebskiJan 16, 2010
  42. Junio C HamanoJan 16, 2010
  43. Jakub NarebskiJan 16, 2010
  44. Junio C HamanoJan 16, 2010
  45. Petr BaudisJan 24, 2010
  46. Jakub NarebskiJan 25, 2010
  47. J.H.Jan 25, 2010
  48. Jakub NarebskiJan 25, 2010
  49. Jakub NarebskiJan 15, 2010
  50. Jakub NarebskiJan 23, 2010
  51. Jakub NarebskiJan 15, 2010
  52. Petr BaudisJan 24, 2010
  53. Jakub NarebskiJan 24, 2010
  54. Jakub NarebskiJan 15, 2010
  55. Jakub NarebskiJan 15, 2010
  56. Jakub NarebskiJan 15, 2010
  57. J.H.Jan 15, 2010
  58. Jakub NarebskiJan 15, 2010
  59. Jakub NarebskiJan 23, 2010
  60. Jakub NarebskiFeb 7, 2010

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.