[PATCH/RFC 4/6] gitweb: Faster project search

From: Jakub Narebski <jnareb@gmail.com>
Date: Jul 29, 2011, 11:52 UTC
Message-ID: <1311940379-9608-5-git-send-email-jnareb@gmail.com>
In-Reply-To: <1311940379-9608-1-git-send-email-jnareb@gmail.com>

Before searching by some field the information we search for must be filled in. For this fill_project_list_info() was enhanced in previous commit to take additional parameters which part of projects info to fill. This way we can limit doing expensive calculations (like running git-for-each-ref to get 'age' / "Last changed" info) only to projects which we will show as search results.

With this commit the number of git commands used to generate search results is 2*<matched projects> + 1, and depends on number of matched projects rather than number of all projects (all repositories).

Note: this is 'git for-each-ref' to find last activity, and 'git config'
for each project, and 'git --version' once.

Signed-off-by: Jakub Narebski <jnareb@gmail.com>
---
I have checked it by running gitweb with 'timed' feature enabled, but
I didn't benchmark it.

This is a win especially for hosting sites with many projects, though they either need to have caching, or not show full project list but only search form by default, or paginate projects list.

BTW. I am thinking about adding configuration which would make gitweb show only search form if number of project is higher than some configured threshold. What do you think of this idea?

 gitweb/gitweb.perl |    9 +++++++--
 1 files changed, 7 insertions(+), 2 deletions(-)

diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
index 4c66307..9c82d79 100755
--- a/gitweb/gitweb.perl
+++ b/gitweb/gitweb.perl
@@ -2933,6 +2933,10 @@ sub search_projects_list {
 	return @$projlist
 		unless ($tagfilter || $searchtext);
 
+	# searching projects require filling to be run before it;
+	fill_project_list_info($projlist,
+	                       $tagfilter  ? 'ctags' : (),
+	                       $searchtext ? ('path', 'descr') : ());
 	my @projects;
  PROJECT:
 	foreach my $pr (@$projlist) {
@@ -5171,12 +5175,13 @@ sub git_project_list_body {
 	# filtering out forks before filling info allows to do less work
 	@projects = filter_forks_from_projects_list(\@projects)
 		if ($check_forks);
-	@projects = fill_project_list_info(\@projects);
-	# searching projects require filling to be run before it
+	# search_projects_list pre-fills required info
 	@projects = search_projects_list(\@projects,
 	                                 'searchtext' => $searchtext,
 	                                 'tagfilter'  => $tagfilter)
 		if ($tagfilter || $searchtext);
+	# fill the rest
+	@projects = fill_project_list_info(\@projects);
 
 	$order ||= $default_projects_order;
 	$from = 0 unless defined $from;

-- 
1.7.5