{"thread":{"id":"10331","subject":"[PATCH] gitweb: speed up project listing by limiting find depth","startedAt":"2007-10-17T00:24:51Z","lastAt":"2007-10-17T00:41:25Z","messageCount":2,"participants":["Luke Lu","Shawn O. Pearce"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"56157","messageId":"1192580691-14308-1-git-send-email-git@vicaya.com","threadId":"10331","inReplyTo":null,"subject":"[PATCH] gitweb: speed up project listing by limiting find depth","fromName":"Luke Lu","fromEmail":"git@vicaya.com","sentAt":"2007-10-17T00:24:51Z","receivedAt":"2007-10-17T00:24:51Z","isPatch":true,"sender":{"key":"git@vicaya.com","avatar":null},"body":"Resubmit patch to make project max depth configurable.\n\nSigned-off-by: Luke Lu <git@vicaya.com>\n---\n Makefile           |    2 ++\n gitweb/gitweb.perl |    8 ++++++++\n 2 files changed, 10 insertions(+), 0 deletions(-)\n\ndiff --git a/Makefile b/Makefile\nindex 8db4dbe..b70ba8c 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -165,6 +165,7 @@ GITWEB_CONFIG = gitweb_config.perl\n GITWEB_HOME_LINK_STR = projects\n GITWEB_SITENAME =\n GITWEB_PROJECTROOT = /pub/git\n+GITWEB_PROJECT_MAXDEPTH = 2\n GITWEB_EXPORT_OK =\n GITWEB_STRICT_EXPORT =\n GITWEB_BASE_URL =\n@@ -831,6 +832,7 @@ gitweb/gitweb.cgi: gitweb/gitweb.perl\n \t    -e 's|++GITWEB_HOME_LINK_STR++|$(GITWEB_HOME_LINK_STR)|g' \\\n \t    -e 's|++GITWEB_SITENAME++|$(GITWEB_SITENAME)|g' \\\n \t    -e 's|++GITWEB_PROJECTROOT++|$(GITWEB_PROJECTROOT)|g' \\\n+\t    -e 's|\"++GITWEB_PROJECT_MAXDEPTH++\"|$(GITWEB_PROJECT_MAXDEPTH)|g' \\\n \t    -e 's|++GITWEB_EXPORT_OK++|$(GITWEB_EXPORT_OK)|g' \\\n \t    -e 's|++GITWEB_STRICT_EXPORT++|$(GITWEB_STRICT_EXPORT)|g' \\\n \t    -e 's|++GITWEB_BASE_URL++|$(GITWEB_BASE_URL)|g' \\\ndiff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl\nindex 3064298..1453101 100755\n--- a/gitweb/gitweb.perl\n+++ b/gitweb/gitweb.perl\n@@ -35,6 +35,10 @@ our $GIT = \"++GIT_BINDIR++/git\";\n #our $projectroot = \"/pub/scm\";\n our $projectroot = \"++GITWEB_PROJECTROOT++\";\n \n+# fs traversing limit for getting project list\n+# the number is relative to the projectroot\n+our $project_maxdepth = \"++GITWEB_PROJECT_MAXDEPTH++\";\n+\n # target of the home link on top of all pages\n our $home_link = $my_uri || \"/\";\n \n@@ -1509,16 +1513,20 @@ sub git_get_projects_list {\n \t\t# remove the trailing \"/\"\n \t\t$dir =~ s!/+$!!;\n \t\tmy $pfxlen = length(\"$dir\");\n+\t\tmy $pfxdepth = ($dir =~ tr!/!!);\n \n \t\tFile::Find::find({\n \t\t\tfollow_fast => 1, # follow symbolic links\n \t\t\tfollow_skip => 2, # ignore duplicates\n+\t\t\tno_chdir => 1, # don't chdir into every directory\n \t\t\tdangling_symlinks => 0, # ignore dangling symlinks, silently\n \t\t\twanted => sub {\n \t\t\t\t# skip project-list toplevel, if we get it.\n \t\t\t\treturn if (m!^[/.]$!);\n \t\t\t\t# only directories can be git repositories\n \t\t\t\treturn unless (-d $_);\n+\t\t\t\t# don't traverse too deep (Find is super slow on os x)\n+\t\t\t\treturn if tr!/!! - $pfxdepth > $project_maxdepth && ($File::Find::prune = 1);\n \n \t\t\t\tmy $subdir = substr($File::Find::name, $pfxlen + 1);\n \t\t\t\t# we check related file in $projectroot\n-- \n1.5.3.4\n"},{"id":"56161","messageId":"20071017004125.GI13801@spearce.org","threadId":"10331","inReplyTo":"1192580691-14308-1-git-send-email-git@vicaya.com","subject":"Re: [PATCH] gitweb: speed up project listing by limiting find depth","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2007-10-17T00:41:25Z","receivedAt":"2007-10-17T00:41:25Z","isPatch":true,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Luke Lu <git@vicaya.com> wrote:\n> diff --git a/Makefile b/Makefile\n> index 8db4dbe..b70ba8c 100644\n> --- a/Makefile\n> +++ b/Makefile\n> @@ -165,6 +165,7 @@ GITWEB_CONFIG = gitweb_config.perl\n>  GITWEB_HOME_LINK_STR = projects\n>  GITWEB_SITENAME =\n>  GITWEB_PROJECTROOT = /pub/git\n> +GITWEB_PROJECT_MAXDEPTH = 2\n\nI'd rather see this default to an unlimited (or maybe insane?) depth.\nCurrent users may be surprised upon upgrading to a more recent git\nwhen their gitweb stops showing projects because the default depth\nis too small.\n\nrepo.or.cz is up at 3 deep, maybe 4 right now, right Pasky?\nI think letting admins control the depth is a good idea, but its\na performance tuning thing and probably shouldn't break existing\nsetups.\n\n> +\t\t\t\t# don't traverse too deep (Find is super slow on os x)\n> +\t\t\t\treturn if tr!/!! - $pfxdepth > $project_maxdepth && ($File::Find::prune = 1);\n\nI don't do much gitweb hacking, but I usually don't like to find\ncode that mutates a value as an important side-effect in the middle\nof a boolean condition that is used to determine if we are breaking\nout of this function now, or falling through to do more work.  yea\nits more lines of code but I think it would be easier to grok if this\nwas a proper if {...}.\n\n-- \nShawn.\n"}]}