{"thread":{"id":"26380","subject":"[PATCH 0/2] gitweb: Begin splitting gitweb","startedAt":"2011-02-01T16:50:12Z","lastAt":"2011-02-11T23:21:56Z","messageCount":7,"participants":["Jakub Narebski","Alejandro R. Sedeño"],"isPatch":true,"patchVersion":1,"patchTotal":2},"messages":[{"id":"160186","messageId":"1296579016-13356-1-git-send-email-jnareb@gmail.com","threadId":"26380","inReplyTo":null,"subject":"[PATCH 0/2] gitweb: Begin splitting gitweb","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2011-02-01T16:50:12Z","receivedAt":"2011-02-01T16:50:12Z","isPatch":true,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"Gitweb is currently next to largest file (after gitk) in git sources,\nmore than 220K with more than 25,000 lines.  Therefore adding any\nlarge feature that would require large amount of code added, like\ngitweb caching by J.H. and my rewrite of it, or \"gitweb admin/write\"\n[failed] GSoC 2010 project by Pavan Kumar Sunkara require for new code\nto be added as a separate module or module.  Otherwise gitweb would\nfast become unmaintainable.\n\nNot in all cases it would require splitting gitweb upfront.  At least\nin the case of gitweb caching it doesn't.  What must be done however\nis preparing the infrastructure for modular gitweb sources; to\nproperly test such infrastructure we need at least one split gitweb\nmodule.\n\n\nThis series is intended to bring such infrastructure to gitweb, to\nprepare way for adding output caching to gitweb. Alternatively it can\nbe thought as beginning of splitting gitweb into smaller submodules,\nfor better maintability.\n\n\nTable of contents:\n~~~~~~~~~~~~~~~~~~\n* [PATCH (version A) 1/2] gitweb: Prepare for splitting gitweb\n\n    sub __DIR__ () {\n    \tFile::Spec->rel2abs(join '', (File::Spec->splitpath(__FILE__))[0, 1]);\n    }\n    use lib __DIR__ . '/lib';\n\n  Advantages:\n  - no changes to t/gitweb-lib.sh, ability to run source version\n    of gitweb without any changes\n  \n  Disadvantages:\n  - supports only modules installed either alongside gitweb, or in one\n    of PERL5LIB directories; no support for installing modules not\n    alongside gitweb\n  - because we cannot rely on FindBin::again being available nor on\n    having Dir::Self installed, __DIR__ must be defined -- more code.\n \n* [PATCH (version B) 1/2] gitweb: Prepare for splitting gitweb\n\n    use lib $ENV{'GITWEBLIBDIR'} || \"++GITWEBLIBDIR++\";\n\n  Advantages:\n  - supports relocating gitweb modules (to gitweblibdir)\n  - shortest code of all the cases\n\n  Disadvantages:\n  - required changes to t/gitweb-lib.sh to pick up gitweb modules\n    by source version of gitweb\n\n* [PATCH (version C) 1/2] gitweb: Prepare for splitting gitweb\n\n    sub __DIR__ () {\n    \tFile::Spec->rel2abs(join '', (File::Spec->splitpath(__FILE__))[0, 1]);\n    }\n    use lib __DIR__ . '/lib';\n    use lib \"++GITWEBLIBDIR++\";\n\n  Advantages:\n  - can run source version of gitweb (gitweb/gitweb.perl) as a script simply\n  - supports relocating gitweb modules (to gitweblibdir)\n\n  Disadvantages:\n  - most complicated code of all cases\n\n\n* [PATCH (proof of concept) 2/2] gitweb: Create Gitweb::Util module\n\n  Something to actually test previous patch(es) with... and I guess\n  good start to splitting gitweb into smaller modules.\n\n  All versions pass \"make -C gitweb test\" and \"make -C gitweb test-installed\"\n  after \"make -C gitweb install\" / \"make install-gitweb\".\n\n\nShortlog:\n~~~~~~~~~\nJakub Narebski (1):\n  gitweb: Prepare for splitting gitweb\n\nPavan Kumar Sunkara (1):\n  gitweb: Create Gitweb::Util module\n\nDiffstat (for version C):\n~~~~~~~~~~~~~~~~~~~~~~~~~\n gitweb/Makefile           |   22 +++++-\n gitweb/gitweb.perl        |  150 ++++-----------------------------------\n gitweb/lib/Gitweb/Util.pm |  177 +++++++++++++++++++++++++++++++++++++++++++++\n 3 files changed, 210 insertions(+), 139 deletions(-)\n create mode 100644 gitweb/lib/Gitweb/Util.pm\n\n-- \n1.7.3\n"},{"id":"160187","messageId":"1296579016-13356-2-git-send-email-jnareb@gmail.com","threadId":"26380","inReplyTo":"1296579016-13356-1-git-send-email-jnareb@gmail.com","subject":"[PATCH (version A) 1/2] gitweb: Prepare for splitting gitweb","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2011-02-01T16:50:13Z","receivedAt":"2011-02-01T16:50:13Z","isPatch":true,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"Prepare gitweb for having been split into modules that are to be\ninstalled alongside gitweb in 'lib/' subdirectory, by adding\n\n  use lib __DIR__.'/lib';\n\nto gitweb.perl (to main gitweb script), and preparing for putting\nmodules (relative path) in $(GITWEB_MODULES) in gitweb/Makefile.\n\nThis preparatory work allows to add new module to gitweb by simply\nadding\n\n  GITWEB_MODULES += <module>\n\nto gitweb/Makefile (assuming that the module is in 'gitweb/lib/'\ndirectory).\n\nWhile at it pass GITWEBLIBDIR in addition to GITWEB_TEST_INSTALLED to\nallow testing installed version of gitweb and installed version of\nmodules (for future tests which would check individual (sub)modules).\n\n\nUsing __DIR__ from Dir::Self module (not in core, that's why currently\ngitweb includes excerpt of code from Dir::Self defining __DIR__) was\nchosen over using FindBin-based solution (in core since perl 5.00307,\nwhile gitweb itself requires at least perl 5.8.0) because FindBin uses\nBEGIN block, which is a problem under mod_perl and other persistent\nPerl environments (thought there are workarounds).\n\nAt Pavan Kumar Sankara suggestion gitweb/Makefile uses\n\n  install [OPTION]... SOURCE... DIRECTORY\n\nformat (2nd format) with single SOURCE rather than\n\n  install [OPTION]... SOURCE DEST\n\nformat (1st format) because of security reasons (race conditions).\nModern GNU install has `-T' / `--no-target-directory' option, but we\ncannot rely that the $(INSTALL) we are using supports this option.\n\nThe install-modules target in gitweb/Makefile uses shell 'for' loop,\ninstead of make's $(foreach) function, to avoid possible problem with\ngenerating a command line that exceeded the maximum argument list\nlength.\n\nSigned-off-by: Jakub Narebski <jnareb@gmail.com>\n---\n gitweb/Makefile    |   17 +++++++++++++++--\n gitweb/gitweb.perl |    8 ++++++++\n 2 files changed, 23 insertions(+), 2 deletions(-)\n\ndiff --git a/gitweb/Makefile b/gitweb/Makefile\nindex 0a6ac00..e6029e1 100644\n--- a/gitweb/Makefile\n+++ b/gitweb/Makefile\n@@ -57,6 +57,7 @@ PERL_PATH  ?= /usr/bin/perl\n bindir_SQ = $(subst ','\\'',$(bindir))#'\n gitwebdir_SQ = $(subst ','\\'',$(gitwebdir))#'\n gitwebstaticdir_SQ = $(subst ','\\'',$(gitwebdir)/static)#'\n+gitweblibdir_SQ = $(subst ','\\'',$(gitwebdir)/lib)#'\n SHELL_PATH_SQ = $(subst ','\\'',$(SHELL_PATH))#'\n PERL_PATH_SQ  = $(subst ','\\'',$(PERL_PATH))#'\n DESTDIR_SQ    = $(subst ','\\'',$(DESTDIR))#'\n@@ -153,20 +154,32 @@ test:\n \n test-installed:\n \tGITWEB_TEST_INSTALLED='$(DESTDIR_SQ)$(gitwebdir_SQ)' \\\n+\tGITWEBLIBDIR='$(DESTDIR_SQ)$(gitweblibdir_SQ)' \\\n \t\t$(MAKE) -C ../t gitweb-test\n \n ### Installation rules\n \n-install: all\n+install: all install-modules\n \t$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(gitwebdir_SQ)'\n \t$(INSTALL) -m 755 $(GITWEB_PROGRAMS) '$(DESTDIR_SQ)$(gitwebdir_SQ)'\n \t$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(gitwebstaticdir_SQ)'\n \t$(INSTALL) -m 644 $(GITWEB_FILES) '$(DESTDIR_SQ)$(gitwebstaticdir_SQ)'\n \n+install-modules:\n+\tinstall_dirs=\"$(sort $(dir $(GITWEB_MODULES)))\" && \\\n+\tfor dir in $$install_dirs; do \\\n+\t\ttest -d '$(DESTDIR_SQ)$(gitweblibdir_SQ)'/\"$$dir\" || \\\n+\t\t$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(gitweblibdir_SQ)'/\"$$dir\"; \\\n+\tdone\n+\tgitweb_modules=\"$(GITWEB_MODULES)\" && \\\n+\tfor mod in $$gitweb_modules; do \\\n+\t\t$(INSTALL) -m 644 \"lib/$$mod\" '$(DESTDIR_SQ)$(gitweblibdir_SQ)'/\"$$(dirname $$mod)\"; \\\n+\tdone\n+\n ### Cleaning rules\n \n clean:\n \t$(RM) gitweb.cgi static/gitweb.min.js static/gitweb.min.css GITWEB-BUILD-OPTIONS\n \n-.PHONY: all clean install test test-installed .FORCE-GIT-VERSION-FILE FORCE\n+.PHONY: all clean install install-modules test test-installed .FORCE-GIT-VERSION-FILE FORCE\n \ndiff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl\nindex 1025c2f..1ccea49 100755\n--- a/gitweb/gitweb.perl\n+++ b/gitweb/gitweb.perl\n@@ -10,6 +10,14 @@\n use 5.008;\n use strict;\n use warnings;\n+\n+use File::Spec;\n+# __DIR__ is taken from Dir::Self __DIR__ fragment\n+sub __DIR__ () {\n+\tFile::Spec->rel2abs(join '', (File::Spec->splitpath(__FILE__))[0, 1]);\n+}\n+use lib __DIR__ . '/lib';\n+\n use CGI qw(:standard :escapeHTML -nosticky);\n use CGI::Util qw(unescape);\n use CGI::Carp qw(fatalsToBrowser set_message);\n-- \n1.6.5.GIT\n"},{"id":"160189","messageId":"1296579016-13356-3-git-send-email-jnareb@gmail.com","threadId":"26380","inReplyTo":"1296579016-13356-1-git-send-email-jnareb@gmail.com","subject":"[PATCH (version B) 1/2] gitweb: Prepare for splitting gitweb","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2011-02-01T16:50:14Z","receivedAt":"2011-02-01T16:50:14Z","isPatch":true,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"Prepare gitweb for being split into modules that would be installed\nin gitweblibdir, by default alongside gitweb in 'lib/' subdirectory,\nby adding\n\n  use lib $ENV{'GITWEBLIBDIR'} || \"++GITWEBLIBDIR++\";\n\nto gitweb.perl (to main gitweb script).  The first part allow to run\ngitweb scripts using source version of gitweb (after small change to\nt/gitweb-lib.sh).  The second part is set to appropriate value during\nbuild (generating gitweb.cgi).  This allows to select where to install\ngitweb modules via 'gitweblibdir' build time configuration variable,\ne.g.\n\n  $ make gitwebdir=/var/www/cgi-bin gitweblibdir=/usr/lib/perl5\n    install-gitweb\n\n\nThis preparatory work allows to add new module to gitweb by simply\nadding\n\n  GITWEB_MODULES += <module>\n\nto gitweb/Makefile (assuming that the module is in 'gitweb/lib/'\ndirectory).\n\nWhile at it pass GITWEBLIBDIR in addition to GITWEB_TEST_INSTALLED to\nallow testing installed version of gitweb and installed version of\nmodules (for future tests which would check individual (sub)modules).\n\n\nAt Pavan Kumar Sankara suggestion gitweb/Makefile uses\n\n  install [OPTION]... SOURCE... DIRECTORY\n\nformat (2nd format) with single SOURCE rather than\n\n  install [OPTION]... SOURCE DEST\n\nformat (1st format) because of security reasons (race conditions).\nModern GNU install has `-T' / `--no-target-directory' option, but we\ncannot rely that the $(INSTALL) we are using supports this option.\n\nThe install-modules target in gitweb/Makefile uses shell 'for' loop,\ninstead of make's $(foreach) function, to avoid possible problem with\ngenerating a command line that exceeded the maximum argument list\nlength.\n\nSigned-off-by: Jakub Narebski <jnareb@gmail.com>\n---\n gitweb/Makefile    |   19 +++++++++++++++++--\n gitweb/gitweb.perl |    4 ++++\n t/gitweb-lib.sh    |    2 ++\n 3 files changed, 23 insertions(+), 2 deletions(-)\n\ndiff --git a/gitweb/Makefile b/gitweb/Makefile\nindex 0a6ac00..abe9db8 100644\n--- a/gitweb/Makefile\n+++ b/gitweb/Makefile\n@@ -13,6 +13,7 @@ all::\n prefix ?= $(HOME)\n bindir ?= $(prefix)/bin\n gitwebdir ?= /var/www/cgi-bin\n+gitweblibdir ?= $(gitwebdir)/lib\n \n RM ?= rm -f\n INSTALL ?= install\n@@ -57,6 +58,7 @@ PERL_PATH  ?= /usr/bin/perl\n bindir_SQ = $(subst ','\\'',$(bindir))#'\n gitwebdir_SQ = $(subst ','\\'',$(gitwebdir))#'\n gitwebstaticdir_SQ = $(subst ','\\'',$(gitwebdir)/static)#'\n+gitweblibdir_SQ = $(subst ','\\'',$(gitweblibdir))#'\n SHELL_PATH_SQ = $(subst ','\\'',$(SHELL_PATH))#'\n PERL_PATH_SQ  = $(subst ','\\'',$(PERL_PATH))#'\n DESTDIR_SQ    = $(subst ','\\'',$(DESTDIR))#'\n@@ -115,6 +117,7 @@ GITWEB_FILES += static/git-logo.png static/git-favicon.png\n GITWEB_REPLACE = \\\n \t-e 's|++GIT_VERSION++|$(GIT_VERSION)|g' \\\n \t-e 's|++GIT_BINDIR++|$(bindir)|g' \\\n+\t-e 's|++GITWEBLIBDIR++|$(gitweblibdir)|g' \\\n \t-e 's|++GITWEB_CONFIG++|$(GITWEB_CONFIG)|g' \\\n \t-e 's|++GITWEB_CONFIG_SYSTEM++|$(GITWEB_CONFIG_SYSTEM)|g' \\\n \t-e 's|++GITWEB_HOME_LINK_STR++|$(GITWEB_HOME_LINK_STR)|g' \\\n@@ -153,20 +156,32 @@ test:\n \n test-installed:\n \tGITWEB_TEST_INSTALLED='$(DESTDIR_SQ)$(gitwebdir_SQ)' \\\n+\tGITWEBLIBDIR='$(DESTDIR_SQ)$(gitweblibdir_SQ)' \\\n \t\t$(MAKE) -C ../t gitweb-test\n \n ### Installation rules\n \n-install: all\n+install: all install-modules\n \t$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(gitwebdir_SQ)'\n \t$(INSTALL) -m 755 $(GITWEB_PROGRAMS) '$(DESTDIR_SQ)$(gitwebdir_SQ)'\n \t$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(gitwebstaticdir_SQ)'\n \t$(INSTALL) -m 644 $(GITWEB_FILES) '$(DESTDIR_SQ)$(gitwebstaticdir_SQ)'\n \n+install-modules:\n+\tinstall_dirs=\"$(sort $(dir $(GITWEB_MODULES)))\" && \\\n+\tfor dir in $$install_dirs; do \\\n+\t\ttest -d '$(DESTDIR_SQ)$(gitweblibdir_SQ)'/\"$$dir\" || \\\n+\t\t$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(gitweblibdir_SQ)'/\"$$dir\"; \\\n+\tdone\n+\tgitweb_modules=\"$(GITWEB_MODULES)\" && \\\n+\tfor mod in $$gitweb_modules; do \\\n+\t\t$(INSTALL) -m 644 \"lib/$$mod\" '$(DESTDIR_SQ)$(gitweblibdir_SQ)'/\"$$(dirname $$mod)\"; \\\n+\tdone\n+\n ### Cleaning rules\n \n clean:\n \t$(RM) gitweb.cgi static/gitweb.min.js static/gitweb.min.css GITWEB-BUILD-OPTIONS\n \n-.PHONY: all clean install test test-installed .FORCE-GIT-VERSION-FILE FORCE\n+.PHONY: all clean install install-modules test test-installed .FORCE-GIT-VERSION-FILE FORCE\n \ndiff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl\nindex 1025c2f..bed1eec 100755\n--- a/gitweb/gitweb.perl\n+++ b/gitweb/gitweb.perl\n@@ -10,6 +10,10 @@\n use 5.008;\n use strict;\n use warnings;\n+\n+use lib $ENV{'GITWEBLIBDIR'} || \"++GITWEBLIBDIR++\";\n+\n+use File::Spec;\n use CGI qw(:standard :escapeHTML -nosticky);\n use CGI::Util qw(unescape);\n use CGI::Carp qw(fatalsToBrowser set_message);\ndiff --git a/t/gitweb-lib.sh b/t/gitweb-lib.sh\nindex b9bb95f..8b98047 100644\n--- a/t/gitweb-lib.sh\n+++ b/t/gitweb-lib.sh\n@@ -48,6 +48,8 @@ EOF\n \t\tsay \"# Testing $SCRIPT_NAME\"\n \telse # normal case, use source version of gitweb\n \t\tSCRIPT_NAME=\"$GIT_BUILD_DIR/gitweb/gitweb.perl\"\n+\t\tGITWEBLIBDIR=\"$GIT_BUILD_DIR/gitweb/lib\"\n+\t\texport GITWEBLIBDIR\n \tfi\n \texport SCRIPT_NAME\n }\n-- \n1.6.5.GIT\n"},{"id":"160188","messageId":"1296579016-13356-4-git-send-email-jnareb@gmail.com","threadId":"26380","inReplyTo":"1296579016-13356-1-git-send-email-jnareb@gmail.com","subject":"[PATCH (version C) 1/2] gitweb: Prepare for splitting gitweb","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2011-02-01T16:50:15Z","receivedAt":"2011-02-01T16:50:15Z","isPatch":true,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"Prepare gitweb for being split into modules that would be installed\nin gitweblibdir, by default alongside gitweb in 'lib/' subdirectory.\n\nGitweb would search first in 'lib/' subdirectory from where it is\ninstalled, via\n\n  use lib __DIR__.'/lib';\n\n(This allow for tests to work with source version of gitweb without\nchanges.)  Then it searches in $(gitweblibdir) directory (set during\nbuild time), by default \"$(gitwebdir)/lib\", via\n\n  use lib \"++GITWEBLIBDIR++\";\n\n\"++GITWEBLIBDIR++\" is set to appropriate value during build time\n(generating gitweb.cgi).  This allows to select where to install\ngitweb modules via 'gitweblibdir' build time configuration variable,\nfor example\n\n  $ make gitwebdir=/var/www/cgi-bin gitweblibdir=/usr/lib/perl5\n    install-gitweb\n\n\nThis preparatory work allows to add new module to gitweb by simply\nadding\n\n  GITWEB_MODULES += <module>\n\nto gitweb/Makefile (assuming that the module is in 'gitweb/lib/'\ndirectory).\n\nWhile at it pass GITWEBLIBDIR in addition to GITWEB_TEST_INSTALLED to\nallow testing installed version of gitweb and installed version of\nmodules (for future tests which would check individual (sub)modules).\n\n\nUsing __DIR__ from Dir::Self module (not in core, that's why currently\ngitweb includes excerpt of code from Dir::Self defining __DIR__) was\nchosen over using FindBin-based solution (in core since perl 5.00307,\nwhile gitweb itself requires at least perl 5.8.0) because FindBin uses\nBEGIN block, which is a problem under mod_perl and other persistent\nPerl environments (thought there are workarounds).\n\nAt Pavan Kumar Sankara suggestion gitweb/Makefile uses\n\n  install [OPTION]... SOURCE... DIRECTORY\n\nformat (2nd format) with single SOURCE rather than\n\n  install [OPTION]... SOURCE DEST\n\nformat (1st format) because of security reasons (race conditions).\nModern GNU install has `-T' / `--no-target-directory' option, but we\ncannot rely that the $(INSTALL) we are using supports this option.\n\nThe install-modules target in gitweb/Makefile uses shell 'for' loop,\ninstead of make's $(foreach) function, to avoid possible problem with\ngenerating a command line that exceeded the maximum argument list\nlength.\n\nSigned-off-by: Jakub Narebski <jnareb@gmail.com>\n---\n gitweb/Makefile    |   19 +++++++++++++++++--\n gitweb/gitweb.perl |   10 ++++++++++\n 2 files changed, 27 insertions(+), 2 deletions(-)\n\ndiff --git a/gitweb/Makefile b/gitweb/Makefile\nindex 0a6ac00..abe9db8 100644\n--- a/gitweb/Makefile\n+++ b/gitweb/Makefile\n@@ -13,6 +13,7 @@ all::\n prefix ?= $(HOME)\n bindir ?= $(prefix)/bin\n gitwebdir ?= /var/www/cgi-bin\n+gitweblibdir ?= $(gitwebdir)/lib\n \n RM ?= rm -f\n INSTALL ?= install\n@@ -57,6 +58,7 @@ PERL_PATH  ?= /usr/bin/perl\n bindir_SQ = $(subst ','\\'',$(bindir))#'\n gitwebdir_SQ = $(subst ','\\'',$(gitwebdir))#'\n gitwebstaticdir_SQ = $(subst ','\\'',$(gitwebdir)/static)#'\n+gitweblibdir_SQ = $(subst ','\\'',$(gitweblibdir))#'\n SHELL_PATH_SQ = $(subst ','\\'',$(SHELL_PATH))#'\n PERL_PATH_SQ  = $(subst ','\\'',$(PERL_PATH))#'\n DESTDIR_SQ    = $(subst ','\\'',$(DESTDIR))#'\n@@ -115,6 +117,7 @@ GITWEB_FILES += static/git-logo.png static/git-favicon.png\n GITWEB_REPLACE = \\\n \t-e 's|++GIT_VERSION++|$(GIT_VERSION)|g' \\\n \t-e 's|++GIT_BINDIR++|$(bindir)|g' \\\n+\t-e 's|++GITWEBLIBDIR++|$(gitweblibdir)|g' \\\n \t-e 's|++GITWEB_CONFIG++|$(GITWEB_CONFIG)|g' \\\n \t-e 's|++GITWEB_CONFIG_SYSTEM++|$(GITWEB_CONFIG_SYSTEM)|g' \\\n \t-e 's|++GITWEB_HOME_LINK_STR++|$(GITWEB_HOME_LINK_STR)|g' \\\n@@ -153,20 +156,32 @@ test:\n \n test-installed:\n \tGITWEB_TEST_INSTALLED='$(DESTDIR_SQ)$(gitwebdir_SQ)' \\\n+\tGITWEBLIBDIR='$(DESTDIR_SQ)$(gitweblibdir_SQ)' \\\n \t\t$(MAKE) -C ../t gitweb-test\n \n ### Installation rules\n \n-install: all\n+install: all install-modules\n \t$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(gitwebdir_SQ)'\n \t$(INSTALL) -m 755 $(GITWEB_PROGRAMS) '$(DESTDIR_SQ)$(gitwebdir_SQ)'\n \t$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(gitwebstaticdir_SQ)'\n \t$(INSTALL) -m 644 $(GITWEB_FILES) '$(DESTDIR_SQ)$(gitwebstaticdir_SQ)'\n \n+install-modules:\n+\tinstall_dirs=\"$(sort $(dir $(GITWEB_MODULES)))\" && \\\n+\tfor dir in $$install_dirs; do \\\n+\t\ttest -d '$(DESTDIR_SQ)$(gitweblibdir_SQ)'/\"$$dir\" || \\\n+\t\t$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(gitweblibdir_SQ)'/\"$$dir\"; \\\n+\tdone\n+\tgitweb_modules=\"$(GITWEB_MODULES)\" && \\\n+\tfor mod in $$gitweb_modules; do \\\n+\t\t$(INSTALL) -m 644 \"lib/$$mod\" '$(DESTDIR_SQ)$(gitweblibdir_SQ)'/\"$$(dirname $$mod)\"; \\\n+\tdone\n+\n ### Cleaning rules\n \n clean:\n \t$(RM) gitweb.cgi static/gitweb.min.js static/gitweb.min.css GITWEB-BUILD-OPTIONS\n \n-.PHONY: all clean install test test-installed .FORCE-GIT-VERSION-FILE FORCE\n+.PHONY: all clean install install-modules test test-installed .FORCE-GIT-VERSION-FILE FORCE\n \ndiff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl\nindex 1025c2f..ea8ab56 100755\n--- a/gitweb/gitweb.perl\n+++ b/gitweb/gitweb.perl\n@@ -10,6 +10,16 @@\n use 5.008;\n use strict;\n use warnings;\n+\n+use File::Spec;\n+\n+# __DIR__ is excerpt from Dir::Self\n+sub __DIR__ () {\n+\tFile::Spec->rel2abs(join '', (File::Spec->splitpath(__FILE__))[0, 1]);\n+}\n+use lib __DIR__ . '/lib';\n+use lib \"++GITWEBLIBDIR++\";\n+\n use CGI qw(:standard :escapeHTML -nosticky);\n use CGI::Util qw(unescape);\n use CGI::Carp qw(fatalsToBrowser set_message);\n-- \n1.7.3\n"},{"id":"160190","messageId":"1296579016-13356-5-git-send-email-jnareb@gmail.com","threadId":"26380","inReplyTo":"1296579016-13356-1-git-send-email-jnareb@gmail.com","subject":"[PATCH (proof of concept) 2/2] gitweb: Create Gitweb::Util module","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2011-02-01T16:50:16Z","receivedAt":"2011-02-01T16:50:16Z","isPatch":true,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"From: Pavan Kumar Sunkara <pavan.sss1991@gmail.com>\n\nCreate a Gitweb::Util module, which is meant to contain internal\nutilities used by gitweb.  Currently it includes all the\nquoting/unquoting and escaping subroutines that are used by the\ngitweb.\n\nUpdate gitweb/Makefile to install Gitweb::Util module alongside gitweb\n\nSigned-off-by: Pavan Kumar Sunkara <pavan.sss1991@gmail.com>\nSigned-off-by: Jakub Narebski <jnareb@gmail.com>\n---\nThis patch serves two purposes.  First, it serves as test that earlier\n\"gitweb: Prepare for splitting gitweb\" patch acually work correctly.\n\nSecond, it might be good starting point to splitting gitweb.\nRefactoring well defined parts into separate modules (Perl packages)\ncould be a better, easier way than trying to come with good separation\n(split) into modules upfront.  Such leisure approach to splitting\ngitweb has more chance to be accepted.  Perhaps if such approach were\nproposed on GSoC 2010, maybe \"gitweb write\" project wouldn't fail\nmidterm evaluations...\n\n\nThis module was taken out of unfinished GSoC 2010 project with\nPavan Kumar Sunkara as a student\n\n  git://repo.or.cz/git/gsoc2010-gitweb.git\n\nThe module was renamed from Gitweb::Escape to Gitweb::Util.  Currently\nthe contents is the same, but it might change.\n\nCode was updated to more modern codebase; since then esc_path_info and\nesc_attr were added to gitweb - both of those are now in Gitweb::Util.\n\nThere were also required some changes and conflicts resolved due to\nthe fact that creating Gitweb::Util (formerly Gitweb::Escape) is no\nlonger in the middle of larger patch series.  In particular lack of\nGitweb::Config means that $fallback_encoding needed to be added to\nGitweb::Util module.\n\nWhile at it do not export quot_cec and quot_upr helper subroutines by\ndefault, but mark them exportable nevrtheless.\n\n gitweb/Makefile           |    3 +\n gitweb/gitweb.perl        |  140 +-----------------------------------\n gitweb/lib/Gitweb/Util.pm |  177 +++++++++++++++++++++++++++++++++++++++++++++\n 3 files changed, 183 insertions(+), 137 deletions(-)\n create mode 100644 gitweb/lib/Gitweb/Util.pm\n\ndiff --git a/gitweb/Makefile b/gitweb/Makefile\nindex abe9db8..9a4053b 100644\n--- a/gitweb/Makefile\n+++ b/gitweb/Makefile\n@@ -114,6 +114,9 @@ endif\n \n GITWEB_FILES += static/git-logo.png static/git-favicon.png\n \n+# Modules: Gitweb::*\n+GITWEB_MODULES += Gitweb/Util.pm\n+\n GITWEB_REPLACE = \\\n \t-e 's|++GIT_VERSION++|$(GIT_VERSION)|g' \\\n \t-e 's|++GIT_BINDIR++|$(bindir)|g' \\\ndiff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl\nindex ea8ab56..e8d8589 100755\n--- a/gitweb/gitweb.perl\n+++ b/gitweb/gitweb.perl\n@@ -23,11 +23,13 @@ use lib \"++GITWEBLIBDIR++\";\n use CGI qw(:standard :escapeHTML -nosticky);\n use CGI::Util qw(unescape);\n use CGI::Carp qw(fatalsToBrowser set_message);\n-use Encode;\n use Fcntl ':mode';\n use File::Find qw();\n use File::Basename qw(basename);\n use Time::HiRes qw(gettimeofday tv_interval);\n+\n+use Gitweb::Util;\n+\n binmode STDOUT, ':utf8';\n \n our $t0 = [ gettimeofday() ];\n@@ -1382,128 +1384,6 @@ sub validate_refname {\n \treturn $input;\n }\n \n-# decode sequences of octets in utf8 into Perl's internal form,\n-# which is utf-8 with utf8 flag set if needed.  gitweb writes out\n-# in utf-8 thanks to \"binmode STDOUT, ':utf8'\" at beginning\n-sub to_utf8 {\n-\tmy $str = shift;\n-\treturn undef unless defined $str;\n-\tif (utf8::valid($str)) {\n-\t\tutf8::decode($str);\n-\t\treturn $str;\n-\t} else {\n-\t\treturn decode($fallback_encoding, $str, Encode::FB_DEFAULT);\n-\t}\n-}\n-\n-# quote unsafe chars, but keep the slash, even when it's not\n-# correct, but quoted slashes look too horrible in bookmarks\n-sub esc_param {\n-\tmy $str = shift;\n-\treturn undef unless defined $str;\n-\t$str =~ s/([^A-Za-z0-9\\-_.~()\\/:@ ]+)/CGI::escape($1)/eg;\n-\t$str =~ s/ /\\+/g;\n-\treturn $str;\n-}\n-\n-# the quoting rules for path_info fragment are slightly different\n-sub esc_path_info {\n-\tmy $str = shift;\n-\treturn undef unless defined $str;\n-\n-\t# path_info doesn't treat '+' as space (specially), but '?' must be escaped\n-\t$str =~ s/([^A-Za-z0-9\\-_.~();\\/;:@&= +]+)/CGI::escape($1)/eg;\n-\n-\treturn $str;\n-}\n-\n-# quote unsafe chars in whole URL, so some characters cannot be quoted\n-sub esc_url {\n-\tmy $str = shift;\n-\treturn undef unless defined $str;\n-\t$str =~ s/([^A-Za-z0-9\\-_.~();\\/;?:@&= ]+)/CGI::escape($1)/eg;\n-\t$str =~ s/ /\\+/g;\n-\treturn $str;\n-}\n-\n-# quote unsafe characters in HTML attributes\n-sub esc_attr {\n-\n-\t# for XHTML conformance escaping '\"' to '&quot;' is not enough\n-\treturn esc_html(@_);\n-}\n-\n-# replace invalid utf8 character with SUBSTITUTION sequence\n-sub esc_html {\n-\tmy $str = shift;\n-\tmy %opts = @_;\n-\n-\treturn undef unless defined $str;\n-\n-\t$str = to_utf8($str);\n-\t$str = $cgi->escapeHTML($str);\n-\tif ($opts{'-nbsp'}) {\n-\t\t$str =~ s/ /&nbsp;/g;\n-\t}\n-\t$str =~ s|([[:cntrl:]])|(($1 ne \"\\t\") ? quot_cec($1) : $1)|eg;\n-\treturn $str;\n-}\n-\n-# quote control characters and escape filename to HTML\n-sub esc_path {\n-\tmy $str = shift;\n-\tmy %opts = @_;\n-\n-\treturn undef unless defined $str;\n-\n-\t$str = to_utf8($str);\n-\t$str = $cgi->escapeHTML($str);\n-\tif ($opts{'-nbsp'}) {\n-\t\t$str =~ s/ /&nbsp;/g;\n-\t}\n-\t$str =~ s|([[:cntrl:]])|quot_cec($1)|eg;\n-\treturn $str;\n-}\n-\n-# Make control characters \"printable\", using character escape codes (CEC)\n-sub quot_cec {\n-\tmy $cntrl = shift;\n-\tmy %opts = @_;\n-\tmy %es = ( # character escape codes, aka escape sequences\n-\t\t\"\\t\" => '\\t',   # tab            (HT)\n-\t\t\"\\n\" => '\\n',   # line feed      (LF)\n-\t\t\"\\r\" => '\\r',   # carrige return (CR)\n-\t\t\"\\f\" => '\\f',   # form feed      (FF)\n-\t\t\"\\b\" => '\\b',   # backspace      (BS)\n-\t\t\"\\a\" => '\\a',   # alarm (bell)   (BEL)\n-\t\t\"\\e\" => '\\e',   # escape         (ESC)\n-\t\t\"\\013\" => '\\v', # vertical tab   (VT)\n-\t\t\"\\000\" => '\\0', # nul character  (NUL)\n-\t);\n-\tmy $chr = ( (exists $es{$cntrl})\n-\t\t    ? $es{$cntrl}\n-\t\t    : sprintf('\\%2x', ord($cntrl)) );\n-\tif ($opts{-nohtml}) {\n-\t\treturn $chr;\n-\t} else {\n-\t\treturn \"<span class=\\\"cntrl\\\">$chr</span>\";\n-\t}\n-}\n-\n-# Alternatively use unicode control pictures codepoints,\n-# Unicode \"printable representation\" (PR)\n-sub quot_upr {\n-\tmy $cntrl = shift;\n-\tmy %opts = @_;\n-\n-\tmy $chr = sprintf('&#%04d;', 0x2400+ord($cntrl));\n-\tif ($opts{-nohtml}) {\n-\t\treturn $chr;\n-\t} else {\n-\t\treturn \"<span class=\\\"cntrl\\\">$chr</span>\";\n-\t}\n-}\n-\n # git may return quoted and escaped filenames\n sub unquote {\n \tmy $str = shift;\n@@ -1540,20 +1420,6 @@ sub unquote {\n \treturn $str;\n }\n \n-# escape tabs (convert tabs to spaces)\n-sub untabify {\n-\tmy $line = shift;\n-\n-\twhile ((my $pos = index($line, \"\\t\")) != -1) {\n-\t\tif (my $count = (8 - ($pos % 8))) {\n-\t\t\tmy $spaces = ' ' x $count;\n-\t\t\t$line =~ s/\\t/$spaces/;\n-\t\t}\n-\t}\n-\n-\treturn $line;\n-}\n-\n sub project_in_list {\n \tmy $project = shift;\n \tmy @list = git_get_projects_list();\ndiff --git a/gitweb/lib/Gitweb/Util.pm b/gitweb/lib/Gitweb/Util.pm\nnew file mode 100644\nindex 0000000..a213d3f\n--- /dev/null\n+++ b/gitweb/lib/Gitweb/Util.pm\n@@ -0,0 +1,177 @@\n+# Gitweb::Util -- Internal utilities used by gitweb (git web interface)\n+#\n+# This module is licensed under the GPLv2\n+\n+package Gitweb::Util;\n+\n+use strict;\n+use warnings;\n+use Exporter qw(import);\n+\n+our @EXPORT = qw(to_utf8\n+                 esc_param esc_path_info esc_url\n+                 esc_html esc_path esc_attr\n+                 untabify\n+                 $fallback_encoding);\n+our @EXPORT_OK = qw(quot_cec quot_upr);\n+\n+use Encode;\n+use CGI;\n+\n+# ......................................................................\n+# Perl encoding (utf-8)\n+\n+# decode sequences of octets in utf8 into Perl's internal form,\n+# which is utf-8 with utf8 flag set if needed.  gitweb writes out\n+# in utf-8 thanks to \"binmode STDOUT, ':utf8'\" at beginning of gitweb.perl\n+our $fallback_encoding = 'latin1';\n+sub to_utf8 {\n+\tmy $str = shift;\n+\treturn undef unless defined $str;\n+\tif (utf8::valid($str)) {\n+\t\tutf8::decode($str);\n+\t\treturn $str;\n+\t} else {\n+\t\treturn decode($fallback_encoding, $str, Encode::FB_DEFAULT);\n+\t}\n+}\n+\n+# ......................................................................\n+# CGI encoding\n+\n+# quote unsafe chars, but keep the slash, even when it's not\n+# correct, but quoted slashes look too horrible in bookmarks\n+sub esc_param {\n+\tmy $str = shift;\n+\treturn undef unless defined $str;\n+\n+\t$str =~ s/([^A-Za-z0-9\\-_.~()\\/:@ ]+)/CGI::escape($1)/eg;\n+\t$str =~ s/ /\\+/g;\n+\n+\treturn $str;\n+}\n+\n+# the quoting rules for path_info fragment are slightly different\n+sub esc_path_info {\n+\tmy $str = shift;\n+\treturn undef unless defined $str;\n+\n+\t# path_info doesn't treat '+' as space (specially), but '?' must be escaped\n+\t$str =~ s/([^A-Za-z0-9\\-_.~();\\/;:@&= +]+)/CGI::escape($1)/eg;\n+\n+\treturn $str;\n+}\n+\n+# quote unsafe chars in whole URL, so some characters cannot be quoted\n+sub esc_url {\n+\tmy $str = shift;\n+\treturn undef unless defined $str;\n+\n+\t$str =~ s/([^A-Za-z0-9\\-_.~();\\/;?:@&= ]+)/CGI::escape($1)/eg;\n+\t$str =~ s/ /\\+/g;\n+\n+\treturn $str;\n+}\n+\n+# ......................................................................\n+# (X)HTML escaping\n+\n+# replace invalid utf8 character with SUBSTITUTION sequence\n+sub esc_html {\n+\tmy $str = shift;\n+\tmy %opts = @_;\n+\n+\treturn undef unless defined $str;\n+\n+\t$str = to_utf8($str);\n+\t$str = CGI::escapeHTML($str);\n+\tif ($opts{'-nbsp'}) {\n+\t\t$str =~ s/ /&nbsp;/g;\n+\t}\n+\t$str =~ s|([[:cntrl:]])|(($1 ne \"\\t\") ? quot_cec($1) : $1)|eg;\n+\treturn $str;\n+}\n+\n+# quote unsafe characters in HTML attributes\n+sub esc_attr {\n+\n+\t# for XHTML conformance escaping '\"' to '&quot;' is not enough\n+\treturn esc_html(@_);\n+}\n+\n+# quote control characters and escape filename to HTML\n+sub esc_path {\n+\tmy $str = shift;\n+\tmy %opts = @_;\n+\n+\treturn undef unless defined $str;\n+\n+\t$str = to_utf8($str);\n+\t$str = CGI::escapeHTML($str);\n+\tif ($opts{'-nbsp'}) {\n+\t\t$str =~ s/ /&nbsp;/g;\n+\t}\n+\t$str =~ s|([[:cntrl:]])|quot_cec($1)|eg;\n+\treturn $str;\n+}\n+\n+# ......................................................................\n+# Other\n+\n+# escape tabs (convert tabs to spaces)\n+sub untabify {\n+\tmy $line = shift;\n+\n+\twhile ((my $pos = index($line, \"\\t\")) != -1) {\n+\t\tif (my $count = (8 - ($pos % 8))) {\n+\t\t\tmy $spaces = ' ' x $count;\n+\t\t\t$line =~ s/\\t/$spaces/;\n+\t\t}\n+\t}\n+\n+\treturn $line;\n+}\n+\n+# ----------------------------------------------------------------------\n+# Showing \"unprintable\" characters (utility functions)\n+\n+# Make control characters \"printable\", using character escape codes (CEC)\n+sub quot_cec {\n+\tmy $cntrl = shift;\n+\tmy %opts = @_;\n+\tmy %es = ( # character escape codes, aka escape sequences\n+\t\t\"\\t\" => '\\t',   # tab            (HT)\n+\t\t\"\\n\" => '\\n',   # line feed      (LF)\n+\t\t\"\\r\" => '\\r',   # carrige return (CR)\n+\t\t\"\\f\" => '\\f',   # form feed      (FF)\n+\t\t\"\\b\" => '\\b',   # backspace      (BS)\n+\t\t\"\\a\" => '\\a',   # alarm (bell)   (BEL)\n+\t\t\"\\e\" => '\\e',   # escape         (ESC)\n+\t\t\"\\013\" => '\\v', # vertical tab   (VT)\n+\t\t\"\\000\" => '\\0', # nul character  (NUL)\n+\t);\n+\tmy $chr = ( (exists $es{$cntrl})\n+\t\t    ? $es{$cntrl}\n+\t\t    : sprintf('\\%2x', ord($cntrl)) );\n+\tif ($opts{-nohtml}) {\n+\t\treturn $chr;\n+\t} else {\n+\t\treturn \"<span class=\\\"cntrl\\\">$chr</span>\";\n+\t}\n+}\n+\n+# Alternatively use unicode control pictures codepoints,\n+# Unicode \"printable representation\" (PR)\n+sub quot_upr {\n+\tmy $cntrl = shift;\n+\tmy %opts = @_;\n+\n+\tmy $chr = sprintf('&#%04d;', 0x2400+ord($cntrl));\n+\tif ($opts{-nohtml}) {\n+\t\treturn $chr;\n+\t} else {\n+\t\treturn \"<span class=\\\"cntrl\\\">$chr</span>\";\n+\t}\n+}\n+\n+1;\n-- \n1.7.3\n"},{"id":"160284","messageId":"4D49AB9C.2050307@mit.edu","threadId":"26380","inReplyTo":"1296579016-13356-4-git-send-email-jnareb@gmail.com","subject":"Re: [PATCH (version C) 1/2] gitweb: Prepare for splitting gitweb","fromName":"Alejandro R. Sedeño","fromEmail":"asedeno@mit.edu","sentAt":"2011-02-02T19:08:12Z","receivedAt":"2011-02-02T19:08:12Z","isPatch":true,"sender":{"key":"asedeno@mit.edu","avatar":"https://avatars.githubusercontent.com/u/28302?v=4"},"body":"On 2/1/2011 11:50 AM, Jakub Narebski wrote:\n> Prepare gitweb for being split into modules that would be installed\n> in gitweblibdir, by default alongside gitweb in 'lib/' subdirectory.\n> \n> Gitweb would search first in 'lib/' subdirectory from where it is\n> installed, via\n> \n>   use lib __DIR__.'/lib';\n> \n> (This allow for tests to work with source version of gitweb without\n> changes.)  Then it searches in $(gitweblibdir) directory (set during\n> build time), by default \"$(gitwebdir)/lib\", via\n> \n>   use lib \"++GITWEBLIBDIR++\";\n> \n> \"++GITWEBLIBDIR++\" is set to appropriate value during build time\n> (generating gitweb.cgi).  This allows to select where to install\n> gitweb modules via 'gitweblibdir' build time configuration variable\n\nI would personally prefer to see this path taken, as it seems the most\nflexible and would fulfill a use case I have.\n\nI maintain a build of git in an AFS volume at MIT. One of its uses is\nsymlinking to the current gitweb.cgi to instantly deploy a gitweb in a\nshared hosting environment (example: http://git.scripts.mit.edu/).\n\n__DIR__ would point to the directory containing a user symlink to\ngitweb, which would allow users to add their own libraries, while\n++GITWEBLIBDIR++ would allow the standard gitweb libraries to be hosted\nat a common path without placing additional burdens on the user at\nupgrade time.\n\n-Alejandro\n"},{"id":"160916","messageId":"201102120021.58250.jnareb@gmail.com","threadId":"26380","inReplyTo":"4D49AB9C.2050307@mit.edu","subject":"Re: [PATCH (version C) 1/2] gitweb: Prepare for splitting gitweb","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2011-02-11T23:21:56Z","receivedAt":"2011-02-11T23:21:56Z","isPatch":true,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"On Wed, 2 Feb 2011, Alejandro R. Sedeño wrote:\n> On 2/1/2011 11:50 AM, Jakub Narebski wrote:\n> >\n> > Prepare gitweb for being split into modules that would be installed\n> > in gitweblibdir, by default alongside gitweb in 'lib/' subdirectory.\n> > \n> > Gitweb would search first in 'lib/' subdirectory from where it is\n> > installed, via\n> > \n> >   use lib __DIR__.'/lib';\n> > \n> > (This allow for tests to work with source version of gitweb without\n> > changes.)  Then it searches in $(gitweblibdir) directory (set during\n> > build time), by default \"$(gitwebdir)/lib\", via\n> > \n> >   use lib \"++GITWEBLIBDIR++\";\n> > \n> > \"++GITWEBLIBDIR++\" is set to appropriate value during build time\n> > (generating gitweb.cgi).  This allows to select where to install\n> > gitweb modules via 'gitweblibdir' build time configuration variable\n> \n> I would personally prefer to see this path taken, as it seems the most\n> flexible and would fulfill a use case I have.\n> \n> I maintain a build of git in an AFS volume at MIT. One of its uses is\n> symlinking to the current gitweb.cgi to instantly deploy a gitweb in a\n> shared hosting environment (example: http://git.scripts.mit.edu/).\n> \n> __DIR__ would point to the directory containing a user symlink to\n> gitweb, which would allow users to add their own libraries, while\n> ++GITWEBLIBDIR++ would allow the standard gitweb libraries to be\n> hosted at a common path without placing additional burdens on the\n> user at upgrade time.\n\nNote however that if you want __DIR__ version to take preference over\ngitweblibdir version, then the 'use lib' statements have to be in \nreverse order, i.e.:\n\n  +use lib \"++GITWEBLIBDIR++\";\n  +use lib __DIR__ . '/lib';\n\n-- \nJakub Narebski\nPoland\n"}]}