{"thread":{"id":"23758","subject":"Re: gitweb, FastCGI and PSGI/Plack","startedAt":"2010-05-09T23:05:48Z","lastAt":"2010-05-11T18:56:43Z","messageCount":14,"participants":["Jakub Narebski","Tatsuhiko Miyagawa","Peter Vereshagin"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"141340","messageId":"201005100105.49985.jnareb@gmail.com","threadId":"23758","inReplyTo":"g2s693254b91005091428ib188cbd1le5ffa90eace741a8@mail.gmail.com","subject":"Re: gitweb, FastCGI and PSGI/Plack","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2010-05-09T23:05:48Z","receivedAt":"2010-05-09T23:05:48Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"On Sun, 9 May 2010, Tatsuhiko Miyagawa wrote:\n\n> I'm not subscribed to the git mailing list but:\n> \n> http://www.spinics.net/lists/git/msg131014.html\n\nYou don't need to be subscribed to post to git mailing list.  I personally\nread it with a news reader using GMane NNTP interface\n\n  nntp://news.gmane.org/gmane.comp.version-control.git\n\nbut sending messages via email, to git@ver.kernel.org.  Usually people\nreply both to original poster and to git mailing list, but just in case\nyou can mention that you are not subscribed, and to not forget to send\nyou a CC copy.\n\n\nAll CC re-added.\n\n> \n> > Actually both CGI and CGI::Fast are in Perl core distribution since\n> > perl 5.004 (Perl 5.4.0).  I assume that CGI::Fast simply degrades to CGI\n> > if FCGI module is not present.\n> \n> No, it just dies if FCGI is not installed\n\nAh, so that is why CGI distribution has FCGI as requirement!\n\n  http://deps.cpantesters.org/?module=CGI;perl=latest\n\n> (so CGI::Fast in core doesn't make any sense).\n\nYes, it doesn't.\n\nNevertheless gitweb would try to require CGI::Fast only when it is run\nwith '--fastcgi' / '--fcgi' / '-f' option.  It should probably check if\nit can be require'd without errors, like it is done for FCGI::ProcManager\nin\n  [RFC/PATCHv2 2/2] gitweb: Add support for FastCGI, using CGI::Fast\n  Message-Id: <201005080959.01800.jnareb@gmail.com>\n  http://article.gmane.org/gmane.comp.version-control.git/146647\n\n> \n> > JN> Yes, it can.  CGI::Compile is used for example by CGI::Emulate::PSGI,\n> > JN> and you can run PSGI app on standalone Perl web server (pure Perl\n> > JN> HTTP::Server::PSGI, or HTTP::Server::Simple::PSGI which in turn uses\n> > JN> HTTP::Server::Simple, or Starman, or Twiggy, or Perlbal)\n> \n> And FastCGI.\n> \n> I don't understand why you implemented FastCGI interface *in addition\n> to* PSGI/Plack interface.\n\nI didn't implement support for PSGI/Plack, at least not in patch send in\n\"[PATCH 0/2] gitweb: Add support for running gitweb as FastCGI script\"\nseries.\n\nAdding support for FastCGI to CGI application is as simple as using\nCGI::Fast object in place of CGI, and wrapping running main subroutine\nin a loop that processes requests, like described in CGI::Fast manpage.\nWell, for gitweb it required a few more small changes, anyway...\n\nMoving to PSGI, or adding possibility to run gitweb as PSGI script (like\nthe series adds *ability* to run gitweb as FastCGI script) by modifying\ngitweb would not be that easy, even with help of CGI::PSGI.\n\n\nCGI::Compile was referring to an alternate approach, where instead of\nmodifying gitweb to be able to run it as FastCGI script (you can run it\nas CGI script and as ModPerl::Registry script from mod_perl) there was\nadded gitweb.fcgi wrapper:\n\nJN> The alternate solution would be to add gitweb.fcgi wrapper, like e.g.:\nJN> in the following patch by Eric Wong\nJN> \nJN>  \"[PATCH 1/2] gitweb: add a simple wrapper for FCGI support\"\nJN>  http://thread.gmane.org/gmane.comp.version-control.git/35920/focus=35921\nJN>\nJN> which was part of the \"[0/2 PATCH] FastCGI and nginx support for gitweb\"\nJN> series.  (Note that the patch does 'do $gitweb_cgi;' without checking for\nJN> errors, see the bottom of `perldoc -f do` documentation on how it should\nJN> be done).\n\nOne can of course use this approach wrapping gitweb to be run on PSGI,\nusing CGI::Emulate::PSGI (via Plack::App::WrapCGI), which in turn uses\nCGI::Compile.  The gitweb.fcgi wrapper could use CGI::Emulate::FCGI...\nif it existed.\n\nNB I use the following gitweb.psgi wrapper to run gitweb from \"plackup\"\nfor tests (after running \"make gitweb\", of course, and with appropriate\ngitweb_config.perl, unversioned, in gitweb/ alongside gitweb.perl, the\ngitweb.psgi wrapper, and generated gitweb.cgi):\n\n-- 8< --\n#!/usr/bin/env plackup\n\n# gitweb - simple web interface to track changes in git repositories\n#          PSGI wrapper (see http://plackperl.org)\n\nuse strict;\nuse warnings;\n\nuse Plack::Builder;\nuse Plack::App::WrapCGI;\nuse CGI::Emulate::PSGI 0.07; # minimum version required to work\n\nuse File::Spec;\n# __DIR__ is taken from Dir::Self __DIR__ fragment\nsub __DIR__ () {\n\tFile::Spec->rel2abs(join '', (File::Spec->splitpath(__FILE__))[0, 1]);\n}\n\nbuilder {\n\tenable 'Static',\n\t\tpath => sub { m!\\.(js|css|png)$! && s!^/gitweb/!! }, root => __DIR__.\"/\";\n\tPlack::App::WrapCGI->new(script => __DIR__.\"/gitweb.cgi\")->to_app;\n}\n\n__END__\n-- >8 --\n\n> \n> `plackup -s FCGI` makes your PSGI app a fastcgi handler using FCGI.pm,\n> or `plackup -s Net::FastCGI` does the same but using Net::FastCGI,\n> pure perl alternative.\n\nIt is a pity that Plack::App::WrapFCGI / FCGI::Emulate::PSGI does not\nexist, so that gitweb.psgi wrapper would not require indirectly \nCGI::Compile.\n\n> \n> Come over to #plack (irc://irc.perl.org/#plack) for more discussion.\n\nEmail not good?\n\n-- \nJakub Narebski\nShadeHawk on #git, jnareb on #plack\nPoland\n"},{"id":"141342","messageId":"t2n693254b91005091759ye577992y32d2b6a41d7d8c45@mail.gmail.com","threadId":"23758","inReplyTo":"201005100105.49985.jnareb@gmail.com","subject":"Re: gitweb, FastCGI and PSGI/Plack","fromName":"Tatsuhiko Miyagawa","fromEmail":"miyagawa@gmail.com","sentAt":"2010-05-10T00:59:11Z","receivedAt":"2010-05-10T00:59:11Z","isPatch":false,"sender":{"key":"miyagawa@gmail.com","avatar":"https://gravatar.com/avatar/951a9e8b1dbd5cae0c39f3ffde3cb20813149cc5b48ffcea6be1955618408cce?d=mp&s=160"},"body":"On Sun, May 9, 2010 at 4:05 PM, Jakub Narebski <jnareb@gmail.com> wrote:\n\n> Moving to PSGI, or adding possibility to run gitweb as PSGI script (like\n> the series adds *ability* to run gitweb as FastCGI script) by modifying\n> gitweb would not be that easy, even with help of CGI::PSGI.\n\nYou don't need it - since you can just use Plack::App::WrapCGI.\n>\n> CGI::Compile was referring to an alternate approach, where instead of\n> modifying gitweb to be able to run it as FastCGI script (you can run it\n> as CGI script and as ModPerl::Registry script from mod_perl) there was\n> added gitweb.fcgi wrapper:\n\nand the .fcgi wrapper can just use Plack::Loader, or the plackup\nexecutable with FCGI environment variable set, to DWIM.\n\n> One can of course use this approach wrapping gitweb to be run on PSGI,\n> using CGI::Emulate::PSGI (via Plack::App::WrapCGI), which in turn uses\n> CGI::Compile.  The gitweb.fcgi wrapper could use CGI::Emulate::FCGI...\n> if it existed.\n\nAgain, you still don't understand - once your CGI script is turned\ninto PSGI, plackup can take over the web server interface, including\nthe FastCGI interface. If you need .fcgi wrapper to be spawned from\nweb servers, just put plackup command line call (or Plack::Runner or\n::Loader) in the .fcgi script.\n\n> NB I use the following gitweb.psgi wrapper to run gitweb from \"plackup\"\n> for tests (after running \"make gitweb\", of course, and with appropriate\n> gitweb_config.perl, unversioned, in gitweb/ alongside gitweb.perl, the\n> gitweb.psgi wrapper, and generated gitweb.cgi):\n\n>> `plackup -s FCGI` makes your PSGI app a fastcgi handler using FCGI.pm,\n>> or `plackup -s Net::FastCGI` does the same but using Net::FastCGI,\n>> pure perl alternative.\n>\n> It is a pity that Plack::App::WrapFCGI / FCGI::Emulate::PSGI does not\n> exist, so that gitweb.psgi wrapper would not require indirectly\n> CGI::Compile.\n\nit's not a pity - it's the simplification by wrapping CGI environment\ninto PSGI, which is a pure perl web server interface that then can be\nturned into ANY web server handlers including standalone, CGI,\nFastCGI, mod_perl and SCGI.\n\n\n\n\n-- \nTatsuhiko Miyagawa\n"},{"id":"141343","messageId":"x2k693254b91005091805re0f63ac5z471e05fa4ab71ca7@mail.gmail.com","threadId":"23758","inReplyTo":"201005100105.49985.jnareb@gmail.com","subject":"Re: gitweb, FastCGI and PSGI/Plack","fromName":"Tatsuhiko Miyagawa","fromEmail":"miyagawa@gmail.com","sentAt":"2010-05-10T01:05:04Z","receivedAt":"2010-05-10T01:05:04Z","isPatch":false,"sender":{"key":"miyagawa@gmail.com","avatar":"https://gravatar.com/avatar/951a9e8b1dbd5cae0c39f3ffde3cb20813149cc5b48ffcea6be1955618408cce?d=mp&s=160"},"body":"On Sun, May 9, 2010 at 4:05 PM, Jakub Narebski <jnareb@gmail.com> wrote:\n\n>> `plackup -s FCGI` makes your PSGI app a fastcgi handler using FCGI.pm,\n>> or `plackup -s Net::FastCGI` does the same but using Net::FastCGI,\n>> pure perl alternative.\n>\n> It is a pity that Plack::App::WrapFCGI / FCGI::Emulate::PSGI does not\n> exist, so that gitweb.psgi wrapper would not require indirectly\n> CGI::Compile.\n\nI *think* we discussed about this on IRC, but you still seems to be\nmisunderstanding:\n\nYou have a CGI script and you want to turn it into a PSGI application,\nhence we have CGI::Emulate::PSGI and CGI::Compile.\n\nYou usually do not have a FCGI \"application\". You're writing a .fcgi\n\"wrapper\" to make your CGI script runnable from a web server (like\nyou're doing with gitweb.fcgi).\n\nWriting an FCGI emulation layer for PSGI would allow you to run the\nFCGI wrapper from PSGI compatible web server - which does not make\nsense AT ALLl. It's even one more indirection.\n\n\n--\nTatsuhiko Miyagawa\n"},{"id":"141377","messageId":"201005101232.32908.jnareb@gmail.com","threadId":"23758","inReplyTo":"x2k693254b91005091805re0f63ac5z471e05fa4ab71ca7@mail.gmail.com","subject":"Re: gitweb, FastCGI and PSGI/Plack","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2010-05-10T10:32:30Z","receivedAt":"2010-05-10T10:32:30Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"On Mon, May 10, 2010, Tatsuhiko Miyagawa wrote:\n> On Sun, May 9, 2010 at 4:05 PM, Jakub Narebski <jnareb@gmail.com> wrote:\n>>> \n>>> `plackup -s FCGI` makes your PSGI app a fastcgi handler using FCGI.pm,\n>>> or `plackup -s Net::FastCGI` does the same but using Net::FastCGI,\n>>> pure perl alternative.\n>>\n>> It is a pity that Plack::App::WrapFCGI / FCGI::Emulate::PSGI does not\n>> exist, so that gitweb.psgi wrapper would not require indirectly\n>> CGI::Compile.\n> \n> I *think* we discussed about this on IRC, but you still seems to be\n> misunderstanding:\n\nI'm sorry for the confusion; I didn't make myself clear (and there also\nwas some misunderstanding of technologies involved on my side).\n\n> \n> You have a CGI script and you want to turn it into a PSGI application,\n> hence we have CGI::Emulate::PSGI and CGI::Compile.\n> \n> You usually do not have a FCGI \"application\". You're writing a .fcgi\n> \"wrapper\" to make your CGI script runnable from a web server (like\n> you're doing with gitweb.fcgi).\n\nAfter thinking about it a bit, I realized that I don't want to have\nPlack::App::WrapFCGI wrapper (which if there are no FastCGI-only Perl\nweb apps, e.g. using FCGI directly and which do not have support for\nrunning as ordinary CGI would be totally unnecessary), but I want to\navoid price of using CGI::Compile.\n\n>From what I understand Plack::App::WrapCGI does something like that\n(example taken from CGI::Compile manpage):\n\n   use CGI::Emulate::PSGI;\n   use CGI::Compile;\n\n   my $cgi_script = \"/path/to/foo.cgi\";\n   my $sub = CGI::Compile->compile($cgi_script);\n   my $app = CGI::Emulate::PSGI->handler($sub);\n\n   # $app is a PSGI application\n\n\nA typical application that uses CGI::Fast to provide support for running\nas FastCGI script looks like this, according to CGI::Fast manpage \n(the example is slightly modified):\n\n   use CGI::Fast;\n\n   do_some_initialization();\n\n   while ($q = new CGI::Fast) {\n      process_request();  # it uses 'my $cgi = CGI->new()' inside\n   }\n\nI'd like to use the fact that per-request part is separated from\ninitialization part in wrapper for PSGI.\n\n  use CGI::Emulate::PSGI;\n  use CGI::Fast::Loader; # or something like that\n\n  my $fcgi_script = \"/path/to/foo.fcgi\";\n  my $fcgi = CGI::Fast::Loader->load($fcgi_script);\n  $fcgi->import(qw(do_some_initialization process_request));\n\n  do_some_initialization();\n  my $app = CGI::Emulate::PSGI->handler(\\&process_request);\n\n   # $app is a PSGI application\n\n\nThat is of course heavy handwaving, and I am not sure if it is something\nthat can be generalized for scripts that use FCGI module (and \nFCGI::Request) directly.\n\n> \n> Writing an FCGI emulation layer for PSGI would allow you to run the\n> FCGI wrapper from PSGI compatible web server - which does not make\n> sense AT ALL!. It's even one more indirection.\n\nWell, it would allow to run FastCGI application in *any* PSGI compatibile\nweb server, including standalone (HTTP::Server::PSGI), mod_perl and\nTest::Plack 'server'.\n\nAlso even if wrapping FCGI application as PSGI application to run it\non FCGI server doesn't make much sense on first glance, it neverheless\nallows to use many, many Plack::Middleware::*.\n\nMind you, I don't think that there are many FastCGI-only Perl web apps...\nif there are any.\n\n\nI hope that clarify what I want (and no, I guess FCGI::Emulate::PSGI\nain't it).\n-- \nJakub Narebski\nPoland\n"},{"id":"141407","messageId":"201005101826.10735.jnareb@gmail.com","threadId":"23758","inReplyTo":"t2n693254b91005091759ye577992y32d2b6a41d7d8c45@mail.gmail.com","subject":"Re: gitweb, FastCGI and PSGI/Plack","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2010-05-10T16:26:07Z","receivedAt":"2010-05-10T16:26:07Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"On Sun, May 9, 2010 at 17:59:11 -0700, Tatsuhiko Miyagawa wrote:\n> On Sun, May 9, 2010 at 4:05 PM, Jakub Narebski <jnareb@gmail.com> wrote:\n> \n> > Moving to PSGI, or adding possibility to run gitweb as PSGI script (like\n> > the series adds *ability* to run gitweb as FastCGI script) by modifying\n> > gitweb would not be that easy, even with help of CGI::PSGI.\n> \n> You don't need it - since you can just use Plack::App::WrapCGI.\n\nIt is simplest method, but I don't think it is best one from the\nperformance and especially latency point of view.  \n\nAs I understand it, CGI::Compile catches all STDOUT output of compiled\nCGI script, and passes it to PSGI layer (PSGI interface).  That doesn't\nallow, I think, for serving request as it is generated.  This might be\nimportant for actions (pages) such as 'snapshot', 'blob_plain' and\n'patches', which simply dump output of external comand to STDOUT.  If\ngitweb was written as PSGI app, or with PSGI-compliant web framework, \nit would return IO::Handle-like object.\n\nAlso I think that currently you can currently see in web browser page as\nit is being generated by gitweb (e.g. 'projects_list' page).  This is\nimpossible (I guess) with CGI::Compile.\n\n\nThere is also a problem that there is no way, I guess, to automatically\nreload / refresh app if underlying CGI script changes (a la \"plackup -r\").\n\n> > CGI::Compile was referring to an alternate approach, where instead of\n> > modifying gitweb to be able to run it as FastCGI script (you can run it\n> > as CGI script and as ModPerl::Registry script from mod_perl) there was\n> > added gitweb.fcgi wrapper:\n> \n> and the .fcgi wrapper can just use Plack::Loader, or the plackup\n> executable with FCGI environment variable set, to DWIM.\n\nRight, having gitweb.psgi PSGI wrapper, which can be run not only via\nFCGI, but also on many other web servers, is certainly superior to\nhaving gitweb.fcgi FastCGI wrapper, which allows to run gitweb as\nFastCGI script (on FCGI server).\n \n> > One can of course use this approach wrapping gitweb to be run on PSGI,\n> > using CGI::Emulate::PSGI (via Plack::App::WrapCGI), which in turn uses\n> > CGI::Compile.  The gitweb.fcgi wrapper could use CGI::Emulate::FCGI...\n> > if it existed.\n> \n> Again, you still don't understand - once your CGI script is turned\n> into PSGI, plackup can take over the web server interface, including\n> the FastCGI interface. If you need .fcgi wrapper to be spawned from\n> web servers, just put plackup command line call (or Plack::Runner or\n> ::Loader) in the .fcgi script.\n\nI didn't made myself clear here.\n\nWhat I want, like I wrote in neighbour subthread, is for the FastCGI app\nthat looks like this:\n\n   use CGI::Fast;\n\n   do_some_initialization();\n\n   while ($q = new CGI::Fast) {\n      process_request();  # it uses 'my $cgi = CGI->new()' inside\n   }\n\nto make PSGI application that runs do_some_initialization() only once,\nand returns converted-to-PSGI process_request() as PSGI subroutine (as\nPSGI $app).\n\n-- \nJakub Narebski\njnareb on #plack\nPoland\n"},{"id":"141410","messageId":"AANLkTikH39rbm64sc0nn7eTBJSfxcqWpz6VXZBCQQIl3@mail.gmail.com","threadId":"23758","inReplyTo":"201005101232.32908.jnareb@gmail.com","subject":"Re: gitweb, FastCGI and PSGI/Plack","fromName":"Tatsuhiko Miyagawa","fromEmail":"miyagawa@gmail.com","sentAt":"2010-05-10T17:03:44Z","receivedAt":"2010-05-10T17:03:44Z","isPatch":false,"sender":{"key":"miyagawa@gmail.com","avatar":"https://gravatar.com/avatar/951a9e8b1dbd5cae0c39f3ffde3cb20813149cc5b48ffcea6be1955618408cce?d=mp&s=160"},"body":"On Mon, May 10, 2010 at 3:32 AM, Jakub Narebski <jnareb@gmail.com> wrote:\n>> You have a CGI script and you want to turn it into a PSGI application,\n>> hence we have CGI::Emulate::PSGI and CGI::Compile.\n>>\n>> You usually do not have a FCGI \"application\". You're writing a .fcgi\n>> \"wrapper\" to make your CGI script runnable from a web server (like\n>> you're doing with gitweb.fcgi).\n>\n> After thinking about it a bit, I realized that I don't want to have\n> Plack::App::WrapFCGI wrapper (which if there are no FastCGI-only Perl\n> web apps, e.g. using FCGI directly and which do not have support for\n> running as ordinary CGI would be totally unnecessary), but I want to\n> avoid price of using CGI::Compile.\n\nYes, that makes sense - if implementing FastCGI is just switching CGI\nto CGI::Fast and a while loop, implementing PSGI interface just for\nthat is far more complicated. I just argued about it because i saw on\na separate thread that PSGI implementation is also on its way.\n\n\n\n-- \nTatsuhiko Miyagawa\n"},{"id":"141413","messageId":"AANLkTimcEjYpwa2xFhnRtnQ4ImfqJrDzQJdpptKNoskh@mail.gmail.com","threadId":"23758","inReplyTo":"201005101826.10735.jnareb@gmail.com","subject":"Re: gitweb, FastCGI and PSGI/Plack","fromName":"Tatsuhiko Miyagawa","fromEmail":"miyagawa@gmail.com","sentAt":"2010-05-10T17:14:09Z","receivedAt":"2010-05-10T17:14:09Z","isPatch":false,"sender":{"key":"miyagawa@gmail.com","avatar":"https://gravatar.com/avatar/951a9e8b1dbd5cae0c39f3ffde3cb20813149cc5b48ffcea6be1955618408cce?d=mp&s=160"},"body":"On Mon, May 10, 2010 at 9:26 AM, Jakub Narebski <jnareb@gmail.com> wrote:\n\n> As I understand it, CGI::Compile catches all STDOUT output of compiled\n> CGI script, and passes it to PSGI layer (PSGI interface).  That doesn't\n> allow, I think, for serving request as it is generated.  This might be\n> important for actions (pages) such as 'snapshot', 'blob_plain' and\n> 'patches', which simply dump output of external comand to STDOUT.\n\nYes, good point.\n\nPSGI allows \"streaming response interface\", which gives you a writer\nobject to call ->write method, to implement the non-buffering output\nfrom your application. (Of course, returning IO::Handle is another way\nto do so, but that's \"pull\" style streaming, not \"push\" which is much\neasier when you convert CGI applications).\n\nIt would still require you to rewrite some of the gitweb.cgi HTML\ngeneration code, such as;\n\n  # original\n  my $cgi = CGI->new;\n  print $cgi->header(...);\n  print \"<html>\";\n  print \"<body>...\";\n\ninto:\n\n  # CGI::PSGI\n  my $app = sub {\n      my $env = shift;\n      return sub {\n          my $r = shift;\n          my $cgi = CGI::PSGI->new($env);\n          my $writer = $r->([ $cgi->header(...) ]);\n          $writer->write(\"<html>\");\n          $writer->write(\"<body>...\");\n      }\n  };\n\nWith some work to do the initialization and turn ->write call into a\ncallback, it should be possible to make one code do both the normal\nCGI and PSGI.\n\n> Also I think that currently you can currently see in web browser page as\n> it is being generated by gitweb (e.g. 'projects_list' page).  This is\n> impossible (I guess) with CGI::Compile.\n\nCorrect, and possible with CGI::PSGI.\n\n> There is also a problem that there is no way, I guess, to automatically\n> reload / refresh app if underlying CGI script changes (a la \"plackup -r\").\n\nYes there is. \"plackup -R\" can specify the path containing your cgi\nscript to refresh whenever they're updated.\n\nThere's also \"plackup -L Shotgun\" to fork your application to get a\nfresh copy in every request, handy for the development.\n\n\n-- \nTatsuhiko Miyagawa\n"},{"id":"141427","messageId":"201005102040.36490.jnareb@gmail.com","threadId":"23758","inReplyTo":"AANLkTikH39rbm64sc0nn7eTBJSfxcqWpz6VXZBCQQIl3@mail.gmail.com","subject":"Re: gitweb, FastCGI and PSGI/Plack","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2010-05-10T18:40:33Z","receivedAt":"2010-05-10T18:40:33Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"On Mon, 10 May 2010, Tatsuhiko Miyagawa wrote:\n\n> Yes, that makes sense - if implementing FastCGI is just switching CGI\n> to CGI::Fast and a while loop, implementing PSGI interface just for\n> that is far more complicated. I just argued about it because I saw on\n> a separate thread that PSGI implementation is also on its way.\n\nWhat!?  No, currently there are no plans to add PSGI support to gitweb,\nnor moving gitweb from CGI (and mod_perl's ModPerl::Registry, and\nFastCGI now) to PSGI.\n\nThis separate thread is about adding support for 'plackup' to\ngit-instaweb (and perhaps also adding gitweb.psgi wrapper).\n\n>From git-instaweb manpage:\n\n  NAME\n       git-instaweb - Instantly browse your working repository in gitweb\n\n  SYNOPSIS\n       git instaweb [--local] [--httpd=<httpd>] [--port=<port>]\n                    [--browser=<browser>]\n       git instaweb [--start] [--stop] [--restart]\n\n  DESCRIPTION\n       A simple script to set up gitweb and a web server for browsing the local\n       repository.\n\n  OPTIONS\n       -d, --httpd\n              The HTTP daemon command-line that  will  be  executed.  Command-line\n              options  may  be  specified here, and the configuration file will be\n              added at the end of the command-line. Currently  apache2,  lighttpd,\n              mongoose and webrick are supported. (Default: lighttpd)\n\nI wanted to add support for '--httpd=plackup', see the following thread\non git mailing list:\n\n  \"Adding support for \"plackup\" and similar web server tools to git-instaweb\"\n  Message-ID: <201005020317.42112.jnareb@gmail.com>\n  http://thread.gmane.org/gmane.comp.version-control.git/146124\n\n*Not* moving gitweb to PSGI/Plack.\n\n-- \nJakub Narebski\nPoland\n"},{"id":"141428","messageId":"AANLkTilXqAUs5-go-JHgTrQuslTyA1sZ3re5yuLtnxkz@mail.gmail.com","threadId":"23758","inReplyTo":"201005102040.36490.jnareb@gmail.com","subject":"Re: gitweb, FastCGI and PSGI/Plack","fromName":"Tatsuhiko Miyagawa","fromEmail":"miyagawa@gmail.com","sentAt":"2010-05-10T18:43:59Z","receivedAt":"2010-05-10T18:43:59Z","isPatch":false,"sender":{"key":"miyagawa@gmail.com","avatar":"https://gravatar.com/avatar/951a9e8b1dbd5cae0c39f3ffde3cb20813149cc5b48ffcea6be1955618408cce?d=mp&s=160"},"body":"On Mon, May 10, 2010 at 11:40 AM, Jakub Narebski <jnareb@gmail.com> wrote:\n> On Mon, 10 May 2010, Tatsuhiko Miyagawa wrote:\n\n>> Yes, that makes sense - if implementing FastCGI is just switching CGI\n>> to CGI::Fast and a while loop, implementing PSGI interface just for\n>> that is far more complicated. I just argued about it because I saw on\n>> a separate thread that PSGI implementation is also on its way.\n>\n> What!?  No, currently there are no plans to add PSGI support to gitweb,\n> nor moving gitweb from CGI (and mod_perl's ModPerl::Registry, and\n> FastCGI now) to PSGI.\n>\n> This separate thread is about adding support for 'plackup' to\n> git-instaweb (and perhaps also adding gitweb.psgi wrapper).\n\nYes, that's what i meant.\n\nIf you get plackup support by converting your CGI application into a\nPSGI app, then adding support to FastCGI is just one command line\noption away.\n\n\n-- \nTatsuhiko Miyagawa\n"},{"id":"141436","messageId":"201005102310.47879.jnareb@gmail.com","threadId":"23758","inReplyTo":"AANLkTilXqAUs5-go-JHgTrQuslTyA1sZ3re5yuLtnxkz@mail.gmail.com","subject":"Re: gitweb, FastCGI and PSGI/Plack","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2010-05-10T21:10:47Z","receivedAt":"2010-05-10T21:10:47Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"On Mon, 10 May 2010, Tatsuhiko Miyagawa wrote:\n> On Mon, May 10, 2010 at 11:40 AM, Jakub Narebski <jnareb@gmail.com> wrote:\n>> On Mon, 10 May 2010, Tatsuhiko Miyagawa wrote:\n>>> \n>>> Yes, that makes sense - if implementing FastCGI is just switching CGI\n>>> to CGI::Fast and a while loop, implementing PSGI interface just for\n>>> that is far more complicated. I just argued about it because I saw on\n>>> a separate thread that PSGI implementation is also on its way.\n>>\n>> What!?  No, currently there are no plans to add PSGI support to gitweb,\n>> nor moving gitweb from CGI (and mod_perl's ModPerl::Registry, and\n>> FastCGI now) to PSGI.\n>>\n>> This separate thread is about adding support for 'plackup' to\n>> git-instaweb (and perhaps also adding gitweb.psgi wrapper).\n> \n> Yes, that's what I meant.\n> \n> If you get plackup support by converting your CGI application into a\n> PSGI app, then adding support to FastCGI is just one command line\n> option away.\n\nWell, the support for 'plackup' in git-instaweb is / would be done not\nby converting gitweb from CGI to PSGI app, but by using gitweb.psgi\nwrapper.  This wrapper uses Plack::App::WrapCGI (which in turn uses\nCGI::Emulate::PSGI, which in turn use CGI::Compile) to wrap gitweb.cgi,\nand Plack::Middleware::Static to serve static files (gitweb.css,\ngitweb.js, git-favicon.png, git-logo.png).\n\nTo be more exact in current, as yet unpublished version, git-instaweb\ngenerates gitweb.psgi on-the-fly.  I am considering adding gitweb.psgi\nwrapper to git repository (to git sources); when building git-instaweb\nit would get embedded in $(gitexecdir)/git-instaweb, just like\ngitweb.cgi, gitweb.css (or gitweb.min.css) and gitweb.js (or\ngitweb.min.js) are.  Running git-instaweb for first time would create\n_configured_ gitweb.cgi in $GIT_DIR/gitweb.  For \"plackup\" git-instaweb\nwould put configured gitweb.psgi (or app.psgi) there; for other web\nservers git-instaweb puts httpd.conf there.\n\nIt is true that with gitweb.psgi wrapper running gitweb as FastCGI\nscript is just one command line option away.  But it does require Plack\nto be installed... well, on the other hand the modified gitweb needs\nFCGI module to be ran as FastCGI script, which is not a CORE Perl module\neither.\n\n\nI also wonder how running via wrapper script affect performance, as\ncompared to modified gitweb running as FastCGI script, using CGI::Fast\nand FCGI.\n\n\nP.S. A bit of history: original patch by Sam Vilain adding FastCGI\nsupport by using CGI::Fast is from 2006.  Eric Wong patch adding\ngitweb.fcgi wrapper (with 'no warnings; do $file' instead of modern\nCGI::Compile->compile($file)) is from 2007.\n\nI have originally replaced 'exit' in die_error() subroutine by non-local\njump to the end of request processing in ultimately a bit failed attempt\nto use die_error() in CGI::Carp::set_message(), because when using\n'exit' the error didn't get logged (unfortunately you can set error\nmessage, but you can set HTTP headers for error message this way).\n\nI have then noticed that this change would also allow to add FastCGI\nsupport to gitweb in a very simple way.  Therefore I have ported Sam\nVilain patch to modern gitweb codebase.  Well, it did require some\nrestructuring of gitweb code (some refactoring), so it wasn't that\nsimple... nevertheless I think that this refactoring is mainly a good\nchange anyway.\n\n\nP.P.S. One of constraints to gitweb development is that it should run\nwith minimal set of non-core modules.  Some people even complain that\ngitweb (or was it about git in general?) requires at least Perl 5.8.6 or\nabout (because of Encode module and Unicode support).\n\nIf one wants Modern Perl git web interface, there is always Gitalist...\n\n-- \nJakub Narebski, Poland\nShadeHawk on #git at FreeNode\njnareb on #plack at irc.perl.org\n"},{"id":"141447","messageId":"AANLkTimypauJKP-ifPKDmM9TK0u0JsSG5PlcpWdkkbVd@mail.gmail.com","threadId":"23758","inReplyTo":"201005102310.47879.jnareb@gmail.com","subject":"Re: gitweb, FastCGI and PSGI/Plack","fromName":"Tatsuhiko Miyagawa","fromEmail":"miyagawa@gmail.com","sentAt":"2010-05-11T00:07:51Z","receivedAt":"2010-05-11T00:07:51Z","isPatch":false,"sender":{"key":"miyagawa@gmail.com","avatar":"https://gravatar.com/avatar/951a9e8b1dbd5cae0c39f3ffde3cb20813149cc5b48ffcea6be1955618408cce?d=mp&s=160"},"body":"On Mon, May 10, 2010 at 2:10 PM, Jakub Narebski <jnareb@gmail.com> wrote:\n>\n> Well, the support for 'plackup' in git-instaweb is / would be done not\n> by converting gitweb from CGI to PSGI app, but by using gitweb.psgi\n> wrapper.\n\nwrapping gitweb.cgi with gitweb.psgi is a legitimate way to say\n\"converting CGI to PSGI app\".\n\n> It is true that with gitweb.psgi wrapper running gitweb as FastCGI\n> script is just one command line option away.  But it does require Plack\n> to be installed... well, on the other hand the modified gitweb needs\n> FCGI module to be ran as FastCGI script, which is not a CORE Perl module\n> either.\n\n*nods*\n\n> I also wonder how running via wrapper script affect performance, as\n> compared to modified gitweb running as FastCGI script, using CGI::Fast\n> and FCGI.\n\nBased on my experience CGI::Emulate::PSGI doesn't have much of an\noverhead because it's just swapping STDIN and STDOUT handles and\ndoesn't require stuff like tie or overload which tends to be slower.\n\n> P.P.S. One of constraints to gitweb development is that it should run\n> with minimal set of non-core modules.  Some people even complain that\n> gitweb (or was it about git in general?) requires at least Perl 5.8.6 or\n> about (because of Encode module and Unicode support).\n\nI know, but git-instaweb is a different story, since you rely on the\nfact that the system has one of web servers like apache, lighttpd or\nmongrel.\n\nAnd that's exactly why I've been suggesting to you use WrapCGI\n(CGI::Emulate::PSGI  + CGI::Compile) instead of converting gitweb.cgi\nto natively support PSGI. We're on the same page and i don't\nunderstand why you keep disagreeing with me :)\n\n\n-- \nTatsuhiko Miyagawa\n"},{"id":"141465","messageId":"201005111129.10931.jnareb@gmail.com","threadId":"23758","inReplyTo":"AANLkTimypauJKP-ifPKDmM9TK0u0JsSG5PlcpWdkkbVd@mail.gmail.com","subject":"Re: gitweb, FastCGI and PSGI/Plack","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2010-05-11T09:29:07Z","receivedAt":"2010-05-11T09:29:07Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"On Mon, 10 May 2010, Tatsuhiko Miyagawa wrote:\n> On Mon, May 10, 2010 at 2:10 PM, Jakub Narebski <jnareb@gmail.com> wrote:\n> >\n> > Well, the support for 'plackup' in git-instaweb is / would be done not\n> > by converting gitweb from CGI to PSGI app, but by using gitweb.psgi\n> > wrapper.\n> \n> Wrapping gitweb.cgi with gitweb.psgi is a legitimate way to say\n> \"converting CGI to PSGI app\".\n\nI have misunderstood you then, I'm sorry.\n \n> > I also wonder how running via wrapper script affect performance, as\n> > compared to modified gitweb running as FastCGI script, using CGI::Fast\n> > and FCGI.\n> \n> Based on my experience CGI::Emulate::PSGI doesn't have much of an\n> overhead because it's just swapping STDIN and STDOUT handles and\n> doesn't require stuff like tie or overload which tends to be slower.\n\nNice to know.\n\nUnfortunately I don't have mod_fcgid / mod_fastcgi installed, so I can't\ndo a benchmark comparing PSGI wrapper + FCGI handler with modified\ngitweb running as FastCGI script (using ab, ApacheBench).  Do you know\nany pure-Perl FastCGI server with minimal dependencies, and pure-Perl\nHTTP server benchmarking tool (like ab and httperf)?\n \n> > P.P.S. One of constraints to gitweb development is that it should run\n> > with minimal set of non-core modules.  Some people even complain that\n> > gitweb (or was it about git in general?) requires at least Perl 5.8.6 or\n> > about (because of Encode module and Unicode support).\n> \n> I know, but git-instaweb is a different story, since you rely on the\n> fact that the system has one of web servers like apache, lighttpd or\n> mongrel.\n> \n> And that's exactly why I've been suggesting to you use WrapCGI\n> (CGI::Emulate::PSGI + CGI::Compile) instead of converting gitweb.cgi\n> to natively support PSGI. We're on the same page and I don't\n> understand why you keep disagreeing with me :)\n\nI might have misunderstand you as arguing against modifying gitweb to\nadd FastCGI support via CGI::Fast...\n\nP.S. discussion != disagreeing ;-)\n-- \nJakub Narębski\nPoland\n"},{"id":"141466","messageId":"20100511094409.GB5220@screwed.box","threadId":"23758","inReplyTo":"201005111129.10931.jnareb@gmail.com","subject":"Re: gitweb, FastCGI and PSGI/Plack","fromName":"Peter Vereshagin","fromEmail":"peter@vereshagin.org","sentAt":"2010-05-11T09:44:09Z","receivedAt":"2010-05-11T09:44:09Z","isPatch":false,"sender":{"key":"peter@vereshagin.org","avatar":"https://gravatar.com/avatar/27a92b8c80743df8621433ca040657c4ac37a78497228d04f703e70731c5f30b?d=mp&s=160"},"body":"I know St. Peter won't call your name, Jakub!\n2010/05/11 11:29:07 +0200 Jakub Narebski <jnareb@gmail.com> => To Tatsuhiko Miyagawa :\n\nJN> gitweb running as FastCGI script (using ab, ApacheBench).  Do you know\nJN> any pure-Perl FastCGI server with minimal dependencies, and pure-Perl\n\nI'd like to know about the pure-perl FastCGI protocol implementation either, at\nleast to make tests for FCGI::Spawn.\n\n73! Peter pgp: A0E26627 (4A42 6841 2871 5EA7 52AB  12F8 0CE1 4AAC A0E2 6627)\n-- \nhttp://vereshagin.org\n"},{"id":"141489","messageId":"AANLkTinDzAgLckK6z08DJwZPWRmXywQ69EImTaiFsDOE@mail.gmail.com","threadId":"23758","inReplyTo":"20100511094409.GB5220@screwed.box","subject":"Re: gitweb, FastCGI and PSGI/Plack","fromName":"Tatsuhiko Miyagawa","fromEmail":"miyagawa@gmail.com","sentAt":"2010-05-11T18:56:43Z","receivedAt":"2010-05-11T18:56:43Z","isPatch":false,"sender":{"key":"miyagawa@gmail.com","avatar":"https://gravatar.com/avatar/951a9e8b1dbd5cae0c39f3ffde3cb20813149cc5b48ffcea6be1955618408cce?d=mp&s=160"},"body":"2010/5/11 Peter Vereshagin <peter@vereshagin.org>:\n> I know St. Peter won't call your name, Jakub!\n> 2010/05/11 11:29:07 +0200 Jakub Narebski <jnareb@gmail.com> => To Tatsuhiko Miyagawa :\n>\n> JN> gitweb running as FastCGI script (using ab, ApacheBench).  Do you know\n> JN> any pure-Perl FastCGI server with minimal dependencies, and pure-Perl\n>\n> I'd like to know about the pure-perl FastCGI protocol implementation either, at\n> least to make tests for FCGI::Spawn.\n\nThere are two pure perl FCGI protocol implementations (caller side)\navailable: Net::FastCGI and FCGI::Client.\n\nPlack::App::FCGIDispatcher is a PSGI application to connect to a\nFastCGI external daemon over TCP or UNIX sockets, so you can use pure\nperl PSGI web servers such as Starman to talk to them. But since it's\nall pure perl and experimental and not suited for the benchmarking,\nbut it could be handy for testing (we actually use FCGI::Client to\ntest our own FastCGI server side implementation in Plack).\n\nWhen I seriously want to benchmark FastCGI i always use nginx or\nlighttpd, which is easy to install and doesn't need external modules\nsuch as the case with Apache.\n\n-- \nTatsuhiko Miyagawa\n"}]}