{"thread":{"id":"26633","subject":"gitweb: cloud tags feature produces malformed XML for errors","startedAt":"2011-03-01T22:21:41Z","lastAt":"2011-06-09T07:11:23Z","messageCount":16,"participants":["Jonathan Nieder","Jakub Narebski","J.H.","Uwe Kleine-König","Petr Baudis"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"162594","messageId":"20110301222141.GB7918@elie","threadId":"26633","inReplyTo":"20110301190229.11297.17767.reportbug@cassiopeia.kleinek","subject":"gitweb: cloud tags feature produces malformed XML for errors","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2011-03-01T22:21:41Z","receivedAt":"2011-03-01T22:21:41Z","isPatch":false,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"(resending, censored for vger; sorry for the noise)\nHi Jakub et al,\n\nUwe Kleine-König wrote [1]:\n\n> I experiment with $feature{'ctags'}. After installing\n> libhtml-tagcloud-perl (0.34-1) and adding\n>\n>\t$feature{'ctags'}{'default'} = [1];\n>\n> to gitweb.conf and doing\n>\n> \tmkdir ctags\n> \techo Linux > ctags/Linux\n>\n> in the only repository served by gitweb makes iceweasel barf on the\n> output (see attachment).\n\nWith chromium I get:\n\n\tThis page contains the following errors:\n\n\terror on line 26 at column 6: XML declaration allowed only at the start of the document\n\tBelow is a rendering of the page up to the first error.\n\nFirst, we hit\n\n\tif ($show_ctags) {\n\t\tmy %ctags;\n\t\tforeach my $p (@projects) {\n\t\t\tforeach my $ct (keys %{$p->{'ctags'}}) {\n\t\t\t\t$ctags{$ct} += $p->{'ctags'}->{$ct};\n\nwhich produces a warning warning in error.log:\n\n\tindex.cgi: Argument \"Linux\" isn't numeric in addition (+) at /usr/share/gitweb/index.cgi line 4819.\n\nin error.log. Then we hit git_show_project_tagcloud, which dies in\n\"$cloud->html_and_css($count);\" with\n\n\tHTML::TagCloud..\n\tindex.cgi: Can't take log of 0 at /usr/share/perl5/HTML/TagCloud.pm line 67.\n\nFor some reason, the result is an _embedded_ error page:\n\n\t \n\tContent-type: {a content type which shall not be named}\n\t \n\t \n [...]\n\t500 - Internal Server Error\n\nSo I suspect there are two bugs here.\n\nIdeas?\nJonathan\n\n[1] http://bugs.debian.org/616005\n"},{"id":"162610","messageId":"201103020105.04440.jnareb@gmail.com","threadId":"26633","inReplyTo":"20110301222141.GB7918@elie","subject":"Re: gitweb: cloud tags feature produces malformed XML for errors","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2011-03-02T00:05:00Z","receivedAt":"2011-03-02T00:05:00Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"On Tue, 1 Mar 2011, Jonathan Nieder wrote:\n> Uwe Kleine-König wrote [1]:\n> [1] http://bugs.debian.org/616005\n\n> > I experiment with $feature{'ctags'}. After installing\n> > libhtml-tagcloud-perl (0.34-1) and adding\n> >\n> >\t$feature{'ctags'}{'default'} = [1];\n> >\n> > to gitweb.conf and doing\n> >\n> > \tmkdir ctags\n> > \techo Linux > ctags/Linux\n\nThis is invalid usage. See below for details.\n\n> >\n> > in the only repository served by gitweb makes iceweasel barf on the\n> > output (see attachment).\n> \n> With chromium I get:\n> \n> \tThis page contains the following errors:\n> \n> \terror on line 26 at column 6: XML declaration allowed only at the start of the document\n> \tBelow is a rendering of the page up to the first error.\n> \n> First, we hit\n> \n> \tif ($show_ctags) {\n> \t\tmy %ctags;\n> \t\tforeach my $p (@projects) {\n> \t\t\tforeach my $ct (keys %{$p->{'ctags'}}) {\n> \t\t\t\t$ctags{$ct} += $p->{'ctags'}->{$ct};\n> \n> which produces a warning warning in error.log:\n> \n> \tindex.cgi: Argument \"Linux\" isn't numeric in addition (+) at /usr/share/gitweb/index.cgi line 4819.\n> \n> in error.log.\n\nWell, blame Pasky for not describing 'ctags' in more detail. Value of\ntag is its weight, so instead of\n\n \tmkdir ctags\n\techo Linux > ctags/Linux\n\none should use\n\n \tmkdir ctags\n\techo 1 > ctags/Linux\n\nAdmittedly gitweb should be more defensive about invalid input...\n\n> Then we hit git_show_project_tagcloud, which dies in \n> \"$cloud->html_and_css($count);\" with\n> \n> \tHTML::TagCloud..\n> \tindex.cgi: Can't take log of 0 at /usr/share/perl5/HTML/TagCloud.pm line 67.\n> \n> For some reason, the result is an _embedded_ error page:\n> \n> \t \n> \tContent-type: {a content type which shall not be named}\n> \t \n> \t \n> [...]\n> \t500 - Internal Server Error\n> \n> So I suspect there are two bugs here.\n\nThis I think is caused by the fact that error (\"die\") occurs after gitweb\nhave send some output to web browser already. That would be harder to fix.\n\n\n-- \nJakub Narebski\nPoland\n"},{"id":"162614","messageId":"4D6D97FB.3050106@kernel.org","threadId":"26633","inReplyTo":"20110301222141.GB7918@elie","subject":"Re: gitweb: cloud tags feature produces malformed XML for errors","fromName":"J.H.","fromEmail":"warthog9@kernel.org","sentAt":"2011-03-02T01:06:03Z","receivedAt":"2011-03-02T01:06:03Z","isPatch":false,"sender":{"key":"warthog9@kernel.org","avatar":"https://avatars.githubusercontent.com/u/2334704?v=4"},"body":"On 03/01/2011 02:21 PM, Jonathan Nieder wrote:\n> (resending, censored for vger; sorry for the noise)\n> Hi Jakub et al,\n> \n> Uwe Kleine-König wrote [1]:\n> \n>> I experiment with $feature{'ctags'}. After installing\n>> libhtml-tagcloud-perl (0.34-1) and adding\n>>\n>> \t$feature{'ctags'}{'default'} = [1];\n>>\n>> to gitweb.conf and doing\n>>\n>> \tmkdir ctags\n>> \techo Linux > ctags/Linux\n>>\n>> in the only repository served by gitweb makes iceweasel barf on the\n>> output (see attachment).\n> \n> With chromium I get:\n> \n> \tThis page contains the following errors:\n> \n> \terror on line 26 at column 6: XML declaration allowed only at the start of the document\n> \tBelow is a rendering of the page up to the first error.\n> \n> First, we hit\n> \n> \tif ($show_ctags) {\n> \t\tmy %ctags;\n> \t\tforeach my $p (@projects) {\n> \t\t\tforeach my $ct (keys %{$p->{'ctags'}}) {\n> \t\t\t\t$ctags{$ct} += $p->{'ctags'}->{$ct};\n> \n> which produces a warning warning in error.log:\n> \n> \tindex.cgi: Argument \"Linux\" isn't numeric in addition (+) at /usr/share/gitweb/index.cgi line 4819.\n> \n> in error.log. Then we hit git_show_project_tagcloud, which dies in\n> \"$cloud->html_and_css($count);\" with\n> \n> \tHTML::TagCloud..\n> \tindex.cgi: Can't take log of 0 at /usr/share/perl5/HTML/TagCloud.pm line 67.\n> \n> For some reason, the result is an _embedded_ error page:\n> \n> \t \n> \tContent-type: {a content type which shall not be named}\n> \t \n> \t \n> [...]\n> \t500 - Internal Server Error\n> \n> So I suspect there are two bugs here.\n> \n\nAs a general note the cloud tags code that's there has a whole slew of\nproblems, including a pretty trivial way to crash gitweb entirely if the\ntags file is malformed in any way to what it's expecting.\n\nI've said it before, I'd rather see that code reverted or completely\noverhauled, and as far as I know repo.or.cz is the only place even using\nthe code at all.\n\nI also think I reported this particular error a couple of years ago\n(with the file being malformed and causing all kinds of problems). The\nfact that the tags don't follow the repository when it's cloned more or\nless makes this entire feature a giant PITA, couple that with no good\ndocumentation on how to create the tag files (I think the file itself\nneeds to have filename of tag, and *ONLY* contain 1 for it's contains\nfor it to work)\n\nLinux:\n1\nEOF\n\n- John 'Warthog9' Hawley\n"},{"id":"162627","messageId":"201103020924.32924.jnareb@gmail.com","threadId":"26633","inReplyTo":"201103020105.04440.jnareb@gmail.com","subject":"Re: gitweb: cloud tags feature produces malformed XML for errors","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2011-03-02T08:24:28Z","receivedAt":"2011-03-02T08:24:28Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"On Wed, 2 Mar 2011, Jakub Narebski wrote:\n> On Tue, 1 Mar 2011, Jonathan Nieder wrote:\n> > Uwe Kleine-König wrote [1]:\n> > [1] http://bugs.debian.org/616005\n> \n> > > I experiment with $feature{'ctags'}. After installing\n> > > libhtml-tagcloud-perl (0.34-1) and adding\n> > >\n> > >\t$feature{'ctags'}{'default'} = [1];\n\nThis is also invalid... but this one actually is described:\n\n # gitweb by itself can show existing tags, but it does not handle\n # tagging itself; you need an external application for that.\n # For an example script, check Girocco's cgi/tagproj.cgi.\n # You may want to install the HTML::TagCloud Perl module to get\n # a pretty tag cloud instead of just a list of tags.\n\n # To enable system wide have in $GITWEB_CONFIG\n # $feature{'ctags'}{'default'} = ['path_to_tag_script'];\n # Project specific override is not supported.\n\nUsing \"$feature{'ctags'}{'default'} = [1];\" would lead to errors when \nyou would want to create a tag from web interface.\n\n-- \nJakub Narebski\nPoland\n"},{"id":"162628","messageId":"20110302084528.GB22310@pengutronix.de","threadId":"26633","inReplyTo":"201103020924.32924.jnareb@gmail.com","subject":"Re: gitweb: cloud tags feature produces malformed XML for errors","fromName":"Uwe Kleine-König","fromEmail":"u.kleine-koenig@pengutronix.de","sentAt":"2011-03-02T08:45:28Z","receivedAt":"2011-03-02T08:45:28Z","isPatch":false,"sender":{"key":"u.kleine-koenig@pengutronix.de","avatar":"https://gravatar.com/avatar/354b5e3ceb2806a2f1e1e382ac29ddbdad18288654da62b61eb13583a857eee7?d=mp&s=160"},"body":"On Wed, Mar 02, 2011 at 09:24:28AM +0100, Jakub Narebski wrote:\n> On Wed, 2 Mar 2011, Jakub Narebski wrote:\n> > On Tue, 1 Mar 2011, Jonathan Nieder wrote:\n> > > Uwe Kleine-König wrote [1]:\n> > > [1] http://bugs.debian.org/616005\n> > \n> > > > I experiment with $feature{'ctags'}. After installing\n> > > > libhtml-tagcloud-perl (0.34-1) and adding\n> > > >\n> > > >\t$feature{'ctags'}{'default'} = [1];\n> \n> This is also invalid... but this one actually is described:\n> \n> # gitweb by itself can show existing tags, but it does not handle\n> # tagging itself; you need an external application for that.\n> # For an example script, check Girocco's cgi/tagproj.cgi.\n> # You may want to install the HTML::TagCloud Perl module to get\n> # a pretty tag cloud instead of just a list of tags.\n> \n> # To enable system wide have in $GITWEB_CONFIG\n> # $feature{'ctags'}{'default'} = ['path_to_tag_script'];\n> # Project specific override is not supported.\n> \n> Using \"$feature{'ctags'}{'default'} = [1];\" would lead to errors when \n> you would want to create a tag from web interface.\nYeah, I was aware of that (but of course I should have written that). I\nalso tried ... = ['/bin/sh'] and a few others. Obviously this is\npre-production :-)\n\nBest regards\nUwe\n\n-- \nPengutronix e.K. | Uwe Kleine-König |\nIndustrial Linux Solutions | http://www.pengutronix.de/ |\n"},{"id":"162672","messageId":"201103022218.46640.jnareb@gmail.com","threadId":"26633","inReplyTo":"4D6D97FB.3050106@kernel.org","subject":"Re: gitweb: cloud tags feature produces malformed XML for errors","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2011-03-02T21:18:44Z","receivedAt":"2011-03-02T21:18:44Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"On Wed, 2 Mar 2011 02:06, J.H. wrote:\n\n> As a general note the cloud tags code that's there has a whole slew of\n> problems, including a pretty trivial way to crash gitweb entirely if the\n> tags file is malformed in any way to what it's expecting.\n\nThat's not a huge problem, I think; it would be enough to add some\nvalidation to parsing ctags code (e.g. treat any contents that is\nnot a single number as a 1).\n \n> I've said it before, I'd rather see that code reverted or completely\n> overhauled, and as far as I know repo.or.cz is the only place even using\n> the code at all.\n> \n> I also think I reported this particular error a couple of years ago\n> (with the file being malformed and causing all kinds of problems). The\n> fact that the tags don't follow the repository when it's cloned more or\n> less makes this entire feature a giant PITA, couple that with no good\n> documentation on how to create the tag files (I think the file itself\n> needs to have filename of tag, and *ONLY* contain 1 for it's contains\n> for it to work)\n> \n> Linux:\n> 1\n> EOF\n\nWell, neither does README.html follow the repository when it is cloned,\nand usually neither does description ('description' file is generated by\ngit default template, and description file overrides `gitweb.description'\nconfig variable).\n\nDocumentation always can be added, either as comments for the 'ctags'\nfeature, or in gitweb/README (or, in the future, in gitweb manpage).\n\n\nWhat is most important that makes this feature to be considered for\nremoval (or rehauling) is that only half of this feature is implemented\nin gitweb: the displaying part. There is half-attempt of providing\nsome web interface for managing tags... which needs external script with\nstrict coupling, doesn't offer any access control as far as I know, do\nnot allow deleting tags, etc.\n\n-- \nJakub Narebski\nPoland\n"},{"id":"162682","messageId":"20110302215556.GL22310@pengutronix.de","threadId":"26633","inReplyTo":"201103022218.46640.jnareb@gmail.com","subject":"Re: gitweb: cloud tags feature produces malformed XML for errors","fromName":"Uwe Kleine-König","fromEmail":"u.kleine-koenig@pengutronix.de","sentAt":"2011-03-02T21:55:56Z","receivedAt":"2011-03-02T21:55:56Z","isPatch":false,"sender":{"key":"u.kleine-koenig@pengutronix.de","avatar":"https://gravatar.com/avatar/354b5e3ceb2806a2f1e1e382ac29ddbdad18288654da62b61eb13583a857eee7?d=mp&s=160"},"body":"Hello Jakub,\n\nOn Wed, Mar 02, 2011 at 10:18:44PM +0100, Jakub Narebski wrote:\n> What is most important that makes this feature to be considered for\n> removal (or rehauling) is that only half of this feature is implemented\n> in gitweb: the displaying part. There is half-attempt of providing\n> some web interface for managing tags... which needs external script with\n> strict coupling, doesn't offer any access control as far as I know, do\n> not allow deleting tags, etc.\nFor a small set of repositories the need to hand-edit the tags is OK\nIMHO. That's what I intended to do.\n\nBest regards\nUwe\n\n-- \nPengutronix e.K. | Uwe Kleine-König |\nIndustrial Linux Solutions | http://www.pengutronix.de/ |\n"},{"id":"162692","messageId":"201103030142.17258.jnareb@gmail.com","threadId":"26633","inReplyTo":"20110302215556.GL22310@pengutronix.de","subject":"Re: gitweb: cloud tags feature produces malformed XML for errors","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2011-03-03T00:42:15Z","receivedAt":"2011-03-03T00:42:15Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"On Wed, 2 Mar 2011, Uwe Kleine-König wrote:\n> On Wed, Mar 02, 2011 at 10:18:44PM +0100, Jakub Narebski wrote:\n\n> > What is most important that makes this feature to be considered for\n> > removal (or rehauling) is that only half of this feature is implemented\n> > in gitweb: the displaying part. There is half-attempt of providing\n> > some web interface for managing tags... which needs external script with\n> > strict coupling, doesn't offer any access control as far as I know, do\n> > not allow deleting tags, etc.\n>\n> For a small set of repositories the need to hand-edit the tags is OK\n> IMHO. That's what I intended to do.\n\nSo what would you like to see?\n\n1. Hardening parsing of ctags files, so that gitweb does not crash on\n malformed entries, but e.g. just ignores them.\n\n2. Generating tag cloud upfront, before sending any output to browser,\n to catch error better (and perhaps separate CSS for HTML::TagCloud).\n\n3. Describe format of ctags files, either in comments in code, or in\n gitweb/README.\n\n4. Either:\n\n A. Remove editing ctags from gitweb, or\n B. Add some simple generation of ctags file to gitweb\n\nor should we remove ctags feature altogether?\n-- \nJakub Narebski\nPoland\n"},{"id":"162707","messageId":"20110303081904.GM22310@pengutronix.de","threadId":"26633","inReplyTo":"201103030142.17258.jnareb@gmail.com","subject":"Re: gitweb: cloud tags feature produces malformed XML for errors","fromName":"Uwe Kleine-König","fromEmail":"u.kleine-koenig@pengutronix.de","sentAt":"2011-03-03T08:19:04Z","receivedAt":"2011-03-03T08:19:04Z","isPatch":false,"sender":{"key":"u.kleine-koenig@pengutronix.de","avatar":"https://gravatar.com/avatar/354b5e3ceb2806a2f1e1e382ac29ddbdad18288654da62b61eb13583a857eee7?d=mp&s=160"},"body":"Hello Jakub,\n\nOn Thu, Mar 03, 2011 at 01:42:15AM +0100, Jakub Narebski wrote:\n> On Wed, 2 Mar 2011, Uwe Kleine-König wrote:\n> > On Wed, Mar 02, 2011 at 10:18:44PM +0100, Jakub Narebski wrote:\n> \n> > > What is most important that makes this feature to be considered for\n> > > removal (or rehauling) is that only half of this feature is implemented\n> > > in gitweb: the displaying part. There is half-attempt of providing\n> > > some web interface for managing tags... which needs external script with\n> > > strict coupling, doesn't offer any access control as far as I know, do\n> > > not allow deleting tags, etc.\n> >\n> > For a small set of repositories the need to hand-edit the tags is OK\n> > IMHO. That's what I intended to do.\n> \n> So what would you like to see?\n> \n> 1. Hardening parsing of ctags files, so that gitweb does not crash on\n> malformed entries, but e.g. just ignores them.\n> \n> 2. Generating tag cloud upfront, before sending any output to browser,\n> to catch error better (and perhaps separate CSS for HTML::TagCloud).\n> \n> 3. Describe format of ctags files, either in comments in code, or in\n> gitweb/README.\n> \n> 4. Either:\n> \n> A. Remove editing ctags from gitweb, or\n> B. Add some simple generation of ctags file to gitweb\n> \nyes :-) (1-3, 4a) I wouldn't mind 4b, but it should be possible to\ndisable the possibility to publically edit tags via gitweb.\n\nBest regards\nUwe\n\n-- \nPengutronix e.K. | Uwe Kleine-König |\nIndustrial Linux Solutions | http://www.pengutronix.de/ |\n"},{"id":"162717","messageId":"20110303115856.GH24584@machine.or.cz","threadId":"26633","inReplyTo":"201103022218.46640.jnareb@gmail.com","subject":"Re: gitweb: cloud tags feature produces malformed XML for errors","fromName":"Petr Baudis","fromEmail":"pasky@suse.cz","sentAt":"2011-03-03T11:58:56Z","receivedAt":"2011-03-03T11:58:56Z","isPatch":false,"sender":{"key":"pasky@ucw.cz","avatar":"https://avatars.githubusercontent.com/u/18439?v=4"},"body":"On Wed, Mar 02, 2011 at 10:18:44PM +0100, Jakub Narebski wrote:\n> Well, neither does README.html follow the repository when it is cloned,\n> and usually neither does description ('description' file is generated by\n> git default template, and description file overrides `gitweb.description'\n> config variable).\n> \n> Documentation always can be added, either as comments for the 'ctags'\n> feature, or in gitweb/README (or, in the future, in gitweb manpage).\n> \n> \n> What is most important that makes this feature to be considered for\n> removal (or rehauling) is that only half of this feature is implemented\n> in gitweb: the displaying part. There is half-attempt of providing\n> some web interface for managing tags... which needs external script with\n> strict coupling, doesn't offer any access control as far as I know, do\n> not allow deleting tags, etc.\n\nWell, but your argument above also applies here - gitweb does not allow\nusers to modify README.html or desrciption either. Who knows where ctags\nmay come from, they might even be autogenerated based on something.\nI admit that the code could be a bit more resilient to invalid contents\nof the files, but I do not see that as a major problem either (it is\npossible to confuse gitweb by putting invalid stuff into other\nrepository files too).\n\n-- \n\t\t\t\tPetr \"Pasky\" Baudis\nComputer science education cannot make an expert programmer any more\nthan studying brushes and pigment can make an expert painter. --esr\n"},{"id":"162721","messageId":"201103031429.32657.jnareb@gmail.com","threadId":"26633","inReplyTo":"20110303115856.GH24584@machine.or.cz","subject":"Re: gitweb: cloud tags feature produces malformed XML for errors","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2011-03-03T13:29:29Z","receivedAt":"2011-03-03T13:29:29Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"On Thu, 3 March 2011, Petr Baudis wrote:\n> On Wed, Mar 02, 2011 at 10:18:44PM +0100, Jakub Narebski wrote:\n\n> > Well, neither does README.html follow the repository when it is cloned,\n> > and usually neither does description ('description' file is generated by\n> > git default template, and description file overrides `gitweb.description'\n> > config variable).\n> > \n> > Documentation always can be added, either as comments for the 'ctags'\n> > feature, or in gitweb/README (or, in the future, in gitweb manpage).\n> > \n> > \n> > What is most important that makes this feature to be considered for\n> > removal (or rehauling) is that only half of this feature is implemented\n> > in gitweb: the displaying part. There is half-attempt of providing\n> > some web interface for managing tags... which needs external script with\n> > strict coupling, doesn't offer any access control as far as I know, do\n> > not allow deleting tags, etc.\n> \n> Well, but your argument above also applies here - gitweb does not allow\n> users to modify README.html or desrciption either. Who knows where ctags\n> may come from, they might even be autogenerated based on something.\n> I admit that the code could be a bit more resilient to invalid contents\n> of the files, but I do not see that as a major problem either (it is\n> possible to confuse gitweb by putting invalid stuff into other\n> repository files too).\n\nWell, if not the whole 'ctags' / folksonomy feature, then perhaps (as\nproposed in other subthread of this discussion, and what Uwe agreed upon)\nto remove \"write\" support for content tags.\n\nCurrent solution with gitweb providing interface, and third-party script\nproviding implementation for creating tags, with unspecified API given by\ngitweb, and no way to edit / delete tags from web interface, seems \nhalf-baked at best.\n\n\nBTW. if I remember correctly [failed] GSoC 2010 project was among others\nto add ability to edit description and README.html from gitweb; is there\nany code about this in Pavan repository?\n\n-- \nJakub Narebski\nPoland\n"},{"id":"162926","messageId":"201103071900.16126.jnareb@gmail.com","threadId":"26633","inReplyTo":"20110303081904.GM22310@pengutronix.de","subject":"[RFC/PATCH] gitweb: Change the way \"content tags\" ('ctags') are handled","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2011-03-07T18:00:13Z","receivedAt":"2011-03-07T18:00:13Z","isPatch":true,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"On Thu, 3 March 2011, Uwe Kleine-König wrote:\n> On Thu, Mar 03, 2011 at 01:42:15AM +0100, Jakub Narebski wrote:\n>> On Wed, 2 Mar 2011, Uwe Kleine-König wrote:\n>>> On Wed, Mar 02, 2011 at 10:18:44PM +0100, Jakub Narebski wrote:\n\n>>>> What is most important that makes this feature to be considered for\n>>>> removal (or rehauling) is that only half of this feature is implemented\n>>>> in gitweb: the displaying part. There is half-attempt of providing\n>>>> some web interface for managing tags... which needs external script with\n>>>> strict coupling, doesn't offer any access control as far as I know, do\n>>>> not allow deleting tags, etc.\n>>>\n>>> For a small set of repositories the need to hand-edit the tags is OK\n>>> IMHO. That's what I intended to do.\n>> \n>> So what would you like to see?\n>> \n>> 1. Hardening parsing of ctags files, so that gitweb does not crash on\n>> malformed entries, but e.g. just ignores them.\n\nDone.\n \n>> 2. Generating tag cloud upfront, before sending any output to browser,\n>> to catch error better (and perhaps separate CSS for HTML::TagCloud).\n\nNot done, but with 1. it should be not very necessary... unless another\nbug is found in related code, that is.\n\n>> 3. Describe format of ctags files, either in comments in code, or in\n>> gitweb/README.\n\nDone (comments in code).\n\n>> 4. Either:\n>> \n>> A. Remove editing ctags from gitweb, or\n>> B. Add some simple generation of ctags file to gitweb\n\nDone 4.A.\n\n> yes :-) (1-3, 4a) I wouldn't mind 4b, but it should be possible to\n> disable the possibility to publically edit tags via gitweb.\n\n\nThe patch below (addressing those issues) is based on\n\n \"[PATCHv2/RFC] gitweb: Restructure projects list generation\"\n http://thread.gmane.org/gmane.comp.version-control.git/167996/focus=168321\n http://repo.or.cz/w/git/jnareb-git.git/commit/420071752d13dcecd59e794d82285e7e142ef75f\n ('gitweb/web' branch in http://repo.or.cz/w/git/jnareb-git.git repository)\n\n-- >8 --\nSubject: [RFC/PATCH] gitweb: Change the way \"content tags\" ('ctags') are handled\n\nThe major change is removing the ability to edit content tags (ctags)\nin a web browser.\n\nThe interface was created by gitweb, while actual editing of tags was\nto be done by external script; the API was not defined, and neither\nwas provided example implementation. Such split is also a bit fragile\n- interface and implementation have to be kept in sync. Gitweb\nprovided only ability to add tags; you could not edit tags nor delete\nthem.\n\n\nFormat of ctags is now described in the comment above git_get_project_ctags\nsubroutine. Gitweb now is more robust with respect to original ctags\nformat; it also accepts two new formats: $GIT_DIR/ctags file, with one\ncontent tag per line, and multi-value `gitweb.ctag' config variable.\n\nGathering all ctags of all project is now put in git_gather_all_ctags\nsubroutine, making git_project_list_body more clear.\n\ngit_populate_project_tagcloud subroutine now generates data used for\ntag cloud, including generation of ctag link, also in the case\nHTML::TagCloud module is unavailable. Links are now generated using\nhref() subroutine - this is more robust, as ctags might contain '?',\n';' and '=' special characters that need to be escaped in query param.\nShown tags are HTML-escaped.\n\nThe generation of tag cloud in git_show_project_tagcloud in the case\nwhen HTML::TagCloud is not available is now changed slightly.\n\nThe 'content tags' field on project summary page is made more in line\nwith other fields in \"projects_list\" table. Because one cannot now\nadd new tags from web interface, this field is no longer displayed\nwhen there are no content tags for given project.\n\nSigned-off-by: Jakub Narebski \n---\n gitweb/gitweb.perl | 141 +++++++++++++++++++++++++++++++++++----------------\n 1 files changed, 97 insertions(+), 44 deletions(-)\n\ndiff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl\nindex 996b647..4d80818 100755\n--- a/gitweb/gitweb.perl\n+++ b/gitweb/gitweb.perl\n@@ -412,20 +412,23 @@ our %feature = (\n \t\t'override' => 0,\n \t\t'default' => []},\n \n-\t# Allow gitweb scan project content tags described in ctags/\n-\t# of project repository, and display the popular Web 2.0-ish\n-\t# \"tag cloud\" near the project list. Note that this is something\n-\t# COMPLETELY different from the normal Git tags.\n+\t# Allow gitweb scan project content tags of project repository,\n+\t# and display the popular Web 2.0-ish \"tag cloud\" near the projects\n+\t# list. Note that this is something COMPLETELY different from the\n+\t# normal Git tags.\n \n \t# gitweb by itself can show existing tags, but it does not handle\n-\t# tagging itself; you need an external application for that.\n-\t# For an example script, check Girocco's cgi/tagproj.cgi.\n+\t# tagging itself; you need to do it externally, outside gitweb.\n+\t# The format is described in git_get_project_ctags() subroutine.\n \t# You may want to install the HTML::TagCloud Perl module to get\n \t# a pretty tag cloud instead of just a list of tags.\n \n \t# To enable system wide have in $GITWEB_CONFIG\n-\t# $feature{'ctags'}{'default'} = ['path_to_tag_script'];\n+\t# $feature{'ctags'}{'default'} = [1];\n \t# Project specific override is not supported.\n+\n+\t# In the future whether ctags editing is enabled might depend\n+\t# on the value, but using 1 should always mean no editing of ctags.\n \t'ctags' => {\n \t\t'override' => 0,\n \t\t'default' => [0]},\n@@ -703,6 +706,7 @@ our @cgi_param_mapping = (\n \tsnapshot_format => \"sf\",\n \textra_options => \"opt\",\n \tsearch_use_regexp => \"sr\",\n+\tctag => \"by_tag\",\n \t# this must be last entry (for manipulation from JavaScript)\n \tjavascript => \"js\"\n );\n@@ -2564,23 +2568,66 @@ sub git_get_project_description {\n \treturn $descr;\n }\n \n+# supported formats:\n+# * $GIT_DIR/ctags/ file (in 'ctags' subdirectory)\n+# - if its contents is a number, use it as tag weight,\n+# - otherwise add a tag with weight 1\n+# * $GIT_DIR/ctags file, each line is a tag (with weight 1)\n+# the same value multiple times increases tag weight\n+# * `gitweb.ctag' multi-valued repo config variable\n sub git_get_project_ctags {\n-\tmy $path = shift;\n+\tmy $project = shift;\n \tmy $ctags = {};\n \n-\t$git_dir = \"$projectroot/$path\";\n-\topendir my $dh, \"$git_dir/ctags\"\n-\t\tor return $ctags;\n-\tforeach (grep { -f $_ } map { \"$git_dir/ctags/$_\" } readdir($dh)) {\n-\t\topen my $ct, '<', $_ or next;\n-\t\tmy $val = <$ct>;\n-\t\tchomp $val;\n-\t\tclose $ct;\n-\t\tmy $ctag = $_; $ctag =~ s#.*/##;\n-\t\t$ctags->{$ctag} = $val;\n+\t$git_dir = \"$projectroot/$project\";\n+\tif (opendir my $dh, \"$git_dir/ctags\") {\n+\t\tmy @files = grep { -f $_ } map { \"$git_dir/ctags/$_\" } readdir($dh);\n+\t\tforeach my $tagfile (@files) {\n+\t\t\topen my $ct, '<', $tagfile\n+\t\t\t\tor next;\n+\t\t\tmy $val = <$ct>;\n+\t\t\tchomp $val if $val;\n+\t\t\tclose $ct;\n+\n+\t\t\t(my $ctag = $tagfile) =~ s#.*/##;\n+\t\t\tif ($val =~ /\\d+/) {\n+\t\t\t\t$ctags->{$ctag} = $val;\n+\t\t\t} else {\n+\t\t\t\t$ctags->{$ctag} = 1;\n+\t\t\t}\n+\t\t}\n+\t\tclosedir $dh;\n+\n+\t} elsif (open my $fh, '<', \"$git_dir/ctags\") {\n+\t\twhile (my $line = <$fh>) {\n+\t\t\tchomp $line;\n+\t\t\t$ctags->{$line}++ if $line;\n+\t\t}\n+\t\tclose $fh;\n+\n+\t} else {\n+\t\tmy $taglist = config_to_multi(git_get_project_config('ctag'));\n+\t\tforeach my $tag (@$taglist) {\n+\t\t\t$ctags->{$tag}++;\n+\t\t}\n \t}\n-\tclosedir $dh;\n-\t$ctags;\n+\n+\treturn $ctags;\n+}\n+\n+# return hash, where keys are content tags ('ctags'),\n+# and values are sum of weights of given tag in every project\n+sub git_gather_all_ctags {\n+\tmy $projects = shift;\n+\tmy $ctags = {};\n+\n+\tforeach my $p (@$projects) {\n+\t\tforeach my $ct (keys %{$p->{'ctags'}}) {\n+\t\t\t$ctags->{$ct} += $p->{'ctags'}->{$ct};\n+\t\t}\n+\t}\n+\n+\treturn $ctags;\n }\n \n sub git_populate_project_tagcloud {\n@@ -2600,31 +2647,41 @@ sub git_populate_project_tagcloud {\n \tmy $cloud;\n \tif (eval { require HTML::TagCloud; 1; }) {\n \t\t$cloud = HTML::TagCloud->new;\n-\t\tforeach (sort keys %ctags_lc) {\n+\t\tforeach my $ctag (sort keys %ctags_lc) {\n \t\t\t# Pad the title with spaces so that the cloud looks\n \t\t\t# less crammed.\n-\t\t\tmy $title = $ctags_lc{$_}->{topname};\n+\t\t\tmy $title = esc_html($ctags_lc{$ctag}->{topname});\n \t\t\t$title =~ s/ / /g;\n \t\t\t$title =~ s/^/ /g;\n \t\t\t$title =~ s/$/ /g;\n-\t\t\t$cloud->add($title, $home_link.\"?by_tag=\".$_, $ctags_lc{$_}->{count});\n+\t\t\t$cloud->add($title, href(project=>undef, ctag=>$ctag),\n+\t\t\t $ctags_lc{$ctag}->{count});\n \t\t}\n \t} else {\n-\t\t$cloud = \\%ctags_lc;\n+\t\t$cloud = {};\n+\t\tforeach my $ctag (keys %ctags_lc) {\n+\t\t\tmy $title = $ctags_lc{$ctag}->{topname};\n+\t\t\t$cloud->{$ctag}{count} = $ctags_lc{$ctag}->{count};\n+\t\t\t$cloud->{$ctag}{ctag} =\n+\t\t\t\t$cgi->a({-href=>href(project=>undef, ctag=>$ctag)},\n+\t\t\t esc_html($title, -nbsp=>1));\n+\t\t}\n \t}\n-\t$cloud;\n+\treturn $cloud;\n }\n \n sub git_show_project_tagcloud {\n \tmy ($cloud, $count) = @_;\n-\tprint STDERR ref($cloud).\"..\\n\";\n \tif (ref $cloud eq 'HTML::TagCloud') {\n \t\treturn $cloud->html_and_css($count);\n \t} else {\n-\t\tmy @tags = sort { $cloud->{$a}->{count} <=> $cloud->{$b}->{count} } keys %$cloud;\n-\t\treturn '
\" if %$ctags;\n-\t\tprint git_show_project_tagcloud($cloud, 48);\n-\t\tprint \"
\";\n+\t\tif (%$ctags) {\n+\t\t\t# without ability to add tags, don't show if there are none\n+\t\t\tmy $cloud = git_populate_project_tagcloud($ctags);\n+\t\t\tprint \"
\" .\n+\t\t\t \"
content tags
\" .\n+\t\t\t \"
\".git_show_project_tagcloud($cloud, 48).\"
\" .\n+\t\t\t \"
\\n\";\n+\t\t}\n \t}\n \n \tprint \"\\n\";\n-- \n1.7.3\n"},{"id":"163056","messageId":"201103091505.03045.jnareb@gmail.com","threadId":"26633","inReplyTo":"201103071900.16126.jnareb@gmail.com","subject":"[PATCH 2/1] gitweb: Mark matched 'ctag' / contents tag (?by_tag=foo)","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2011-03-09T14:04:59Z","receivedAt":"2011-03-09T14:04:59Z","isPatch":true,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"It might have been hard to discover that current view is limited to\nprojects with given content tag (ctag), as it was distinquished only\nin gitweb URL. Mark matched contents tag in the tag cloud using\n\"match\" class, for easier discovery.\n\nThis commit introduces a bit of further code duplication in\ngit_populate_project_tagcloud().\n\nSigned-off-by: Jakub Narebski \n---\n gitweb/gitweb.perl | 12 +++++++++---\n 1 files changed, 9 insertions(+), 3 deletions(-)\n\ndiff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl\nindex 4d80818..7ba8a72 100755\n--- a/gitweb/gitweb.perl\n+++ b/gitweb/gitweb.perl\n@@ -2645,6 +2645,7 @@ sub git_populate_project_tagcloud {\n \t}\n \n \tmy $cloud;\n+\tmy $matched = $cgi->param('by_tag');\n \tif (eval { require HTML::TagCloud; 1; }) {\n \t\t$cloud = HTML::TagCloud->new;\n \t\tforeach my $ctag (sort keys %ctags_lc) {\n@@ -2654,17 +2655,22 @@ sub git_populate_project_tagcloud {\n \t\t\t$title =~ s/ / /g;\n \t\t\t$title =~ s/^/ /g;\n \t\t\t$title =~ s/$/ /g;\n+\t\t\tif (defined $matched && $matched eq $ctag) {\n+\t\t\t\t$title = qq($title);\n+\t\t\t}\n \t\t\t$cloud->add($title, href(project=>undef, ctag=>$ctag),\n \t\t\t $ctags_lc{$ctag}->{count});\n \t\t}\n \t} else {\n \t\t$cloud = {};\n \t\tforeach my $ctag (keys %ctags_lc) {\n-\t\t\tmy $title = $ctags_lc{$ctag}->{topname};\n+\t\t\tmy $title = esc_html($ctags_lc{$ctag}->{topname}, -nbsp=>1);\n+\t\t\tif (defined $matched && $matched eq $ctag) {\n+\t\t\t\t$title = qq($title);\n+\t\t\t}\n \t\t\t$cloud->{$ctag}{count} = $ctags_lc{$ctag}->{count};\n \t\t\t$cloud->{$ctag}{ctag} =\n-\t\t\t\t$cgi->a({-href=>href(project=>undef, ctag=>$ctag)},\n-\t\t\t esc_html($title, -nbsp=>1));\n+\t\t\t\t$cgi->a({-href=>href(project=>undef, ctag=>$ctag)}, $title);\n \t\t}\n \t}\n \treturn $cloud;\n-- \n1.7.3\n"},{"id":"163057","messageId":"20110309140939.GV24584@machine.or.cz","threadId":"26633","inReplyTo":"201103091505.03045.jnareb@gmail.com","subject":"Re: [PATCH 2/1] gitweb: Mark matched 'ctag' / contents tag (?by_tag=foo)","fromName":"Petr Baudis","fromEmail":"pasky@suse.cz","sentAt":"2011-03-09T14:09:40Z","receivedAt":"2011-03-09T14:09:40Z","isPatch":true,"sender":{"key":"pasky@ucw.cz","avatar":"https://avatars.githubusercontent.com/u/18439?v=4"},"body":"On Wed, Mar 09, 2011 at 03:04:59PM +0100, Jakub Narebski wrote:\n> It might have been hard to discover that current view is limited to\n> projects with given content tag (ctag), as it was distinquished only\n> in gitweb URL. Mark matched contents tag in the tag cloud using\n> \"match\" class, for easier discovery.\n> \n> This commit introduces a bit of further code duplication in\n> git_populate_project_tagcloud().\n> \n> Signed-off-by: Jakub Narebski \n\nAcked-by: Petr Baudis \n"},{"id":"169734","messageId":"20110609070857.GA735@elie","threadId":"26633","inReplyTo":"201103071900.16126.jnareb@gmail.com","subject":"[PATCH] gitweb: do misparse nonnumeric content tag files that contain a digit","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2011-06-09T07:08:57Z","receivedAt":"2011-06-09T07:08:57Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"v1.7.6-rc0~27^2~4 (gitweb: Change the way \"content tags\" ('ctags') are\nhandled, 2011-04-29) tried to make gitweb's tag cloud feature more\nintuitive for webmasters by checking whether the ctags/