{"thread":{"id":"28132","subject":"[PATCH/RFC] gitweb: highlight: strip non-printable characters via col(1)","startedAt":"2011-08-16T18:16:44Z","lastAt":"2011-08-16T21:32:12Z","messageCount":3,"participants":["Christopher M. Fuhrman","J.H."],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"173629","messageId":"1313518605-26460-1-git-send-email-cfuhrman@panix.com","threadId":"28132","inReplyTo":null,"subject":"[PATCH/RFC] gitweb: highlight: strip non-printable characters via col(1)","fromName":"Christopher M. Fuhrman","fromEmail":"cfuhrman@panix.com","sentAt":"2011-08-16T18:16:44Z","receivedAt":"2011-08-16T18:16:44Z","isPatch":true,"sender":{"key":"cfuhrman@panix.com","avatar":"https://gravatar.com/avatar/ec9fed21fc8fadbb9fab491480c80a464ac25a0632eedfa594ca6c2f39f04c2d?d=mp&s=160"},"body":"From: \"Christopher M. Fuhrman\" \n\nThe current code, as is, passes control characters, such as form-feed\n(^L) to highlight which then passes it through to the browser. This\nwill cause the browser to display one of the following warnings:\n\nSafari v5.1 (6534.50) & Google Chrome v13.0.782.112:\n\n This page contains the following errors:\n\n error on line 657 at column 38: PCDATA invalid Char value 12\n Below is a rendering of the page up to the first error.\n\nMozilla Firefox 3.6.19 & Mozilla Firefox 5.0:\n\n XML Parsing Error: not well-formed\n Location:\n http://path/to/git/repo/blah/blah\n\nBoth errors were generated by gitweb.perl v1.7.3.4 w/ highlight 2.7\nusing arch/ia64/kernel/unwind.c from the Linux kernel.\n\nStrip non-printable control-characters by piping the output produced\nby git-cat-file(1) to col(1) as follows:\n\n git cat-file blob deadbeef314159 | col -bx | highlight \n\nTested under OpenSuSE 11.4 & NetBSD 5.1 using perl 5.12.3 and perl\n5.12.2 respectively using Safari, Firefox, and Google Chrome.\n\nSigned-off-by: Christopher M. Fuhrman \n---\n\nFor an example of this bug in action, see:\n\n *\n http://git.fuhrbear.com/~cfuhrman/?p=linux/.git;a=blob;f=arch/alpha/kernel/core_titan.c;h=219bf271c0ba2e5f2d668af707df57fbbd00ccfd;hb=HEAD\n *\n http://git.fuhrbear.com/~cfuhrman/?p=linux/.git;a=blob;f=arch/ia64/kernel/unwind.c;h=fed6afa2e8a9014e65229e51e64fa4b1c13cc284;hb=HEAD\n\nWRT the col(1) command, I've verified that the binary is installed in\n/usr/bin on OpenSuSE, NetBSD, OpenBSD, Solaris 10, and AIX. This\npatch assumes that /usr/bin is in $PATH.\n\nCheers!\n\n gitweb/gitweb.perl | 1 +\n 1 files changed, 1 insertions(+), 0 deletions(-)\n\ndiff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl\nindex 81dacf2..38d5d4e 100755\n--- a/gitweb/gitweb.perl\n+++ b/gitweb/gitweb.perl\n@@ -3656,6 +3656,7 @@ sub run_highlighter {\n \n \tclose $fd;\n \topen $fd, quote_command(git_cmd(), \"cat-file\", \"blob\", $hash).\" | \".\n+\t \"col -bx | \".\n \t quote_command($highlight_bin).\n \t \" --replace-tabs=8 --fragment --syntax $syntax |\"\n \t\tor die_error(500, \"Couldn't open file or run syntax highlighter\");\n-- \n1.7.5.4\n"},{"id":"173637","messageId":"4E4AD35E.8060907@eaglescrag.net","threadId":"28132","inReplyTo":"1313518605-26460-1-git-send-email-cfuhrman@panix.com","subject":"Re: [PATCH/RFC] gitweb: highlight: strip non-printable characters via col(1)","fromName":"J.H.","fromEmail":"warthog9@eaglescrag.net","sentAt":"2011-08-16T20:30:22Z","receivedAt":"2011-08-16T20:30:22Z","isPatch":true,"sender":{"key":"warthog9@kernel.org","avatar":"https://avatars.githubusercontent.com/u/2334704?v=4"},"body":"On 08/16/2011 11:16 AM, Christopher M. Fuhrman wrote:\n> From: \"Christopher M. Fuhrman\" \n> \n> The current code, as is, passes control characters, such as form-feed\n> (^L) to highlight which then passes it through to the browser. This\n> will cause the browser to display one of the following warnings:\n> \n> Safari v5.1 (6534.50) & Google Chrome v13.0.782.112:\n> \n> This page contains the following errors:\n> \n> error on line 657 at column 38: PCDATA invalid Char value 12\n> Below is a rendering of the page up to the first error.\n> \n> Mozilla Firefox 3.6.19 & Mozilla Firefox 5.0:\n> \n> XML Parsing Error: not well-formed\n> Location:\n> http://path/to/git/repo/blah/blah\n> \n> Both errors were generated by gitweb.perl v1.7.3.4 w/ highlight 2.7\n> using arch/ia64/kernel/unwind.c from the Linux kernel.\n> \n> Strip non-printable control-characters by piping the output produced\n> by git-cat-file(1) to col(1) as follows:\n> \n> git cat-file blob deadbeef314159 | col -bx | highlight \n\nSo my only real concern here is that `col` itself is going to munge\nwhitespace. Quoting from the col man page:\n\n\t[...] and replaces white-space characters with tabs where\n\t possible. [...]\n\nHave you actually run into a situation where something like ^L was\npresent in a blob that was being passed to highlight?\n\n- John\n"},{"id":"173639","messageId":"alpine.NEB.2.01.1108161414030.7527@vc75.vc.panix.com","threadId":"28132","inReplyTo":"4E4AD35E.8060907@eaglescrag.net","subject":"Re: [PATCH/RFC] gitweb: highlight: strip non-printable characters via col(1)","fromName":"Christopher M. Fuhrman","fromEmail":"cfuhrman@panix.com","sentAt":"2011-08-16T21:32:12Z","receivedAt":"2011-08-16T21:32:12Z","isPatch":true,"sender":{"key":"cfuhrman@panix.com","avatar":"https://gravatar.com/avatar/ec9fed21fc8fadbb9fab491480c80a464ac25a0632eedfa594ca6c2f39f04c2d?d=mp&s=160"},"body":"On Tue, 16 Aug 2011 at 1:30pm, J.H. wrote:\n\n> On 08/16/2011 11:16 AM, Christopher M. Fuhrman wrote:\n> > From: \"Christopher M. Fuhrman\" \n> >\n> > The current code, as is, passes control characters, such as form-feed\n> > (^L) to highlight which then passes it through to the browser. This\n> > will cause the browser to display one of the following warnings:\n> >\n\n\n\n> > Strip non-printable control-characters by piping the output produced\n> > by git-cat-file(1) to col(1) as follows:\n> >\n> > git cat-file blob deadbeef314159 | col -bx | highlight \n>\n> So my only real concern here is that `col` itself is going to munge\n> whitespace. Quoting from the col man page:\n>\n> \t[...] and replaces white-space characters with tabs where\n> \t possible. [...]\n\nI figured that would be a concern which is why I added the -x option.\nFrom the col(1) man page:\n\n -x Output multiple spaces instead of tabs.\n\nI also took a diff between two XHTML files. One that used col -bx and one\nthat didn't. Here's the results:\n\n--- withoutcol.xhtml\t2011-08-16 14:11:39.000000000 -0700\n+++ withcol.xhtml\t2011-08-16 14:11:26.000000000 -0700\n@@ -52,7 +52,7 @@\n # define DBG_CFG(args)\n #endif\n\n-\n+\n /*\n * Routines to access TIG registers.\n */\n@@ -76,7 +76,7 @@\n *tig_addr = (unsigned long)value;\n }\n\n-\n+\n /*\n * Given a bus, device, and function number, compute resulting\n * configuration space address\n@@ -197,7 +197,7 @@\n .write = titan_write_config,\n };\n\n(remainder stripped)\n\n>\n> Have you actually run into a situation where something like ^L was\n> present in a blob that was being passed to highlight?\n>\n\nI've seen ^L is the Linux kernel source tree as well as the NetBSD src\ntree. I've not encountered it elsewhere although I would think it would\nbe present depending on personal/corporate coding preferences.\n\n> - John\n\nCheers!\n\n-- \nChris Fuhrman\ncfuhrman@panix.com\n"}]}