threads / patch / 57938

patchgitweb: switch to a modern DOCTYPE

Subject: [PATCH] gitweb: switch to a modern DOCTYPE

## tl;dr

9 messages between Jun 1, 2022 and Jun 2, 2022. Diffs are folded; open one to read it.

replies: 8people: 4as markdown or json

Jason Yundt· Jun 1, 2022, 01:26 UTC · lore
According to the HTML Standard FAQ:
	“What is the DOCTYPE for modern HTML documents?
	In text/html documents:
		<!DOCTYPE html>
	In documents delivered with an XML media type: no DOCTYPE is required
	and its use is generally unnecessary. However, you may use one if you
	want (see the following question). Note that the above is well-formed
	XML.”
	Source: [1]
Gitweb uses an XHTML 1.0 DOCTYPE:
	<!DOCTYPE html PUBLIC
	"-//W3C//DTD XHTML 1.0 Strict//EN"
	"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
While that DOCTYPE is still valid [2], it has several disadvantages:
1. It’s misleading. The DTD that browsers are supposed to use with that
   DOCTYPE has nothing to do with XHTML 1.0 and isn’t available at the URL
   that is given [2].
2. It’s obsolete. XHTML 1.0 was last revised in 2002 and was superseded in
   2018 [3].
3. It’s unreliable. Gitweb uses &nbsp; and &sdot; but lets an external file
   define them. “[…U]using entity references for characters in XML documents
   is unsafe if they are defined in an external file (except for &lt;, &gt;,
   &amp;, &quot;, and &apos;).” [4]

[1]: <https://github.com/whatwg/html/blob/main/FAQ.md#what-is-the-doctype-for-modern-html-documents> [2]: <https://html.spec.whatwg.org/multipage/xhtml.html#parsing-xhtml-documents> [3]: <https://www.w3.org/TR/xhtml1/#xhtml> [4]: <https://html.spec.whatwg.org/multipage/xhtml.html#writing-xhtml-documents>

Signed-off-by: Jason Yundt <jason@jasonyundt.email>
---
 gitweb/gitweb.perl                        |  5 ++++-
 t/t9502-gitweb-standalone-parse-output.sh | 14 ++++++++++++++
 2 files changed, 18 insertions(+), 1 deletion(-)
Show changes to 2 files +18 −1

gitweb/gitweb.perl, t/t9502-gitweb-standalone-parse-output.sh

diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
index 606b50104c..1835487ab2 100755
--- a/gitweb/gitweb.perl
+++ b/gitweb/gitweb.perl
@@ -4219,7 +4219,10 @@ sub git_header_html {
 	my $mod_perl_version = $ENV{'MOD_PERL'} ? " $ENV{'MOD_PERL'}" : '';
 	print <<EOF;
 <?xml version="1.0" encoding="utf-8"?>
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
+<!DOCTYPE html [
+	<!ENTITY nbsp "&#xA0;">
+	<!ENTITY sdot "&#x22C5;">
+]>
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en-US" lang="en-US">
 <!-- git web interface version $version, (C) 2005-2006, Kay Sievers <kay.sievers\@vrfy.org>, Christian Gierke -->
 <!-- git core binaries version $git_version -->
diff --git a/t/t9502-gitweb-standalone-parse-output.sh b/t/t9502-gitweb-standalone-parse-output.sh
index 8cb582f0e6..81d5625557 100755
--- a/t/t9502-gitweb-standalone-parse-output.sh
+++ b/t/t9502-gitweb-standalone-parse-output.sh
@@ -220,4 +220,18 @@ test_expect_success 'no http-equiv="content-type" in XHTML' '
 	no_http_equiv_content_type "p=.git;a=tree"
 '
 
+proper_doctype() {
+	gitweb_run "$@" &&
+	grep -F "<!DOCTYPE html [" gitweb.body &&
+	grep "<!ENTITY nbsp" gitweb.body &&
+	grep "<!ENTITY sdot" gitweb.body
+}
+
+test_expect_success 'Proper DOCTYPE with entity declarations' '
+	proper_doctype &&
+	proper_doctype "p=.git" &&
+	proper_doctype "p=.git;a=log" &&
+	proper_doctype "p=.git;a=tree"
+'
+
 test_done
-- 
2.36.1
brian m. carlson· Jun 2, 2022, 00:41 UTC · re: Jason Yundt · lore

Re: [PATCH] gitweb: switch to a modern DOCTYPE

On 2022-06-01 at 01:26:47, Jason Yundt wrote:
Show 26 quoted lines
> According to the HTML Standard FAQ:
> 
> 	“What is the DOCTYPE for modern HTML documents?
> 
> 	In text/html documents:
> 
> 		<!DOCTYPE html>
> 
> 	In documents delivered with an XML media type: no DOCTYPE is required
> 	and its use is generally unnecessary. However, you may use one if you
> 	want (see the following question). Note that the above is well-formed
> 	XML.”
> 
> 	Source: [1]
> 
> Gitweb uses an XHTML 1.0 DOCTYPE:
> 
> 	<!DOCTYPE html PUBLIC
> 	"-//W3C//DTD XHTML 1.0 Strict//EN"
> 	"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
> 
> While that DOCTYPE is still valid [2], it has several disadvantages:
> 
> 1. It’s misleading. The DTD that browsers are supposed to use with that
>    DOCTYPE has nothing to do with XHTML 1.0 and isn’t available at the URL
>    that is given [2].

While the WHATWG may claim that, an XML parser is absolutely within its rights to refer to and use that DTD, and in fact should do so unless its catalog directs it elsewhere. It may be that some browsers use an internal catalog that refers to a different DTD, however.

Show 13 quoted lines
> diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
> index 606b50104c..1835487ab2 100755
> --- a/gitweb/gitweb.perl
> +++ b/gitweb/gitweb.perl
> @@ -4219,7 +4219,10 @@ sub git_header_html {
>  	my $mod_perl_version = $ENV{'MOD_PERL'} ? " $ENV{'MOD_PERL'}" : '';
>  	print <<EOF;
>  <?xml version="1.0" encoding="utf-8"?>
> -<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
> +<!DOCTYPE html [
> +	<!ENTITY nbsp "&#xA0;">
> +	<!ENTITY sdot "&#x22C5;">
> +]>

I think this should be fine. It defines the entities we need and appears to be valid XML. I don't think there should be any problem upgrading to XHTML 5 here.

-- 
brian m. carlson (he/him or they/them)
Toronto, Ontario, CA
Junio C Hamano· Jun 2, 2022, 06:10 UTC · re: brian m. carlson · lore

Re: [PATCH] gitweb: switch to a modern DOCTYPE

"brian m. carlson" <sandals@crustytoothpaste.net> writes:
Show 28 quoted lines
>> While that DOCTYPE is still valid [2], it has several disadvantages:
>> 
>> 1. It’s misleading. The DTD that browsers are supposed to use with that
>>    DOCTYPE has nothing to do with XHTML 1.0 and isn’t available at the URL
>>    that is given [2].
>
> While the WHATWG may claim that, an XML parser is absolutely within its
> rights to refer to and use that DTD, and in fact should do so unless its
> catalog directs it elsewhere.  It may be that some browsers use an
> internal catalog that refers to a different DTD, however.
>
>> diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
>> index 606b50104c..1835487ab2 100755
>> --- a/gitweb/gitweb.perl
>> +++ b/gitweb/gitweb.perl
>> @@ -4219,7 +4219,10 @@ sub git_header_html {
>>  	my $mod_perl_version = $ENV{'MOD_PERL'} ? " $ENV{'MOD_PERL'}" : '';
>>  	print <<EOF;
>>  <?xml version="1.0" encoding="utf-8"?>
>> -<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
>> +<!DOCTYPE html [
>> +	<!ENTITY nbsp "&#xA0;">
>> +	<!ENTITY sdot "&#x22C5;">
>> +]>
>
> I think this should be fine.  It defines the entities we need and
> appears to be valid XML.  I don't think there should be any problem
> upgrading to XHTML 5 here.

OK, so in short, the patch text looks OK and the proposed log message needs a bit more work?

Thanks.
Bagas Sanjaya· Jun 2, 2022, 07:26 UTC · re: Jason Yundt · lore

Re: [PATCH] gitweb: switch to a modern DOCTYPE

On 6/1/22 08:26, Jason Yundt wrote:
Show 39 quoted lines
> According to the HTML Standard FAQ:
> 
> 	“What is the DOCTYPE for modern HTML documents?
> 
> 	In text/html documents:
> 
> 		<!DOCTYPE html>
> 
> 	In documents delivered with an XML media type: no DOCTYPE is required
> 	and its use is generally unnecessary. However, you may use one if you
> 	want (see the following question). Note that the above is well-formed
> 	XML.”
> 
> 	Source: [1]
> 
> Gitweb uses an XHTML 1.0 DOCTYPE:
> 
> 	<!DOCTYPE html PUBLIC
> 	"-//W3C//DTD XHTML 1.0 Strict//EN"
> 	"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
> 
> While that DOCTYPE is still valid [2], it has several disadvantages:
> 
> 1. It’s misleading. The DTD that browsers are supposed to use with that
>    DOCTYPE has nothing to do with XHTML 1.0 and isn’t available at the URL
>    that is given [2].
> 2. It’s obsolete. XHTML 1.0 was last revised in 2002 and was superseded in
>    2018 [3].
> 3. It’s unreliable. Gitweb uses &nbsp; and &sdot; but lets an external file
>    define them. “[…U]using entity references for characters in XML documents
>    is unsafe if they are defined in an external file (except for &lt;, &gt;,
>    &amp;, &quot;, and &apos;).” [4]
> 
> [1]: <https://github.com/whatwg/html/blob/main/FAQ.md#what-is-the-doctype-for-modern-html-documents>
> [2]: <https://html.spec.whatwg.org/multipage/xhtml.html#parsing-xhtml-documents>
> [3]: <https://www.w3.org/TR/xhtml1/#xhtml>
> [4]: <https://html.spec.whatwg.org/multipage/xhtml.html#writing-xhtml-documents>
> 
> Signed-off-by: Jason Yundt <jason@jasonyundt.email>

So basically what this patch does is switch to HTML5, right? That is because I can see DOCTYPE "upgrade" to use "<!DOCTYPE html>", which is the DOCTYPE for HTML5. If it does, then mention HTML5 in v2.

-- 
An old man doll... just what I always wanted! - Clara
Jason Yundt· Jun 2, 2022, 11:43 UTC · re: Jason Yundt · lore

[PATCH v2] gitweb: switch to an XHTML5 DOCTYPE

According to the HTML Standard FAQ:
	“What is the DOCTYPE for modern HTML documents?
	In text/html documents:
		<!DOCTYPE html>
	In documents delivered with an XML media type: no DOCTYPE is required
	and its use is generally unnecessary. However, you may use one if you
	want (see the following question). Note that the above is well-formed
	XML.”
	Source: [1]
Gitweb uses an XHTML 1.0 DOCTYPE:
	<!DOCTYPE html PUBLIC
	"-//W3C//DTD XHTML 1.0 Strict//EN"
	"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
While that DOCTYPE is still valid [2], it has several disadvantages:
1. It’s misleading. If an XML parser uses the DTD at the given link,
   then the entities &nbsp; and &sdot; won’t get declared. Instead, the
   parser has to use a DTD from the HTML Standard that has nothing to do
   with XHTML 1.0 [2].
2. It’s obsolete. XHTML 1.0 was last revised in 2002 and was superseded in
   2018 [3].
3. It’s unreliable. Gitweb uses &nbsp; and &sdot; but lets an external file
   define them. “[…U]using entity references for characters in XML documents
   is unsafe if they are defined in an external file (except for &lt;, &gt;,
   &amp;, &quot;, and &apos;).” [4]

[1]: <https://github.com/whatwg/html/blob/main/FAQ.md#what-is-the-doctype-for-modern-html-documents> [2]: <https://html.spec.whatwg.org/multipage/xhtml.html#parsing-xhtml-documents> [3]: <https://www.w3.org/TR/xhtml1/#xhtml> [4]: <https://html.spec.whatwg.org/multipage/xhtml.html#writing-xhtml-documents>

Signed-off-by: Jason Yundt <jason@jasonyundt.email>
---
Compared to the first version of this patch, this version:
1. makes it clear that XML parsers may used the linked DTD like brian
   mentioned.
2. mentions HTML5 like Bagas suggested.
 gitweb/gitweb.perl                        |  5 ++++-
 t/t9502-gitweb-standalone-parse-output.sh | 14 ++++++++++++++
 2 files changed, 18 insertions(+), 1 deletion(-)
Show changes to 2 files +18 −1

gitweb/gitweb.perl, t/t9502-gitweb-standalone-parse-output.sh

diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
index 606b50104c..1835487ab2 100755
--- a/gitweb/gitweb.perl
+++ b/gitweb/gitweb.perl
@@ -4219,7 +4219,10 @@ sub git_header_html {
 	my $mod_perl_version = $ENV{'MOD_PERL'} ? " $ENV{'MOD_PERL'}" : '';
 	print <<EOF;
 <?xml version="1.0" encoding="utf-8"?>
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
+<!DOCTYPE html [
+	<!ENTITY nbsp "&#xA0;">
+	<!ENTITY sdot "&#x22C5;">
+]>
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en-US" lang="en-US">
 <!-- git web interface version $version, (C) 2005-2006, Kay Sievers <kay.sievers\@vrfy.org>, Christian Gierke -->
 <!-- git core binaries version $git_version -->
diff --git a/t/t9502-gitweb-standalone-parse-output.sh b/t/t9502-gitweb-standalone-parse-output.sh
index 8cb582f0e6..81d5625557 100755
--- a/t/t9502-gitweb-standalone-parse-output.sh
+++ b/t/t9502-gitweb-standalone-parse-output.sh
@@ -220,4 +220,18 @@ test_expect_success 'no http-equiv="content-type" in XHTML' '
 	no_http_equiv_content_type "p=.git;a=tree"
 '
 
+proper_doctype() {
+	gitweb_run "$@" &&
+	grep -F "<!DOCTYPE html [" gitweb.body &&
+	grep "<!ENTITY nbsp" gitweb.body &&
+	grep "<!ENTITY sdot" gitweb.body
+}
+
+test_expect_success 'Proper DOCTYPE with entity declarations' '
+	proper_doctype &&
+	proper_doctype "p=.git" &&
+	proper_doctype "p=.git;a=log" &&
+	proper_doctype "p=.git;a=tree"
+'
+
 test_done
-- 
2.36.1
Junio C Hamano· Jun 2, 2022, 18:50 UTC · re: Jason Yundt · lore

Re: [PATCH v2] gitweb: switch to an XHTML5 DOCTYPE

Jason Yundt <jason@jasonyundt.email> writes:
Show 10 quoted lines
> Subject: Re: [PATCH v2] gitweb: switch to an XHTML5 DOCTYPE
>
> According to the HTML Standard FAQ:
>
> 	“What is the DOCTYPE for modern HTML documents?
> ...
> Compared to the first version of this patch, this version:
> 1. makes it clear that XML parsers may used the linked DTD like brian
>    mentioned.
> 2. mentions HTML5 like Bagas suggested.
So, is it XHTML5, or HTML5, we want to see on the title?
Show 6 quoted lines
> +proper_doctype() {
> +	gitweb_run "$@" &&
> +	grep -F "<!DOCTYPE html [" gitweb.body &&
> +	grep "<!ENTITY nbsp" gitweb.body &&
> +	grep "<!ENTITY sdot" gitweb.body
> +}

Hmph, this test does not care what other cruft appears in the file, does not care in what order the three lines that match the patterns appear, and the second and third patterns are even allowed to match the same line. I think that is OK (we do not even mind if the two ENTITY definitions get squashed on the same line).

Show 6 quoted lines
> +test_expect_success 'Proper DOCTYPE with entity declarations' '
> +	proper_doctype &&
> +	proper_doctype "p=.git" &&
> +	proper_doctype "p=.git;a=log" &&
> +	proper_doctype "p=.git;a=tree"
> +'

As far as I can tell, git_header_html() is the only helper that deals with DOCTYPE, and responses to any request must call git_header_html() to produce the header (or the handler for a particular request type is buggy), but I do not think it is part of this topic's job to ensure that all request handlers call the git_header_html(). So we _could_ do with just a single test without trying different request types if we wanted to, as long as there are existing tests that make sure everybody uses git_header_html().

Was there a particular reason why these four requests were chosen? Do they have different entry points and show the doctype from different codepath?

Thanks.
Jason Yundt· Jun 2, 2022, 19:51 UTC · re: Junio C Hamano · lore

Re: [PATCH v2] gitweb: switch to an XHTML5 DOCTYPE

On Thursday, June 2, 2022 2:50:55 PM EDT Junio C Hamano wrote:
Show 15 quoted lines
> Jason Yundt <jason@jasonyundt.email> writes:
> > Subject: Re: [PATCH v2] gitweb: switch to an XHTML5 DOCTYPE
> > 
> > According to the HTML Standard FAQ:
> > 	“What is the DOCTYPE for modern HTML documents?
> > 
> > ...
> > Compared to the first version of this patch, this version:
> > 1. makes it clear that XML parsers may used the linked DTD like brian
> > 
> >    mentioned.
> > 
> > 2. mentions HTML5 like Bagas suggested.
> 
> So, is it XHTML5, or HTML5, we want to see on the title?

I chose XHTML5 since I didn’t think that it was accurate to say “HTML5 DOCTYPE”. The DOCTYPE that this patch uses is valid in the XML syntax, but not the HTML syntax.

Show 12 quoted lines
> > +proper_doctype() {
> > +	gitweb_run "$@" &&
> > +	grep -F "<!DOCTYPE html [" gitweb.body &&
> > +	grep "<!ENTITY nbsp" gitweb.body &&
> > +	grep "<!ENTITY sdot" gitweb.body
> > +}
> 
> Hmph, this test does not care what other cruft appears in the file,
> does not care in what order the three lines that match the patterns
> appear, and the second and third patterns are even allowed to match
> the same line.  I think that is OK (we do not even mind if the two
> ENTITY definitions get squashed on the same line).

While I was writing this patch, I was thinking something similar. Grep is not a good tool for validating (X)HTML. I thought about creating a test that uses the Nu Html Checker [1] to validate pages that Gitweb generates, but I decided that that should be the topic of a separate patch.

Show 19 quoted lines
> > +test_expect_success 'Proper DOCTYPE with entity declarations' '
> > +	proper_doctype &&
> > +	proper_doctype "p=.git" &&
> > +	proper_doctype "p=.git;a=log" &&
> > +	proper_doctype "p=.git;a=tree"
> > +'
> 
> As far as I can tell, git_header_html() is the only helper that
> deals with DOCTYPE, and responses to any request must call
> git_header_html() to produce the header (or the handler for a
> particular request type is buggy), but I do not think it is part of
> this topic's job to ensure that all request handlers call the
> git_header_html().  So we _could_ do with just a single test without
> trying different request types if we wanted to, as long as there are
> existing tests that make sure everybody uses git_header_html().
> 
> Was there a particular reason why these four requests were chosen?
> Do they have different entry points and show the doctype from
> different codepath?

Not really. When I created a262585d81 (gitweb: remove invalid http- equiv="content-type", 2022-03-08), I chose those requests by running git instaweb and then clicking on the first four links I saw. For this patch, I just copied what I had done previously. I don’t know if they use different codepaths (I don’t understand Perl very well).

> Thanks.
[1]: <https://validator.w3.org/nu/>
brian m. carlson· Jun 2, 2022, 21:47 UTC · re: Junio C Hamano · lore

Re: [PATCH v2] gitweb: switch to an XHTML5 DOCTYPE

On 2022-06-02 at 18:50:55, Junio C Hamano wrote:
Show 14 quoted lines
> Jason Yundt <jason@jasonyundt.email> writes:
> 
> > Subject: Re: [PATCH v2] gitweb: switch to an XHTML5 DOCTYPE
> >
> > According to the HTML Standard FAQ:
> >
> > 	“What is the DOCTYPE for modern HTML documents?
> > ...
> > Compared to the first version of this patch, this version:
> > 1. makes it clear that XML parsers may used the linked DTD like brian
> >    mentioned.
> > 2. mentions HTML5 like Bagas suggested.
> 
> So, is it XHTML5, or HTML5, we want to see on the title?

It's XHTML 5. Technically, it's the XML syntax of HTML 5, so I don't think we care much about the nit here.

Show 12 quoted lines
> > +proper_doctype() {
> > +	gitweb_run "$@" &&
> > +	grep -F "<!DOCTYPE html [" gitweb.body &&
> > +	grep "<!ENTITY nbsp" gitweb.body &&
> > +	grep "<!ENTITY sdot" gitweb.body
> > +}
> 
> Hmph, this test does not care what other cruft appears in the file,
> does not care in what order the three lines that match the patterns
> appear, and the second and third patterns are even allowed to match
> the same line.  I think that is OK (we do not even mind if the two
> ENTITY definitions get squashed on the same line).

Yeah, that should be fine because IIRC this syntax isn't going to be allowed elsewhere in the file. If it did appear elsewhere, then the user would receive an XML parsing error.

We _could_ make this check for beginning of file, the XML declaration, and then the specified lines (e.g., with head(1)), but I don't really think that's necessary. We should be fine here for now, and if we decide we need that in the future, we can add it. I'd be happy with this as it stands.

-- 
brian m. carlson (he/him or they/them)
Toronto, Ontario, CA
Junio C Hamano· Jun 2, 2022, 22:09 UTC · re: brian m. carlson · lore

Re: [PATCH v2] gitweb: switch to an XHTML5 DOCTYPE

"brian m. carlson" <sandals@crustytoothpaste.net> writes:
Show 8 quoted lines
>> > Subject: Re: [PATCH v2] gitweb: switch to an XHTML5 DOCTYPE
>> ...
>> > 2. mentions HTML5 like Bagas suggested.
>> 
>> So, is it XHTML5, or HTML5, we want to see on the title?
>
> It's XHTML 5.  Technically, it's the XML syntax of HTML 5, so I don't
> think we care much about the nit here.
I too don't care either way very much.

I just noticed the discrepancy between the title and the body, but the mention of HTML5 is after the three-dash lines so nobody gets hurt anyway ;-)

Show 5 quoted lines
> We _could_ make this check for beginning of file, the XML declaration,
> and then the specified lines (e.g., with head(1)), but I don't really
> think that's necessary. We should be fine here for now, and if we decide
> we need that in the future, we can add it.  I'd be happy with this as it
> stands.
Yup, thanks for sanity checking.
Will queue.

← back to recent threads