threads / patch / 44934

patch, 7 partsMacro for Asciidoctor support

Subject: [PATCH 0/7] Macro for Asciidoctor support

## tl;dr

9 messages between Jan 21, 2017 and Jan 22, 2017. Diffs are folded; open one to read it.

replies: 8people: 2as markdown or json

brian m. carlson· Jan 21, 2017, 21:59 UTC · lore

There are two major processors of AsciiDoc: AsciiDoc itself, and Asciidoctor. Both have advantages and disadvantages, but traditionally the documentation has been built with AsciiDoc, leading to some surprising breakage when building with Asciidoctor. Partially, this is due to the need to specify a significant number of macros on the command line when building with Asciidoctor.

This series cleans up some issues building the documentation with Asciidoctor and provides two knobs, USE_ASCIIDOCTOR, which controls building with Asciidoctor, and ASCIIDOCTOR_EXTENSIONS_LAB, which controls the location of the Asciidoctor Extensions Lab, which is necessary to expand the linkgit macro.

The need for the extensions could be replaced with a small amount of Ruby code, if that's considered desirable. Previous opinions on doing so were negative, however.

In the process, I found several issues with cat-texi.perl, which have been fixed. It has also been modernized to use strict, warnings, and lexical file handles. I also made an attempt to produce more diffable texi files; I may follow up with additional series along this line to make the documentation build reproducibly.

brian m. carlson (7):
  Documentation: fix warning in cat-texi.perl
  Documentation: modernize cat-texi.perl
  Documentation: remove unneeded argument in cat-texi.perl
  Documentation: sort sources for gitman.texi
  Documentation: add XSLT to fix DocBook for Texinfo
  Documentation: move dblatex arguments into variable
  Makefile: add a knob to enable the use of Asciidoctor
 Documentation/Makefile      | 22 ++++++++++++++++++----
 Documentation/cat-texi.perl | 21 ++++++++++++---------
 Documentation/texi.xsl      | 26 ++++++++++++++++++++++++++
 Makefile                    |  6 ++++++
 4 files changed, 62 insertions(+), 13 deletions(-)
 create mode 100644 Documentation/texi.xsl
brian m. carlson· Jan 21, 2017, 21:59 UTC · re: brian m. carlson · lore

[PATCH 3/7] Documentation: remove unneeded argument in cat-texi.perl

The newly-added use of the warnings pragma exposes that the $menu[0] argument to printf has long been silently ignored, since there is no format specifier for it. It doesn't appear that the argument is actually needed, either: there is no reason to insert the name of one particular documentation page anywhere in the header that's being generated.

Remove the unused argument, and since the format specification functionality is no longer needed, convert the printf to a simple print.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 Documentation/cat-texi.perl | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
Show changes to Documentation/cat-texi.perl +2 −2
diff --git a/Documentation/cat-texi.perl b/Documentation/cat-texi.perl
index 1bc84d3c7..14d2f8341 100755
--- a/Documentation/cat-texi.perl
+++ b/Documentation/cat-texi.perl
@@ -20,7 +20,7 @@ while (<STDIN>) {
 }
 close $tmp;
 
-printf '\input texinfo
+print '\input texinfo
 @setfilename gitman.info
 @documentencoding UTF-8
 @dircategory Development
@@ -31,7 +31,7 @@ printf '\input texinfo
 @top Git Manual Pages
 @documentlanguage en
 @menu
-', $menu[0];
+';
 
 for (@menu) {
 	print "* ${_}::\n";
brian m. carlson· Jan 21, 2017, 21:59 UTC · re: brian m. carlson · lore

[PATCH 2/7] Documentation: modernize cat-texi.perl

Good style for Perl includes using the strict and warnings pragmas, and preferring lexical file handles over bareword file handles. Using lexical file handles necessitates being explicit when $_ is printed, so that Perl does not get confused and instead print the glob ref.

The benefit of this modernization is that a formerly obscured bug is now visible, which will be fixed in a followup patch.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 Documentation/cat-texi.perl | 15 +++++++++------
 1 file changed, 9 insertions(+), 6 deletions(-)
Show changes to Documentation/cat-texi.perl +9 −6
diff --git a/Documentation/cat-texi.perl b/Documentation/cat-texi.perl
index b1fe52e8b..1bc84d3c7 100755
--- a/Documentation/cat-texi.perl
+++ b/Documentation/cat-texi.perl
@@ -1,9 +1,12 @@
 #!/usr/bin/perl -w
 
+use strict;
+use warnings;
+
 my @menu = ();
 my $output = $ARGV[0];
 
-open TMP, '>', "$output.tmp";
+open my $tmp, '>', "$output.tmp";
 
 while (<STDIN>) {
 	next if (/^\\input texinfo/../\@node Top/);
@@ -13,9 +16,9 @@ while (<STDIN>) {
 	}
 	s/\(\@pxref\{\[(URLS|REMOTES)\]}\)//;
 	s/\@anchor\{[^{}]*\}//g;
-	print TMP;
+	print $tmp $_;
 }
-close TMP;
+close $tmp;
 
 printf '\input texinfo
 @setfilename gitman.info
@@ -34,10 +37,10 @@ for (@menu) {
 	print "* ${_}::\n";
 }
 print "\@end menu\n";
-open TMP, '<', "$output.tmp";
-while (<TMP>) {
+open $tmp, '<', "$output.tmp";
+while (<$tmp>) {
 	print;
 }
-close TMP;
+close $tmp;
 print "\@bye\n";
 unlink "$output.tmp";
brian m. carlson· Jan 21, 2017, 21:59 UTC · re: brian m. carlson · lore

[PATCH 1/7] Documentation: fix warning in cat-texi.perl

Newer versions of Perl produce the warning "Unescaped left brace in regex is deprecated, passed through in regex" when an unescaped left brace occurs in a regex. Escape the brace to avoid this warning.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 Documentation/cat-texi.perl | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Show changes to Documentation/cat-texi.perl +1 −1
diff --git a/Documentation/cat-texi.perl b/Documentation/cat-texi.perl
index 87437f8a9..b1fe52e8b 100755
--- a/Documentation/cat-texi.perl
+++ b/Documentation/cat-texi.perl
@@ -11,7 +11,7 @@ while (<STDIN>) {
 	if (s/^\@top (.*)/\@node $1,,,Top/) {
 		push @menu, $1;
 	}
-	s/\(\@pxref{\[(URLS|REMOTES)\]}\)//;
+	s/\(\@pxref\{\[(URLS|REMOTES)\]}\)//;
 	s/\@anchor\{[^{}]*\}//g;
 	print TMP;
 }
brian m. carlson· Jan 21, 2017, 21:59 UTC · re: brian m. carlson · lore

[PATCH 4/7] Documentation: sort sources for gitman.texi

Sorting the sources makes it easier to compare the output using diff. In addition, it aids groups creating reproducible builds, as the order of the files is no longer dependent on the file system or other irrelevant factors.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 Documentation/Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Show changes to Documentation/Makefile +1 −1
diff --git a/Documentation/Makefile b/Documentation/Makefile
index a9fb497b8..6e6c82409 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -373,7 +373,7 @@ user-manual.pdf: user-manual.xml
 
 gitman.texi: $(MAN_XML) cat-texi.perl
 	$(QUIET_DB2TEXI)$(RM) $@+ $@ && \
-	($(foreach xml,$(MAN_XML),$(DOCBOOK2X_TEXI) --encoding=UTF-8 \
+	($(foreach xml,$(sort $(MAN_XML)),$(DOCBOOK2X_TEXI) --encoding=UTF-8 \
 		--to-stdout $(xml) &&) true) > $@++ && \
 	$(PERL_PATH) cat-texi.perl $@ <$@++ >$@+ && \
 	rm $@++ && \
brian m. carlson· Jan 21, 2017, 21:59 UTC · re: brian m. carlson · lore

[PATCH 7/7] Makefile: add a knob to enable the use of Asciidoctor

While Git has traditionally built its documentation using AsciiDoc, some people wish to use Asciidoctor for speed or other reasons. Add a Makefile knob, USE_ASCIIDOCTOR, that sets various options in order to produce acceptable output. For HTML output, XHTML5 was chosen, since the AsciiDoc options also produce XHTML, albeit XHTML 1.1.

Asciidoctor does not have built-in support for the linkgit macro, but it is available using the Asciidoctor Extensions Lab. Add a macro to enable the use of this extension if it is available. Without it, the linkgit macros are emitted into the output.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 Documentation/Makefile | 12 ++++++++++++
 Makefile               |  6 ++++++
 2 files changed, 18 insertions(+)
Show changes to 2 files +18 −0

Documentation/Makefile, Makefile

diff --git a/Documentation/Makefile b/Documentation/Makefile
index 0f4db48eb..5cbecfa99 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -174,6 +174,18 @@ ifdef GNU_ROFF
 XMLTO_EXTRA += -m manpage-quote-apos.xsl
 endif
 
+ifdef USE_ASCIIDOCTOR
+ASCIIDOC = asciidoctor
+ASCIIDOC_CONF =
+ASCIIDOC_HTML = xhtml5
+ASCIIDOC_DOCBOOK = docbook45
+ifdef ASCIIDOCTOR_EXTENSIONS_LAB
+ASCIIDOC_EXTRA = -I$(ASCIIDOCTOR_EXTENSIONS_LAB) -rasciidoctor/extensions -rman-inline-macro
+endif
+ASCIIDOC_EXTRA += -alitdd='&\#x2d;&\#x2d;'
+DBLATEX_COMMON =
+endif
+
 SHELL_PATH ?= $(SHELL)
 # Shell quote;
 SHELL_PATH_SQ = $(subst ','\'',$(SHELL_PATH))
diff --git a/Makefile b/Makefile
index 27afd0f37..7ed9d4d4b 100644
--- a/Makefile
+++ b/Makefile
@@ -250,6 +250,12 @@ all::
 # apostrophes to be ASCII so that cut&pasting examples to the shell
 # will work.
 #
+# Define USE_ASCIIDOCTOR to use Asciidoctor instead of AsciiDoc to build the
+# documentation.
+#
+# Define ASCIIDOCTOR_EXTENSIONS_LAB to point to the location of the Asciidoctor
+# Extensions Lab if you have it available.
+#
 # Define PERL_PATH to the path of your Perl binary (usually /usr/bin/perl).
 #
 # Define NO_PERL_MAKEMAKER if you cannot use Makefiles generated by perl's
brian m. carlson· Jan 21, 2017, 21:59 UTC · re: brian m. carlson · lore

[PATCH 5/7] Documentation: add XSLT to fix DocBook for Texinfo

There are two ways to create a section in a reference document (i.e., manpage) in DocBook 4: refsection elements and refsect, refsect2, and refsect3 elements. Either form is acceptable as of DocBook 4.2, but they cannot be mixed. Prior to DocBook 4.2, only the numbered forms were acceptable.

docbook2texi only accepts the numbered forms, and this has not generally been a problem, since AsciiDoc produces the numbered forms. Asciidoctor, on the other hand, uses a shared backend for DocBook 4 and 5, and uses the unnumbered refsection elements instead.

If we don't convert the unnumbered form to the numbered form, docbook2texi omits section headings, which is undesirable. Add an XSLT stylesheet to transform the unnumbered forms to the numbered forms automatically, and preprocess the DocBook XML as part of the transformation to Texinfo format.

Note that this transformation is only necessary for Texinfo, since docbook2texi provides its own stylesheets. The DocBook stylesheets, which we use for other formats, provide the full range of DocBook 4 and 5 compatibility, and don't have this issue.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 Documentation/Makefile |  7 ++++---
 Documentation/texi.xsl | 26 ++++++++++++++++++++++++++
 2 files changed, 30 insertions(+), 3 deletions(-)
 create mode 100644 Documentation/texi.xsl
Show changes to 2 files +30 −3

Documentation/Makefile, Documentation/texi.xsl

diff --git a/Documentation/Makefile b/Documentation/Makefile
index 6e6c82409..76be7017c 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -371,10 +371,11 @@ user-manual.pdf: user-manual.xml
 	$(DBLATEX) -o $@+ -p $(ASCIIDOC_DBLATEX_DIR)/asciidoc-dblatex.xsl -s $(ASCIIDOC_DBLATEX_DIR)/asciidoc-dblatex.sty $< && \
 	mv $@+ $@
 
-gitman.texi: $(MAN_XML) cat-texi.perl
+gitman.texi: $(MAN_XML) cat-texi.perl texi.xsl
 	$(QUIET_DB2TEXI)$(RM) $@+ $@ && \
-	($(foreach xml,$(sort $(MAN_XML)),$(DOCBOOK2X_TEXI) --encoding=UTF-8 \
-		--to-stdout $(xml) &&) true) > $@++ && \
+	($(foreach xml,$(sort $(MAN_XML)),xsltproc -o $(xml)+ texi.xsl $(xml) && \
+		$(DOCBOOK2X_TEXI) --encoding=UTF-8 --to-stdout $(xml)+ && \
+		rm $(xml)+ &&) true) > $@++ && \
 	$(PERL_PATH) cat-texi.perl $@ <$@++ >$@+ && \
 	rm $@++ && \
 	mv $@+ $@
diff --git a/Documentation/texi.xsl b/Documentation/texi.xsl
new file mode 100644
index 000000000..0f8ff07ec
--- /dev/null
+++ b/Documentation/texi.xsl
@@ -0,0 +1,26 @@
+<!-- texi.xsl:
+     convert refsection elements into refsect elements that docbook2texi can
+     understand -->
+<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
+		version="1.0">
+
+<xsl:output method="xml"
+	    encoding="UTF-8"
+	    doctype-public="-//OASIS//DTD DocBook XML V4.5//EN"
+	    doctype-system="http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" />
+
+<xsl:template match="//refsection">
+	<xsl:variable name="element">refsect<xsl:value-of select="count(ancestor-or-self::refsection)" /></xsl:variable>
+	<xsl:element name="{$element}">
+		<xsl:apply-templates select="@*|node()" />
+	</xsl:element>
+</xsl:template>
+
+<!-- Copy all other nodes through. -->
+<xsl:template match="node()|@*">
+	<xsl:copy>
+		<xsl:apply-templates select="@*|node()" />
+	</xsl:copy>
+</xsl:template>
+
+</xsl:stylesheet>
brian m. carlson· Jan 21, 2017, 21:59 UTC · re: brian m. carlson · lore

[PATCH 6/7] Documentation: move dblatex arguments into variable

Our dblatex invocation uses several style components from the AsciiDoc distribution, but those components are not available when building with Asciidoctor. Move the command line arguments into a variable so it can be overridden by the user or makefile configuration options.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 Documentation/Makefile | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
Show changes to Documentation/Makefile +2 −1
diff --git a/Documentation/Makefile b/Documentation/Makefile
index 76be7017c..0f4db48eb 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -120,6 +120,7 @@ INSTALL_INFO = install-info
 DOCBOOK2X_TEXI = docbook2x-texi
 DBLATEX = dblatex
 ASCIIDOC_DBLATEX_DIR = /etc/asciidoc/dblatex
+DBLATEX_COMMON = -p $(ASCIIDOC_DBLATEX_DIR)/asciidoc-dblatex.xsl -s $(ASCIIDOC_DBLATEX_DIR)/asciidoc-dblatex.sty
 ifndef PERL_PATH
 	PERL_PATH = /usr/bin/perl
 endif
@@ -368,7 +369,7 @@ user-manual.texi: user-manual.xml
 
 user-manual.pdf: user-manual.xml
 	$(QUIET_DBLATEX)$(RM) $@+ $@ && \
-	$(DBLATEX) -o $@+ -p $(ASCIIDOC_DBLATEX_DIR)/asciidoc-dblatex.xsl -s $(ASCIIDOC_DBLATEX_DIR)/asciidoc-dblatex.sty $< && \
+	$(DBLATEX) $-o $@+ (DBLATEX_COMMON) $< && \
 	mv $@+ $@
 
 gitman.texi: $(MAN_XML) cat-texi.perl texi.xsl
Øyvind A. Holm· Jan 22, 2017, 01:03 UTC · re: brian m. carlson · lore

Re: [PATCH 6/7] Documentation: move dblatex arguments into variable

On 2017-01-21 21:59:11, brian m. carlson wrote:
Show 7 quoted lines
> Our dblatex invocation uses several style components from the AsciiDoc
> distribution, but those components are not available when building with
> Asciidoctor.  Move the command line arguments into a variable so it can
> be overridden by the user or makefile configuration options.
> [...]
> -	$(DBLATEX) -o $@+ -p $(ASCIIDOC_DBLATEX_DIR)/asciidoc-dblatex.xsl -s $(ASCIIDOC_DBLATEX_DIR)/asciidoc-dblatex.sty $< && \
> +	$(DBLATEX) $-o $@+ (DBLATEX_COMMON) $< && \

It looks as the dollar sign is in the wrong place, should be in front of (DBLATEX_COMMON).

Regards, Øyvind

+-| Øyvind A. Holm <sunny@sunbase.org> - N 60.37604° E 5.33339° |-+
| OpenPGP: 0xFB0CBEE894A506E5 - http://www.sunbase.org/pubkey.asc |
| Fingerprint: A006 05D6 E676 B319 55E2  E77E FB0C BEE8 94A5 06E5 |
+------------| 61815930-e03e-11e6-b4f4-db5caa6d21d3 |-------------+

← back to recent threads