{"thread":{"id":"50765","subject":"[PATCH] asciidoctor-extensions: provide ``","startedAt":"2019-03-17T14:48:34Z","lastAt":"2019-09-16T19:00:59Z","messageCount":71,"participants":["Martin Ågren","Todd Zullinger","Jeff King","Junio C Hamano","SZEDER Gábor","brian m. carlson"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"371729","messageId":"20190317144747.2418514-1-martin.agren@gmail.com","threadId":"50765","inReplyTo":null,"subject":"[PATCH] asciidoctor-extensions: provide ``","fromName":"Martin Ågren","fromEmail":"martin.agren@gmail.com","sentAt":"2019-03-17T14:47:47Z","receivedAt":"2019-03-17T14:48:34Z","isPatch":true,"sender":{"key":"martin.agren@gmail.com","avatar":null},"body":"When we build with AsciiDoc, asciidoc.conf ensures that each xml-file we\ngenerate contains some meta-information which `xmlto` can act on, based\non the following template:\n\n \n {mantitle}\n {manvolnum}\n Git\n {git_version}\n Git Manual\n \n\nWhen we build with Asciidoctor, it does not honor this configuration file\nand we end up with only this (for a hypothetical git-foo.xml):\n\n \n git-foo\n 1\n \n\nThat is, we miss out on the `` tags. As a result, the\nheader of each man page doesn't say \"Git Manual\", but \"git-foo(1)\"\ninstead. Worse, the footers don't give the Git version number and\ninstead provide the fairly ugly \"[FIXME: source]\".\n\nThat Asciidoctor ignores asciidoc.conf is nothing new. This is why we\nimplement the `linkgit:` macro in asciidoc.conf *and* in\nasciidoctor-extensions.rb. Follow suit and provide these tags in\nasciidoctor-extensions.rb, using a \"postprocessor\" extension.\n\nWe may consider a few alternatives:\n\n * Provide the `mansource` attribute to Asciidoctor. This attribute\n looks promising until one realizes that it can only be given inside\n the source file (the .txt file in our case), *not* on the command\n line using `-a mansource=foobar`. I toyed with the idea of injecting\n this attribute while feeding Asciidoctor the input on stdin, but it\n didn't feel like it was worth the complexity in the Makefile.\n\n * Similar to that last idea, we could inject these lines into the\n xml-files from the Makefile, e.g., using `sed`. This reduces\n repetition, but seems fairly brittle. It feels wrong to impose\n another step and another risk on the Asciidoc-processing only to\n benefit the Asciidoctor-one.\n\n * Considering the above abandoned ideas, it seems better to put any\n complexity inside asciidoctor-extensions.rb. It is after all\n supposed to be the \"equivalent\" of asciidoc.conf. I considered\n providing a \"tree processor\" extension and use it to set, e.g.,\n `mansource` mentioned above.\n\nLet's instead try to stay as close as possible to what asciidoc.conf\ndoes. We'll make it fairly obvious that we aim to inject the exact same\nthree lines of `` that asciidoc.conf provides. The only\nsomewhat tricky part is that we inject them *post*-processing so we need\nto do the variable expansion ourselves.\n\nSigned-off-by: Martin Ågren \n---\n Cc Todd and Peff who had a brief exchange [1] a while ago. Apparently\n Todd saw this \"[FIXME: source]\" on Fedora but Peff did not on Debian.\n I'm on Ubuntu 18.04 and used to see this also on 16.04. I'm not sure\n what might make Debian so special here.\n\n [1] https://public-inbox.org/git/20180627164443.GK20217@zaya.teonanacatl.net/\n\n I've based this on ma/asciidoctor-fixes, not because it's needed for\n the patch itself, but because it provides a15ef383e7\n (\"Documentation/Makefile: add missing dependency on\n asciidoctor-extensions\", 2019-02-27), which ensures that the\n documentation will be rebuilt. I'm hoping this was the right call.\n\n Documentation/asciidoctor-extensions.rb | 16 ++++++++++++++++\n 1 file changed, 16 insertions(+)\n\ndiff --git a/Documentation/asciidoctor-extensions.rb b/Documentation/asciidoctor-extensions.rb\nindex 0089e0cfb8..059279dee1 100644\n--- a/Documentation/asciidoctor-extensions.rb\n+++ b/Documentation/asciidoctor-extensions.rb\n@@ -20,9 +20,25 @@ module Git\n end\n end\n end\n+\n+ class DocumentPostProcessor < Asciidoctor::Extensions::Postprocessor\n+ def process document, output\n+ if document.basebackend? 'docbook'\n+ git_version = document.attributes['git_version']\n+ replacement = \"\" \\\n+ \"Git\\n\" \\\n+ \"#{git_version}\\n\" \\\n+ \"Git Manual\\n\" \\\n+ \"<\\/refmeta>\"\n+ output = output.sub(/<\\/refmeta>/, replacement)\n+ end\n+ output\n+ end\n+ end\n end\n end\n \n Asciidoctor::Extensions.register do\n inline_macro Git::Documentation::LinkGitProcessor, :linkgit\n+ postprocessor Git::Documentation::DocumentPostProcessor\n end\n-- \n2.21.0\n\n"},{"id":"371749","messageId":"20190317194431.GY31362@pobox.com","threadId":"50765","inReplyTo":"20190317144747.2418514-1-martin.agren@gmail.com","subject":"Re: [PATCH] asciidoctor-extensions: provide ``","fromName":"Todd Zullinger","fromEmail":"tmz@pobox.com","sentAt":"2019-03-17T19:44:31Z","receivedAt":"2019-03-17T19:44:42Z","isPatch":true,"sender":{"key":"tmz@pobox.com","avatar":"https://avatars.githubusercontent.com/u/806319?v=4"},"body":"Hi Martin,\n\nMartin Ågren wrote:\n> When we build with AsciiDoc, asciidoc.conf ensures that each xml-file we\n> generate contains some meta-information which `xmlto` can act on, based\n> on the following template:\n> \n> \n> {mantitle}\n> {manvolnum}\n> Git\n> {git_version}\n> Git Manual\n> \n> \n> When we build with Asciidoctor, it does not honor this configuration file\n> and we end up with only this (for a hypothetical git-foo.xml):\n> \n> \n> git-foo\n> 1\n> \n> \n> That is, we miss out on the `` tags. As a result, the\n> header of each man page doesn't say \"Git Manual\", but \"git-foo(1)\"\n> instead. Worse, the footers don't give the Git version number and\n> instead provide the fairly ugly \"[FIXME: source]\".\n> \n> That Asciidoctor ignores asciidoc.conf is nothing new. This is why we\n> implement the `linkgit:` macro in asciidoc.conf *and* in\n> asciidoctor-extensions.rb. Follow suit and provide these tags in\n> asciidoctor-extensions.rb, using a \"postprocessor\" extension.\n\nNice! I looked at this a long time ago and didn't figure\nout how to use a postprocessor extension. From my notes, I\nfound discussions about using ruby's tilt for templating and\nit all seemed way too convoluted.\n\nYour method looks quite simple and elegant.\n\n> We may consider a few alternatives:\n> \n> * Provide the `mansource` attribute to Asciidoctor. This attribute\n> looks promising until one realizes that it can only be given inside\n> the source file (the .txt file in our case), *not* on the command\n> line using `-a mansource=foobar`. I toyed with the idea of injecting\n> this attribute while feeding Asciidoctor the input on stdin, but it\n> didn't feel like it was worth the complexity in the Makefile.\n\nI played with this direction before. Using Asciidoctor we\ncan convert directly from .txt to man without docbook\nand xmlto. That does have some other issues which need to\nbe worked out though. Here's what I had as a start:\n\n-- 8< --\nSubject: [PATCH] WIP: doc: improve asciidoctor manpage generation\n\nAvoid 'FIXME: Source' by setting mansource. Skip xmlto step and render\nmanpages directly with asciidoctor.\n\nTODO:\n - apply to all man pages\n - fix links to html docs, like user-manual.html in git.1 (currently\n it is listed in brackets inline rather than as a footnote)\n\nReference:\nhttps://lore.kernel.org/lkml/20180424150456.17353-1-tiwai@suse.de/\nSigned-off-by: Todd Zullinger \n---\n Documentation/Makefile | 8 ++++++++\n Documentation/asciidoctor-extensions.rb | 2 ++\n 2 files changed, 10 insertions(+)\n\ndiff --git a/Documentation/Makefile b/Documentation/Makefile\nindex a9697f5146..494f8c9464 100644\n--- a/Documentation/Makefile\n+++ b/Documentation/Makefile\n@@ -197,6 +197,7 @@ ASCIIDOC_DOCBOOK = docbook45\n ASCIIDOC_EXTRA += -acompat-mode -atabsize=8\n ASCIIDOC_EXTRA += -I. -rasciidoctor-extensions\n ASCIIDOC_EXTRA += -alitdd='&\\#x2d;&\\#x2d;'\n+ASCIIDOC_EXTRA += -a mansource=\"Git $(GIT_VERSION)\" -a manmanual=\"Git Manual\"\n DBLATEX_COMMON =\n endif\n \n@@ -354,9 +355,16 @@ $(OBSOLETE_HTML): %.html : %.txto asciidoc.conf\n manpage-base-url.xsl: manpage-base-url.xsl.in\n \t$(QUIET_GEN)sed \"s|@@MAN_BASE_URL@@|$(MAN_BASE_URL)|\" $< > $@\n \n+ifndef USE_ASCIIDOCTOR\n %.1 %.5 %.7 : %.xml manpage-base-url.xsl $(wildcard manpage*.xsl)\n \t$(QUIET_XMLTO)$(RM) $@ && \\\n \t$(XMLTO) -m $(MANPAGE_XSL) $(XMLTO_EXTRA) man $<\n+else\n+%.1 %.5 %.7 : %.txt\n+\t$(QUIET_ASCIIDOC)$(RM) $@+ $@ && \\\n+\t$(ASCIIDOC_COMMON) -b manpage -d manpage -o $@+ $< && \\\n+\tmv $@+ $@\n+endif\n \n %.xml : %.txt asciidoc.conf asciidoctor-extensions.rb\n \t$(QUIET_ASCIIDOC)$(RM) $@+ $@ && \\\ndiff --git a/Documentation/asciidoctor-extensions.rb b/Documentation/asciidoctor-extensions.rb\nindex f7a5982f8b..ebb078807a 100644\n--- a/Documentation/asciidoctor-extensions.rb\n+++ b/Documentation/asciidoctor-extensions.rb\n@@ -12,6 +12,8 @@ module Git\n if parent.document.basebackend? 'html'\n prefix = parent.document.attr('git-relative-html-prefix')\n %(#{target}(#{attrs[1]})\\n)\n+ elsif parent.document.basebackend? 'manpage'\n+ \"#{target}(#{attrs[1]})\"\n elsif parent.document.basebackend? 'docbook'\n \"\\n\" \\\n \"#{target}\" \\\n-- 8< --\n\nThat was based on ma/asciidoctor-extensions, but it may be\nmissing other recent improvements you've made to the make\ntargets. It's been a month or so since I worked on it.\n\nI munged up doc-diff to set MANDWIDTH=1000 and set one\nbranch to default to asciidoctor to compare. (Your other\nrecent series looks like it'll make doing asciidoc and\nasciidoctor comparisons easier.)\n\nThere were a number of differences that I didn't work\nthrough though. Most importantly was the change in the\nlinks noted in the commit message.\n\nThanks for working on asciidoctor. I've been trying to poke\nit off and on to help ensure the docs can be built if\nasciidoc ever gets dropped from Fedora.\n\n-- \nTodd\n"},{"id":"371750","messageId":"CAN0heSr3Q48Dfskc7A55Z6MUEk0A3MPQAn3b25+=J1Ezrwk9Hg@mail.gmail.com","threadId":"50765","inReplyTo":"20190317194431.GY31362@pobox.com","subject":"Re: [PATCH] asciidoctor-extensions: provide ``","fromName":"Martin Ågren","fromEmail":"martin.agren@gmail.com","sentAt":"2019-03-17T20:03:30Z","receivedAt":"2019-03-17T20:03:46Z","isPatch":true,"sender":{"key":"martin.agren@gmail.com","avatar":null},"body":"Hi Todd,\n\nOn Sun, 17 Mar 2019 at 20:44, Todd Zullinger wrote:\n>\n> Hi Martin,\n>\n> Martin Ågren wrote:\n> > That is, we miss out on the `` tags. As a result, the\n> > header of each man page doesn't say \"Git Manual\", but \"git-foo(1)\"\n> > instead. Worse, the footers don't give the Git version number and\n> > instead provide the fairly ugly \"[FIXME: source]\".\n> >\n> > That Asciidoctor ignores asciidoc.conf is nothing new. This is why we\n> > implement the `linkgit:` macro in asciidoc.conf *and* in\n> > asciidoctor-extensions.rb. Follow suit and provide these tags in\n> > asciidoctor-extensions.rb, using a \"postprocessor\" extension.\n>\n> > We may consider a few alternatives:\n> >\n> > * Provide the `mansource` attribute to Asciidoctor. This attribute\n> > looks promising until one realizes that it can only be given inside\n> > the source file (the .txt file in our case), *not* on the command\n> > line using `-a mansource=foobar`. I toyed with the idea of injecting\n> > this attribute while feeding Asciidoctor the input on stdin, but it\n> > didn't feel like it was worth the complexity in the Makefile.\n>\n> I played with this direction before. Using Asciidoctor we\n> can convert directly from .txt to man without docbook\n> and xmlto. That does have some other issues which need to\n> be worked out though. Here's what I had as a start:\n>\n> -- 8< --\n> Subject: [PATCH] WIP: doc: improve asciidoctor manpage generation\n>\n> Avoid 'FIXME: Source' by setting mansource. Skip xmlto step and render\n> manpages directly with asciidoctor.\n>\n> TODO:\n> - apply to all man pages\n> - fix links to html docs, like user-manual.html in git.1 (currently\n> it is listed in brackets inline rather than as a footnote)\n>\n> Reference:\n> https://lore.kernel.org/lkml/20180424150456.17353-1-tiwai@suse.de/\n> Signed-off-by: Todd Zullinger \n> ---\n> Documentation/Makefile | 8 ++++++++\n> Documentation/asciidoctor-extensions.rb | 2 ++\n> 2 files changed, 10 insertions(+)\n>\n> diff --git a/Documentation/Makefile b/Documentation/Makefile\n> index a9697f5146..494f8c9464 100644\n> --- a/Documentation/Makefile\n> +++ b/Documentation/Makefile\n> @@ -197,6 +197,7 @@ ASCIIDOC_DOCBOOK = docbook45\n> ASCIIDOC_EXTRA += -acompat-mode -atabsize=8\n> ASCIIDOC_EXTRA += -I. -rasciidoctor-extensions\n> ASCIIDOC_EXTRA += -alitdd='&\\#x2d;&\\#x2d;'\n> +ASCIIDOC_EXTRA += -a mansource=\"Git $(GIT_VERSION)\" -a manmanual=\"Git Manual\"\n> DBLATEX_COMMON =\n> endif\n>\n> @@ -354,9 +355,16 @@ $(OBSOLETE_HTML): %.html : %.txto asciidoc.conf\n> manpage-base-url.xsl: manpage-base-url.xsl.in\n> $(QUIET_GEN)sed \"s|@@MAN_BASE_URL@@|$(MAN_BASE_URL)|\" $< > $@\n>\n> +ifndef USE_ASCIIDOCTOR\n> %.1 %.5 %.7 : %.xml manpage-base-url.xsl $(wildcard manpage*.xsl)\n> $(QUIET_XMLTO)$(RM) $@ && \\\n> $(XMLTO) -m $(MANPAGE_XSL) $(XMLTO_EXTRA) man $<\n> +else\n> +%.1 %.5 %.7 : %.txt\n> + $(QUIET_ASCIIDOC)$(RM) $@+ $@ && \\\n> + $(ASCIIDOC_COMMON) -b manpage -d manpage -o $@+ $< && \\\n> + mv $@+ $@\n> +endif\n>\n> %.xml : %.txt asciidoc.conf asciidoctor-extensions.rb\n> $(QUIET_ASCIIDOC)$(RM) $@+ $@ && \\\n> diff --git a/Documentation/asciidoctor-extensions.rb b/Documentation/asciidoctor-extensions.rb\n> index f7a5982f8b..ebb078807a 100644\n> --- a/Documentation/asciidoctor-extensions.rb\n> +++ b/Documentation/asciidoctor-extensions.rb\n> @@ -12,6 +12,8 @@ module Git\n> if parent.document.basebackend? 'html'\n> prefix = parent.document.attr('git-relative-html-prefix')\n> %(#{target}(#{attrs[1]})\\n)\n> + elsif parent.document.basebackend? 'manpage'\n> + \"#{target}(#{attrs[1]})\"\n> elsif parent.document.basebackend? 'docbook'\n> \"\\n\" \\\n> \"#{target}\" \\\n> -- 8< --\n>\n> That was based on ma/asciidoctor-extensions, but it may be\n> missing other recent improvements you've made to the make\n> targets. It's been a month or so since I worked on it.\n>\n> I munged up doc-diff to set MANDWIDTH=1000 and set one\n> branch to default to asciidoctor to compare. (Your other\n> recent series looks like it'll make doing asciidoc and\n> asciidoctor comparisons easier.)\n>\n> There were a number of differences that I didn't work\n> through though. Most importantly was the change in the\n> links noted in the commit message.\n\nYour approach looks like the correct one long term, at least.\n\nI'll try to play with this to see if I can figure out the differences.\nThat will have to wait until tomorrow though.\n\nThanks for sharing your progress.\n\nMartin\n"},{"id":"371870","messageId":"20190319024645.GA6173@sigill.intra.peff.net","threadId":"50765","inReplyTo":"20190317144747.2418514-1-martin.agren@gmail.com","subject":"Re: [PATCH] asciidoctor-extensions: provide ``","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2019-03-19T02:46:46Z","receivedAt":"2019-03-19T02:46:50Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Sun, Mar 17, 2019 at 03:47:47PM +0100, Martin Ågren wrote:\n\n> Cc Todd and Peff who had a brief exchange [1] a while ago. Apparently\n> Todd saw this \"[FIXME: source]\" on Fedora but Peff did not on Debian.\n> I'm on Ubuntu 18.04 and used to see this also on 16.04. I'm not sure\n> what might make Debian so special here.\n\nI think it was just that my version of asciidoctor had\n\n https://github.com/asciidoctor/asciidoctor/pull/2636\n\nand Todd's did not. However, mine still does not do the _right_ thing,\nbecause we didn't pass the right attributes in to asciidoctor. It just\ndidn't print an ugly \"FIXME\". Looking at the XML, I have:\n\n git-add\n 1\n  \n  \n \n\nSo it's just an nbsp instead of the real content, and the \"version\"\nfield is missing entirely.\n\n> That Asciidoctor ignores asciidoc.conf is nothing new. This is why we\n> implement the `linkgit:` macro in asciidoc.conf *and* in\n> asciidoctor-extensions.rb. Follow suit and provide these tags in\n> asciidoctor-extensions.rb, using a \"postprocessor\" extension.\n\nYeah, that seems sensible overall. Some thoughts on your approach:\n\n> * Provide the `mansource` attribute to Asciidoctor. This attribute\n> looks promising until one realizes that it can only be given inside\n> the source file (the .txt file in our case), *not* on the command\n> line using `-a mansource=foobar`. I toyed with the idea of injecting\n> this attribute while feeding Asciidoctor the input on stdin, but it\n> didn't feel like it was worth the complexity in the Makefile.\n\nIt does seem like \"mansource\" is the way asciidoctor expects us to do\nthis. Why doesn't it work from the command line? Is it a bug in\nasciidoctor, or is there something more subtle going on?\n\nI think even if it is a bug and gets fixed, though, it still wouldn't\nhave the version field (though that seems like something we could\ncontribute to asciidoctor).\n\n> * Similar to that last idea, we could inject these lines into the\n> xml-files from the Makefile, e.g., using `sed`. This reduces\n> repetition, but seems fairly brittle. It feels wrong to impose\n> another step and another risk on the Asciidoc-processing only to\n> benefit the Asciidoctor-one.\n\nThat's more or less what your ruby code is doing on. That said, I'd just\nas soon do it in ruby as with a separate sed invocation. At least then\nthe external build is the same.\n\n> * Considering the above abandoned ideas, it seems better to put any\n> complexity inside asciidoctor-extensions.rb. It is after all\n> supposed to be the \"equivalent\" of asciidoc.conf. I considered\n> providing a \"tree processor\" extension and use it to set, e.g.,\n> `mansource` mentioned above.\n\nThis seems like the least bad option, at least for now. Your code does\ndo a generic regex substitution. The promise of XML is that we're\nsupposed to be able to do structured, robust transformations of the\ndocument. But my experience has been that the tooling is sufficiently\ndifficult to work with that you just end up writing a regex.\n\nSo I'm curious if you tried to use an actual XML parser (or god forbid,\nXSLT) to do the transformation. But if you spent more than 5 minutes on\nit and got disgusted, I wouldn't ask you to look deeper than that. :)\n\nI doubt we'd see any other refmeta tags (and any non-tag content would\nbe quoted).\n\n> Let's instead try to stay as close as possible to what asciidoc.conf\n> does. We'll make it fairly obvious that we aim to inject the exact same\n> three lines of `` that asciidoc.conf provides. The only\n> somewhat tricky part is that we inject them *post*-processing so we need\n> to do the variable expansion ourselves.\n\nOne thing that asciidoctor buys us that asciidoc does not is that we\nmight eventually move to directly generating the manpages, without the\nXML / Docbook step in between. And if we do, then all of this XML\nhackery is going to have to get replaced with something else. I guess we\ncan cross that bridge when we come to it.\n\n> diff --git a/Documentation/asciidoctor-extensions.rb b/Documentation/asciidoctor-extensions.rb\n> index 0089e0cfb8..059279dee1 100644\n> --- a/Documentation/asciidoctor-extensions.rb\n> +++ b/Documentation/asciidoctor-extensions.rb\n> @@ -20,9 +20,25 @@ module Git\n> end\n> end\n> end\n> +\n> + class DocumentPostProcessor < Asciidoctor::Extensions::Postprocessor\n> + def process document, output\n> + if document.basebackend? 'docbook'\n> + git_version = document.attributes['git_version']\n> + replacement = \"\" \\\n> + \"Git\\n\" \\\n> + \"#{git_version}\\n\" \\\n> + \"Git Manual\\n\" \\\n> + \"<\\/refmeta>\"\n> + output = output.sub(/<\\/refmeta>/, replacement)\n> + end\n> + output\n> + end\n> + end\n\nThe patch itself looks sane. Would we ever need to XML-quote the\ncontents of git_version? I guess the asciidoc.conf version doesn't\nbother. Technically the user running \"make\" could put whatever they want\ninto it, but I think this is a case of \"if it hurts, don't do it\", and\nwe can ignore it.\n\n-Peff\n"},{"id":"371871","messageId":"20190319025945.GB6173@sigill.intra.peff.net","threadId":"50765","inReplyTo":"20190319024645.GA6173@sigill.intra.peff.net","subject":"Re: [PATCH] asciidoctor-extensions: provide ``","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2019-03-19T02:59:46Z","receivedAt":"2019-03-19T02:59:49Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Mar 18, 2019 at 10:46:46PM -0400, Jeff King wrote:\n\n> > Let's instead try to stay as close as possible to what asciidoc.conf\n> > does. We'll make it fairly obvious that we aim to inject the exact same\n> > three lines of `` that asciidoc.conf provides. The only\n> > somewhat tricky part is that we inject them *post*-processing so we need\n> > to do the variable expansion ourselves.\n> \n> One thing that asciidoctor buys us that asciidoc does not is that we\n> might eventually move to directly generating the manpages, without the\n> XML / Docbook step in between. And if we do, then all of this XML\n> hackery is going to have to get replaced with something else. I guess we\n> can cross that bridge when we come to it.\n\nI see there was some discussion of that in the side-thread, too. Just to\ngive my two cents: I think that's where we want to go eventually, but I\nalso think that while we are maintaining a dual asciidoc/asciidoctor\ntool chain, it's probably crazy not to go through docbook, just because\nit keeps so much of the pipeline the same for both tools.\n\nSo in my mind, the endgame is that we eventually drop asciidoc in favor\nof asciidoctor. The repo at:\n\n https://github.com/asciidoc/asciidoc\n\nsays:\n\n NOTE: This implementation is written in Python 2, which EOLs in Jan\n 2020. AsciiDoc development is being continued under @asciidoctor.\n\nI'm not sure when is the right time to switch. If we can get the output\nat parity, I don't think asciidoctor is too onerous to install (and we\ndon't have to worry about ancient platforms, as they can use\npre-formatted manpages).\n\n-Peff\n"},{"id":"371874","messageId":"20190319033023.GA25772@sigill.intra.peff.net","threadId":"50765","inReplyTo":"20190319024645.GA6173@sigill.intra.peff.net","subject":"Re: [PATCH] asciidoctor-extensions: provide ``","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2019-03-19T03:30:24Z","receivedAt":"2019-03-19T03:30:27Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Mar 18, 2019 at 10:46:45PM -0400, Jeff King wrote:\n\n> > * Provide the `mansource` attribute to Asciidoctor. This attribute\n> > looks promising until one realizes that it can only be given inside\n> > the source file (the .txt file in our case), *not* on the command\n> > line using `-a mansource=foobar`. I toyed with the idea of injecting\n> > this attribute while feeding Asciidoctor the input on stdin, but it\n> > didn't feel like it was worth the complexity in the Makefile.\n> \n> It does seem like \"mansource\" is the way asciidoctor expects us to do\n> this. Why doesn't it work from the command line? Is it a bug in\n> asciidoctor, or is there something more subtle going on?\n> \n> I think even if it is a bug and gets fixed, though, it still wouldn't\n> have the version field (though that seems like something we could\n> contribute to asciidoctor).\n\nI just tried with asciidoc 2.0.0.rc.2, which came out last week. It does\nseem to work from the command line:\n\n $ make USE_ASCIIDOCTOR=Yes \\\n ASCIIDOC_DOCBOOK=docbook5 \\\n ASCIIDOC='asciidoctor -amansource=Git -amanmanual=\"Git Manual\"' \\\n git-add.xml\n $ sed -n '/refmeta/,/refmeta/p' git-add.xml\n \n git-add\n 1\n Git\n Git Manual\n \n\nStill no \"manversion\" attribute, though.\n\n-Peff\n"},{"id":"371875","messageId":"xmqqr2b3y0tx.fsf@gitster-ct.c.googlers.com","threadId":"50765","inReplyTo":"20190319025945.GB6173@sigill.intra.peff.net","subject":"Re: [PATCH] asciidoctor-extensions: provide ``","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2019-03-19T03:55:54Z","receivedAt":"2019-03-19T03:56:02Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King writes:\n\n> So in my mind, the endgame is that we eventually drop asciidoc in favor\n> of asciidoctor. The repo at:\n>\n> https://github.com/asciidoc/asciidoc\n>\n> says:\n>\n> NOTE: This implementation is written in Python 2, which EOLs in Jan\n> 2020. AsciiDoc development is being continued under @asciidoctor.\n\n;-)\n\n> I'm not sure when is the right time to switch. If we can get the output\n> at parity, I don't think asciidoctor is too onerous to install (and we\n> don't have to worry about ancient platforms, as they can use\n> pre-formatted manpages).\n\nOne minor thing that bothers me abit is the continuity of the\npre-formatted pages when I switch to asciidoctor to update them.\n\nI do not mind having to see a huge diff in the \"git log -p\" output\nrun in pre-formatted manpages and htmldocs repositories at the\nboundary due to e.g. the differences how lines are broken or folded\nbetween the formatters, but by the time we have to transition, the\nefforts by you, Martin and friends to allow us compare the formatted\ndocs would have made the real differences to empty (or at least\nnegligible). Knock knock...\n\n\n\n"},{"id":"371885","messageId":"CAN0heSrajiswzpm+au_5nmZMZG9406iZa-CK9p5CaHLTuxm8nw@mail.gmail.com","threadId":"50765","inReplyTo":"20190317194431.GY31362@pobox.com","subject":"Re: [PATCH] asciidoctor-extensions: provide ``","fromName":"Martin Ågren","fromEmail":"martin.agren@gmail.com","sentAt":"2019-03-19T07:02:14Z","receivedAt":"2019-03-19T07:02:30Z","isPatch":true,"sender":{"key":"martin.agren@gmail.com","avatar":null},"body":"On Sun, 17 Mar 2019 at 20:44, Todd Zullinger wrote:\n> Martin Ågren wrote:\n> > * Provide the `mansource` attribute to Asciidoctor. This attribute\n> > looks promising until one realizes that it can only be given inside\n> > the source file (the .txt file in our case), *not* on the command\n> > line using `-a mansource=foobar`. I toyed with the idea of injecting\n> > this attribute while feeding Asciidoctor the input on stdin, but it\n> > didn't feel like it was worth the complexity in the Makefile.\n>\n> I played with this direction before. Using Asciidoctor we\n> can convert directly from .txt to man without docbook\n> and xmlto. That does have some other issues which need to\n> be worked out though. Here's what I had as a start:\n\n> +ASCIIDOC_EXTRA += -a mansource=\"Git $(GIT_VERSION)\" -a manmanual=\"Git Manual\"\n\nSo to be honest, I still don't understand how this works, but it does,\ngreat. I really need to improve my documentation-reading skills.\n\n> I munged up doc-diff to set MANDWIDTH=1000 and set one\n> branch to default to asciidoctor to compare. (Your other\n> recent series looks like it'll make doing asciidoc and\n> asciidoctor comparisons easier.)\n>\n> There were a number of differences that I didn't work\n> through though. Most importantly was the change in the\n> links noted in the commit message.\n\nI had some more time to look at this. Thanks for getting started on this\nswitch. A few things I noticed:\n\n{litdd} now renders as --. We should find some other way to\nproduce '--'. This should then be a simple change, as we're already\nproviding this attribute inside an `ifdef USE_ASCIIDOCTOR`.\n\n\"+\" becomes \"+\". I didn't immediately find where we do that.\n\nBackticks should result in monospace.\n\n`./doc-diff HEAD^ HEAD` shows how several \"git-\\nfoo\" become\n\"\\ngit-foo\", i.e., linkgit expansions are now treated as non-breaking.\nThat's arguably good, but it brings some noise to the diff. Maybe one\nshould try and see if it's possible to break that to have a nicer\ndiff, then remove that breakage in a follow-up commit. Or, if it's\npossible to make \"git-foo\" non-breaking before the switch. Hmm, was this\nwhy you increased MANWIDTH?\n\nA double-space between sentences turns into a single space -- at least\nin constructions such as \"... to foobar. `git-foo` does ...\". Not a\nproblem perhaps, but noise in the diff.\n\nAnd I'm sure there's more lurking in that huge diff. Whether any of that\nis significant or not is another matter. ;-)\n\n\nMartin\n"},{"id":"371887","messageId":"CAN0heSoS_M-nYCTaL0jKBot8PWv1k-nscD9tgf3pgAz-Ddn-7g@mail.gmail.com","threadId":"50765","inReplyTo":"20190319024645.GA6173@sigill.intra.peff.net","subject":"Re: [PATCH] asciidoctor-extensions: provide ``","fromName":"Martin Ågren","fromEmail":"martin.agren@gmail.com","sentAt":"2019-03-19T07:10:52Z","receivedAt":"2019-03-19T07:11:07Z","isPatch":true,"sender":{"key":"martin.agren@gmail.com","avatar":null},"body":"On Tue, 19 Mar 2019 at 03:46, Jeff King wrote:\n>\n> On Sun, Mar 17, 2019 at 03:47:47PM +0100, Martin Ågren wrote:\n>\n> > Cc Todd and Peff who had a brief exchange [1] a while ago. Apparently\n> > Todd saw this \"[FIXME: source]\" on Fedora but Peff did not on Debian.\n> > I'm on Ubuntu 18.04 and used to see this also on 16.04. I'm not sure\n> > what might make Debian so special here.\n>\n> I think it was just that my version of asciidoctor had\n>\n> https://github.com/asciidoctor/asciidoctor/pull/2636\n>\n> and Todd's did not. However, mine still does not do the _right_ thing,\n> because we didn't pass the right attributes in to asciidoctor. It just\n> didn't print an ugly \"FIXME\". Looking at the XML, I have:\n>\n> git-add\n> 1\n>  \n>  \n> \n>\n> So it's just an nbsp instead of the real content, and the \"version\"\n> field is missing entirely.\n\nHuh, yeah, that's a big improvement already.\n\n> > That Asciidoctor ignores asciidoc.conf is nothing new. This is why we\n> > implement the `linkgit:` macro in asciidoc.conf *and* in\n> > asciidoctor-extensions.rb. Follow suit and provide these tags in\n> > asciidoctor-extensions.rb, using a \"postprocessor\" extension.\n>\n> Yeah, that seems sensible overall. Some thoughts on your approach:\n>\n> > * Provide the `mansource` attribute to Asciidoctor. This attribute\n> > looks promising until one realizes that it can only be given inside\n> > the source file (the .txt file in our case), *not* on the command\n> > line using `-a mansource=foobar`. I toyed with the idea of injecting\n> > this attribute while feeding Asciidoctor the input on stdin, but it\n> > didn't feel like it was worth the complexity in the Makefile.\n>\n> It does seem like \"mansource\" is the way asciidoctor expects us to do\n> this. Why doesn't it work from the command line? Is it a bug in\n> asciidoctor, or is there something more subtle going on?\n>\n> I think even if it is a bug and gets fixed, though, it still wouldn't\n> have the version field (though that seems like something we could\n> contribute to asciidoctor).\n\nThe bug is in my docs-reading, see below.\n\n> > * Considering the above abandoned ideas, it seems better to put any\n> > complexity inside asciidoctor-extensions.rb. It is after all\n> > supposed to be the \"equivalent\" of asciidoc.conf. I considered\n> > providing a \"tree processor\" extension and use it to set, e.g.,\n> > `mansource` mentioned above.\n>\n> This seems like the least bad option, at least for now. Your code does\n> do a generic regex substitution. The promise of XML is that we're\n> supposed to be able to do structured, robust transformations of the\n> document. But my experience has been that the tooling is sufficiently\n> difficult to work with that you just end up writing a regex.\n>\n> So I'm curious if you tried to use an actual XML parser (or god forbid,\n> XSLT) to do the transformation. But if you spent more than 5 minutes on\n> it and got disgusted, I wouldn't ask you to look deeper than that. :)\n\nWell, I didn't spend 5 minutes on it, but my experience told me\nsomething like that would happen. ;-)\n\nNow I realize that I'm wrong in my \"it doesn't work from the command\nline\". Somehow, I read the following in the user manual: \"Many\nattributes can only be defined in the document header (or via the API or\nCLI).\" And *repeatedly* read is as \"(not via the API or CLI)\", somehow\nalways expecting a \"not\" to follow an \"only\". Oh well.\n\nBut of course, I did try it out before reaching for the docs, like\nanyone would. The true reason it doesn't work for me is probably this\nheader from the listing that contains \"mansource\": \"Manpage attributes\n(relevant only when using the manpage doctype and/or converter)\".\n\n> I doubt we'd see any other refmeta tags (and any non-tag content would\n> be quoted).\n>\n> > Let's instead try to stay as close as possible to what asciidoc.conf\n> > does. We'll make it fairly obvious that we aim to inject the exact same\n> > three lines of `` that asciidoc.conf provides. The only\n> > somewhat tricky part is that we inject them *post*-processing so we need\n> > to do the variable expansion ourselves.\n>\n> One thing that asciidoctor buys us that asciidoc does not is that we\n> might eventually move to directly generating the manpages, without the\n> XML / Docbook step in between. And if we do, then all of this XML\n> hackery is going to have to get replaced with something else. I guess we\n> can cross that bridge when we come to it.\n\nTodd has made a promising start in another part of this thread. There\nseems to be a few wrinkles that need some care, but hopefully nothing\nimpossible (famous last words).\n\n> The patch itself looks sane. Would we ever need to XML-quote the\n> contents of git_version? I guess the asciidoc.conf version doesn't\n> bother.\n\nGood point. Hadn't thought of it. You're right that the asciidoc.conf\nversion has the same problem and a version string like \"<>\" goes\nunescaped into the xml.\n\n> Technically the user running \"make\" could put whatever they want\n> into it, but I think this is a case of \"if it hurts, don't do it\", and\n> we can ignore it.\n\n:-)\n\nMartin\n"},{"id":"371888","messageId":"CAN0heSo92kS6mhYH0HM-z9jOKNR0wzDynu3bnq=FtBc4_+zCpQ@mail.gmail.com","threadId":"50765","inReplyTo":"20190319033023.GA25772@sigill.intra.peff.net","subject":"Re: [PATCH] asciidoctor-extensions: provide ``","fromName":"Martin Ågren","fromEmail":"martin.agren@gmail.com","sentAt":"2019-03-19T07:12:54Z","receivedAt":"2019-03-19T07:13:09Z","isPatch":true,"sender":{"key":"martin.agren@gmail.com","avatar":null},"body":"On Tue, 19 Mar 2019 at 04:30, Jeff King wrote:\n>\n> On Mon, Mar 18, 2019 at 10:46:45PM -0400, Jeff King wrote:\n>\n> > It does seem like \"mansource\" is the way asciidoctor expects us to do\n> > this. Why doesn't it work from the command line? Is it a bug in\n> > asciidoctor, or is there something more subtle going on?\n> >\n> > I think even if it is a bug and gets fixed, though, it still wouldn't\n> > have the version field (though that seems like something we could\n> > contribute to asciidoctor).\n>\n> I just tried with asciidoc 2.0.0.rc.2, which came out last week. It does\n> seem to work from the command line:\n>\n> $ make USE_ASCIIDOCTOR=Yes \\\n> ASCIIDOC_DOCBOOK=docbook5 \\\n> ASCIIDOC='asciidoctor -amansource=Git -amanmanual=\"Git Manual\"' \\\n> git-add.xml\n> $ sed -n '/refmeta/,/refmeta/p' git-add.xml\n> \n> git-add\n> 1\n> Git\n> Git Manual\n> \n\nNo such luck with asciidoctor 1.5.5. Seems like it really wants\n\"manpage\" before it considers these attributes.\n\n(That's still me holding the tool, so factor that into it.)\n\nMartin\n"},{"id":"371889","messageId":"CAN0heSo2hdkvwnZPipgqnOnb4fmhdY6d+iHqXPfVyiLsf8SkOQ@mail.gmail.com","threadId":"50765","inReplyTo":"xmqqr2b3y0tx.fsf@gitster-ct.c.googlers.com","subject":"Re: [PATCH] asciidoctor-extensions: provide ``","fromName":"Martin Ågren","fromEmail":"martin.agren@gmail.com","sentAt":"2019-03-19T07:33:57Z","receivedAt":"2019-03-19T07:34:13Z","isPatch":true,"sender":{"key":"martin.agren@gmail.com","avatar":null},"body":"On Tue, 19 Mar 2019 at 04:55, Junio C Hamano wrote:\n>\n> Jeff King writes:\n>\n> > So in my mind, the endgame is that we eventually drop asciidoc in favor\n> > of asciidoctor. The repo at:\n> >\n> > https://github.com/asciidoc/asciidoc\n> >\n> > says:\n> >\n> > NOTE: This implementation is written in Python 2, which EOLs in Jan\n> > 2020. AsciiDoc development is being continued under @asciidoctor.\n>\n> ;-)\n>\n> > I'm not sure when is the right time to switch. If we can get the output\n> > at parity, I don't think asciidoctor is too onerous to install (and we\n> > don't have to worry about ancient platforms, as they can use\n> > pre-formatted manpages).\n>\n> One minor thing that bothers me abit is the continuity of the\n> pre-formatted pages when I switch to asciidoctor to update them.\n>\n> I do not mind having to see a huge diff in the \"git log -p\" output\n> run in pre-formatted manpages and htmldocs repositories at the\n> boundary due to e.g. the differences how lines are broken or folded\n> between the formatters, but by the time we have to transition, the\n> efforts by you, Martin and friends to allow us compare the formatted\n> docs would have made the real differences to empty (or at least\n> negligible). Knock knock...\n\nThis might be a good spot to provide a bit of current status on this.\n\nThe patch under discussion here and 185f9a0ea0 (\"asciidoctor-extensions:\nfix spurious space after linkgit\", 2019-02-27), reduce the diff\nbetween asciidoc and asciidoctor considerably. Let's assume for the\nmoment that these patches or something like them enter master...\n\nThere's one larger difference in git-checkout.txt which I'm staying away\nfrom at the moment, since Duy is doing a lot of work there at the time.\n(He's not making that asciidoc/tor issue any worse, and he's not\nspreading it to say git-switch.txt, so I'd rather just not rush to it.)\nLet's assume I get around to this soonish...\n\nLet's also assume that `./doc-diff --cut-header-footer --from-asciidoc\n--to-asciidoctor HEAD HEAD` enters master [1]. Then what remains is a\nfairly small and understandable diff where most issues are \"yeah, I\nguess that looks nicer\" or even \"they're just as fine\", rather than \"oh\nwow, that's ugly\". (IMHO.)\n\nMartin\n\n[1] You could already now run it to diff \"master master\", but you'd\ntrip on the Makefile thinking there's nothing to do. That'd be fixed by\n9a71722b4d (\"Doc: auto-detect changed build flags\", 2019-03-17).\n"},{"id":"371890","messageId":"CAN0heSr2zCQMM6wOM0UnD28qj_VygQ5CQHGHhMR9+H23snpt5Q@mail.gmail.com","threadId":"50765","inReplyTo":"CAN0heSo2hdkvwnZPipgqnOnb4fmhdY6d+iHqXPfVyiLsf8SkOQ@mail.gmail.com","subject":"Re: [PATCH] asciidoctor-extensions: provide ``","fromName":"Martin Ågren","fromEmail":"martin.agren@gmail.com","sentAt":"2019-03-19T07:36:19Z","receivedAt":"2019-03-19T07:36:33Z","isPatch":true,"sender":{"key":"martin.agren@gmail.com","avatar":null},"body":"Hi Junio\n\nOn Tue, 19 Mar 2019 at 08:33, Martin Ågren wrote:\n> between asciidoc and asciidoctor considerably. Let's assume for the\n> moment that these patches or something like them enter master...\n\nTo be clear. *This* patch has a sufficiently incorrect commit message\nthat it really needs a makeover. You can expect a v2.\n\nMartin\n"},{"id":"371891","messageId":"20190319074321.GA2189@sigill.intra.peff.net","threadId":"50765","inReplyTo":"CAN0heSo92kS6mhYH0HM-z9jOKNR0wzDynu3bnq=FtBc4_+zCpQ@mail.gmail.com","subject":"Re: [PATCH] asciidoctor-extensions: provide ``","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2019-03-19T07:43:21Z","receivedAt":"2019-03-19T07:43:25Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Mar 19, 2019 at 08:12:54AM +0100, Martin Ågren wrote:\n\n> > I just tried with asciidoc 2.0.0.rc.2, which came out last week. It does\n> > seem to work from the command line:\n> >\n> > $ make USE_ASCIIDOCTOR=Yes \\\n> > ASCIIDOC_DOCBOOK=docbook5 \\\n> > ASCIIDOC='asciidoctor -amansource=Git -amanmanual=\"Git Manual\"' \\\n> > git-add.xml\n> > $ sed -n '/refmeta/,/refmeta/p' git-add.xml\n> > \n> > git-add\n> > 1\n> > Git\n> > Git Manual\n> > \n> \n> No such luck with asciidoctor 1.5.5. Seems like it really wants\n> \"manpage\" before it considers these attributes.\n> \n> (That's still me holding the tool, so factor that into it.)\n\nThe refmiscinfo stuff didn't arrive until asciidoctor v1.5.7.\n\n-Peff\n"},{"id":"372017","messageId":"20190320181715.GJ31362@pobox.com","threadId":"50765","inReplyTo":"CAN0heSrajiswzpm+au_5nmZMZG9406iZa-CK9p5CaHLTuxm8nw@mail.gmail.com","subject":"Re: [PATCH] asciidoctor-extensions: provide ``","fromName":"Todd Zullinger","fromEmail":"tmz@pobox.com","sentAt":"2019-03-20T18:17:15Z","receivedAt":"2019-03-20T18:17:30Z","isPatch":true,"sender":{"key":"tmz@pobox.com","avatar":"https://avatars.githubusercontent.com/u/806319?v=4"},"body":"Hi,\n\nMartin Ågren wrote:\n> On Sun, 17 Mar 2019 at 20:44, Todd Zullinger wrote:\n>> Martin Ågren wrote:\n>> +ASCIIDOC_EXTRA += -a mansource=\"Git $(GIT_VERSION)\" -a manmanual=\"Git Manual\"\n> \n> So to be honest, I still don't understand how this works, but it does,\n> great. I really need to improve my documentation-reading skills.\n\nLet me know if you find any good methods for perfect\nretention. I've re-read enough documentation for a\nlifetime. ;)\n\n> I had some more time to look at this. Thanks for getting started on this\n> switch. A few things I noticed:\n> \n> {litdd} now renders as --. We should find some other way to\n> produce '--'. This should then be a simple change, as we're already\n> providing this attribute inside an `ifdef USE_ASCIIDOCTOR`.\n\nI noticed that one and didn't work out a good fix, but it\nsounds like you have one in mind. That's great.\n\n> \"+\" becomes \"+\". I didn't immediately find where we do that.\n\nFor this one, I was working on replacing \"{plus}\" with `+`\n(along with \" \" and \"-\"). That's probably not ideal though.\n\nThis is what I had to test:\n\n-- 8< --\nSubject: [PATCH] WIP: wrap \"[ +-]\" in backticks (NOT FOR SUBMISSION)\n\nasciidoctor's manpage output html-encodes \"{plus}\" which seems like a\nbug. At the least it needs some option I've yet to learn.\n---\n Documentation/git-add.txt | 26 +++++++++++++-------------\n Documentation/gitweb.txt | 6 +++---\n 2 files changed, 16 insertions(+), 16 deletions(-)\n\ndiff --git a/Documentation/git-add.txt b/Documentation/git-add.txt\nindex 37bcab94d5..dc1dd3a91b 100644\n--- a/Documentation/git-add.txt\n+++ b/Documentation/git-add.txt\n@@ -363,20 +363,20 @@ may see in a patch, and which editing operations make sense on them.\n --\n added content::\n \n-Added content is represented by lines beginning with \"{plus}\". You can\n+Added content is represented by lines beginning with `\"+\"`. You can\n prevent staging any addition lines by deleting them.\n \n removed content::\n \n-Removed content is represented by lines beginning with \"-\". You can\n-prevent staging their removal by converting the \"-\" to a \" \" (space).\n+Removed content is represented by lines beginning with `\"-\"`. You can\n+prevent staging their removal by converting the `\"-\"` to a `\" \"` (space).\n \n modified content::\n \n-Modified content is represented by \"-\" lines (removing the old content)\n-followed by \"{plus}\" lines (adding the replacement content). You can\n-prevent staging the modification by converting \"-\" lines to \" \", and\n-removing \"{plus}\" lines. Beware that modifying only half of the pair is\n+Modified content is represented by `\"-\"` lines (removing the old content)\n+followed by `\"+\"` lines (adding the replacement content). You can\n+prevent staging the modification by converting `\"-\"` lines to `\" \"`, and\n+removing `\"+\"` lines. Beware that modifying only half of the pair is\n likely to introduce confusing changes to the index.\n --\n \n@@ -393,29 +393,29 @@ Avoid using these constructs, or do so with extreme caution.\n removing untouched content::\n \n Content which does not differ between the index and working tree may be\n-shown on context lines, beginning with a \" \" (space). You can stage\n-context lines for removal by converting the space to a \"-\". The\n+shown on context lines, beginning with a `\" \"` (space). You can stage\n+context lines for removal by converting the space to a `\"-\"`. The\n resulting working tree file will appear to re-add the content.\n \n modifying existing content::\n \n One can also modify context lines by staging them for removal (by\n-converting \" \" to \"-\") and adding a \"{plus}\" line with the new content.\n-Similarly, one can modify \"{plus}\" lines for existing additions or\n+converting `\" \"` to `\"-\"`) and adding a `\"+\"` line with the new content.\n+Similarly, one can modify `\"+\"` lines for existing additions or\n modifications. In all cases, the new modification will appear reverted\n in the working tree.\n \n new content::\n \n You may also add new content that does not exist in the patch; simply\n-add new lines, each starting with \"{plus}\". The addition will appear\n+add new lines, each starting with `\"+\"`. The addition will appear\n reverted in the working tree.\n --\n \n There are also several operations which should be avoided entirely, as\n they will make the patch impossible to apply:\n \n-* adding context (\" \") or removal (\"-\") lines\n+* adding context (`\" \"`) or removal (`\"-\"`) lines\n * deleting context or removal lines\n * modifying the contents of context or removal lines\n \ndiff --git a/Documentation/gitweb.txt b/Documentation/gitweb.txt\nindex 88450589af..d27f239242 100644\n--- a/Documentation/gitweb.txt\n+++ b/Documentation/gitweb.txt\n@@ -80,15 +80,15 @@ continuation (newline escaping).\n * Leading and trailing whitespace are ignored.\n \n * Whitespace separated fields; any run of whitespace can be used as field\n-separator (rules for Perl's \"`split(\" \", $line)`\").\n+separator (rules for Perl's \"`split(`\" \"`, $line)`\").\n \n * Fields use modified URI encoding, defined in RFC 3986, section 2.1\n (Percent-Encoding), or rather \"Query string encoding\" (see\n https://en.wikipedia.org/wiki/Query_string#URL_encoding[]), the difference\n-being that SP (\" \") can be encoded as \"{plus}\" (and therefore \"{plus}\" has to be\n+being that SP (`\" \"`) can be encoded as `\"+\"` (and therefore `\"+\"` has to be\n also percent-encoded).\n +\n-Reserved characters are: \"%\" (used for encoding), \"{plus}\" (can be used to\n+Reserved characters are: \"%\" (used for encoding), `\"+\"` (can be used to\n encode SPACE), all whitespace characters as defined in Perl, including SP,\n TAB and LF, (used to separate fields in a record).\n\n-- 8< --\n\n> `./doc-diff HEAD^ HEAD` shows how several \"git-\\nfoo\" become\n> \"\\ngit-foo\", i.e., linkgit expansions are now treated as non-breaking.\n> That's arguably good, but it brings some noise to the diff. Maybe one\n> should try and see if it's possible to break that to have a nicer\n> diff, then remove that breakage in a follow-up commit. Or, if it's\n> possible to make \"git-foo\" non-breaking before the switch. Hmm, was this\n> why you increased MANWIDTH?\n\nYeah, I noticed a number of places where asciidoc and\nasciidoctor wrapped lines at slightly different places. I\ndidn't see if they were all due to wrapping at a dashed git\ncommand, but that could certainly have been the main cause.\n\nI set the large MANWIDTH and then I think I added\n--color-words to the diff call in doc-diff.\n\n> A double-space between sentences turns into a single space -- at least\n> in constructions such as \"... to foobar. `git-foo` does ...\". Not a\n> problem perhaps, but noise in the diff.\n> \n> And I'm sure there's more lurking in that huge diff. Whether any of that\n> is significant or not is another matter. ;-)\n\nOh my yes, I'm sure. :)\n\n-- \nTodd\n"},{"id":"372018","messageId":"20190320183229.GK31362@pobox.com","threadId":"50765","inReplyTo":"20190319074321.GA2189@sigill.intra.peff.net","subject":"Re: [PATCH] asciidoctor-extensions: provide ``","fromName":"Todd Zullinger","fromEmail":"tmz@pobox.com","sentAt":"2019-03-20T18:32:29Z","receivedAt":"2019-03-20T18:32:35Z","isPatch":true,"sender":{"key":"tmz@pobox.com","avatar":"https://avatars.githubusercontent.com/u/806319?v=4"},"body":"Jeff King wrote:\n> On Tue, Mar 19, 2019 at 08:12:54AM +0100, Martin Ågren wrote:\n> \n>>> I just tried with asciidoc 2.0.0.rc.2, which came out last week. It does\n>>> seem to work from the command line:\n>>>\n>>> $ make USE_ASCIIDOCTOR=Yes \\\n>>> ASCIIDOC_DOCBOOK=docbook5 \\\n>>> ASCIIDOC='asciidoctor -amansource=Git -amanmanual=\"Git Manual\"' \\\n>>> git-add.xml\n>>> $ sed -n '/refmeta/,/refmeta/p' git-add.xml\n>>> \n>>> git-add\n>>> 1\n>>> Git\n>>> Git Manual\n>>> \n>> \n>> No such luck with asciidoctor 1.5.5. Seems like it really wants\n>> \"manpage\" before it considers these attributes.\n>> \n>> (That's still me holding the tool, so factor that into it.)\n> \n> The refmiscinfo stuff didn't arrive until asciidoctor v1.5.7.\n\nNow I don't feel as bad that I didn't find any good way to\nhandle refmiscinfo when I first tried to use asciidoctor in\nNovember 2017 at least.\n\nThis made me look closer at the fedora asciidoctor packages.\nThey have been stuck on 1.5.6.1. So all my recent testing\nhas been with 1.5.6.1.\n\nI spent some time yesterday working toward getting the\nfedora packages updated to 1.5.8. With luck that will reach\nthe stable updates channels before too long.\n\n(I'm guessing the upcoming 2.0 release won't be suitable as\nan update for released fedora versions, being a major\nrelease with potentially backwards-incompatible changes.)\n\nThere are differences in the output from 1.5.6.1 and 1.5.8,\nbut I haven't looked closely yet.\n\n-- \nTodd\n"},{"id":"372276","messageId":"CAN0heSpJvsPm_qq63VumokmyOG6N=6fwMZRqf_9CzoCeHsdiyQ@mail.gmail.com","threadId":"50765","inReplyTo":"20190320181715.GJ31362@pobox.com","subject":"Re: [PATCH] asciidoctor-extensions: provide ``","fromName":"Martin Ågren","fromEmail":"martin.agren@gmail.com","sentAt":"2019-03-22T21:01:37Z","receivedAt":"2019-03-22T21:01:53Z","isPatch":true,"sender":{"key":"martin.agren@gmail.com","avatar":null},"body":"On Wed, 20 Mar 2019 at 19:17, Todd Zullinger wrote:\n> Martin Ågren wrote:\n> > {litdd} now renders as --. We should find some other way to\n> > produce '--'. This should then be a simple change, as we're already\n> > providing this attribute inside an `ifdef USE_ASCIIDOCTOR`.\n>\n> I noticed that one and didn't work out a good fix, but it\n> sounds like you have one in mind. That's great.\n>\n> > \"+\" becomes \"+\". I didn't immediately find where we do that.\n>\n> For this one, I was working on replacing \"{plus}\" with `+`\n> (along with \" \" and \"-\"). That's probably not ideal though.\n\nThe \"plus\" and \"litdd\" issues seem like they can be solved by doing:\n\n ASCIIDOC_EXTRA += -aplus='+'\n ASCIIDOC_EXTRA += -alitdd='\\--'\n\n> > `./doc-diff HEAD^ HEAD` shows how several \"git-\\nfoo\" become\n> > \"\\ngit-foo\", i.e., linkgit expansions are now treated as non-breaking.\n> > [...] Hmm, was this\n> > why you increased MANWIDTH?\n>\n> Yeah, I noticed a number of places where asciidoc and\n> asciidoctor wrapped lines at slightly different places. I\n> didn't see if they were all due to wrapping at a dashed git\n> command, but that could certainly have been the main cause.\n>\n> I set the large MANWIDTH and then I think I added\n> --color-words to the diff call in doc-diff.\n>\n> > A double-space between sentences turns into a single space -- at least\n> > in constructions such as \"... to foobar. `git-foo` does ...\". Not a\n> > problem perhaps, but noise in the diff.\n> >\n> > And I'm sure there's more lurking in that huge diff. Whether any of that\n> > is significant or not is another matter. ;-)\n>\n> Oh my yes, I'm sure. :)\n\n:)\n\nUsing origin/ma/doc-diff-doc-vs-doctor-comparison, you can do something\nlike\n\n ./doc-diff --asciidoctor --cut-header-footer foo bar --\n--ignore-space-change --word-diff\n\nto reduce the amount of noise in the diff by a great deal. Going through\nthe result, I noted two issues which should be fixed by Asciidoctor\n1.5.7, according to some initial searching (but I haven't tested):\n\n\"--\" renders as an em dash followed by a \";\". Looks like it's\nhttps://github.com/asciidoctor/asciidoctor/issues/2604\n\n\"&\" turns into \"&\". Seems to be known and fixed:\nhttps://github.com/asciidoctor/asciidoctor/issues/2525\n\nPlus a few other problems. There's the odd problem here and there, but\nmost of the diff seems to be a large number of occurences of just a few\nissues.\n\nIt would probably be worthwhile to try 1.5.7+ to see how much that\nimproves things. Seems like you're already underway there.\n\nMartin\n"},{"id":"372311","messageId":"20190323192756.GK4047@pobox.com","threadId":"50765","inReplyTo":"CAN0heSpJvsPm_qq63VumokmyOG6N=6fwMZRqf_9CzoCeHsdiyQ@mail.gmail.com","subject":"Re: [PATCH] asciidoctor-extensions: provide ``","fromName":"Todd Zullinger","fromEmail":"tmz@pobox.com","sentAt":"2019-03-23T19:27:56Z","receivedAt":"2019-03-23T19:28:04Z","isPatch":true,"sender":{"key":"tmz@pobox.com","avatar":"https://avatars.githubusercontent.com/u/806319?v=4"},"body":"Hi,\n\nMartin Ågren wrote:\n> On Wed, 20 Mar 2019 at 19:17, Todd Zullinger wrote:\n>> Martin Ågren wrote:\n>>> {litdd} now renders as --. We should find some other way to\n>>> produce '--'. This should then be a simple change, as we're already\n>>> providing this attribute inside an `ifdef USE_ASCIIDOCTOR`.\n>>\n>> I noticed that one and didn't work out a good fix, but it\n>> sounds like you have one in mind. That's great.\n>>\n>>> \"+\" becomes \"+\". I didn't immediately find where we do that.\n>>\n>> For this one, I was working on replacing \"{plus}\" with `+`\n>> (along with \" \" and \"-\"). That's probably not ideal though.\n> \n> The \"plus\" and \"litdd\" issues seem like they can be solved by doing:\n> \n> ASCIIDOC_EXTRA += -aplus='+'\n> ASCIIDOC_EXTRA += -alitdd='\\--'\n\nHmm, setting litdd makes the html generate an em dash\nfollowed by a zero width space (in 1.5.8 and 2.0.0)\n\nI updated my system to asciidoctor-2.0.0 last night and now\nI can't even generate the man pages properly, because the\ndocbook45 converter was removed. I'll have to see if I\nmissed some other required update. :/\n\n> It would probably be worthwhile to try 1.5.7+ to see how much that\n> improves things. Seems like you're already underway there.\n\nYeah, before I knew how soon 2.0.0 was coming, I updated to\n1.5.8 and built the various Fedora packages which require it\nto see how they handled it. Almost all of the changes were\nfixes to bugs in previous versions. And the one which was\nnot is likely to be fixed in 2.0.0 according to asciidoctor\nmaintainer Dan Allen.\n\nHave you looked at diffing the html output as well? It\nseems like we'll need to check it as well to be sure any\nfixes to the manpage output don't have a negative impact on\nthe html output, and vice versa.\n\nI used 'links -dump' output for comparison of the various\nFedora packages which currently build with asciidoctor.\nIt's not perfect. It could miss visual changes which might\nbe important when viewed in a graphical browser. But it was\nbetter than trying to diff the html files directly. :)\n\nWe probably also want to check the validity of links within\nthe docs, as one of the changes in 1.5.8 caused breakage of\ncross interdocument references. (This is the change I noted\nabove which should be fixed in 2.0.0.)\n\nIt'll be quite a while until most systems with asciidoctor\n1.5.x are gone. I doubt that upgrading to even 1.5.8 is\nsuitable for the currently released Fedora versions due to\nincompatible changes. I am going to try and get 2.0.0 into\nFedora 30, but the deadline for changes has just passed, so\nI may not be able to do so. If not, it'll be 6-8 months\nuntil a released version of Fedora has an asciidoctor newer\nthan 1.5.6.1.\n\nAll that is just to say that even if newer asciidoctor fixes\nmany of the issues we've seen it will still be a long time\nbefore we can reasonable default to asciidoctor or drop\nasciidoc support.\n\nFor what it's worth, the Fedora asciidoc packages moved to\npython3 using https://github.com/asciidoc/asciidoc-py3.\nThat's not very active, but it should at least keep asciidoc\nalive beyond the approaching python2 EOL.\n\n-- \nTodd\n"},{"id":"372355","messageId":"20190324121619.GD312@sigill.intra.peff.net","threadId":"50765","inReplyTo":"20190323192756.GK4047@pobox.com","subject":"Re: [PATCH] asciidoctor-extensions: provide ``","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2019-03-24T12:16:19Z","receivedAt":"2019-03-24T12:16:23Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Sat, Mar 23, 2019 at 03:27:56PM -0400, Todd Zullinger wrote:\n\n> I updated my system to asciidoctor-2.0.0 last night and now\n> I can't even generate the man pages properly, because the\n> docbook45 converter was removed. I'll have to see if I\n> missed some other required update. :/\n\nI ran into that, too. Using ASCIIDOC_DOCBOOK=docbook5 worked. The output\nlooked OK to me, but I only eyeballed it briefly. It's possible there\nare subtle problems.\n\n-Peff\n"},{"id":"372373","messageId":"20190324162131.GL4047@pobox.com","threadId":"50765","inReplyTo":"20190324121619.GD312@sigill.intra.peff.net","subject":"Re: [PATCH] asciidoctor-extensions: provide ``","fromName":"Todd Zullinger","fromEmail":"tmz@pobox.com","sentAt":"2019-03-24T16:21:31Z","receivedAt":"2019-03-24T16:21:43Z","isPatch":true,"sender":{"key":"tmz@pobox.com","avatar":"https://avatars.githubusercontent.com/u/806319?v=4"},"body":"Hi,\n\nJeff King wrote:\n> On Sat, Mar 23, 2019 at 03:27:56PM -0400, Todd Zullinger wrote:\n> \n>> I updated my system to asciidoctor-2.0.0 last night and now\n>> I can't even generate the man pages properly, because the\n>> docbook45 converter was removed. I'll have to see if I\n>> missed some other required update. :/\n> \n> I ran into that, too. Using ASCIIDOC_DOCBOOK=docbook5 worked. The output\n> looked OK to me, but I only eyeballed it briefly. It's possible there\n> are subtle problems.\n\nInteresting. I did that last night as well, but it only led\nto more errors. I'm curious why manpage builds work for you\nand not for me.\n\nOn my fedora 29 test system, ASCIIDOC_DOCBOOK=docbook5 leads\nto a validation failure from xmlto, since docbook5 doesn't\nuse a DTD¹. I added XMLTO_EXTRA = --skip-validation to the\nUSE_ASCIIDOCTOR block, which can build many of the man\npages, but fails when it gets to git-blame due to use of\nliteral < > characters in the xml:\n\n git-blame.xml:423: parser error : StartTag: invalid element name\n <40-byte hex sha1> <\n ^\n\nWhile this may be a real issue in the formatting of\ngit-blame.txt which we should fix, I'm suspicious of that\ndue to the number of other files similarly affected:\n\n git-blame.txt\n git-diff-files.txt\n git-diff-index.txt\n git-diff-tree.txt\n git-diff.txt\n git-format-patch.txt\n git-http-fetch.txt\n git-log.txt\n git-rebase.txt\n git-rev-list.txt\n git-show.txt\n git-svn.txt\n\n¹ Here's the failure I get without --skip-validation:\n[tmz@f29 Documentation]$ make USE_ASCIIDOCTOR=1 git.1\n SUBDIR ../\nmake[1]: 'GIT-VERSION-FILE' is up to date.\n ASCIIDOC git.xml\n XMLTO git.1\nxmlto: /home/tmz/src/git/Documentation/git.xml does not validate (status 3)\nxmlto: Fix document syntax or use --skip-validation option\nvalidity error : no DTD found!\nDocument /home/tmz/src/git/Documentation/git.xml does not validate\nmake: *** [Makefile:359: git.1] Error 13\n\nThanks,\n\n-- \nTodd\n"},{"id":"372431","messageId":"20190325150633.GC19929@sigill.intra.peff.net","threadId":"50765","inReplyTo":"20190324162131.GL4047@pobox.com","subject":"Re: [PATCH] asciidoctor-extensions: provide ``","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2019-03-25T15:06:33Z","receivedAt":"2019-03-25T15:06:37Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Sun, Mar 24, 2019 at 12:21:31PM -0400, Todd Zullinger wrote:\n\n> > I ran into that, too. Using ASCIIDOC_DOCBOOK=docbook5 worked. The output\n> > looked OK to me, but I only eyeballed it briefly. It's possible there\n> > are subtle problems.\n> \n> Interesting. I did that last night as well, but it only led\n> to more errors. I'm curious why manpage builds work for you\n> and not for me.\n\nI think it's because I'm an idiot. I must have only been using 2.0.0\nwhen I was looking at the XML output manually (for the refmiscinfo\nlines), and never actually rendered it to roff. I get the same problem\nwhen I try a full build.\n\n> On my fedora 29 test system, ASCIIDOC_DOCBOOK=docbook5 leads\n> to a validation failure from xmlto, since docbook5 doesn't\n> use a DTD¹. I added XMLTO_EXTRA = --skip-validation to the\n> USE_ASCIIDOCTOR block, which can build many of the man\n> pages, but fails when it gets to git-blame due to use of\n> literal < > characters in the xml:\n> \n> git-blame.xml:423: parser error : StartTag: invalid element name\n> <40-byte hex sha1> <\n> ^\n\nThat seems like a bug in asciidoctor, which ought to be quoting the \"<\".\nWe certainly can't quote it ourselves (we don't even know that our\nbackend output is going to a format that needs angle brackets quoted).\n\n-Peff\n"},{"id":"372456","messageId":"20190325190041.GM4047@pobox.com","threadId":"50765","inReplyTo":"20190325150633.GC19929@sigill.intra.peff.net","subject":"Re: [PATCH] asciidoctor-extensions: provide ``","fromName":"Todd Zullinger","fromEmail":"tmz@pobox.com","sentAt":"2019-03-25T19:00:41Z","receivedAt":"2019-03-25T19:00:48Z","isPatch":true,"sender":{"key":"tmz@pobox.com","avatar":"https://avatars.githubusercontent.com/u/806319?v=4"},"body":"Hi,\n\nJeff King wrote:\n> On Sun, Mar 24, 2019 at 12:21:31PM -0400, Todd Zullinger wrote:\n>> I'm curious why manpage builds work for you and not for me.\n> \n> I think it's because I'm an idiot. I must have only been using 2.0.0\n> when I was looking at the XML output manually (for the refmiscinfo\n> lines), and never actually rendered it to roff. I get the same problem\n> when I try a full build.\n\nAhh. I was hoping you'd tell me that I was a fool. :)\n\n>> On my fedora 29 test system, ASCIIDOC_DOCBOOK=docbook5 leads\n>> to a validation failure from xmlto, since docbook5 doesn't\n>> use a DTD¹. I added XMLTO_EXTRA = --skip-validation to the\n>> USE_ASCIIDOCTOR block, which can build many of the man\n>> pages, but fails when it gets to git-blame due to use of\n>> literal < > characters in the xml:\n>> \n>> git-blame.xml:423: parser error : StartTag: invalid element name\n>> <40-byte hex sha1> <\n>> ^\n> \n> That seems like a bug in asciidoctor, which ought to be quoting the \"<\".\n> We certainly can't quote it ourselves (we don't even know that our\n> backend output is going to a format that needs angle brackets quoted).\n\nYep, it seems so. I filed this upstream:\n\nhttps://github.com/asciidoctor/asciidoctor/issues/3205\n\nI updated to asciidoctor-2.0.1 this morning to test, in case\nit was one of the issues fixed since the 2.0.0 release.\nAlas, we're the first to hit it and report it.\n\n-- \nTodd\n"},{"id":"372555","messageId":"20190327010603.GR4047@pobox.com","threadId":"50765","inReplyTo":"20190325190041.GM4047@pobox.com","subject":"Re: [PATCH] asciidoctor-extensions: provide ``","fromName":"Todd Zullinger","fromEmail":"tmz@pobox.com","sentAt":"2019-03-27T01:06:03Z","receivedAt":"2019-03-27T01:06:12Z","isPatch":true,"sender":{"key":"tmz@pobox.com","avatar":"https://avatars.githubusercontent.com/u/806319?v=4"},"body":"Hi,\n\nI wrote:\n> Jeff King wrote:\n>> That seems like a bug in asciidoctor, which ought to be quoting the \"<\".\n>> We certainly can't quote it ourselves (we don't even know that our\n>> backend output is going to a format that needs angle brackets quoted).\n> \n> Yep, it seems so. I filed this upstream:\n> \n> https://github.com/asciidoctor/asciidoctor/issues/3205\n> \n> I updated to asciidoctor-2.0.1 this morning to test, in case\n> it was one of the issues fixed since the 2.0.0 release.\n> Alas, we're the first to hit it and report it.\n\nDan Allen fixed this upstream and released 2.0.2 today.\nIt's very good to know that asciidoctor upstream is\nincredibly responsive. If anyone runs into Dan at a\nconference, please buy him a beer. ;)\n\nThere's still the matter of 2.0 dropping docbook45. I'll\ntry to get around to testing 1.5.x releases with docbook5 to\nsee if they work reasonably well. If not, we can add a\nversion check and set ASCIIDOC_DOCBOOK appropriately.\n\nOne other issue that crops up with docbook5 is that the\nxmlto toolchain now outputs:\n\n Note: namesp. cut : stripped namespace before processing\n\nas documented at\n\n https://docbook.org/docs/howto/howto.html#dbxsl\n\nIt's mostly an annoyance which we either want to strip out,\nor pass an alternate docbook5 xsl, I think. But I'm not\nvery familiar with the guts of the xml/docbook toolchains.\n\n-- \nTodd\n"},{"id":"372561","messageId":"20190327100522.GA32732@szeder.dev","threadId":"50765","inReplyTo":"20190327010603.GR4047@pobox.com","subject":"Re: [PATCH] asciidoctor-extensions: provide ``","fromName":"SZEDER Gábor","fromEmail":"szeder.dev@gmail.com","sentAt":"2019-03-27T10:05:22Z","receivedAt":"2019-03-27T10:05:33Z","isPatch":true,"sender":{"key":"szeder.dev@gmail.com","avatar":"https://avatars.githubusercontent.com/u/116324?v=4"},"body":"On Tue, Mar 26, 2019 at 09:06:03PM -0400, Todd Zullinger wrote:\n> Dan Allen fixed this upstream and released 2.0.2 today.\n> It's very good to know that asciidoctor upstream is\n> incredibly responsive. If anyone runs into Dan at a\n> conference, please buy him a beer. ;)\n\nI noticed the \"Release beer\" lines in the Asciidoctor relnotes... :)\n\n> One other issue that crops up with docbook5 is that the\n> xmlto toolchain now outputs:\n> \n> Note: namesp. cut : stripped namespace before processing\n> \n> as documented at\n> \n> https://docbook.org/docs/howto/howto.html#dbxsl\n> \n> It's mostly an annoyance which we either want to strip out,\n> or pass an alternate docbook5 xsl, I think. But I'm not\n> very familiar with the guts of the xml/docbook toolchains.\n\nIn our documentation CI build jobs we check that nothing is written to\nAsciidoc/Asciidoctor's standard error [1]. These \"Note:\" lines go to\nstderr, and will trigger that check and cause the build to fail. So\nwither we should find a way to silence these notes, or filter them out\nin the CI builds.\n\n\n[1] 505ad91304 (travis-ci: check AsciiDoc/AsciiDoctor stderr output,\n 2017-04-26), though it never actually worked as intended, but that\n is about to get fixed:\n\n https://public-inbox.org/git/20190324215534.9495-7-szeder.dev@gmail.com/\n\n\n"},{"id":"372596","messageId":"20190328000612.GA12419@genre.crustytoothpaste.net","threadId":"50765","inReplyTo":"20190327010603.GR4047@pobox.com","subject":"Re: [PATCH] asciidoctor-extensions: provide ``","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2019-03-28T00:06:12Z","receivedAt":"2019-03-28T00:06:25Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On Tue, Mar 26, 2019 at 09:06:03PM -0400, Todd Zullinger wrote:\n> There's still the matter of 2.0 dropping docbook45. I'll\n> try to get around to testing 1.5.x releases with docbook5 to\n> see if they work reasonably well. If not, we can add a\n> version check and set ASCIIDOC_DOCBOOK appropriately.\n> \n> One other issue that crops up with docbook5 is that the\n> xmlto toolchain now outputs:\n> \n> Note: namesp. cut : stripped namespace before processing\n> \n> as documented at\n> \n> https://docbook.org/docs/howto/howto.html#dbxsl\n> \n> It's mostly an annoyance which we either want to strip out,\n> or pass an alternate docbook5 xsl, I think. But I'm not\n> very familiar with the guts of the xml/docbook toolchains.\n\nI'm quite familiar with this area, because I used DocBook and its\nstylesheets for my college papers. There are two sets of stylesheets,\nthe namespaced ones (for DocBook 5) and the non-namespaced ones (for\nDocBook 4). They can be distinguished because the URLs (and typically\nthe paths) use \"xsl\" (for non-namespaced) or \"xsl-ns\" (for namespaced).\n\nxmlto by default uses the older ones, and assuming you have a reasonably\ncapable XSLT processor, either set of stylesheets can be used, since\nthey simply transform the document into the right type (although with a\nwarning, as you've noticed).\n\nUnfortunately, xmlto is hard-coded to use the non-namespaced stylesheets\nwith no option to specify which to use, and it doesn't appear to be\nactively developed. There's an option to specify the stylesheet (-x),\nbut it can't take a URL, since xmlto \"helpfully\" fully qualifies the\npath. That means we'd need to know the location on disk of those\nstylesheets in order to use them, since we can't rely on catalog\nresolution.\n-- \nbrian m. carlson: Houston, Texas, US\nOpenPGP: https://keybase.io/bk2204\n"},{"id":"372602","messageId":"20190328025434.GA10032@sigill.intra.peff.net","threadId":"50765","inReplyTo":"20190327010603.GR4047@pobox.com","subject":"Re: [PATCH] asciidoctor-extensions: provide ``","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2019-03-28T02:54:34Z","receivedAt":"2019-03-28T02:54:37Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Mar 26, 2019 at 09:06:03PM -0400, Todd Zullinger wrote:\n\n> > I updated to asciidoctor-2.0.1 this morning to test, in case\n> > it was one of the issues fixed since the 2.0.0 release.\n> > Alas, we're the first to hit it and report it.\n> \n> Dan Allen fixed this upstream and released 2.0.2 today.\n> It's very good to know that asciidoctor upstream is\n> incredibly responsive. If anyone runs into Dan at a\n> conference, please buy him a beer. ;)\n\nCool. I've interacted a few times with the asciidoctor project due to\nour use on git-scm.com, and have always found them pleasant and\nresponsive.\n\n> There's still the matter of 2.0 dropping docbook45. I'll\n> try to get around to testing 1.5.x releases with docbook5 to\n> see if they work reasonably well. If not, we can add a\n> version check and set ASCIIDOC_DOCBOOK appropriately.\n\nWith the hacky patch below, I was able to doc-diff the v1.5.8 versus\nv2.0.2 output. It looks like there are quite a few cosmetic whitespace\nchanges. Some probably intentional (extra blank between terms and\nparagraphs in a definition list) and some probably not (extra whitespace\nbefore start of content in a bulleted list).\n\ndiff --git a/Documentation/doc-diff b/Documentation/doc-diff\nindex 3355be4798..4c2dccf516 100755\n--- a/Documentation/doc-diff\n+++ b/Documentation/doc-diff\n@@ -17,9 +17,11 @@ f\t\t\tforce rebuild; do not rely on cached results\n c,clean\t\t\tcleanup temporary working files\n from-asciidoc\t\tuse asciidoc with the 'from'-commit\n from-asciidoctor\tuse asciidoctor with the 'from'-commit\n+from-asciidoctor2\tuse asciidoctor2 with the 'from'-commit\n asciidoc\t\tuse asciidoc with both commits\n to-asciidoc\t\tuse asciidoc with the 'to'-commit\n to-asciidoctor\t\tuse asciidoctor with the 'to'-commit\n+to-asciidoctor2\t\tuse asciidoctor with the 'to'-commit\n asciidoctor\t\tuse asciidoctor with both commits\n cut-header-footer\tcut away header and footer\n \"\n@@ -55,6 +57,10 @@ do\n \t--asciidoc)\n \t\tfrom_program=-asciidoc\n \t\tto_program=-asciidoc ;;\n+\t--from-asciidoctor2)\n+\t\tfrom_program=-asciidoctor2 ;;\n+\t--to-asciidoctor2)\n+\t\tto_program=-asciidoctor2 ;;\n \t--cut-header-footer)\n \t\tcut_header_footer=-cut-header-footer ;;\n \t--)\n@@ -112,6 +118,13 @@ construct_makemanflags () {\n \telif test \"$1\" = \"-asciidoctor\"\n \tthen\n \t\techo USE_ASCIIDOCTOR=YesPlease\n+\telif test \"$1\" = \"-asciidoctor2\"\n+\tthen\n+\t\techo USE_ASCIIDOCTOR=YesPlease\n+\t\techo ASCIIDOC_DOCBOOK=docbook5\n+\t\techo XMLTO_EXTRA=--skip-validation\n+\t\t# not really portable :)\n+\t\techo ASCIIDOCTOR=/tmp/run-asciidoctor-2\n \tfi\n }\n \n@@ -182,6 +195,6 @@ render_tree () {\n \tfi\n }\n \n-render_tree $from_oid $from_dir $from_makemanflags &&\n-render_tree $to_oid $to_dir $to_makemanflags &&\n+render_tree $from_oid $from_dir \"$from_makemanflags\" &&\n+render_tree $to_oid $to_dir \"$to_makemanflags\" &&\n git -C $tmp/rendered diff --no-index \"$@\" $from_dir $to_dir\n"},{"id":"372605","messageId":"20190328033345.GA18544@sigill.intra.peff.net","threadId":"50765","inReplyTo":"20190328025434.GA10032@sigill.intra.peff.net","subject":"Re: [PATCH] asciidoctor-extensions: provide ``","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2019-03-28T03:33:45Z","receivedAt":"2019-03-28T03:33:48Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Mar 27, 2019 at 10:54:34PM -0400, Jeff King wrote:\n\n> @@ -112,6 +118,13 @@ construct_makemanflags () {\n> \telif test \"$1\" = \"-asciidoctor\"\n> \tthen\n> \t\techo USE_ASCIIDOCTOR=YesPlease\n> +\telif test \"$1\" = \"-asciidoctor2\"\n> +\tthen\n> +\t\techo USE_ASCIIDOCTOR=YesPlease\n> +\t\techo ASCIIDOC_DOCBOOK=docbook5\n> +\t\techo XMLTO_EXTRA=--skip-validation\n> +\t\t# not really portable :)\n> +\t\techo ASCIIDOCTOR=/tmp/run-asciidoctor-2\n> \tfi\n\nOops, this last line should be ASCIIDOC=..., of course. Which meant that\nI was actually just testing the docbook5 output from v1.5.8. The results\nwith v2.0.2 seem similar, though.\n\n-Peff\n"},{"id":"372808","messageId":"20190330180014.GB4047@pobox.com","threadId":"50765","inReplyTo":"20190328000612.GA12419@genre.crustytoothpaste.net","subject":"Re: [PATCH] asciidoctor-extensions: provide ``","fromName":"Todd Zullinger","fromEmail":"tmz@pobox.com","sentAt":"2019-03-30T18:00:14Z","receivedAt":"2019-03-30T18:00:26Z","isPatch":true,"sender":{"key":"tmz@pobox.com","avatar":"https://avatars.githubusercontent.com/u/806319?v=4"},"body":"Hi,\n\nbrian m. carlson wrote:\n> On Tue, Mar 26, 2019 at 09:06:03PM -0400, Todd Zullinger wrote:\n>> There's still the matter of 2.0 dropping docbook45. I'll\n>> try to get around to testing 1.5.x releases with docbook5 to\n>> see if they work reasonably well. If not, we can add a\n>> version check and set ASCIIDOC_DOCBOOK appropriately.\n>> \n>> One other issue that crops up with docbook5 is that the\n>> xmlto toolchain now outputs:\n>> \n>> Note: namesp. cut : stripped namespace before processing\n>> \n>> as documented at\n>> \n>> https://docbook.org/docs/howto/howto.html#dbxsl\n>> \n>> It's mostly an annoyance which we either want to strip out,\n>> or pass an alternate docbook5 xsl, I think. But I'm not\n>> very familiar with the guts of the xml/docbook toolchains.\n> \n> I'm quite familiar with this area, because I used DocBook and its\n> stylesheets for my college papers. There are two sets of stylesheets,\n> the namespaced ones (for DocBook 5) and the non-namespaced ones (for\n> DocBook 4). They can be distinguished because the URLs (and typically\n> the paths) use \"xsl\" (for non-namespaced) or \"xsl-ns\" (for namespaced).\n> \n> xmlto by default uses the older ones, and assuming you have a reasonably\n> capable XSLT processor, either set of stylesheets can be used, since\n> they simply transform the document into the right type (although with a\n> warning, as you've noticed).\n> \n> Unfortunately, xmlto is hard-coded to use the non-namespaced stylesheets\n> with no option to specify which to use, and it doesn't appear to be\n> actively developed. There's an option to specify the stylesheet (-x),\n> but it can't take a URL, since xmlto \"helpfully\" fully qualifies the\n> path. That means we'd need to know the location on disk of those\n> stylesheets in order to use them, since we can't rely on catalog\n> resolution.\n\nThanks for the very useful docbook5/xmlto details!\n\nThe hard-coded use of the non-namespaced stylesheets in\nxmlto is unfortunate. I hadn't gotten far enough along to\nrun into that limitation of xmlto, so many thanks for saving\nme from finding it myself. I'm sure it would have taken me\nfar longer.\n\nIf it turns out that docbook5 causes us more pain than it's\nworth, I suppose we could choose to use the builtin manpage\nbackend when using asciidoctor >= 2.\n\nOr we could see about adding a docbook45 converter, which\nupstream noted as a possibility in the ticket¹ which dropped\ndocbook45 by default.\n\nIt'll be some time before we can reasonably expect to only\nsupport asciidoctor-2.x. We'll have to see what method\ninvolves the least ugly hacks to support asciidoc and the\nvarious 1.5.x and 2.x versions of asciidoctor.\n\n¹ https://github.com/asciidoctor/asciidoctor/issues/3005\n\n-- \nTodd\n"},{"id":"372816","messageId":"20190330210457.GC12419@genre.crustytoothpaste.net","threadId":"50765","inReplyTo":"20190330180014.GB4047@pobox.com","subject":"Re: [PATCH] asciidoctor-extensions: provide ``","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2019-03-30T21:04:57Z","receivedAt":"2019-03-30T21:05:06Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On Sat, Mar 30, 2019 at 02:00:14PM -0400, Todd Zullinger wrote:\n> Thanks for the very useful docbook5/xmlto details!\n> \n> The hard-coded use of the non-namespaced stylesheets in\n> xmlto is unfortunate. I hadn't gotten far enough along to\n> run into that limitation of xmlto, so many thanks for saving\n> me from finding it myself. I'm sure it would have taken me\n> far longer.\n> \n> If it turns out that docbook5 causes us more pain than it's\n> worth, I suppose we could choose to use the builtin manpage\n> backend when using asciidoctor >= 2.\n\nI suspect this will be the easiest way forward. If we produce\nsemantically equivalent manpages, then this is also a lot nicer for\npeople who would prefer not to have a full XML toolchain installed.\n\n> Or we could see about adding a docbook45 converter, which\n> upstream noted as a possibility in the ticket¹ which dropped\n> docbook45 by default.\n\nAnother possibility, depending on how responsive the xmlto upstream is,\nis to add some sort of DocBook 5 support to it. This shouldn't be\nterribly difficult, although we'd have to disable validation. DocBook 5\nuses RELAX NG, and libxml2/xmllint's support for this has some bugs,\nsuch that it will mark some valid documents using complex interleaves as\ninvalid. Still, if this is the direction we want to go, I have no\nproblem adding this support.\n\nSince we'd have a quite new Asciidoctor and a new xmlto, distros should\nhave both around the same time.\n-- \nbrian m. carlson: Houston, Texas, US\nOpenPGP: https://keybase.io/bk2204\n"},{"id":"373170","messageId":"20190405021721.GS4047@pobox.com","threadId":"50765","inReplyTo":"20190330210457.GC12419@genre.crustytoothpaste.net","subject":"Re: [PATCH] asciidoctor-extensions: provide ``","fromName":"Todd Zullinger","fromEmail":"tmz@pobox.com","sentAt":"2019-04-05T02:17:21Z","receivedAt":"2019-04-05T02:17:28Z","isPatch":true,"sender":{"key":"tmz@pobox.com","avatar":"https://avatars.githubusercontent.com/u/806319?v=4"},"body":"brian m. carlson wrote:\n> On Sat, Mar 30, 2019 at 02:00:14PM -0400, Todd Zullinger wrote:\n>> Thanks for the very useful docbook5/xmlto details!\n>> \n>> The hard-coded use of the non-namespaced stylesheets in\n>> xmlto is unfortunate. I hadn't gotten far enough along to\n>> run into that limitation of xmlto, so many thanks for saving\n>> me from finding it myself. I'm sure it would have taken me\n>> far longer.\n>> \n>> If it turns out that docbook5 causes us more pain than it's\n>> worth, I suppose we could choose to use the builtin manpage\n>> backend when using asciidoctor >= 2.\n> \n> I suspect this will be the easiest way forward. If we produce\n> semantically equivalent manpages, then this is also a lot nicer for\n> people who would prefer not to have a full XML toolchain installed.\n\nIt's a good end goal. There are a number of differences in\nthe output when using the manpage backend directly verus\ndocbook and then xmlto. The way links to external html\ndocuments are presented is the main one which comes to mind.\nMaybe we can adjust that via asciidoctor-extensions.rb or\nsomething, I don't know.\n\nElsewhere in this thread, Jeff made the very valid point\nthat we're probably wise to keep using the docbook/xmlto\nchain as long as we're supporting both asciidoc and\nasciidoctor. Unless it turns out that it's more work to\ncoax asciidoctor (and the various 1.5 and 2.0 releases in\ncommon use) to work with that same docbook/xmlto chain than\nit is to do it directly, that is.\n\n>> Or we could see about adding a docbook45 converter, which\n>> upstream noted as a possibility in the ticket¹ which dropped\n>> docbook45 by default.\n> \n> Another possibility, depending on how responsive the xmlto upstream is,\n> is to add some sort of DocBook 5 support to it. This shouldn't be\n> terribly difficult, although we'd have to disable validation. DocBook 5\n> uses RELAX NG, and libxml2/xmllint's support for this has some bugs,\n> such that it will mark some valid documents using complex interleaves as\n> invalid. Still, if this is the direction we want to go, I have no\n> problem adding this support.\n> \n> Since we'd have a quite new Asciidoctor and a new xmlto, distros should\n> have both around the same time.\n\nGood point. It can't hurt to push for improvements across\nthe tools. (Well, other than time being a limited resource\nand time you may spend on doc tools being time you can't\nspend on the hash conversion, which I imagine is a more\ninteresting project.)\n\nThanks,\n\n-- \nTodd\n"},{"id":"373247","messageId":"20190405184629.GF2284@sigill.intra.peff.net","threadId":"50765","inReplyTo":"20190405021721.GS4047@pobox.com","subject":"Re: [PATCH] asciidoctor-extensions: provide ``","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2019-04-05T18:46:30Z","receivedAt":"2019-04-05T18:46:33Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Apr 04, 2019 at 10:17:21PM -0400, Todd Zullinger wrote:\n\n> Elsewhere in this thread, Jeff made the very valid point\n> that we're probably wise to keep using the docbook/xmlto\n> chain as long as we're supporting both asciidoc and\n> asciidoctor. Unless it turns out that it's more work to\n> coax asciidoctor (and the various 1.5 and 2.0 releases in\n> common use) to work with that same docbook/xmlto chain than\n> it is to do it directly, that is.\n\nOne of my secret (maybe not so secret?) implications there was that it\nmight be worth dropping asciidoc support sooner rather than later. I.e.,\nif it is a burden to make it work with both old and new systems, then\nlet's make the jump to having it work with the new system.\n\nIMHO we can be a bit more cavalier with saying \"you must have a\nrecent-ish asciidoctor to build the docs\", because it's so easy for us\nto provide a binary distribution of the built HTML and manpages (in\nfact, we already do so for the install-man-quick target).\n\nSo it doesn't really leave any platforms out in the cold; it just means\nthey have to tweak their build procedure.\n\n-Peff\n"},{"id":"381749","messageId":"cover.1567534373.git.martin.agren@gmail.com","threadId":"50765","inReplyTo":"CAN0heSr2zCQMM6wOM0UnD28qj_VygQ5CQHGHhMR9+H23snpt5Q@mail.gmail.com","subject":"[PATCH v2 0/2] asciidoctor-extensions: provide ``","fromName":"Martin Ågren","fromEmail":"martin.agren@gmail.com","sentAt":"2019-09-03T18:51:19Z","receivedAt":"2019-09-03T18:51:50Z","isPatch":true,"sender":{"key":"martin.agren@gmail.com","avatar":null},"body":"Almost half a year ago, I wrote:\n> To be clear. *This* patch has a sufficiently incorrect commit message\n> that it really needs a makeover. You can expect a v2.\n\nFinally, here's that v2. I should probably refresh memories: The goal of\nthe main patch here is to make the headers and footers of our manpages\nbuilt by Asciidoctor look a lot more like those generated by AsciiDoc.\nIn particular, this gets rid of the ugly \"[FIXME: source]\".\n\nI spent a little bit of time trying to work on the XML as XML, and\nquite a lot of time procrastinating on that. In the end, I decided that\nthe outcome of my attempts wouldn't get better and that there is some\nvalue to the stupid approach from v1 of doing a simple search-and-replace\nin the text. I've preserved my attempts in the commit message.\n\nWhen I posted v1, it turned into quite a thread [1] on AsciiDoc vs\nAsciidoctor vs Asciidoctor 2.0 and differences in rendering. (I am on\nAsciidoctor 1.5.5.)\n\nAmong other things, the v1-thread discussed switching the rendering\ntoolchain entirely to avoid the detour over xmlto. Doing that would\nrender this patch obsolete. While I agree that such a switch is the\ncorrect long-term goal and that we can be fairly aggressive about it, I\ndo also think it makes sense to first make the \"softer\" switch to\nAsciidoctor-by-default and get that particular hurdle behind us. Then,\nonce we're ok with dropping AsciiDoc entirely, we can do the switch to\nan Asciidoctor-only toolchain.\n\n(I'm preparing a second, independent series of patches that should halve\nthe difference (sans headers/footers) between these two engines -- at\nleast the versions of them that I'm using. The remaining differences are\nthen mainly, but not exclusively, in favor of Asciidoctor. That series\nshould appear on the list within the next couple of days. After that,\nthere's user-manual.html/pdf that needs looking into...)\n\n[1] https://public-inbox.org/git/20190317144747.2418514-1-martin.agren@gmail.com/\n\nMartin Ågren (2):\n asciidoctor-extensions: provide ``\n doc-diff: replace --cut-header-footer with --cut-footer\n\n Documentation/asciidoctor-extensions.rb | 15 +++++++++++++++\n Documentation/doc-diff | 17 ++++++++---------\n 2 files changed, 23 insertions(+), 9 deletions(-)\n\n-- \n2.23.0\n\n"},{"id":"381750","messageId":"7545b16bd845580548dee380a54891ed1f75290f.1567534373.git.martin.agren@gmail.com","threadId":"50765","inReplyTo":"cover.1567534373.git.martin.agren@gmail.com","subject":"[PATCH v2 1/2] asciidoctor-extensions: provide ``","fromName":"Martin Ågren","fromEmail":"martin.agren@gmail.com","sentAt":"2019-09-03T18:51:20Z","receivedAt":"2019-09-03T18:51:59Z","isPatch":true,"sender":{"key":"martin.agren@gmail.com","avatar":null},"body":"When we build with AsciiDoc, asciidoc.conf ensures that each xml-file we\ngenerate contains some meta-information which `xmlto` can act on, based\non the following template:\n\n \n {mantitle}\n {manvolnum}\n Git\n {git_version}\n Git Manual\n \n\nWhen we build with Asciidoctor, it does not honor this configuration file\nand we end up with only this (for a hypothetical git-foo.xml):\n\n \n git-foo\n 1\n \n\nThat is, we miss out on the `` tags. As a result, the\nheader of each man page doesn't say \"Git Manual\", but \"git-foo(1)\"\ninstead. Worse, the footers don't give the Git version number and\ninstead provide the fairly ugly \"[FIXME: source]\".\n\nThat Asciidoctor ignores asciidoc.conf is nothing new. This is why we\nimplement the `linkgit:` macro in asciidoc.conf *and* in\nasciidoctor-extensions.rb. Follow suit and provide these tags in\nasciidoctor-extensions.rb, using a \"postprocessor\" extension where we\njust search and replace in the XML, treated as text.\n\nWe may consider a few alternatives:\n\n * Provide the `mansource` attribute to Asciidoctor. This attribute\n is only respected (i.e., turned into those tags) with\n 1) the \"manpage\" doctype and/or converter [1], which we currently do\n not use, or 2) using Asciidoctor 1.5.7 or newer [2].\n\n * We could inject these lines into the xml-files from the *Makefile*,\n e.g., using `sed`. That would reduce repetition, but it feels wrong\n to impose another step and another risk on the AsciiDoc-processing\n only to benefit the Asciidoctor-one.\n\n * I tried providing a \"docinfo processor\" to inject these tags, but\n could not figure out how to \"merge\" the two sections that\n resulted. To avoid xmlto barfing on the result, I needed need to use\n `xmlto --skip-validation ...`, which seems unfortunate.\n\nLet's instead inject the missing tags using a postprocessor. We'll make\nit fairly obvious that we aim to inject the exact same three lines of\n`` that asciidoc.conf provides. We inject them in\n*post*-processing so we need to do the variable expansion ourselves. We\ndo introduce the bug that asciidoc.conf already has in that we won't do\nany escaping, e.g., of funky versions like \"some v <2.25, >2.20\".\n\nThe postprocessor we add here works on the XML as raw text and doesn't\nreally use the full potential of XML to do a more structured injection.\nThis is actually precisely what the Asciidoctor User Manual does in its\npostprocessor example [3]. I looked into two other approaches:\n\n 1. The nokogiri library is apparently the \"modern\" way of doing XML\n in ruby. I got it working fairly easily:\n require 'nokogiri'\n doc = Nokogiri::XML(output)\n doc.search(\"refmeta\").each { |n| n.add_child(new_tags) }\n output = doc.to_xml\n However, this adds another dependency (e.g., the \"ruby-nokogiri\"\n package on Ubuntu). Using Asciidoctor is not our default, but it\n will probably need to become so soon (AsciiDoc relies on Python 2,\n which is going away). Let's avoid adding a dependency just so that\n we can say \"search...add_child\" rather than \"sub(regex...)\".\n\n 2. The older REXML is apparently always(?) bundled with ruby, but I\n couldn't even parse the original document:\n require 'rexml/document'\n doc = REXML::Document.new(output)\n ...\n The error was \"no implicit conversion of nil into String\" and I\n stopped there.\n\nHaving never dabbled with ruby outside of this very file, I might be\nmissing something obvious, but I don't think it's unlikely that doing a\nplain old search-and-replace will work just as fine or better compared\nto parsing XML and worrying about libraries and library versions.\n\n[1] https://asciidoctor.org/docs/user-manual/#builtin-attributes\n[2] https://public-inbox.org/git/20190319074321.GA2189@sigill.intra.peff.net/\n[3] https://asciidoctor.org/docs/user-manual/#postprocessor-example\n\nSigned-off-by: Martin Ågren \n---\n Documentation/asciidoctor-extensions.rb | 15 +++++++++++++++\n 1 file changed, 15 insertions(+)\n\ndiff --git a/Documentation/asciidoctor-extensions.rb b/Documentation/asciidoctor-extensions.rb\nindex 0089e0cfb8..4ae130d2c6 100644\n--- a/Documentation/asciidoctor-extensions.rb\n+++ b/Documentation/asciidoctor-extensions.rb\n@@ -20,9 +20,24 @@ module Git\n end\n end\n end\n+\n+ class DocumentPostProcessor < Asciidoctor::Extensions::Postprocessor\n+ def process document, output\n+ if document.basebackend? 'docbook'\n+ git_version = document.attributes['git_version']\n+ new_tags = \"\" \\\n+ \"Git\\n\" \\\n+ \"#{git_version}\\n\" \\\n+ \"Git Manual\\n\"\n+ output = output.sub(/<\\/refmeta>/, new_tags + \"\")\n+ end\n+ output\n+ end\n+ end\n end\n end\n \n Asciidoctor::Extensions.register do\n inline_macro Git::Documentation::LinkGitProcessor, :linkgit\n+ postprocessor Git::Documentation::DocumentPostProcessor\n end\n-- \n2.23.0\n\n"},{"id":"381751","messageId":"40f3c4c3a9ce5e9152477b66016831be0b040310.1567534373.git.martin.agren@gmail.com","threadId":"50765","inReplyTo":"cover.1567534373.git.martin.agren@gmail.com","subject":"[PATCH v2 2/2] doc-diff: replace --cut-header-footer with --cut-footer","fromName":"Martin Ågren","fromEmail":"martin.agren@gmail.com","sentAt":"2019-09-03T18:51:21Z","receivedAt":"2019-09-03T18:52:03Z","isPatch":true,"sender":{"key":"martin.agren@gmail.com","avatar":null},"body":"After the previous commit, AsciiDoc and Asciidoctor render the manpage\nheaders identically, so we no longer need the \"cut the header\" part of\nour `--cut-header-footer` option. We do still need the \"cut the footer\"\npart, though. The previous commit improved the rendering of the footer\nin Asciidoctor by quite a bit, but the two programs still disagree on\nhow to format the date in the footer: 01/01/1970 vs 1970-01-01.\n\nWe could keep using `--cut-header-footer`, but it would be nice if we\nhad a slightly smaller hammer `--cut-footer` that would be less likely\nto hide regressions. Rather than simply adding such an option, let's\nalso drop `--cut-header-footer`, i.e., rework it to lose the \"header\"\npart of its name and functionality.\n\n`--cut-header-footer` is just a developer tool and it probably has no\nmore than a handful of users, so we can afford be aggressive.\n\nSigned-off-by: Martin Ågren \n---\n Documentation/doc-diff | 17 ++++++++---------\n 1 file changed, 8 insertions(+), 9 deletions(-)\n\ndiff --git a/Documentation/doc-diff b/Documentation/doc-diff\nindex 3355be4798..88a9b20168 100755\n--- a/Documentation/doc-diff\n+++ b/Documentation/doc-diff\n@@ -21,7 +21,7 @@ asciidoc\t\tuse asciidoc with both commits\n to-asciidoc\t\tuse asciidoc with the 'to'-commit\n to-asciidoctor\t\tuse asciidoctor with the 'to'-commit\n asciidoctor\t\tuse asciidoctor with both commits\n-cut-header-footer\tcut away header and footer\n+cut-footer\t\tcut away footer\n \"\n SUBDIRECTORY_OK=1\n . \"$(git --exec-path)/git-sh-setup\"\n@@ -31,7 +31,7 @@ force=\n clean=\n from_program=\n to_program=\n-cut_header_footer=\n+cut_footer=\n while test $# -gt 0\n do\n \tcase \"$1\" in\n@@ -55,8 +55,8 @@ do\n \t--asciidoc)\n \t\tfrom_program=-asciidoc\n \t\tto_program=-asciidoc ;;\n-\t--cut-header-footer)\n-\t\tcut_header_footer=-cut-header-footer ;;\n+\t--cut-footer)\n+\t\tcut_footer=-cut-footer ;;\n \t--)\n \t\tshift; break ;;\n \t*)\n@@ -118,8 +118,8 @@ construct_makemanflags () {\n from_makemanflags=$(construct_makemanflags \"$from_program\") &&\n to_makemanflags=$(construct_makemanflags \"$to_program\") &&\n \n-from_dir=$from_oid$from_program$cut_header_footer &&\n-to_dir=$to_oid$to_program$cut_header_footer &&\n+from_dir=$from_oid$from_program$cut_footer &&\n+to_dir=$to_oid$to_program$cut_footer &&\n \n # generate_render_makefile \n generate_render_makefile () {\n@@ -169,12 +169,11 @@ render_tree () {\n \t\tmake -j$parallel -f - &&\n \t\tmv \"$tmp/rendered/$dname+\" \"$tmp/rendered/$dname\"\n \n-\t\tif test \"$cut_header_footer\" = \"-cut-header-footer\"\n+\t\tif test \"$cut_footer\" = \"-cut-footer\"\n \t\tthen\n \t\t\tfor f in $(find \"$tmp/rendered/$dname\" -type f)\n \t\t\tdo\n-\t\t\t\ttail -n +3 \"$f\" | head -n -2 |\n-\t\t\t\tsed -e '1{/^$/d}' -e '${/^$/d}' >\"$f+\" &&\n+\t\t\t\thead -n -2 \"$f\" | sed -e '${/^$/d}' >\"$f+\" &&\n \t\t\t\tmv \"$f+\" \"$f\" ||\n \t\t\t\treturn 1\n \t\t\tdone\n-- \n2.23.0\n\n"},{"id":"381778","messageId":"20190903231610.GI11334@genre.crustytoothpaste.net","threadId":"50765","inReplyTo":"cover.1567534373.git.martin.agren@gmail.com","subject":"Re: [PATCH v2 0/2] asciidoctor-extensions: provide ``","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2019-09-03T23:16:11Z","receivedAt":"2019-09-03T23:16:19Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2019-09-03 at 18:51:19, Martin Ågren wrote:\n> Almost half a year ago, I wrote:\n> > To be clear. *This* patch has a sufficiently incorrect commit message\n> > that it really needs a makeover. You can expect a v2.\n> \n> Finally, here's that v2. I should probably refresh memories: The goal of\n> the main patch here is to make the headers and footers of our manpages\n> built by Asciidoctor look a lot more like those generated by AsciiDoc.\n> In particular, this gets rid of the ugly \"[FIXME: source]\".\n> \n> I spent a little bit of time trying to work on the XML as XML, and\n> quite a lot of time procrastinating on that. In the end, I decided that\n> the outcome of my attempts wouldn't get better and that there is some\n> value to the stupid approach from v1 of doing a simple search-and-replace\n> in the text. I've preserved my attempts in the commit message.\n> \n> When I posted v1, it turned into quite a thread [1] on AsciiDoc vs\n> Asciidoctor vs Asciidoctor 2.0 and differences in rendering. (I am on\n> Asciidoctor 1.5.5.)\n> \n> Among other things, the v1-thread discussed switching the rendering\n> toolchain entirely to avoid the detour over xmlto. Doing that would\n> render this patch obsolete. While I agree that such a switch is the\n> correct long-term goal and that we can be fairly aggressive about it, I\n> do also think it makes sense to first make the \"softer\" switch to\n> Asciidoctor-by-default and get that particular hurdle behind us. Then,\n> once we're ok with dropping AsciiDoc entirely, we can do the switch to\n> an Asciidoctor-only toolchain.\n\nYeah, this seems like a good approach. xmlto is a neat tool, but it's\nessentially unmaintained and is designed for DocBook 4. Avoiding it\nwhere possible seems like the right choice.\n\nI looked at this series and it seems sane. I agree that adding a\ndependency on nokogiri isn't really desirable. It is an extremely\ncommon Ruby package, but it has native extensions, which causes problems\nfor some people if their distro doesn't support it.\n-- \nbrian m. carlson: Houston, Texas, US\nOpenPGP: https://keybase.io/bk2204\n"},{"id":"381789","messageId":"20190904032609.GD28836@sigill.intra.peff.net","threadId":"50765","inReplyTo":"cover.1567534373.git.martin.agren@gmail.com","subject":"Re: [PATCH v2 0/2] asciidoctor-extensions: provide ``","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2019-09-04T03:26:10Z","receivedAt":"2019-09-04T03:26:12Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Sep 03, 2019 at 08:51:19PM +0200, Martin Ågren wrote:\n\n> Almost half a year ago, I wrote:\n> > To be clear. *This* patch has a sufficiently incorrect commit message\n> > that it really needs a makeover. You can expect a v2.\n> \n> Finally, here's that v2. I should probably refresh memories: The goal of\n> the main patch here is to make the headers and footers of our manpages\n> built by Asciidoctor look a lot more like those generated by AsciiDoc.\n> In particular, this gets rid of the ugly \"[FIXME: source]\".\n\nThanks for resuming this work. The patches both look good to me.\n\n> I spent a little bit of time trying to work on the XML as XML, and\n> quite a lot of time procrastinating on that. In the end, I decided that\n> the outcome of my attempts wouldn't get better and that there is some\n> value to the stupid approach from v1 of doing a simple search-and-replace\n> in the text. I've preserved my attempts in the commit message.\n\nI think your hack is better than introducing yet another dependency.\nAnd because it's isolated and you've documented it well, it will be easy\nto undo later if we choose.\n\nThe doc-diff tool should tell us if we catch any false positives inside\nthe text (though it seems quite unlikely to me).\n\n> When I posted v1, it turned into quite a thread [1] on AsciiDoc vs\n> Asciidoctor vs Asciidoctor 2.0 and differences in rendering. (I am on\n> Asciidoctor 1.5.5.)\n\nYes, sadly I still can't format the docs at all with 2.0.10 (which is\nwhat ships in Debian unstable).\n\n> Among other things, the v1-thread discussed switching the rendering\n> toolchain entirely to avoid the detour over xmlto. Doing that would\n> render this patch obsolete. While I agree that such a switch is the\n> correct long-term goal and that we can be fairly aggressive about it, I\n> do also think it makes sense to first make the \"softer\" switch to\n> Asciidoctor-by-default and get that particular hurdle behind us. Then,\n> once we're ok with dropping AsciiDoc entirely, we can do the switch to\n> an Asciidoctor-only toolchain.\n\nYeah, I do still like that as an endgame, but I like what you have here\nas an intermediate step in the right direction.\n\n> (I'm preparing a second, independent series of patches that should halve\n> the difference (sans headers/footers) between these two engines -- at\n> least the versions of them that I'm using. The remaining differences are\n> then mainly, but not exclusively, in favor of Asciidoctor. That series\n> should appear on the list within the next couple of days. After that,\n> there's user-manual.html/pdf that needs looking into...)\n\nYay. :)\n\n-Peff\n"},{"id":"381902","messageId":"CAN0heSrQAHtjJa88utUNEqBsX-gwjp7nWA_QskDDQ8FawD8mag@mail.gmail.com","threadId":"50765","inReplyTo":"20190903231610.GI11334@genre.crustytoothpaste.net","subject":"Re: [PATCH v2 0/2] asciidoctor-extensions: provide ``","fromName":"Martin Ågren","fromEmail":"martin.agren@gmail.com","sentAt":"2019-09-05T19:28:16Z","receivedAt":"2019-09-05T19:28:29Z","isPatch":true,"sender":{"key":"martin.agren@gmail.com","avatar":null},"body":"On Wed, 4 Sep 2019 at 01:16, brian m. carlson\n wrote:\n>\n> I looked at this series and it seems sane. I agree that adding a\n> dependency on nokogiri isn't really desirable. It is an extremely\n> common Ruby package, but it has native extensions, which causes problems\n> for some people if their distro doesn't support it.\n\nOk, nice to know I wasn't too far off with avoiding the added\ndependency. I saw statements like \"oh, it's available everywhere\", but\nalso more nuanced ones like \"not necessarily...\". I'm glad I have a\ntrustworthy source now. ;-)\n\nMartin\n"},{"id":"381904","messageId":"CAN0heSpbRvNG9okz5pqkHqDMB2BM5T+FzAbaK3sVwzGC6fjpPA@mail.gmail.com","threadId":"50765","inReplyTo":"20190904032609.GD28836@sigill.intra.peff.net","subject":"Re: [PATCH v2 0/2] asciidoctor-extensions: provide ``","fromName":"Martin Ågren","fromEmail":"martin.agren@gmail.com","sentAt":"2019-09-05T19:35:10Z","receivedAt":"2019-09-05T19:35:23Z","isPatch":true,"sender":{"key":"martin.agren@gmail.com","avatar":null},"body":"On Wed, 4 Sep 2019 at 05:26, Jeff King wrote:\n>\n> On Tue, Sep 03, 2019 at 08:51:19PM +0200, Martin Ågren wrote:\n>\n> > When I posted v1, it turned into quite a thread [1] on AsciiDoc vs\n> > Asciidoctor vs Asciidoctor 2.0 and differences in rendering. (I am on\n> > Asciidoctor 1.5.5.)\n>\n> Yes, sadly I still can't format the docs at all with 2.0.10 (which is\n> what ships in Debian unstable).\n>\n> > do also think it makes sense to first make the \"softer\" switch to\n> > Asciidoctor-by-default and get that particular hurdle behind us. Then,\n> > once we're ok with dropping AsciiDoc entirely, we can do the switch to\n> > an Asciidoctor-only toolchain.\n>\n> Yeah, I do still like that as an endgame, but I like what you have here\n> as an intermediate step in the right direction.\n\nHmm, so this sounds like once I am happy with replacing AsciiDoc with\nAsciidoctor 1(.5.5), I should rather not propose a series \"let's default\nto Asciidoctor!!!\" but instead a slightly more careful \"go with\nAsciidoctor, but document that we work badly with v2 and that the 2nd\nchoice after Asciidoctor 1 should be AsciiDoc\". Or do you see it\ndifferently? (I wonder which Asciidoctor-version Junio would be on..)\n\nMartin\n"},{"id":"381989","messageId":"20190906232947.GJ11334@genre.crustytoothpaste.net","threadId":"50765","inReplyTo":"20190904032609.GD28836@sigill.intra.peff.net","subject":"Re: [PATCH v2 0/2] asciidoctor-extensions: provide ``","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2019-09-06T23:29:47Z","receivedAt":"2019-09-06T23:29:56Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2019-09-04 at 03:26:10, Jeff King wrote:\n> On Tue, Sep 03, 2019 at 08:51:19PM +0200, Martin Ågren wrote:\n> > When I posted v1, it turned into quite a thread [1] on AsciiDoc vs\n> > Asciidoctor vs Asciidoctor 2.0 and differences in rendering. (I am on\n> > Asciidoctor 1.5.5.)\n> \n> Yes, sadly I still can't format the docs at all with 2.0.10 (which is\n> what ships in Debian unstable).\n\nI'll look into this. I requested the upgrade in sid.\n\nIf it's the lack of DocBook 4.5 support, then I'll probably need to\npatch xmlto for that. DocBook 5 has been around for a decade now, so\nit's time for xmlto to support it properly.\n-- \nbrian m. carlson: Houston, Texas, US\nOpenPGP: https://keybase.io/bk2204\n"},{"id":"381992","messageId":"20190907044021.GA24463@sigill.intra.peff.net","threadId":"50765","inReplyTo":"20190906232947.GJ11334@genre.crustytoothpaste.net","subject":"Re: [PATCH v2 0/2] asciidoctor-extensions: provide ``","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2019-09-07T04:40:22Z","receivedAt":"2019-09-07T04:40:24Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Fri, Sep 06, 2019 at 11:29:47PM +0000, brian m. carlson wrote:\n\n> On 2019-09-04 at 03:26:10, Jeff King wrote:\n> > On Tue, Sep 03, 2019 at 08:51:19PM +0200, Martin Ågren wrote:\n> > > When I posted v1, it turned into quite a thread [1] on AsciiDoc vs\n> > > Asciidoctor vs Asciidoctor 2.0 and differences in rendering. (I am on\n> > > Asciidoctor 1.5.5.)\n> > \n> > Yes, sadly I still can't format the docs at all with 2.0.10 (which is\n> > what ships in Debian unstable).\n> \n> I'll look into this. I requested the upgrade in sid.\n> \n> If it's the lack of DocBook 4.5 support, then I'll probably need to\n> patch xmlto for that. DocBook 5 has been around for a decade now, so\n> it's time for xmlto to support it properly.\n\nYes, it's the docbook45 thing. Switching to docbook5 lets asciidoc run,\nbut then xmlto chokes.\n\n-Peff\n"},{"id":"382001","messageId":"20190907064508.GC28860@sigill.intra.peff.net","threadId":"50765","inReplyTo":"CAN0heSpbRvNG9okz5pqkHqDMB2BM5T+FzAbaK3sVwzGC6fjpPA@mail.gmail.com","subject":"Re: [PATCH v2 0/2] asciidoctor-extensions: provide ``","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2019-09-07T06:45:08Z","receivedAt":"2019-09-07T06:45:10Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Sep 05, 2019 at 09:35:10PM +0200, Martin Ågren wrote:\n\n> > > do also think it makes sense to first make the \"softer\" switch to\n> > > Asciidoctor-by-default and get that particular hurdle behind us. Then,\n> > > once we're ok with dropping AsciiDoc entirely, we can do the switch to\n> > > an Asciidoctor-only toolchain.\n> >\n> > Yeah, I do still like that as an endgame, but I like what you have here\n> > as an intermediate step in the right direction.\n> \n> Hmm, so this sounds like once I am happy with replacing AsciiDoc with\n> Asciidoctor 1(.5.5), I should rather not propose a series \"let's default\n> to Asciidoctor!!!\" but instead a slightly more careful \"go with\n> Asciidoctor, but document that we work badly with v2 and that the 2nd\n> choice after Asciidoctor 1 should be AsciiDoc\". Or do you see it\n> differently? (I wonder which Asciidoctor-version Junio would be on..)\n\nYeah, that seems reasonable.\n\nTBH, if making things in the middle step work turns out to be too hard,\nI'm not entirely opposed to a hard switch.\n\nThe \"does not work with 2.0\" thing has to be a temporary step, though, I\nthink, since using the older versions will get harder and harder as time\ngoes on. I think it's OK to take such a temporary step as long as we\nunderstand where it leads (and presumably its to directly generating the\nroff with asciidoctor). The middle step of having asciidoctor+xmlto\nhelps us understand and isolate which changes are responsible for which\nparts of the output.\n\n-Peff\n"},{"id":"382012","messageId":"CAN0heSoOcpiPiRcYwLxxByC49Q-yXipFzBGm=XKW-AZk4n=8Jg@mail.gmail.com","threadId":"50765","inReplyTo":"20190907064508.GC28860@sigill.intra.peff.net","subject":"Re: [PATCH v2 0/2] asciidoctor-extensions: provide ``","fromName":"Martin Ågren","fromEmail":"martin.agren@gmail.com","sentAt":"2019-09-07T14:06:48Z","receivedAt":"2019-09-07T14:07:05Z","isPatch":true,"sender":{"key":"martin.agren@gmail.com","avatar":null},"body":"On Sat, 7 Sep 2019 at 08:45, Jeff King wrote:\n>\n> On Thu, Sep 05, 2019 at 09:35:10PM +0200, Martin Ågren wrote:\n>\n> > > Yeah, I do still like that as an endgame, but I like what you have here\n> > > as an intermediate step in the right direction.\n> >\n> > Hmm, so this sounds like once I am happy with replacing AsciiDoc with\n> > Asciidoctor 1(.5.5), I should rather not propose a series \"let's default\n> > to Asciidoctor!!!\" but instead a slightly more careful \"go with\n> > Asciidoctor, but document that we work badly with v2 and that the 2nd\n> > choice after Asciidoctor 1 should be AsciiDoc\". Or do you see it\n> > differently? (I wonder which Asciidoctor-version Junio would be on..)\n>\n> Yeah, that seems reasonable.\n>\n> TBH, if making things in the middle step work turns out to be too hard,\n> I'm not entirely opposed to a hard switch.\n>\n> The \"does not work with 2.0\" thing has to be a temporary step, though, I\n> think, since using the older versions will get harder and harder as time\n> goes on. I think it's OK to take such a temporary step as long as we\n> understand where it leads (and presumably its to directly generating the\n> roff with asciidoctor). The middle step of having asciidoctor+xmlto\n> helps us understand and isolate which changes are responsible for which\n> parts of the output.\n\nSo of these steps:\n\n 0. Get Asciidoctor (v1) in shape.\n\n 1. Switch the default to Asciidoctor (v1).\n\n 2. Drop AsciiDoc to have faster Asciidoctor-processing, avoid xmlto\n and support Asciidoctor 2. And to avoid the Python 2 EOL, too.\n\nStep 0 is not far away, so step 1 could be done fairly soon IMHO. Step 2\nwould \"hopefully\" happen soon after -- maybe even in the same release\ncycle as step 1, and if not the same then the one just after. But I\nmight be the wrong person to trust on that one. I currently don't even\ntry to build with Asciidoctor 2. I might perhaps look into installing\nit, but it could also be that I'll only start using it when it happens\nto arrive through my distro.\n\nSo as long as I'm not looking into Asciidoctor 2, maybe I shouldn't be\nthe one to impose \"default to asciidoctor\" on the world. Dunno. In any\ncase, I should be able to bring the asciidoc/tor1 differences to a state\nwhere we trust asciidoctor 1 to be in a good shape, so that \"someone\nelse\" could pick up the ball and work on asciidoctor 2 vs 1, knowing\nthat it's ok if they regress AsciiDoc support or even drop it entirely\nin the process.\n\nMartin\n"},{"id":"382021","messageId":"20190907165302.GK11334@genre.crustytoothpaste.net","threadId":"50765","inReplyTo":"20190907044021.GA24463@sigill.intra.peff.net","subject":"Re: [PATCH v2 0/2] asciidoctor-extensions: provide ``","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2019-09-07T16:53:02Z","receivedAt":"2019-09-07T16:53:14Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2019-09-07 at 04:40:22, Jeff King wrote:\n> On Fri, Sep 06, 2019 at 11:29:47PM +0000, brian m. carlson wrote:\n> > I'll look into this. I requested the upgrade in sid.\n> > \n> > If it's the lack of DocBook 4.5 support, then I'll probably need to\n> > patch xmlto for that. DocBook 5 has been around for a decade now, so\n> > it's time for xmlto to support it properly.\n> \n> Yes, it's the docbook45 thing. Switching to docbook5 lets asciidoc run,\n> but then xmlto chokes.\n\nI already have a patch for Git to fix this. I'll send it out this weekend.\n-- \nbrian m. carlson: Houston, Texas, US\nOpenPGP: https://keybase.io/bk2204\n"},{"id":"382022","messageId":"20190907170746.273984-1-sandals@crustytoothpaste.net","threadId":"50765","inReplyTo":"20190906232947.GJ11334@genre.crustytoothpaste.net","subject":"[PATCH] Documentation: fix build with Asciidoctor 2","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2019-09-07T17:07:46Z","receivedAt":"2019-09-07T17:08:05Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"Our documentation toolchain has traditionally been built around DocBook\n4.5. This version of DocBook is the last DTD-based version of DocBook.\nIn 2009, DocBook 5 was introduced using namespaces and its syntax is\nexpressed in RELAX-NG, which is more expressive and allows a wider\nvariety of syntax forms.\n\nAsciidoctor, one of the alternatives for building our documentation,\ndropped support for DocBook 4.5 in its recent 2.0 release and now only\nsupports DocBook 5. This would not be a problem but for the fact that\nwe use xmlto, which is still stuck in the DocBook 4.5 era.\n\nxmlto performs DTD validation as part of the build process. This is not\nproblematic for DocBook 4.5, which has a valid DTD, but it clearly\ncannot work for DocBook 5, since no DTD can adequately express its full\nsyntax. In addition, even if xmlto did support RELAX-NG validation,\nthat wouldn't be sufficient because it uses the libxml2-based xmllint to\ndo so, which has known problems with validating interleaves in RELAX-NG.\n\nFortunately, there's an easy way forward: ask Asciidoctor to use its\nDocBook 5 backend and tell xmlto to skip validation. Asciidoctor has\nsupported DocBook 5 since v0.1.4 in 2013 and xmlto has supported\nskipping validation for probably longer than that.\n\nxmlto will still use the non-namespaced DocBook XSL stylesheets (which\nare designed for DocBook 4.5) for building the documentation instead of\nthe namespaced ones (which are designed for DocBook 5). This isn't\nreally a problem, since both forms are built from the same source and\ncan handle both versions, but it does mean that an ugly message about\nthe stylesheets stripping the namespace will be printed to standard\nerror.\n\nOverall, however, this message is a relatively minor price to pay and it\nmeans that we can continue to use the aging xmlto to drive our build\nprocess, while still supporting newer tooling like Asciidoctor 2.\n\nThe differences in output between AsciiDoc 8.6.10 on master and\nAsciidoctor 2.0.10 with this patch are, with one exception, all due to\nwhitespace, wrapping, or quoting and do not affect substantive content.\n\nSigned-off-by: brian m. carlson \n---\n Documentation/Makefile | 3 ++-\n 1 file changed, 2 insertions(+), 1 deletion(-)\n\ndiff --git a/Documentation/Makefile b/Documentation/Makefile\nindex 76f2ecfc1b..485f365cbf 100644\n--- a/Documentation/Makefile\n+++ b/Documentation/Makefile\n@@ -197,11 +197,12 @@ ifdef USE_ASCIIDOCTOR\n ASCIIDOC = asciidoctor\n ASCIIDOC_CONF =\n ASCIIDOC_HTML = xhtml5\n-ASCIIDOC_DOCBOOK = docbook45\n+ASCIIDOC_DOCBOOK = docbook5\n ASCIIDOC_EXTRA += -acompat-mode -atabsize=8\n ASCIIDOC_EXTRA += -I. -rasciidoctor-extensions\n ASCIIDOC_EXTRA += -alitdd='&\\#x2d;&\\#x2d;'\n DBLATEX_COMMON =\n+XMLTO_EXTRA += --skip-validation\n endif\n \n SHELL_PATH ?= $(SHELL)\n"},{"id":"382043","messageId":"20190908104833.GE15641@sigill.intra.peff.net","threadId":"50765","inReplyTo":"20190907170746.273984-1-sandals@crustytoothpaste.net","subject":"Re: [PATCH] Documentation: fix build with Asciidoctor 2","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2019-09-08T10:48:33Z","receivedAt":"2019-09-08T10:48:36Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Sat, Sep 07, 2019 at 05:07:46PM +0000, brian m. carlson wrote:\n\n> Our documentation toolchain has traditionally been built around DocBook\n> 4.5. This version of DocBook is the last DTD-based version of DocBook.\n> In 2009, DocBook 5 was introduced using namespaces and its syntax is\n> expressed in RELAX-NG, which is more expressive and allows a wider\n> variety of syntax forms.\n\nWow, this patch turned out to be much simpler than I had feared. And the\nexplanation turned out to be much longer and more interesting than I\nexpected. :) Thanks for writing it all down.\n\nThe patch worked well for me.\n\n> Fortunately, there's an easy way forward: ask Asciidoctor to use its\n> DocBook 5 backend and tell xmlto to skip validation. Asciidoctor has\n> supported DocBook 5 since v0.1.4 in 2013 and xmlto has supported\n> skipping validation for probably longer than that.\n> \n> xmlto will still use the non-namespaced DocBook XSL stylesheets (which\n> are designed for DocBook 4.5) for building the documentation instead of\n> the namespaced ones (which are designed for DocBook 5). This isn't\n> really a problem, since both forms are built from the same source and\n> can handle both versions, but it does mean that an ugly message about\n> the stylesheets stripping the namespace will be printed to standard\n> error.\n\nSo I think this does introduce the ugly namespace message even for\npeople who are using older versions of asciidoctor that understand\ndocbook45. But it's probably sensible to keep all of our asciidoctor\nbuilds using the same technique. We have enough trouble tracking the\nminor differences between the variants as it is.\n\n> The differences in output between AsciiDoc 8.6.10 on master and\n> Asciidoctor 2.0.10 with this patch are, with one exception, all due to\n> whitespace, wrapping, or quoting and do not affect substantive content.\n\nWe know already there are a lot of differences between asciidoc and\nasciidoctor here. What's more interesting to me is how it changes the\noutput between two versions of asciidoctor. Of course we can't check for\n2.0.10, because it doesn't build at all. But I did do a \"doc-diff\"\nwith and without this patch using 1.5.5. There are quite a few\nwhitespace changes. Some of them seem good or at least neutral, like:\n\n --- a/745f6812895b31c02b29bdfe4ae8e5498f776c26-asciidoctor/home/peff/share/man/man1/git-add.1\n +++ b/303729d86b69657777222bf4b3a6f95932e12648-asciidoctor/home/peff/share/man/man1/git-add.1\n [...]\n @@ -43,6 +43,7 @@ DESCRIPTION\n\n OPTIONS\n ...\n +\n Files to add content from. Fileglobs (e.g. *.c) can be given to\n add all matching files. Also a leading directory name (e.g. dir to\n add dir/file1 and dir/file2) can be given to update the index to\n\nSome of them seem bad, though:\n\n --- a/745f6812895b31c02b29bdfe4ae8e5498f776c26-asciidoctor/home/peff/share/man/man1/git-am.1\n +++ b/303729d86b69657777222bf4b3a6f95932e12648-asciidoctor/home/peff/share/man/man1/git-am.1\n [...]\n @@ -175,10 +201,10 @@ DISCUSSION\n\n to process. Upon seeing the first patch that does not apply, it aborts\n in the middle. You can recover from this in one of two ways:\n \n - 1. skip the current patch by re-running the command with the --skip\n + 1. skip the current patch by re-running the command with the --skip\n option.\n \n - 2. hand resolve the conflict in the working directory, and update the\n + 2. hand resolve the conflict in the working directory, and update the\n index file to bring it into a state that the patch should have\n produced. Then run the command with the --continue option.\n\nI tricked doc-diff into doing a comparison against 1.5.5 without the\npatch and 2.0.10 with the patch, and the diff is similar.\n\n-Peff\n"},{"id":"382044","messageId":"20190908141308.GA7255@szeder.dev","threadId":"50765","inReplyTo":"20190907170746.273984-1-sandals@crustytoothpaste.net","subject":"Re: [PATCH] Documentation: fix build with Asciidoctor 2","fromName":"SZEDER Gábor","fromEmail":"szeder.dev@gmail.com","sentAt":"2019-09-08T14:13:08Z","receivedAt":"2019-09-08T14:13:19Z","isPatch":true,"sender":{"key":"szeder.dev@gmail.com","avatar":"https://avatars.githubusercontent.com/u/116324?v=4"},"body":"On Sat, Sep 07, 2019 at 05:07:46PM +0000, brian m. carlson wrote:\n> Our documentation toolchain has traditionally been built around DocBook\n> 4.5. This version of DocBook is the last DTD-based version of DocBook.\n> In 2009, DocBook 5 was introduced using namespaces and its syntax is\n> expressed in RELAX-NG, which is more expressive and allows a wider\n> variety of syntax forms.\n> \n> Asciidoctor, one of the alternatives for building our documentation,\n> dropped support for DocBook 4.5 in its recent 2.0 release and now only\n> supports DocBook 5. This would not be a problem but for the fact that\n> we use xmlto, which is still stuck in the DocBook 4.5 era.\n> \n> xmlto performs DTD validation as part of the build process. This is not\n> problematic for DocBook 4.5, which has a valid DTD, but it clearly\n> cannot work for DocBook 5, since no DTD can adequately express its full\n> syntax. In addition, even if xmlto did support RELAX-NG validation,\n> that wouldn't be sufficient because it uses the libxml2-based xmllint to\n> do so, which has known problems with validating interleaves in RELAX-NG.\n> \n> Fortunately, there's an easy way forward: ask Asciidoctor to use its\n> DocBook 5 backend and tell xmlto to skip validation. Asciidoctor has\n> supported DocBook 5 since v0.1.4 in 2013 and xmlto has supported\n> skipping validation for probably longer than that.\n> \n> xmlto will still use the non-namespaced DocBook XSL stylesheets (which\n> are designed for DocBook 4.5) for building the documentation instead of\n> the namespaced ones (which are designed for DocBook 5). This isn't\n> really a problem, since both forms are built from the same source and\n> can handle both versions, but it does mean that an ugly message about\n> the stylesheets stripping the namespace will be printed to standard\n> error.\n\nThese messages from 'xmlto' look like these, and there are a lot of\nthem:\n\n Note: namesp. cut : stripped namespace before processing Git User Manual\n Note: namesp. cut : stripped namespace before processing git-sh-setup(1)\n Note: namesp. cut : stripped namespace before processing git-get-tar-commit-id(1)\n\nUnfortunately, these messages to standard error cause our CI builds to\nfail [1].\n\nIn the Documentation build job we check that there was nothing printed\nto standard error during 'make doc', in order to catch warnings that\nare potential signs of a mis-rendered documentation, but do not cause\nany asciidoc/asciidoctor/xmlto commands (and thus 'make doc') to fail.\n\nNow, a few recent messages to standard error that indeed were signs of\nmis-rendered docs [2] looked like this:\n\n asciidoctor: WARNING: api-config.txt: line 232: unterminated listing block\n\ni.e. they came from Asciidoctor and were all-caps warnings.\n\nOTOH, these \"stripped namespace\" messages come from 'xmlto', are not\nwarnings but have that \"Note:\" prefix, and, trusting that you did\ncheck the results thoroughly, are apparently not a sign of any\nrendering issues. So I think it's safe to ignore them and this patch\nshould strip them from 'make doc's output in\n'ci/test-documentation.sh'.\n\nRelated: after this patch we might want to update the CI builds to use\nAsciidoctor 2 instead of sticking with 1.5.8.\n\n\n[1] With Asciidoctor 1.5.8:\n\n https://travis-ci.org/szeder/git/jobs/582294090#L2085\n\n With an additional patch on top to use Asciidoctor 2:\n\n https://travis-ci.org/szeder/git/jobs/582294243#L2066\n\n[2] See 'git log -3 b373e4d29b'\n\n\n"},{"id":"382048","messageId":"20190908171807.GL11334@genre.crustytoothpaste.net","threadId":"50765","inReplyTo":"20190908104833.GE15641@sigill.intra.peff.net","subject":"Re: [PATCH] Documentation: fix build with Asciidoctor 2","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2019-09-08T17:18:07Z","receivedAt":"2019-09-08T17:18:19Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2019-09-08 at 10:48:33, Jeff King wrote:\n> Some of them seem bad, though:\n> \n> --- a/745f6812895b31c02b29bdfe4ae8e5498f776c26-asciidoctor/home/peff/share/man/man1/git-am.1\n> +++ b/303729d86b69657777222bf4b3a6f95932e12648-asciidoctor/home/peff/share/man/man1/git-am.1\n> [...]\n> @@ -175,10 +201,10 @@ DISCUSSION\n> \n> to process. Upon seeing the first patch that does not apply, it aborts\n> in the middle. You can recover from this in one of two ways:\n> \n> - 1. skip the current patch by re-running the command with the --skip\n> + 1. skip the current patch by re-running the command with the --skip\n> option.\n> \n> - 2. hand resolve the conflict in the working directory, and update the\n> + 2. hand resolve the conflict in the working directory, and update the\n> index file to bring it into a state that the patch should have\n> produced. Then run the command with the --continue option.\n> \n> I tricked doc-diff into doing a comparison against 1.5.5 without the\n> patch and 2.0.10 with the patch, and the diff is similar.\n\nI see the same thing in doc-diff, but this issue doesn't appear to\nactually render in the manual pages themselves, even with MANWIDTH=80.\nI'm not sure why, but it doesn't seem to misrender in the actual man\noutput.\n\nI also tried with the DocBook 5 (docbook-xsl-ns) stylesheets, but that\ndoesn't appear to make a difference in doc-diff.\n-- \nbrian m. carlson: Houston, Texas, US\nOpenPGP: https://keybase.io/bk2204\n"},{"id":"382052","messageId":"20190908212122.GA8514@sigill.intra.peff.net","threadId":"50765","inReplyTo":"20190908171807.GL11334@genre.crustytoothpaste.net","subject":"Re: [PATCH] Documentation: fix build with Asciidoctor 2","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2019-09-08T21:21:22Z","receivedAt":"2019-09-08T21:21:25Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Sun, Sep 08, 2019 at 05:18:07PM +0000, brian m. carlson wrote:\n\n> > - 2. hand resolve the conflict in the working directory, and update the\n> > + 2. hand resolve the conflict in the working directory, and update the\n> > index file to bring it into a state that the patch should have\n> > produced. Then run the command with the --continue option.\n> > \n> > I tricked doc-diff into doing a comparison against 1.5.5 without the\n> > patch and 2.0.10 with the patch, and the diff is similar.\n> \n> I see the same thing in doc-diff, but this issue doesn't appear to\n> actually render in the manual pages themselves, even with MANWIDTH=80.\n> I'm not sure why, but it doesn't seem to misrender in the actual man\n> output.\n\nHrm. But doc-diff _is_ diffing the output of man, so that implies a bug\nin that script. However, if I manually go into tmp-doc-diff/installed\nand run \"man -l git-am.1\", I do see the extra whitespace, which is what\nI'd expect. Are you sure you did your manual inspection on the right\nthing?\n\nI was also curious about the root cause at the roff level, so I did this\nhack:\n\ndiff --git a/Documentation/doc-diff b/Documentation/doc-diff\nindex 88a9b20168..fde3ac7154 100755\n--- a/Documentation/doc-diff\n+++ b/Documentation/doc-diff\n@@ -22,6 +22,7 @@ to-asciidoc\t\tuse asciidoc with the 'to'-commit\n to-asciidoctor\t\tuse asciidoctor with the 'to'-commit\n asciidoctor\t\tuse asciidoctor with both commits\n cut-footer\t\tcut away footer\n+raw\t\t\tdiff roff instead of rendered manpages\n \"\n SUBDIRECTORY_OK=1\n . \"$(git --exec-path)/git-sh-setup\"\n@@ -32,6 +33,7 @@ clean=\n from_program=\n to_program=\n cut_footer=\n+raw=\n while test $# -gt 0\n do\n \tcase \"$1\" in\n@@ -57,6 +59,8 @@ do\n \t\tto_program=-asciidoc ;;\n \t--cut-footer)\n \t\tcut_footer=-cut-footer ;;\n+\t--raw)\n+\t\traw=t ;;\n \t--)\n \t\tshift; break ;;\n \t*)\n@@ -159,6 +163,11 @@ render_tree () {\n \t\tmv \"$tmp/installed/$dname+\" \"$tmp/installed/$dname\"\n \tfi &&\n \n+\tif test -n \"$raw\"\n+\tthen\n+\t\treturn 0\n+\tfi &&\n+\n \t# As with \"installed\" above, we skip the render if it's already been\n \t# done. So using make here is primarily just about running in\n \t# parallel.\n@@ -181,6 +190,13 @@ render_tree () {\n \tfi\n }\n \n+if test -z \"$raw\"\n+then\n+\tdiffdir=$tmp/rendered\n+else\n+\tdiffdir=$tmp/installed\n+fi\n+\n render_tree $from_oid $from_dir $from_makemanflags &&\n render_tree $to_oid $to_dir $to_makemanflags &&\n-git -C $tmp/rendered diff --no-index \"$@\" $from_dir $to_dir\n+git -C $diffdir diff --no-index \"$@\" $from_dir $to_dir\n\n\nThe results are quite noisy compared to the rendered output (i.e.,\nstylistic choices in the roff output that have equivalent outputs). But\nit did let me easily find the difference in one of these cases:\n\n\n @@ -312,6 +458,7 @@ option\\&.\n .sp -1\n .IP \" 2.\" 4.2\n .\\}\n +\n hand resolve the conflict in the working directory, and update the index file to bring it into a state that the patch should have produced\\&. Then run the command with the\n \\fB\\-\\-continue\\fR\n option\\&.\n\n\nBut it's not clear from that whether asciidoc's docbook5 backend inserts\nan extra newline, or if it's part of the xml translation. Looking at the\nactual XML, I see:\n\n \n hand resolve the conflict in the working directory, and update\n the index file to bring it into a state that the patch should\n have produced. Then run the command with the --continue option.\n \n\nwhich looks OK. I wondered if there should not be a break between\n and , but it's there in the asciidoc version, too.\nSo I'm inclined to blame xmlto/docbook5 here.\n\n-Peff\n"},{"id":"382053","messageId":"20190908213028.GB8514@sigill.intra.peff.net","threadId":"50765","inReplyTo":"CAN0heSoOcpiPiRcYwLxxByC49Q-yXipFzBGm=XKW-AZk4n=8Jg@mail.gmail.com","subject":"Re: [PATCH v2 0/2] asciidoctor-extensions: provide ``","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2019-09-08T21:30:28Z","receivedAt":"2019-09-08T21:30:31Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Sat, Sep 07, 2019 at 04:06:48PM +0200, Martin Ågren wrote:\n\n> So of these steps:\n> \n> 0. Get Asciidoctor (v1) in shape.\n> \n> 1. Switch the default to Asciidoctor (v1).\n> \n> 2. Drop AsciiDoc to have faster Asciidoctor-processing, avoid xmlto\n> and support Asciidoctor 2. And to avoid the Python 2 EOL, too.\n> \n> Step 0 is not far away, so step 1 could be done fairly soon IMHO. Step 2\n> would \"hopefully\" happen soon after -- maybe even in the same release\n> cycle as step 1, and if not the same then the one just after. But I\n> might be the wrong person to trust on that one. I currently don't even\n> try to build with Asciidoctor 2. I might perhaps look into installing\n> it, but it could also be that I'll only start using it when it happens\n> to arrive through my distro.\n> \n> So as long as I'm not looking into Asciidoctor 2, maybe I shouldn't be\n> the one to impose \"default to asciidoctor\" on the world. Dunno. In any\n> case, I should be able to bring the asciidoc/tor1 differences to a state\n> where we trust asciidoctor 1 to be in a good shape, so that \"someone\n> else\" could pick up the ball and work on asciidoctor 2 vs 1, knowing\n> that it's ok if they regress AsciiDoc support or even drop it entirely\n> in the process.\n\nWe could also drop xmlto for the asciidoctor codepaths, without making\nit the default. That requires a little bit more Makefile massaging, but\nhere's a hacky way to do it (this is on top of brian's asciidoctor 2\npatch, which I've been experimenting with, but obviously we could skip\nthat entirely, too):\n\n---\ndiff --git a/Documentation/Makefile b/Documentation/Makefile\nindex 485f365cbf..c3ebca6e36 100644\n--- a/Documentation/Makefile\n+++ b/Documentation/Makefile\n@@ -197,12 +197,12 @@ ifdef USE_ASCIIDOCTOR\n ASCIIDOC = asciidoctor\n ASCIIDOC_CONF =\n ASCIIDOC_HTML = xhtml5\n-ASCIIDOC_DOCBOOK = docbook5\n+ASCIIDOC_DOCBOOK = manpage\n ASCIIDOC_EXTRA += -acompat-mode -atabsize=8\n ASCIIDOC_EXTRA += -I. -rasciidoctor-extensions\n ASCIIDOC_EXTRA += -alitdd='&\\#x2d;&\\#x2d;'\n DBLATEX_COMMON =\n-XMLTO_EXTRA += --skip-validation\n+XMLTO = ./fake-xmlto\n endif\n \n SHELL_PATH ?= $(SHELL)\ndiff --git a/Documentation/asciidoctor-extensions.rb b/Documentation/asciidoctor-extensions.rb\nindex 4ae130d2c6..aae891e8ff 100644\n--- a/Documentation/asciidoctor-extensions.rb\n+++ b/Documentation/asciidoctor-extensions.rb\n@@ -17,6 +17,8 @@ module Git\n \"#{target}\" \\\n \"#{attrs[1]}\\n\" \\\n \"\"\n+ elsif parent.document.basebackend? 'manpage'\n+ \"#{target}(#{attrs[1]})\"\n end\n end\n end\ndiff --git a/Documentation/fake-xmlto b/Documentation/fake-xmlto\nnew file mode 100755\nindex 0000000000..eca7ba289d\n--- /dev/null\n+++ b/Documentation/fake-xmlto\n@@ -0,0 +1,9 @@\n+#!/bin/sh\n+\n+# pick last arg as file\n+for file in \"$@\"; do\n+ : nothing\n+done\n+\n+nr=$(perl -lne '/^\\.TH \".*?\" \"(\\d)\"/ and print $1' $file)\n+cp $file ${file%.xml}.$nr\n-- \n2.23.0.554.g6dbe768f61\n\n\nThat's enough to let you do:\n\n ./doc-diff --from-asciidoc --to-asciidoctor HEAD^ HEAD\n\nand see what jumping to direct-manpage generation would look like. After\napplying all of your recent patches, I see some improvements. E.g.:\n\n - Files to add content from. Fileglobs (e.g. *.c) can be given to\n + Files to add content from. Fileglobs (e.g. *.c) can be given to add\n\nand some regressions:\n\n - of nothing). The other file, git-add--interactive.perl, has 403\n + of nothing). The other file, git-add--interactive.perl,\n\nThere's also a lot of noise. One curiosity is that it refuses to break\nlinkgit output when wrapping:\n\n - Remove everything in body before a scissors line (see git-\n - mailinfo(1)). Can be activated by default using the\n + Remove everything in body before a scissors line (see\n + git-mailinfo(1)). Can be activated by default using the\n\nI think that's fine (and probably even better), but it makes the\ndoc-diff quite hard to read. ;) Setting MANWIDTH=1000 in doc-diff is a\nhacky way to eliminate these, but I'm not sure it's a good idea in\ngeneral.\n\n-Peff\n"},{"id":"382054","messageId":"20190908213233.GC8514@sigill.intra.peff.net","threadId":"50765","inReplyTo":"20190908141308.GA7255@szeder.dev","subject":"Re: [PATCH] Documentation: fix build with Asciidoctor 2","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2019-09-08T21:32:34Z","receivedAt":"2019-09-08T21:32:36Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Sun, Sep 08, 2019 at 04:13:08PM +0200, SZEDER Gábor wrote:\n\n> OTOH, these \"stripped namespace\" messages come from 'xmlto', are not\n> warnings but have that \"Note:\" prefix, and, trusting that you did\n> check the results thoroughly, are apparently not a sign of any\n> rendering issues. So I think it's safe to ignore them and this patch\n> should strip them from 'make doc's output in\n> 'ci/test-documentation.sh'.\n\nYeah, I'd agree (though I do think there are some other issues with this\napproach, at least in my tests).\n\nIf we do go this route, it might even be nice to use a small xmlto\nwrapper that filters the stderr for us. Then the ci code doesn't have to\nknow about this distinction, and it prevents users from seeing the ugly\nmessages.\n\n-Peff\n"},{"id":"382055","messageId":"20190908222423.GN11334@genre.crustytoothpaste.net","threadId":"50765","inReplyTo":"20190908212122.GA8514@sigill.intra.peff.net","subject":"Re: [PATCH] Documentation: fix build with Asciidoctor 2","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2019-09-08T22:24:24Z","receivedAt":"2019-09-08T22:24:37Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2019-09-08 at 21:21:22, Jeff King wrote:\n> But it's not clear from that whether asciidoc's docbook5 backend inserts\n> an extra newline, or if it's part of the xml translation. Looking at the\n> actual XML, I see:\n> \n> \n> hand resolve the conflict in the working directory, and update\n> the index file to bring it into a state that the patch should\n> have produced. Then run the command with the --continue option.\n> \n> \n> which looks OK. I wondered if there should not be a break between\n> and , but it's there in the asciidoc version, too.\n> So I'm inclined to blame xmlto/docbook5 here.\n\nTrying again, I'm able to reproduce this. I found the cause, which is\nin the stylesheets. XSLT stylesheets have the ability to specify\nelements from which whitespace should be stripped (using the\nxsl:strip-space directive). In the DocBook stylesheets, listitem is\nspecified as such an element, so the whitespace there should be\nstripped.\n\nHowever, in DocBook 5, our elements are in a namespace. Therefore, the\nunnamespaced stylesheets specify only \"listitem\", not \"d:listitem\", like\nthe namespaced stylesheets do. Because this happens right after the\ntree has been constructed \"but before it is otherwise processed by XSLT\"\nand isn't affected by the EXSLT extension that allows re-parsing the\nmodified tree, then we end up with the whitespace that we don't want.\n\nThere are a couple ways around this.\n\n1. We can force xmlto to use the DocBook 5 stylesheets with the \"-x\"\nflag, but we have to know where they are. Debian and Fedora have them\nin different places, so we'd need a knob to figure out where they are.\n\n2. We can force xmlto to use a custom stylesheet with \"-x\" that merely\nimports the DocBook 5 stylesheets using a URL. If the user has the\nDocBook 5 stylesheets installed and XML catalogs configured (the default\non Linux distributions), then everything will just work and the system\nwill resolve it to the local copy. If, however, things are not properly\nconfigured, this will result in multiple network downloads for each\nmanual page.\n\n3. We can give up on xmlto and do things ourselves. This has the same\nproblem as option 1, since we need to learn how to find the stylesheets.\n\n4. We can send a patch to xmlto to make it use the proper stylesheets\nfor DocBook 5 and hope upstream does a release and everyone upgrades\nshortly. Since xmlto is not at all active upstream, this seems like it\nmay be an imprudent choice.\n\n5. We can send a patch to the DocBook stylesheets and have them include\nboth the namespaced and unnamespaced versions of the element names in\nboth sets of stylesheets and hope everyone upgrades.\n\nMy personal preference is #2; I think that seems like the best choice\nforward. XML catalogs are well understood and well configured on Linux\ndistributions. Homebrew supports them adequately, but you have to add\nan environment variable to your shell configuration to enable them. Of\ncourse, if you're doing _anything_ with XML, you'll have them enabled.\n-- \nbrian m. carlson: Houston, Texas, US\nOpenPGP: https://keybase.io/bk2204\n"},{"id":"382077","messageId":"xmqqr24pl6b9.fsf@gitster-ct.c.googlers.com","threadId":"50765","inReplyTo":"20190908222423.GN11334@genre.crustytoothpaste.net","subject":"Re: [PATCH] Documentation: fix build with Asciidoctor 2","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2019-09-09T17:37:14Z","receivedAt":"2019-09-09T17:37:24Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"brian m. carlson\" writes:\n\n> There are a couple ways around this.\n>\n> 1. We can force xmlto to use the DocBook 5 stylesheets with the \"-x\"\n> flag, but we have to know where they are. Debian and Fedora have them\n> in different places, so we'd need a knob to figure out where they are.\n>\n> 2. We can force xmlto to use a custom stylesheet with \"-x\" that merely\n> imports the DocBook 5 stylesheets using a URL. If the user has the\n> DocBook 5 stylesheets installed and XML catalogs configured (the default\n> on Linux distributions), then everything will just work and the system\n> will resolve it to the local copy. If, however, things are not properly\n> configured, this will result in multiple network downloads for each\n> manual page.\n>\n> 3. We can give up on xmlto and do things ourselves. This has the same\n> problem as option 1, since we need to learn how to find the stylesheets.\n>\n> 4. We can send a patch to xmlto to make it use the proper stylesheets\n> for DocBook 5 and hope upstream does a release and everyone upgrades\n> shortly. Since xmlto is not at all active upstream, this seems like it\n> may be an imprudent choice.\n>\n> 5. We can send a patch to the DocBook stylesheets and have them include\n> both the namespaced and unnamespaced versions of the element names in\n> both sets of stylesheets and hope everyone upgrades.\n>\n> My personal preference is #2; I think that seems like the best choice\n> forward. XML catalogs are well understood and well configured on Linux\n> distributions. Homebrew supports them adequately, but you have to add\n> an environment variable to your shell configuration to enable them. Of\n> course, if you're doing _anything_ with XML, you'll have them enabled.\n\nSounds sensible and well reasoned.\n"},{"id":"382143","messageId":"20190910184422.GA17446@sigill.intra.peff.net","threadId":"50765","inReplyTo":"20190908222423.GN11334@genre.crustytoothpaste.net","subject":"Re: [PATCH] Documentation: fix build with Asciidoctor 2","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2019-09-10T18:44:22Z","receivedAt":"2019-09-10T18:44:25Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Sun, Sep 08, 2019 at 10:24:24PM +0000, brian m. carlson wrote:\n\n> Trying again, I'm able to reproduce this. I found the cause, which is\n> in the stylesheets. XSLT stylesheets have the ability to specify\n> elements from which whitespace should be stripped (using the\n> xsl:strip-space directive). In the DocBook stylesheets, listitem is\n> specified as such an element, so the whitespace there should be\n> stripped.\n> \n> However, in DocBook 5, our elements are in a namespace. Therefore, the\n> unnamespaced stylesheets specify only \"listitem\", not \"d:listitem\", like\n> the namespaced stylesheets do. Because this happens right after the\n> tree has been constructed \"but before it is otherwise processed by XSLT\"\n> and isn't affected by the EXSLT extension that allows re-parsing the\n> modified tree, then we end up with the whitespace that we don't want.\n\nFirst off, thank you again for your explanations. I dread digging into\nhow anything related to docbook or xml works, so having you serve it up\non a silver platter is a delight. :)\n\n> 2. We can force xmlto to use a custom stylesheet with \"-x\" that merely\n> imports the DocBook 5 stylesheets using a URL. If the user has the\n> DocBook 5 stylesheets installed and XML catalogs configured (the default\n> on Linux distributions), then everything will just work and the system\n> will resolve it to the local copy. If, however, things are not properly\n> configured, this will result in multiple network downloads for each\n> manual page.\n\nIsn't this already the case just with the docbook DTDs? I.e., if you\ndon't have a catalog entry, it is up to the tool (xmlto in this case) to\neither fail or try to fetch it. That seems like the best we can do. And\nas you note, this typically just works out of the box on modern\ninstalls. Of course people may want to build on non-modern ones, but\nIMHO we should probably be more aggressive about dropping legacy support\nin the documentation and pointing people to the pre-formatted pages.\n\n> My personal preference is #2; I think that seems like the best choice\n> forward. XML catalogs are well understood and well configured on Linux\n> distributions. Homebrew supports them adequately, but you have to add\n> an environment variable to your shell configuration to enable them. Of\n> course, if you're doing _anything_ with XML, you'll have them enabled.\n\nYeah, agreed.\n\n-Peff\n"},{"id":"382190","messageId":"20190911231928.GQ11334@genre.crustytoothpaste.net","threadId":"50765","inReplyTo":"20190910184422.GA17446@sigill.intra.peff.net","subject":"Re: [PATCH] Documentation: fix build with Asciidoctor 2","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2019-09-11T23:19:28Z","receivedAt":"2019-09-11T23:19:36Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2019-09-10 at 18:44:22, Jeff King wrote:\n> First off, thank you again for your explanations. I dread digging into\n> how anything related to docbook or xml works, so having you serve it up\n> on a silver platter is a delight. :)\n\nI'm happy to do it. I was an English major in college and virtually\nevery assignment I wrote in DocBook, so I'm far more familiar with it\nthan I'm sure most people ever want to be.\n\n> Isn't this already the case just with the docbook DTDs? I.e., if you\n> don't have a catalog entry, it is up to the tool (xmlto in this case) to\n> either fail or try to fetch it. That seems like the best we can do. And\n> as you note, this typically just works out of the box on modern\n> installs. Of course people may want to build on non-modern ones, but\n> IMHO we should probably be more aggressive about dropping legacy support\n> in the documentation and pointing people to the pre-formatted pages.\n\nYeah, that's a good point. Since we know that most people will have\ncatalogs working (since usually fetching the DTD repeatedly over the\nnetwork gets you rate-limited resulting in a failure), I think it's fair\nfor us to do this. And anyway, DocBook 5 isn't exactly new: it's been\naround since 2009, IIRC, so distros will have it.\n\nI know Homebrew doesn't have this problem, since they fetch the prebuilt\ndocs, even for HEAD (master) builds.\n\nI'll try to get the patch written up and sent out today or tomorrow.\n-- \nbrian m. carlson: Houston, Texas, US\nOpenPGP: https://keybase.io/bk2204\n"},{"id":"382277","messageId":"20190913015240.686522-1-sandals@crustytoothpaste.net","threadId":"50765","inReplyTo":"20190906232947.GJ11334@genre.crustytoothpaste.net","subject":"[PATCH v2] Documentation: fix build with Asciidoctor 2","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2019-09-13T01:52:40Z","receivedAt":"2019-09-13T01:52:51Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"Our documentation toolchain has traditionally been built around DocBook\n4.5. This version of DocBook is the last DTD-based version of DocBook.\nIn 2009, DocBook 5 was introduced using namespaces and its syntax is\nexpressed in RELAX NG, which is more expressive and allows a wider\nvariety of syntax forms.\n\nAsciidoctor, one of the alternatives for building our documentation,\nmoved support for DocBook 4.5 out of core in its recent 2.0 release and\nnow only supports DocBook 5 in the main release. The DocBoook 4.5\nconverter is still available as a separate component, but this is not\navailable in most distro packages. This would not be a problem but for\nthe fact that we use xmlto, which is still stuck in the DocBook 4.5 era.\n\nxmlto performs DTD validation as part of the build process. This is not\nproblematic for DocBook 4.5, which has a valid DTD, but it clearly\ncannot work for DocBook 5, since no DTD can adequately express its full\nsyntax. In addition, even if xmlto did support RELAX NG validation,\nthat wouldn't be sufficient because it uses the libxml2-based xmllint to\ndo so, which has known problems with validating interleaves in RELAX NG.\n\nFortunately, there's an easy way forward: ask Asciidoctor to use its\nDocBook 5 backend and tell xmlto to skip validation. Asciidoctor has\nsupported DocBook 5 since v0.1.4 in 2013 and xmlto has supported\nskipping validation for probably longer than that.\n\nWe also need to teach xmlto how to use the namespaced DocBook XSLT\nstylesheets instead of the non-namespaced ones it usually uses.\nNormally these stylesheets are interchangeable, but the non-namespaced\nones have a bug that causes them not to strip whitespace automatically\nfrom certain elements when namespaces are in use. This results in\nadditional whitespace at the beginning of list elements, which is\njarring and unsightly.\n\nWe can do this by passing a custom stylesheet with the -x option that\nsimply imports the namespaced stylesheets via a URL. Any system with\nsupport for XML catalogs will automatically look this URL up and\nreference a local copy instead without us having to know where this\nlocal copy is located. We know that anyone using xmlto will already\nhave catalogs set up properly since the DocBook 4.5 DTD used during\nvalidation is also looked up via catalogs. All major Linux\ndistributions distribute the necessary stylesheets and have built-in\ncatalog support, and Homebrew does as well, albeit with a requirement to\nset an environment variable to enable catalog support.\n\nOn the off chance that someone lacks support for catalogs, it is\npossible for xmlto (via xmllint) to download the stylesheets from the\nURLs in question, although this will likely perform poorly enough to\nattract attention. People still have the option of using the prebuilt\ndocumentation that we ship, so happily this should not be an impediment.\n\nFinally, we need to filter out some messages from other stylesheets that\nwhen invoking dblatex in the CI job. This tool strips namespaces much\nlike the unnamespaced DocBook stylesheets and prints similar messages.\nIf we permit these messages to be printed to standard error, our\ndocumentation CI job will because we check standard error for unexpected\noutput. Due to dblatex's reliance on Python 2, we may need to revisit\nits use in the future, in which case this problem may go away, but this\ncan be delayed until a future patch.\n\nSigned-off-by: brian m. carlson \n---\n Documentation/Makefile | 4 +++-\n Documentation/manpage.xsl | 3 +++\n ci/test-documentation.sh | 2 ++\n 3 files changed, 8 insertions(+), 1 deletion(-)\n create mode 100644 Documentation/manpage.xsl\n\ndiff --git a/Documentation/Makefile b/Documentation/Makefile\nindex 76f2ecfc1b..d94f47c5c9 100644\n--- a/Documentation/Makefile\n+++ b/Documentation/Makefile\n@@ -197,11 +197,13 @@ ifdef USE_ASCIIDOCTOR\n ASCIIDOC = asciidoctor\n ASCIIDOC_CONF =\n ASCIIDOC_HTML = xhtml5\n-ASCIIDOC_DOCBOOK = docbook45\n+ASCIIDOC_DOCBOOK = docbook5\n ASCIIDOC_EXTRA += -acompat-mode -atabsize=8\n ASCIIDOC_EXTRA += -I. -rasciidoctor-extensions\n ASCIIDOC_EXTRA += -alitdd='&\\#x2d;&\\#x2d;'\n DBLATEX_COMMON =\n+XMLTO_EXTRA += --skip-validation\n+XMLTO_EXTRA += -x manpage.xsl\n endif\n \n SHELL_PATH ?= $(SHELL)\ndiff --git a/Documentation/manpage.xsl b/Documentation/manpage.xsl\nnew file mode 100644\nindex 0000000000..ef64bab17a\n--- /dev/null\n+++ b/Documentation/manpage.xsl\n@@ -0,0 +1,3 @@\n+\n+\t\n+\ndiff --git a/ci/test-documentation.sh b/ci/test-documentation.sh\nindex d49089832d..b3e76ef863 100755\n--- a/ci/test-documentation.sh\n+++ b/ci/test-documentation.sh\n@@ -8,6 +8,8 @@\n filter_log () {\n \tsed -e '/^GIT_VERSION = /d' \\\n \t -e '/^ \\* new asciidoc flags$/d' \\\n+\t -e '/stripped namespace before processing/d' \\\n+\t -e '/Attributed.*IDs for element/d' \\\n \t \"$1\"\n }\n \n"},{"id":"382280","messageId":"20190913050634.GB21172@sigill.intra.peff.net","threadId":"50765","inReplyTo":"20190913015240.686522-1-sandals@crustytoothpaste.net","subject":"Re: [PATCH v2] Documentation: fix build with Asciidoctor 2","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2019-09-13T05:06:35Z","receivedAt":"2019-09-13T05:06:37Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Fri, Sep 13, 2019 at 01:52:40AM +0000, brian m. carlson wrote:\n\n> We also need to teach xmlto how to use the namespaced DocBook XSLT\n> stylesheets instead of the non-namespaced ones it usually uses.\n> Normally these stylesheets are interchangeable, but the non-namespaced\n> ones have a bug that causes them not to strip whitespace automatically\n> from certain elements when namespaces are in use. This results in\n> additional whitespace at the beginning of list elements, which is\n> jarring and unsightly.\n\nThanks, this fixed most of the rendering problems I saw from the earlier\npatch.\n\n> We can do this by passing a custom stylesheet with the -x option that\n> simply imports the namespaced stylesheets via a URL. Any system with\n> support for XML catalogs will automatically look this URL up and\n> reference a local copy instead without us having to know where this\n> local copy is located. We know that anyone using xmlto will already\n> have catalogs set up properly since the DocBook 4.5 DTD used during\n> validation is also looked up via catalogs. All major Linux\n> distributions distribute the necessary stylesheets and have built-in\n> catalog support, and Homebrew does as well, albeit with a requirement to\n> set an environment variable to enable catalog support.\n\nThis did give me one minor hiccup: I had the debian docbook-xsl package\ninstalled, but not docbook-xsl-ns. The error message was pretty standard\nfor XML: obvious if you know what catalogs are, and utterly confusing\notherwise. :)\n\nEverything worked fine after installing docbook-xsl-ns. I wonder if\ncould/should provide some guidance somewhere (maybe in INSTALL, which\ndiscusses some catalog issues?).\n\n> Finally, we need to filter out some messages from other stylesheets that\n> when invoking dblatex in the CI job. This tool strips namespaces much\n\ns/that/that occur/ or something?\n\n> like the unnamespaced DocBook stylesheets and prints similar messages.\n> If we permit these messages to be printed to standard error, our\n> documentation CI job will because we check standard error for unexpected\n\ns/will/will fail/?\n\n> ---\n> Documentation/Makefile | 4 +++-\n> Documentation/manpage.xsl | 3 +++\n> ci/test-documentation.sh | 2 ++\n> 3 files changed, 8 insertions(+), 1 deletion(-)\n> create mode 100644 Documentation/manpage.xsl\n\nRunning with this patch on asciidoctor 2.0.10, plus Martin's recent\nliteral-block cleanups, plus his refmiscinfo fix, I get pretty decent\noutput from:\n\n ./doc-diff --from-asciidoc --to-asciidoctor origin HEAD\n\nThe header/footer are still a little funny (but I think Martin said that\nhe needs to update the refmiscinfo patches for later versions of\nasciidoctor, which is probably what's going on here):\n\n --- a/f1d4a28250629ae469fc5dd59ab843cb2fd68e12-asciidoc/home/peff/share/man/man1/git-add.1\n +++ b/6c08635fd1d38c83d3765ff05fabbfbd25ef4943-asciidoctor/home/peff/share/man/man1/git-add.1\n @@ -1,4 +1,4 @@\n -GIT-ADD(1) Git Manual GIT-ADD(1)\n +GIT-ADD(1) GIT-ADD(1)\n \n NAME\n git-add - Add file contents to the index\n @@ -356,4 +356,4 @@ SEE ALSO\n GIT\n Part of the git(1) suite\n \n -Git omitted 01/01/1970 GIT-ADD(1)\n + omitted 1970-01-01 GIT-ADD(1)\n\n\nOne curiosity is that any ``smart-quotes'' now get two spaces between them\nand the period of the last sentence (whereas in asciidoc they got only\none):\n\n - and are optional. “-L ” or “-L ,” spans\n - from to end of file. “-L ,” spans from start of file\n - to .\n + and are optional. “-L ” or “-L ,”\n + spans from to end of file. “-L ,” spans from start of\n + file to .\n\nI don't think this is a big deal, but I think most of these should\nactually be backticks these days (the text above is from\ngit-annotate.txt, which hasn't been touched in quite a while).\n\nThere are other miscellaneous indentation fixes. Most of them look\nbetter in asciidoctor, IMHO. For example, some lists now wrap more\nneatly (it looks like it's usually lists after an indented listing\nblock? Maybe a continuation thing?):\n\n - 1. This step and the next one could be combined into a single step\n - with \"checkout -b my2.6.14 v2.6.14\".\n + 1. This step and the next one could be combined into a single\n + step with \"checkout -b my2.6.14 v2.6.14\".\n\nAnother curiosity is that single-quote `smart-quotes' are rendered as\nreal smart-quotes by asciidoctor:\n\n - The following features from ‘svn log’ are supported:\n + The following features from “svn log” are supported:\n\nThe only other case I found was this one, where I think the asciidoctor\nversion is better (the source has literal backticks, so there shouldn't\nbe a visible quote; I'm guessing asciidoc got confused by the apostrophe\nin \"variable's\"):\n\n - The ‘merge.*.driver` variable’s value is used to construct a\n + The merge.*.driver variable’s value is used to construct a command\n\nSo overall, I think we're getting very close to parity.\n\n-Peff\n"},{"id":"382309","messageId":"xmqq1rwkf7mu.fsf@gitster-ct.c.googlers.com","threadId":"50765","inReplyTo":"20190913050634.GB21172@sigill.intra.peff.net","subject":"Re: [PATCH v2] Documentation: fix build with Asciidoctor 2","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2019-09-13T17:06:33Z","receivedAt":"2019-09-13T17:06:45Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King writes:\n\n> On Fri, Sep 13, 2019 at 01:52:40AM +0000, brian m. carlson wrote:\n>\n>> We also need to teach xmlto how to use the namespaced DocBook XSLT\n>> stylesheets instead of the non-namespaced ones it usually uses.\n>> Normally these stylesheets are interchangeable, but the non-namespaced\n>> ones have a bug that causes them not to strip whitespace automatically\n>> from certain elements when namespaces are in use. This results in\n>> additional whitespace at the beginning of list elements, which is\n>> jarring and unsightly.\n>\n> Thanks, this fixed most of the rendering problems I saw from the earlier\n> patch.\n>\n>> We can do this by passing a custom stylesheet with the -x option that\n>> simply imports the namespaced stylesheets via a URL. Any system with\n>> support for XML catalogs will automatically look this URL up and\n>> reference a local copy instead without us having to know where this\n>> local copy is located. We know that anyone using xmlto will already\n>> have catalogs set up properly since the DocBook 4.5 DTD used during\n>> validation is also looked up via catalogs. All major Linux\n>> distributions distribute the necessary stylesheets and have built-in\n>> catalog support, and Homebrew does as well, albeit with a requirement to\n>> set an environment variable to enable catalog support.\n>\n> This did give me one minor hiccup: I had the debian docbook-xsl package\n> installed, but not docbook-xsl-ns. The error message was pretty standard\n> for XML: obvious if you know what catalogs are, and utterly confusing\n> otherwise. :)\n>\n> Everything worked fine after installing docbook-xsl-ns. I wonder if\n> could/should provide some guidance somewhere (maybe in INSTALL, which\n> discusses some catalog issues?).\n>\n>> Finally, we need to filter out some messages from other stylesheets that\n>> when invoking dblatex in the CI job. This tool strips namespaces much\n>\n> s/that/that occur/ or something?\n>\n>> like the unnamespaced DocBook stylesheets and prints similar messages.\n>> If we permit these messages to be printed to standard error, our\n>> documentation CI job will because we check standard error for unexpected\n>\n> s/will/will fail/?\n>\n>> ---\n>> Documentation/Makefile | 4 +++-\n>> Documentation/manpage.xsl | 3 +++\n>> ci/test-documentation.sh | 2 ++\n>> 3 files changed, 8 insertions(+), 1 deletion(-)\n>> create mode 100644 Documentation/manpage.xsl\n>\n> Running with this patch on asciidoctor 2.0.10, plus Martin's recent\n> literal-block cleanups, plus his refmiscinfo fix, I get pretty decent\n> output from:\n>\n> ./doc-diff --from-asciidoc --to-asciidoctor origin HEAD\n>\n> The header/footer are still a little funny (but I think Martin said that\n> he needs to update the refmiscinfo patches for later versions of\n> asciidoctor, which is probably what's going on here):\n> ...\n> So overall, I think we're getting very close to parity.\n\nThanks, both. Have queued with your log message typofixes.\n\n"},{"id":"382360","messageId":"20190914075301.GR32087@szeder.dev","threadId":"50765","inReplyTo":"20190913015240.686522-1-sandals@crustytoothpaste.net","subject":"Re: [PATCH v2] Documentation: fix build with Asciidoctor 2","fromName":"SZEDER Gábor","fromEmail":"szeder.dev@gmail.com","sentAt":"2019-09-14T07:53:01Z","receivedAt":"2019-09-14T07:53:09Z","isPatch":true,"sender":{"key":"szeder.dev@gmail.com","avatar":"https://avatars.githubusercontent.com/u/116324?v=4"},"body":"On Fri, Sep 13, 2019 at 01:52:40AM +0000, brian m. carlson wrote:\n\n> We also need to teach xmlto how to use the namespaced DocBook XSLT\n> stylesheets instead of the non-namespaced ones it usually uses.\n> Normally these stylesheets are interchangeable, but the non-namespaced\n> ones have a bug that causes them not to strip whitespace automatically\n> from certain elements when namespaces are in use. This results in\n> additional whitespace at the beginning of list elements, which is\n> jarring and unsightly.\n> \n> We can do this by passing a custom stylesheet with the -x option that\n> simply imports the namespaced stylesheets via a URL. Any system with\n> support for XML catalogs will automatically look this URL up and\n> reference a local copy instead without us having to know where this\n> local copy is located. We know that anyone using xmlto will already\n> have catalogs set up properly since the DocBook 4.5 DTD used during\n> validation is also looked up via catalogs. All major Linux\n> distributions distribute the necessary stylesheets and have built-in\n> catalog support, and Homebrew does as well, albeit with a requirement to\n> set an environment variable to enable catalog support.\n> \n> On the off chance that someone lacks support for catalogs, it is\n> possible for xmlto (via xmllint) to download the stylesheets from the\n> URLs in question, although this will likely perform poorly enough to\n> attract attention. People still have the option of using the prebuilt\n> documentation that we ship, so happily this should not be an impediment.\n\n\n> diff --git a/Documentation/Makefile b/Documentation/Makefile\n> index 76f2ecfc1b..d94f47c5c9 100644\n> --- a/Documentation/Makefile\n> +++ b/Documentation/Makefile\n> @@ -197,11 +197,13 @@ ifdef USE_ASCIIDOCTOR\n> ASCIIDOC = asciidoctor\n> ASCIIDOC_CONF =\n> ASCIIDOC_HTML = xhtml5\n> -ASCIIDOC_DOCBOOK = docbook45\n> +ASCIIDOC_DOCBOOK = docbook5\n> ASCIIDOC_EXTRA += -acompat-mode -atabsize=8\n> ASCIIDOC_EXTRA += -I. -rasciidoctor-extensions\n> ASCIIDOC_EXTRA += -alitdd='&\\#x2d;&\\#x2d;'\n> DBLATEX_COMMON =\n> +XMLTO_EXTRA += --skip-validation\n> +XMLTO_EXTRA += -x manpage.xsl\n> endif\n> \n> SHELL_PATH ?= $(SHELL)\n> diff --git a/Documentation/manpage.xsl b/Documentation/manpage.xsl\n> new file mode 100644\n> index 0000000000..ef64bab17a\n> --- /dev/null\n> +++ b/Documentation/manpage.xsl\n> @@ -0,0 +1,3 @@\n> +\n> +\t\n> +\n\nUnfortunately, five out of five CI builds failed with the following:\n\n XMLTO git-revert.1\n I/O error : Attempt to load network entity http://docbook.sourceforge.net/release/xsl-ns/current/manpages/docbook.xsl\n warning: failed to load external entity \"http://docbook.sourceforge.net/release/xsl-ns/current/manpages/docbook.xsl\"\n compilation error: file /home/travis/build/git/git/Documentation/manpage.xsl line 2 element import\n xsl:import : unable to load http://docbook.sourceforge.net/release/xsl-ns/current/manpages/docbook.xsl\n Makefile:375: recipe for target 'git-revert.1' failed\n\nhttps://travis-ci.org/git/git/jobs/584794387#L1552\n\n"},{"id":"382373","messageId":"20190914194445.GT11334@genre.crustytoothpaste.net","threadId":"50765","inReplyTo":"20190914075301.GR32087@szeder.dev","subject":"Re: [PATCH v2] Documentation: fix build with Asciidoctor 2","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2019-09-14T19:44:45Z","receivedAt":"2019-09-14T19:44:53Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2019-09-14 at 07:53:01, SZEDER Gábor wrote:\n> Unfortunately, five out of five CI builds failed with the following:\n> \n> XMLTO git-revert.1\n> I/O error : Attempt to load network entity http://docbook.sourceforge.net/release/xsl-ns/current/manpages/docbook.xsl\n> warning: failed to load external entity \"http://docbook.sourceforge.net/release/xsl-ns/current/manpages/docbook.xsl\"\n> compilation error: file /home/travis/build/git/git/Documentation/manpage.xsl line 2 element import\n> xsl:import : unable to load http://docbook.sourceforge.net/release/xsl-ns/current/manpages/docbook.xsl\n> Makefile:375: recipe for target 'git-revert.1' failed\n> \n> https://travis-ci.org/git/git/jobs/584794387#L1552\n\nAh, I forgot to install the packages in CI. I'll send a v3 with that\nfixed.\n-- \nbrian m. carlson: Houston, Texas, US\nOpenPGP: https://keybase.io/bk2204\n"},{"id":"382374","messageId":"20190914194919.748935-1-sandals@crustytoothpaste.net","threadId":"50765","inReplyTo":"20190906232947.GJ11334@genre.crustytoothpaste.net","subject":"[PATCH v3] Documentation: fix build with Asciidoctor 2","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2019-09-14T19:49:19Z","receivedAt":"2019-09-14T19:49:37Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"Our documentation toolchain has traditionally been built around DocBook\n4.5. This version of DocBook is the last DTD-based version of DocBook.\nIn 2009, DocBook 5 was introduced using namespaces and its syntax is\nexpressed in RELAX NG, which is more expressive and allows a wider\nvariety of syntax forms.\n\nAsciidoctor, one of the alternatives for building our documentation,\nmoved support for DocBook 4.5 out of core in its recent 2.0 release and\nnow only supports DocBook 5 in the main release. The DocBoook 4.5\nconverter is still available as a separate component, but this is not\navailable in most distro packages. This would not be a problem but for\nthe fact that we use xmlto, which is still stuck in the DocBook 4.5 era.\n\nxmlto performs DTD validation as part of the build process. This is not\nproblematic for DocBook 4.5, which has a valid DTD, but it clearly\ncannot work for DocBook 5, since no DTD can adequately express its full\nsyntax. In addition, even if xmlto did support RELAX NG validation,\nthat wouldn't be sufficient because it uses the libxml2-based xmllint to\ndo so, which has known problems with validating interleaves in RELAX NG.\n\nFortunately, there's an easy way forward: ask Asciidoctor to use its\nDocBook 5 backend and tell xmlto to skip validation. Asciidoctor has\nsupported DocBook 5 since v0.1.4 in 2013 and xmlto has supported\nskipping validation for probably longer than that.\n\nWe also need to teach xmlto how to use the namespaced DocBook XSLT\nstylesheets instead of the non-namespaced ones it usually uses.\nNormally these stylesheets are interchangeable, but the non-namespaced\nones have a bug that causes them not to strip whitespace automatically\nfrom certain elements when namespaces are in use. This results in\nadditional whitespace at the beginning of list elements, which is\njarring and unsightly.\n\nWe can do this by passing a custom stylesheet with the -x option that\nsimply imports the namespaced stylesheets via a URL. Any system with\nsupport for XML catalogs will automatically look this URL up and\nreference a local copy instead without us having to know where this\nlocal copy is located. We know that anyone using xmlto will already\nhave catalogs set up properly since the DocBook 4.5 DTD used during\nvalidation is also looked up via catalogs. All major Linux\ndistributions distribute the necessary stylesheets and have built-in\ncatalog support, and Homebrew does as well, albeit with a requirement to\nset an environment variable to enable catalog support.\n\nOn the off chance that someone lacks support for catalogs, it is\npossible for xmlto (via xmllint) to download the stylesheets from the\nURLs in question, although this will likely perform poorly enough to\nattract attention. People still have the option of using the prebuilt\ndocumentation that we ship, so happily this should not be an impediment.\n\nFinally, we need to filter out some messages from other stylesheets that\noccur when invoking dblatex in the CI job. This tool strips namespaces\nmuch like the unnamespaced DocBook stylesheets and prints similar\nmessages. If we permit these messages to be printed to standard error,\nour documentation CI job will fail because we check standard error for\nunexpected output. Due to dblatex's reliance on Python 2, we may need\nto revisit its use in the future, in which case this problem may go\naway, but this can be delayed until a future patch.\n\nSigned-off-by: brian m. carlson \n---\nRange-diff against v2:\n1: 6f11a79d03 ! 1: c4935af5de Documentation: fix build with Asciidoctor 2\n @@ Commit message\n documentation that we ship, so happily this should not be an impediment.\n \n Finally, we need to filter out some messages from other stylesheets that\n - when invoking dblatex in the CI job. This tool strips namespaces much\n - like the unnamespaced DocBook stylesheets and prints similar messages.\n - If we permit these messages to be printed to standard error, our\n - documentation CI job will because we check standard error for unexpected\n - output. Due to dblatex's reliance on Python 2, we may need to revisit\n - its use in the future, in which case this problem may go away, but this\n - can be delayed until a future patch.\n + occur when invoking dblatex in the CI job. This tool strips namespaces\n + much like the unnamespaced DocBook stylesheets and prints similar\n + messages. If we permit these messages to be printed to standard error,\n + our documentation CI job will fail because we check standard error for\n + unexpected output. Due to dblatex's reliance on Python 2, we may need\n + to revisit its use in the future, in which case this problem may go\n + away, but this can be delayed until a future patch.\n \n Signed-off-by: brian m. carlson \n \n @@ Documentation/manpage.xsl (new)\n +\t\n +\n \n + ## azure-pipelines.yml ##\n +@@ azure-pipelines.yml: jobs:\n + test \"$GITFILESHAREPWD\" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare \"$GITFILESHAREPWD\" \"$HOME/test-cache\" || exit 1\n + \n + sudo apt-get update &&\n +- sudo apt-get install -y asciidoc xmlto asciidoctor &&\n ++ sudo apt-get install -y asciidoc xmlto asciidoctor docbook-xsl-ns &&\n + \n + export ALREADY_HAVE_ASCIIDOCTOR=yes. &&\n + export jobname=Documentation &&\n +\n + ## ci/install-dependencies.sh ##\n +@@ ci/install-dependencies.sh: StaticAnalysis)\n + \t;;\n + Documentation)\n + \tsudo apt-get -q update\n +-\tsudo apt-get -q -y install asciidoc xmlto\n ++\tsudo apt-get -q -y install asciidoc xmlto docbook-xsl-ns\n + \n + \ttest -n \"$ALREADY_HAVE_ASCIIDOCTOR\" ||\n + \tgem install --version 1.5.8 asciidoctor\n +\n ## ci/test-documentation.sh ##\n @@\n filter_log () {\n\n Documentation/Makefile | 4 +++-\n Documentation/manpage.xsl | 3 +++\n azure-pipelines.yml | 2 +-\n ci/install-dependencies.sh | 2 +-\n ci/test-documentation.sh | 2 ++\n 5 files changed, 10 insertions(+), 3 deletions(-)\n create mode 100644 Documentation/manpage.xsl\n\ndiff --git a/Documentation/Makefile b/Documentation/Makefile\nindex 76f2ecfc1b..d94f47c5c9 100644\n--- a/Documentation/Makefile\n+++ b/Documentation/Makefile\n@@ -197,11 +197,13 @@ ifdef USE_ASCIIDOCTOR\n ASCIIDOC = asciidoctor\n ASCIIDOC_CONF =\n ASCIIDOC_HTML = xhtml5\n-ASCIIDOC_DOCBOOK = docbook45\n+ASCIIDOC_DOCBOOK = docbook5\n ASCIIDOC_EXTRA += -acompat-mode -atabsize=8\n ASCIIDOC_EXTRA += -I. -rasciidoctor-extensions\n ASCIIDOC_EXTRA += -alitdd='&\\#x2d;&\\#x2d;'\n DBLATEX_COMMON =\n+XMLTO_EXTRA += --skip-validation\n+XMLTO_EXTRA += -x manpage.xsl\n endif\n \n SHELL_PATH ?= $(SHELL)\ndiff --git a/Documentation/manpage.xsl b/Documentation/manpage.xsl\nnew file mode 100644\nindex 0000000000..ef64bab17a\n--- /dev/null\n+++ b/Documentation/manpage.xsl\n@@ -0,0 +1,3 @@\n+\n+\t\n+\ndiff --git a/azure-pipelines.yml b/azure-pipelines.yml\nindex c329b7218b..34031b182a 100644\n--- a/azure-pipelines.yml\n+++ b/azure-pipelines.yml\n@@ -374,7 +374,7 @@ jobs:\n test \"$GITFILESHAREPWD\" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare \"$GITFILESHAREPWD\" \"$HOME/test-cache\" || exit 1\n \n sudo apt-get update &&\n- sudo apt-get install -y asciidoc xmlto asciidoctor &&\n+ sudo apt-get install -y asciidoc xmlto asciidoctor docbook-xsl-ns &&\n \n export ALREADY_HAVE_ASCIIDOCTOR=yes. &&\n export jobname=Documentation &&\ndiff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh\nindex 8cc72503cb..a76f348484 100755\n--- a/ci/install-dependencies.sh\n+++ b/ci/install-dependencies.sh\n@@ -53,7 +53,7 @@ StaticAnalysis)\n \t;;\n Documentation)\n \tsudo apt-get -q update\n-\tsudo apt-get -q -y install asciidoc xmlto\n+\tsudo apt-get -q -y install asciidoc xmlto docbook-xsl-ns\n \n \ttest -n \"$ALREADY_HAVE_ASCIIDOCTOR\" ||\n \tgem install --version 1.5.8 asciidoctor\ndiff --git a/ci/test-documentation.sh b/ci/test-documentation.sh\nindex d49089832d..b3e76ef863 100755\n--- a/ci/test-documentation.sh\n+++ b/ci/test-documentation.sh\n@@ -8,6 +8,8 @@\n filter_log () {\n \tsed -e '/^GIT_VERSION = /d' \\\n \t -e '/^ \\* new asciidoc flags$/d' \\\n+\t -e '/stripped namespace before processing/d' \\\n+\t -e '/Attributed.*IDs for element/d' \\\n \t \"$1\"\n }\n \n"},{"id":"382388","messageId":"20190915095952.GA6190@szeder.dev","threadId":"50765","inReplyTo":"20190914194919.748935-1-sandals@crustytoothpaste.net","subject":"Re: [PATCH v3] Documentation: fix build with Asciidoctor 2","fromName":"SZEDER Gábor","fromEmail":"szeder.dev@gmail.com","sentAt":"2019-09-15T09:59:52Z","receivedAt":"2019-09-15T10:00:09Z","isPatch":true,"sender":{"key":"szeder.dev@gmail.com","avatar":"https://avatars.githubusercontent.com/u/116324?v=4"},"body":"On Sat, Sep 14, 2019 at 07:49:19PM +0000, brian m. carlson wrote:\n> Our documentation toolchain has traditionally been built around DocBook\n> 4.5. This version of DocBook is the last DTD-based version of DocBook.\n> In 2009, DocBook 5 was introduced using namespaces and its syntax is\n> expressed in RELAX NG, which is more expressive and allows a wider\n> variety of syntax forms.\n> \n> Asciidoctor, one of the alternatives for building our documentation,\n> moved support for DocBook 4.5 out of core in its recent 2.0 release and\n> now only supports DocBook 5 in the main release. The DocBoook 4.5\n> converter is still available as a separate component, but this is not\n> available in most distro packages. This would not be a problem but for\n> the fact that we use xmlto, which is still stuck in the DocBook 4.5 era.\n> \n> xmlto performs DTD validation as part of the build process. This is not\n> problematic for DocBook 4.5, which has a valid DTD, but it clearly\n> cannot work for DocBook 5, since no DTD can adequately express its full\n> syntax. In addition, even if xmlto did support RELAX NG validation,\n> that wouldn't be sufficient because it uses the libxml2-based xmllint to\n> do so, which has known problems with validating interleaves in RELAX NG.\n> \n> Fortunately, there's an easy way forward: ask Asciidoctor to use its\n> DocBook 5 backend and tell xmlto to skip validation. Asciidoctor has\n> supported DocBook 5 since v0.1.4 in 2013 and xmlto has supported\n> skipping validation for probably longer than that.\n> \n> We also need to teach xmlto how to use the namespaced DocBook XSLT\n> stylesheets instead of the non-namespaced ones it usually uses.\n> Normally these stylesheets are interchangeable, but the non-namespaced\n> ones have a bug that causes them not to strip whitespace automatically\n> from certain elements when namespaces are in use. This results in\n> additional whitespace at the beginning of list elements, which is\n> jarring and unsightly.\n> \n> We can do this by passing a custom stylesheet with the -x option that\n> simply imports the namespaced stylesheets via a URL. Any system with\n> support for XML catalogs will automatically look this URL up and\n> reference a local copy instead without us having to know where this\n> local copy is located. We know that anyone using xmlto will already\n> have catalogs set up properly since the DocBook 4.5 DTD used during\n> validation is also looked up via catalogs. All major Linux\n> distributions distribute the necessary stylesheets and have built-in\n> catalog support, and Homebrew does as well, albeit with a requirement to\n> set an environment variable to enable catalog support.\n> \n> On the off chance that someone lacks support for catalogs, it is\n> possible for xmlto (via xmllint) to download the stylesheets from the\n> URLs in question, although this will likely perform poorly enough to\n> attract attention. People still have the option of using the prebuilt\n> documentation that we ship, so happily this should not be an impediment.\n> \n> Finally, we need to filter out some messages from other stylesheets that\n> occur when invoking dblatex in the CI job. This tool strips namespaces\n> much like the unnamespaced DocBook stylesheets and prints similar\n> messages. If we permit these messages to be printed to standard error,\n> our documentation CI job will fail because we check standard error for\n> unexpected output. Due to dblatex's reliance on Python 2, we may need\n> to revisit its use in the future, in which case this problem may go\n> away, but this can be delayed until a future patch.\n> \n> Signed-off-by: brian m. carlson \n\n> Documentation/Makefile | 4 +++-\n> Documentation/manpage.xsl | 3 +++\n> azure-pipelines.yml | 2 +-\n> ci/install-dependencies.sh | 2 +-\n> ci/test-documentation.sh | 2 ++\n> 5 files changed, 10 insertions(+), 3 deletions(-)\n> create mode 100644 Documentation/manpage.xsl\n> \n> diff --git a/Documentation/Makefile b/Documentation/Makefile\n> index 76f2ecfc1b..d94f47c5c9 100644\n> --- a/Documentation/Makefile\n> +++ b/Documentation/Makefile\n> @@ -197,11 +197,13 @@ ifdef USE_ASCIIDOCTOR\n> ASCIIDOC = asciidoctor\n> ASCIIDOC_CONF =\n> ASCIIDOC_HTML = xhtml5\n> -ASCIIDOC_DOCBOOK = docbook45\n> +ASCIIDOC_DOCBOOK = docbook5\n> ASCIIDOC_EXTRA += -acompat-mode -atabsize=8\n> ASCIIDOC_EXTRA += -I. -rasciidoctor-extensions\n> ASCIIDOC_EXTRA += -alitdd='&\\#x2d;&\\#x2d;'\n> DBLATEX_COMMON =\n> +XMLTO_EXTRA += --skip-validation\n> +XMLTO_EXTRA += -x manpage.xsl\n> endif\n> \n> SHELL_PATH ?= $(SHELL)\n> diff --git a/Documentation/manpage.xsl b/Documentation/manpage.xsl\n> new file mode 100644\n> index 0000000000..ef64bab17a\n> --- /dev/null\n> +++ b/Documentation/manpage.xsl\n> @@ -0,0 +1,3 @@\n> +\n> +\t\n> +\n> diff --git a/azure-pipelines.yml b/azure-pipelines.yml\n> index c329b7218b..34031b182a 100644\n> --- a/azure-pipelines.yml\n> +++ b/azure-pipelines.yml\n> @@ -374,7 +374,7 @@ jobs:\n> test \"$GITFILESHAREPWD\" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare \"$GITFILESHAREPWD\" \"$HOME/test-cache\" || exit 1\n> \n> sudo apt-get update &&\n> - sudo apt-get install -y asciidoc xmlto asciidoctor &&\n> + sudo apt-get install -y asciidoc xmlto asciidoctor docbook-xsl-ns &&\n> \n> export ALREADY_HAVE_ASCIIDOCTOR=yes. &&\n> export jobname=Documentation &&\n> diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh\n> index 8cc72503cb..a76f348484 100755\n> --- a/ci/install-dependencies.sh\n> +++ b/ci/install-dependencies.sh\n> @@ -53,7 +53,7 @@ StaticAnalysis)\n> \t;;\n> Documentation)\n> \tsudo apt-get -q update\n> -\tsudo apt-get -q -y install asciidoc xmlto\n> +\tsudo apt-get -q -y install asciidoc xmlto docbook-xsl-ns\n\nOk, with this package installed the build passed on Travis CI.\n\n> \ttest -n \"$ALREADY_HAVE_ASCIIDOCTOR\" ||\n> \tgem install --version 1.5.8 asciidoctor\n\nSo, since the documentation can now be built with Asciidoctor v2, is\nit already time to remove this '--version 1.5.8'?\n\n> diff --git a/ci/test-documentation.sh b/ci/test-documentation.sh\n> index d49089832d..b3e76ef863 100755\n> --- a/ci/test-documentation.sh\n> +++ b/ci/test-documentation.sh\n> @@ -8,6 +8,8 @@\n> filter_log () {\n> \tsed -e '/^GIT_VERSION = /d' \\\n> \t -e '/^ \\* new asciidoc flags$/d' \\\n> +\t -e '/stripped namespace before processing/d' \\\n> +\t -e '/Attributed.*IDs for element/d' \\\n\nI haven't seen this latter message in the CI builds, neither with\nAsciidoctor v1.5.8 nor with v2. Do we really need this filter, then?\nWhere does this message come from?\n\n> \t \"$1\"\n> }\n> \n"},{"id":"382400","messageId":"20190915212621.GV11334@genre.crustytoothpaste.net","threadId":"50765","inReplyTo":"20190915095952.GA6190@szeder.dev","subject":"Re: [PATCH v3] Documentation: fix build with Asciidoctor 2","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2019-09-15T21:26:21Z","receivedAt":"2019-09-15T21:26:41Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2019-09-15 at 09:59:52, SZEDER Gábor wrote:\n> On Sat, Sep 14, 2019 at 07:49:19PM +0000, brian m. carlson wrote:\n> > \ttest -n \"$ALREADY_HAVE_ASCIIDOCTOR\" ||\n> > \tgem install --version 1.5.8 asciidoctor\n> \n> So, since the documentation can now be built with Asciidoctor v2, is\n> it already time to remove this '--version 1.5.8'?\n\nI think Martin was going to send in some more patches before we did\nthat.\n\n> > diff --git a/ci/test-documentation.sh b/ci/test-documentation.sh\n> > index d49089832d..b3e76ef863 100755\n> > --- a/ci/test-documentation.sh\n> > +++ b/ci/test-documentation.sh\n> > @@ -8,6 +8,8 @@\n> > filter_log () {\n> > \tsed -e '/^GIT_VERSION = /d' \\\n> > \t -e '/^ \\* new asciidoc flags$/d' \\\n> > +\t -e '/stripped namespace before processing/d' \\\n> > +\t -e '/Attributed.*IDs for element/d' \\\n> \n> I haven't seen this latter message in the CI builds, neither with\n> Asciidoctor v1.5.8 nor with v2. Do we really need this filter, then?\n> Where does this message come from?\n\nI see it and it definitely fails on my system without it. It comes from\nlibxslt, which has been patched in Debian to produce deterministic IDs.\nI suspect we may not have seen it on Ubuntu systems because they are\nrunning 16.04, which is likely older than the patch. If Travis updates\nto 18.04, we may be more likely to have a problem.\n-- \nbrian m. carlson: Houston, Texas, US\nOpenPGP: https://keybase.io/bk2204\n"},{"id":"382402","messageId":"20190915220555.GB6190@szeder.dev","threadId":"50765","inReplyTo":"20190915212621.GV11334@genre.crustytoothpaste.net","subject":"Re: [PATCH v3] Documentation: fix build with Asciidoctor 2","fromName":"SZEDER Gábor","fromEmail":"szeder.dev@gmail.com","sentAt":"2019-09-15T22:05:55Z","receivedAt":"2019-09-15T22:06:02Z","isPatch":true,"sender":{"key":"szeder.dev@gmail.com","avatar":"https://avatars.githubusercontent.com/u/116324?v=4"},"body":"On Sun, Sep 15, 2019 at 09:26:21PM +0000, brian m. carlson wrote:\n> > > diff --git a/ci/test-documentation.sh b/ci/test-documentation.sh\n> > > index d49089832d..b3e76ef863 100755\n> > > --- a/ci/test-documentation.sh\n> > > +++ b/ci/test-documentation.sh\n> > > @@ -8,6 +8,8 @@\n> > > filter_log () {\n> > > \tsed -e '/^GIT_VERSION = /d' \\\n> > > \t -e '/^ \\* new asciidoc flags$/d' \\\n> > > +\t -e '/stripped namespace before processing/d' \\\n> > > +\t -e '/Attributed.*IDs for element/d' \\\n> > \n> > I haven't seen this latter message in the CI builds, neither with\n> > Asciidoctor v1.5.8 nor with v2. Do we really need this filter, then?\n> > Where does this message come from?\n> \n> I see it and it definitely fails on my system without it. It comes from\n> libxslt, which has been patched in Debian to produce deterministic IDs.\n> I suspect we may not have seen it on Ubuntu systems because they are\n> running 16.04, which is likely older than the patch. If Travis updates\n> to 18.04, we may be more likely to have a problem.\n\nThanks. Indeed, I kicked off a Travis CI build using their Ubuntu\n18.04 image, and that \"Attributed...\" message was there.\n\nI think this future-proofing is a good idea, but I also think that\nthis should be clarified in the commit message.\n\n"},{"id":"382403","messageId":"20190915221420.GW11334@genre.crustytoothpaste.net","threadId":"50765","inReplyTo":"20190915220555.GB6190@szeder.dev","subject":"Re: [PATCH v3] Documentation: fix build with Asciidoctor 2","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2019-09-15T22:14:20Z","receivedAt":"2019-09-15T22:14:29Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2019-09-15 at 22:05:55, SZEDER Gábor wrote:\n> On Sun, Sep 15, 2019 at 09:26:21PM +0000, brian m. carlson wrote:\n> > > > diff --git a/ci/test-documentation.sh b/ci/test-documentation.sh\n> > > > index d49089832d..b3e76ef863 100755\n> > > > --- a/ci/test-documentation.sh\n> > > > +++ b/ci/test-documentation.sh\n> > > > @@ -8,6 +8,8 @@\n> > > > filter_log () {\n> > > > \tsed -e '/^GIT_VERSION = /d' \\\n> > > > \t -e '/^ \\* new asciidoc flags$/d' \\\n> > > > +\t -e '/stripped namespace before processing/d' \\\n> > > > +\t -e '/Attributed.*IDs for element/d' \\\n> > > \n> > > I haven't seen this latter message in the CI builds, neither with\n> > > Asciidoctor v1.5.8 nor with v2. Do we really need this filter, then?\n> > > Where does this message come from?\n> > \n> > I see it and it definitely fails on my system without it. It comes from\n> > libxslt, which has been patched in Debian to produce deterministic IDs.\n> > I suspect we may not have seen it on Ubuntu systems because they are\n> > running 16.04, which is likely older than the patch. If Travis updates\n> > to 18.04, we may be more likely to have a problem.\n> \n> Thanks. Indeed, I kicked off a Travis CI build using their Ubuntu\n> 18.04 image, and that \"Attributed...\" message was there.\n> \n> I think this future-proofing is a good idea, but I also think that\n> this should be clarified in the commit message.\n\nI can do that. I just noticed it failed on my laptop and added it,\nassuming it was the stylesheets. I had to search Google for the output\nto find out that it was libxslt.\n-- \nbrian m. carlson: Houston, Texas, US\nOpenPGP: https://keybase.io/bk2204\n"},{"id":"382405","messageId":"20190915224332.103930-1-sandals@crustytoothpaste.net","threadId":"50765","inReplyTo":"20190906232947.GJ11334@genre.crustytoothpaste.net","subject":"[PATCH v4] Documentation: fix build with Asciidoctor 2","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2019-09-15T22:43:32Z","receivedAt":"2019-09-15T22:43:47Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"Our documentation toolchain has traditionally been built around DocBook\n4.5. This version of DocBook is the last DTD-based version of DocBook.\nIn 2009, DocBook 5 was introduced using namespaces and its syntax is\nexpressed in RELAX NG, which is more expressive and allows a wider\nvariety of syntax forms.\n\nAsciidoctor, one of the alternatives for building our documentation,\nmoved support for DocBook 4.5 out of core in its recent 2.0 release and\nnow only supports DocBook 5 in the main release. The DocBoook 4.5\nconverter is still available as a separate component, but this is not\navailable in most distro packages. This would not be a problem but for\nthe fact that we use xmlto, which is still stuck in the DocBook 4.5 era.\n\nxmlto performs DTD validation as part of the build process. This is not\nproblematic for DocBook 4.5, which has a valid DTD, but it clearly\ncannot work for DocBook 5, since no DTD can adequately express its full\nsyntax. In addition, even if xmlto did support RELAX NG validation,\nthat wouldn't be sufficient because it uses the libxml2-based xmllint to\ndo so, which has known problems with validating interleaves in RELAX NG.\n\nFortunately, there's an easy way forward: ask Asciidoctor to use its\nDocBook 5 backend and tell xmlto to skip validation. Asciidoctor has\nsupported DocBook 5 since v0.1.4 in 2013 and xmlto has supported\nskipping validation for probably longer than that.\n\nWe also need to teach xmlto how to use the namespaced DocBook XSLT\nstylesheets instead of the non-namespaced ones it usually uses.\nNormally these stylesheets are interchangeable, but the non-namespaced\nones have a bug that causes them not to strip whitespace automatically\nfrom certain elements when namespaces are in use. This results in\nadditional whitespace at the beginning of list elements, which is\njarring and unsightly.\n\nWe can do this by passing a custom stylesheet with the -x option that\nsimply imports the namespaced stylesheets via a URL. Any system with\nsupport for XML catalogs will automatically look this URL up and\nreference a local copy instead without us having to know where this\nlocal copy is located. We know that anyone using xmlto will already\nhave catalogs set up properly since the DocBook 4.5 DTD used during\nvalidation is also looked up via catalogs. All major Linux\ndistributions distribute the necessary stylesheets and have built-in\ncatalog support, and Homebrew does as well, albeit with a requirement to\nset an environment variable to enable catalog support.\n\nOn the off chance that someone lacks support for catalogs, it is\npossible for xmlto (via xmllint) to download the stylesheets from the\nURLs in question, although this will likely perform poorly enough to\nattract attention. People still have the option of using the prebuilt\ndocumentation that we ship, so happily this should not be an impediment.\n\nFinally, we need to filter out some messages from other stylesheets that\noccur when invoking dblatex in the CI job. This tool strips namespaces\nmuch like the unnamespaced DocBook stylesheets and prints similar\nmessages. If we permit these messages to be printed to standard error,\nour documentation CI job will fail because we check standard error for\nunexpected output. Due to dblatex's reliance on Python 2, we may need\nto revisit its use in the future, in which case this problem may go\naway, but this can be delayed until a future patch.\n\nThe final message we filter is due to libxslt on modern Debian and\nUbuntu. The patch which they use to implement reproducible ID\ngeneration also prints messages about the ID generation. While this\ndoesn't affect our current CI images since they use Ubuntu 16.04 which\nlacks this patch, if we upgrade to Ubuntu 18.04 or a modern Debian,\nthese messages will appear and, like the above messages, cause a CI\nfailure.\n\nSigned-off-by: brian m. carlson \n---\nChanges since v3:\n* Further expanded the commit message to include information about\n libxslt. It is now just over a hundred words away from classification\n as \"sudden nonfiction\".\n\n Documentation/Makefile | 4 +++-\n Documentation/manpage.xsl | 3 +++\n azure-pipelines.yml | 2 +-\n ci/install-dependencies.sh | 2 +-\n ci/test-documentation.sh | 2 ++\n 5 files changed, 10 insertions(+), 3 deletions(-)\n create mode 100644 Documentation/manpage.xsl\n\ndiff --git a/Documentation/Makefile b/Documentation/Makefile\nindex 76f2ecfc1b..d94f47c5c9 100644\n--- a/Documentation/Makefile\n+++ b/Documentation/Makefile\n@@ -197,11 +197,13 @@ ifdef USE_ASCIIDOCTOR\n ASCIIDOC = asciidoctor\n ASCIIDOC_CONF =\n ASCIIDOC_HTML = xhtml5\n-ASCIIDOC_DOCBOOK = docbook45\n+ASCIIDOC_DOCBOOK = docbook5\n ASCIIDOC_EXTRA += -acompat-mode -atabsize=8\n ASCIIDOC_EXTRA += -I. -rasciidoctor-extensions\n ASCIIDOC_EXTRA += -alitdd='&\\#x2d;&\\#x2d;'\n DBLATEX_COMMON =\n+XMLTO_EXTRA += --skip-validation\n+XMLTO_EXTRA += -x manpage.xsl\n endif\n \n SHELL_PATH ?= $(SHELL)\ndiff --git a/Documentation/manpage.xsl b/Documentation/manpage.xsl\nnew file mode 100644\nindex 0000000000..ef64bab17a\n--- /dev/null\n+++ b/Documentation/manpage.xsl\n@@ -0,0 +1,3 @@\n+\n+\t\n+\ndiff --git a/azure-pipelines.yml b/azure-pipelines.yml\nindex c329b7218b..34031b182a 100644\n--- a/azure-pipelines.yml\n+++ b/azure-pipelines.yml\n@@ -374,7 +374,7 @@ jobs:\n test \"$GITFILESHAREPWD\" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare \"$GITFILESHAREPWD\" \"$HOME/test-cache\" || exit 1\n \n sudo apt-get update &&\n- sudo apt-get install -y asciidoc xmlto asciidoctor &&\n+ sudo apt-get install -y asciidoc xmlto asciidoctor docbook-xsl-ns &&\n \n export ALREADY_HAVE_ASCIIDOCTOR=yes. &&\n export jobname=Documentation &&\ndiff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh\nindex 8cc72503cb..a76f348484 100755\n--- a/ci/install-dependencies.sh\n+++ b/ci/install-dependencies.sh\n@@ -53,7 +53,7 @@ StaticAnalysis)\n \t;;\n Documentation)\n \tsudo apt-get -q update\n-\tsudo apt-get -q -y install asciidoc xmlto\n+\tsudo apt-get -q -y install asciidoc xmlto docbook-xsl-ns\n \n \ttest -n \"$ALREADY_HAVE_ASCIIDOCTOR\" ||\n \tgem install --version 1.5.8 asciidoctor\ndiff --git a/ci/test-documentation.sh b/ci/test-documentation.sh\nindex d49089832d..b3e76ef863 100755\n--- a/ci/test-documentation.sh\n+++ b/ci/test-documentation.sh\n@@ -8,6 +8,8 @@\n filter_log () {\n \tsed -e '/^GIT_VERSION = /d' \\\n \t -e '/^ \\* new asciidoc flags$/d' \\\n+\t -e '/stripped namespace before processing/d' \\\n+\t -e '/Attributed.*IDs for element/d' \\\n \t \"$1\"\n }\n \n"},{"id":"382415","messageId":"CAN0heSqcKR_HV3jt2vVU2iafA77cqTyjFaTmiyXaHChmmUBhgg@mail.gmail.com","threadId":"50765","inReplyTo":"20190913050634.GB21172@sigill.intra.peff.net","subject":"Re: [PATCH v2] Documentation: fix build with Asciidoctor 2","fromName":"Martin Ågren","fromEmail":"martin.agren@gmail.com","sentAt":"2019-09-16T10:47:46Z","receivedAt":"2019-09-16T10:48:03Z","isPatch":true,"sender":{"key":"martin.agren@gmail.com","avatar":null},"body":"On Fri, 13 Sep 2019 at 07:06, Jeff King wrote:\n>\n> On Fri, Sep 13, 2019 at 01:52:40AM +0000, brian m. carlson wrote:\n>\n>\n> > Documentation/Makefile | 4 +++-\n> > Documentation/manpage.xsl | 3 +++\n> > ci/test-documentation.sh | 2 ++\n> > 3 files changed, 8 insertions(+), 1 deletion(-)\n> > create mode 100644 Documentation/manpage.xsl\n>\n> Running with this patch on asciidoctor 2.0.10, plus Martin's recent\n> literal-block cleanups, plus his refmiscinfo fix, I get pretty decent\n> output from:\n>\n> ./doc-diff --from-asciidoc --to-asciidoctor origin HEAD\n>\n> The header/footer are still a little funny (but I think Martin said that\n> he needs to update the refmiscinfo patches for later versions of\n> asciidoctor, which is probably what's going on here):\n>\n> --- a/f1d4a28250629ae469fc5dd59ab843cb2fd68e12-asciidoc/home/peff/share/man/man1/git-add.1\n> +++ b/6c08635fd1d38c83d3765ff05fabbfbd25ef4943-asciidoctor/home/peff/share/man/man1/git-add.1\n> @@ -1,4 +1,4 @@\n> -GIT-ADD(1) Git Manual GIT-ADD(1)\n> +GIT-ADD(1) GIT-ADD(1)\n>\n> NAME\n> git-add - Add file contents to the index\n> @@ -356,4 +356,4 @@ SEE ALSO\n> GIT\n> Part of the git(1) suite\n>\n> -Git omitted 01/01/1970 GIT-ADD(1)\n> + omitted 1970-01-01 GIT-ADD(1)\n\nYeah, I should be able to post v3 of my refmiscinfo-series this evening,\nwhich should fix this, so that the only difference that remains here is\nhow the date is formatted.\n\nMartin\n"},{"id":"382416","messageId":"CAN0heSqWx09w_fVOBdaYCzz4Rse+sRPFLX1nu4iT9q0N5uzAmw@mail.gmail.com","threadId":"50765","inReplyTo":"20190915212621.GV11334@genre.crustytoothpaste.net","subject":"Re: [PATCH v3] Documentation: fix build with Asciidoctor 2","fromName":"Martin Ågren","fromEmail":"martin.agren@gmail.com","sentAt":"2019-09-16T10:51:55Z","receivedAt":"2019-09-16T10:52:10Z","isPatch":true,"sender":{"key":"martin.agren@gmail.com","avatar":null},"body":"On Sun, 15 Sep 2019 at 23:26, brian m. carlson\n wrote:\n>\n> On 2019-09-15 at 09:59:52, SZEDER Gábor wrote:\n> > On Sat, Sep 14, 2019 at 07:49:19PM +0000, brian m. carlson wrote:\n> > > test -n \"$ALREADY_HAVE_ASCIIDOCTOR\" ||\n> > > gem install --version 1.5.8 asciidoctor\n> >\n> > So, since the documentation can now be built with Asciidoctor v2, is\n> > it already time to remove this '--version 1.5.8'?\n>\n> I think Martin was going to send in some more patches before we did\n> that.\n\nI've got two series floating around [1] [2]. I'll be rerolling [2]\nhopefully tonight and then I feel pretty good about the manpages. I've\ngot a third series that I'll get to then, which fixes up the rendering\nof user-manual.pdf/html with Asciidoctor. Assuming those three series\ngraduate, I'm not aware of any reason to hold off on the switch to\n\"Asciidoctor by default\".\n\nAs for \"v2.x vs 1.5.8\", that seems like a separate issue to me, though\n-- and one that your patch does a very good job at! My series [2] will\nmake the rendering with v2.x /prettier/, but since your series makes the\ndocs build /at all/, maybe it's worth switching the CI-job sooner rather\nthan later to make sure we keep it so.\n\n[1] https://public-inbox.org/git/cover.1567707999.git.martin.agren@gmail.com/\n\n[2] https://public-inbox.org/git/cover.1567534373.git.martin.agren@gmail.com/\n\nMartin\n"},{"id":"382431","messageId":"xmqqd0g0b0gz.fsf@gitster-ct.c.googlers.com","threadId":"50765","inReplyTo":"CAN0heSqcKR_HV3jt2vVU2iafA77cqTyjFaTmiyXaHChmmUBhgg@mail.gmail.com","subject":"Re: [PATCH v2] Documentation: fix build with Asciidoctor 2","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2019-09-16T17:43:56Z","receivedAt":"2019-09-16T17:44:05Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Martin Ågren writes:\n\n>> -Git omitted 01/01/1970 GIT-ADD(1)\n>> + omitted 1970-01-01 GIT-ADD(1)\n>\n> Yeah, I should be able to post v3 of my refmiscinfo-series this evening,\n> which should fix this, so that the only difference that remains here is\n> how the date is formatted.\n\nThanks.\n"},{"id":"382439","messageId":"cover.1568658269.git.martin.agren@gmail.com","threadId":"50765","inReplyTo":"cover.1567534373.git.martin.agren@gmail.com","subject":"[PATCH v3 0/3] asciidoctor-extensions: provide ``","fromName":"Martin Ågren","fromEmail":"martin.agren@gmail.com","sentAt":"2019-09-16T19:00:25Z","receivedAt":"2019-09-16T19:00:50Z","isPatch":true,"sender":{"key":"martin.agren@gmail.com","avatar":null},"body":"Here's v3 of ma/asciidoctor-refmiscinfo to align the headers and footers\nof Asciidoctor-rendered manpages with those from AsciiDoc. Patch 1 is\nnew.\n\nIt turns out that with newer (>=1.5.7) versions of Asciidoctor, we get\ndefault values of these refmiscinfo attributes put into the xml. The\nhack in patch 2/3 (previously patch 1/2) happened to inject these\nattributes after the default ones and it seems like the earlier ones\nhappen to win when xmlto processes the xml. This explains what Peff\nnoticed in [1], where the attribute values introduced by the hacky patch\n2 are effectively ignored and blank values are used instead.\n\nRather than trying to game the order of those attributes, I decided to\ndo a patch 1 which uses a saner way of providing these attributes to\nAsciidoctor. This allowed me to reframe patch 2 from \"fix all of these\nattributes with Asciidoctor\" to \"fix some of them with some Asciidoctor\nversions\".\n\nMy test results:\n\n AsciiDoc (8.6.10): Each patch in this series is a no-op.\n\n Asciidoctor 1.5.5 and 1.5.6: Patch 1/3 is a no-op. Patch 2/3 fixes the\n header/footer to be as with AsciiDoc, except for how the date is\n rendered.\n\n Aciidoctor 1.5.7 and 2.0.10 [2]: Patch 1/3 fixes the header/footer\n partially -- the version number is still missing. Patch 2/3 fixes the\n version number, after which the header/footer are just as with\n AsciiDoc, except for how the date is rendered.\n\n So in short, \"all\" versions of Asciidoctor now render the headers and\n footers like AsciiDoc does, except for the date formatting.\n\n[1] https://public-inbox.org/git/20190913050634.GB21172@sigill.intra.peff.net/\n\n[2] The tests with 2.0.10 were done on top of brian's recent patch to\nbuild at all with Asciidoctor 2:\nhttps://public-inbox.org/git/20190915224332.103930-1-sandals@crustytoothpaste.net/\n\nMartin Ågren (3):\n Doc/Makefile: give mansource/-version/-manual attributes\n asciidoctor-extensions: provide ``\n doc-diff: replace --cut-header-footer with --cut-footer\n\n Documentation/Makefile | 3 ++-\n Documentation/asciidoc.conf | 6 +++---\n Documentation/asciidoctor-extensions.rb | 17 +++++++++++++++++\n Documentation/doc-diff | 17 ++++++++---------\n 4 files changed, 30 insertions(+), 13 deletions(-)\n\n-- \n2.23.0\n\n"},{"id":"382440","messageId":"bf80b9a66f23a287c447509a33dbb2ec97ac2a64.1568658269.git.martin.agren@gmail.com","threadId":"50765","inReplyTo":"cover.1568658269.git.martin.agren@gmail.com","subject":"[PATCH v3 1/3] Doc/Makefile: give mansource/-version/-manual attributes","fromName":"Martin Ågren","fromEmail":"martin.agren@gmail.com","sentAt":"2019-09-16T19:00:26Z","receivedAt":"2019-09-16T19:00:53Z","isPatch":true,"sender":{"key":"martin.agren@gmail.com","avatar":null},"body":"Rather than hardcoding \"Git Manual\" and \"Git\" as the manual and source\nin asciidoc.conf, provide them as attributes `manmanual` and\n`mansource`. Rename the `git_version` attribute to `manversion`.\n\nThese new attribute names are not arbitrary, see, e.g., [1].\n\nFor AsciiDoc (8.6.10) and Asciidoctor <1.5.7, this is a no-op. Starting\nwith Asciidoctor 1.5.7, `manmanual` and `mansource` actually end up in\nthe xml-files and eventually in the rendered manpages. In particular,\nthe manpage headers now render just as with AsciiDoc.\n\nNo versions of Asciidoctor pick up the `manversion` [2], and older\nversions don't pick up any of these attributes. -- We'll fix that with a\nbit of a hack in the next commit.\n\n[1] https://asciidoctor.org/docs/user-manual/#man-pages\n\n[2] Note how [1] says \"Not used by Asciidoctor\".\n\nSigned-off-by: Martin Ågren \n---\n Documentation/Makefile | 3 ++-\n Documentation/asciidoc.conf | 6 +++---\n 2 files changed, 5 insertions(+), 4 deletions(-)\n\ndiff --git a/Documentation/Makefile b/Documentation/Makefile\nindex dbf5a0f276..a67bc04ae8 100644\n--- a/Documentation/Makefile\n+++ b/Documentation/Makefile\n@@ -122,7 +122,8 @@ ASCIIDOC_HTML = xhtml11\n ASCIIDOC_DOCBOOK = docbook\n ASCIIDOC_CONF = -f asciidoc.conf\n ASCIIDOC_COMMON = $(ASCIIDOC) $(ASCIIDOC_EXTRA) $(ASCIIDOC_CONF) \\\n-\t\t-agit_version=$(GIT_VERSION)\n+\t\t-amanversion=$(GIT_VERSION) \\\n+\t\t-amanmanual='Git Manual' -amansource='Git'\n TXT_TO_HTML = $(ASCIIDOC_COMMON) -b $(ASCIIDOC_HTML)\n TXT_TO_XML = $(ASCIIDOC_COMMON) -b $(ASCIIDOC_DOCBOOK)\n MANPAGE_XSL = manpage-normal.xsl\ndiff --git a/Documentation/asciidoc.conf b/Documentation/asciidoc.conf\nindex 2c16c536ba..8fc4b67081 100644\n--- a/Documentation/asciidoc.conf\n+++ b/Documentation/asciidoc.conf\n@@ -78,9 +78,9 @@ template::[header-declarations]\n \n {mantitle}\n {manvolnum}\n-Git\n-{git_version}\n-Git Manual\n+{mansource}\n+{manversion}\n+{manmanual}\n \n \n {manname}\n-- \n2.23.0\n\n"},{"id":"382441","messageId":"4eb89fda71a647bd44aa1d308674e7694db7b17d.1568658269.git.martin.agren@gmail.com","threadId":"50765","inReplyTo":"cover.1568658269.git.martin.agren@gmail.com","subject":"[PATCH v3 2/3] asciidoctor-extensions: provide ``","fromName":"Martin Ågren","fromEmail":"martin.agren@gmail.com","sentAt":"2019-09-16T19:00:27Z","receivedAt":"2019-09-16T19:00:58Z","isPatch":true,"sender":{"key":"martin.agren@gmail.com","avatar":null},"body":"As can be seen from the previous commit, there are three attributes that\nwe provide to AsciiDoc through asciidoc.conf. Asciidoctor ignores that\nfile. After that patch, newer versions of Asciidoctor pick up the\n`manmanual` and `mansource` attributes as we invoke `asciidoctor`, but\nthey don't pick up `manversion`. ([1] says: \"Not used by Asciidoctor.\")\nOlder versions (<1.5.7) don't handle these attributes at all. As a\nresult, we are missing one or three `` tags in each\nxml-file produced when we build with Asciidoctor.\n\nBecause of this, xmlto doesn't include the Git version number in the\nrendered manpages. And in particular, with versions <1.5.7, the manpage\nfooters instead contain the fairly ugly \"[FIXME: source]\".\n\nThat Asciidoctor ignores asciidoc.conf is nothing new. This is why we\nimplement the `linkgit:` macro in asciidoc.conf *and* in\nasciidoctor-extensions.rb. Follow suit and provide these tags in\nasciidoctor-extensions.rb, using a \"postprocessor\" extension where we\njust search and replace in the XML, treated as text.\n\nWe may consider a few alternatives:\n\n * Inject these lines into the xml-files from the *Makefile*, e.g.,\n using `sed`. That would reduce repetition, but it feels wrong to\n impose another step and another risk on the AsciiDoc-processing only\n to benefit the Asciidoctor-one.\n\n * I tried providing a \"docinfo processor\" to inject these tags, but\n could not figure out how to \"merge\" the two sections that\n resulted. To avoid xmlto barfing on the result, I needed to use\n `xmlto --skip-validation ...`, which seems unfortunate.\n\nLet's instead inject the missing tags using a postprocessor. We'll make\nit fairly obvious that we aim to inject the exact same three lines of\n`` that asciidoc.conf provides. We inject them in\n*post*-processing so we need to do the variable expansion ourselves. We\ndo introduce the bug that asciidoc.conf already has in that we won't do\nany escaping, e.g., of funky versions like \"some v <2.25, >2.20\".\n\nThe postprocessor we add here works on the XML as raw text and doesn't\nreally use the full potential of XML to do a more structured injection.\nThis is actually precisely what the Asciidoctor User Manual does in its\npostprocessor example [2]. I looked into two other approaches:\n\n 1. The nokogiri library is apparently the \"modern\" way of doing XML\n in ruby. I got it working fairly easily:\n require 'nokogiri'\n doc = Nokogiri::XML(output)\n doc.search(\"refmeta\").each { |n| n.add_child(new_tags) }\n output = doc.to_xml\n However, this adds another dependency (e.g., the \"ruby-nokogiri\"\n package on Ubuntu). Using Asciidoctor is not our default, but it\n will probably need to become so soon. Let's avoid adding a\n dependency just so that we can say \"search...add_child\" rather than\n \"sub(regex...)\".\n\n 2. The older REXML is apparently always(?) bundled with ruby, but I\n couldn't even parse the original document:\n require 'rexml/document'\n doc = REXML::Document.new(output)\n ...\n The error was \"no implicit conversion of nil into String\" and I\n stopped there.\n\nI don't think it's unlikely that doing a plain old search-and-replace\nwill work just as fine or better compared to parsing XML and worrying\nabout libraries and library versions.\n\n[1] https://asciidoctor.org/docs/user-manual/#builtin-attributes\n\n[2] https://asciidoctor.org/docs/user-manual/#postprocessor-example\n\nSigned-off-by: Martin Ågren \n---\n Documentation/asciidoctor-extensions.rb | 17 +++++++++++++++++\n 1 file changed, 17 insertions(+)\n\ndiff --git a/Documentation/asciidoctor-extensions.rb b/Documentation/asciidoctor-extensions.rb\nindex 0089e0cfb8..85f14c7c11 100644\n--- a/Documentation/asciidoctor-extensions.rb\n+++ b/Documentation/asciidoctor-extensions.rb\n@@ -20,9 +20,26 @@ module Git\n end\n end\n end\n+\n+ class DocumentPostProcessor < Asciidoctor::Extensions::Postprocessor\n+ def process document, output\n+ if document.basebackend? 'docbook'\n+ mansource = document.attributes['mansource']\n+ manversion = document.attributes['manversion']\n+ manmanual = document.attributes['manmanual']\n+ new_tags = \"\" \\\n+ \"#{mansource}\\n\" \\\n+ \"#{manversion}\\n\" \\\n+ \"#{manmanual}\\n\"\n+ output = output.sub(/<\\/refmeta>/, new_tags + \"\")\n+ end\n+ output\n+ end\n+ end\n end\n end\n \n Asciidoctor::Extensions.register do\n inline_macro Git::Documentation::LinkGitProcessor, :linkgit\n+ postprocessor Git::Documentation::DocumentPostProcessor\n end\n-- \n2.23.0\n\n"},{"id":"382442","messageId":"e5a2e78886d9d23bcd25c03b461ec19b29979c6d.1568658269.git.martin.agren@gmail.com","threadId":"50765","inReplyTo":"cover.1568658269.git.martin.agren@gmail.com","subject":"[PATCH v3 3/3] doc-diff: replace --cut-header-footer with --cut-footer","fromName":"Martin Ågren","fromEmail":"martin.agren@gmail.com","sentAt":"2019-09-16T19:00:28Z","receivedAt":"2019-09-16T19:00:59Z","isPatch":true,"sender":{"key":"martin.agren@gmail.com","avatar":null},"body":"After the previous commit, AsciiDoc and Asciidoctor render the manpage\nheaders identically, so we no longer need the \"cut the header\" part of\nour `--cut-header-footer` option. We do still need the \"cut the footer\"\npart, though. The previous commits improved the rendering of the footer\nin Asciidoctor by quite a bit, but the two programs still disagree on\nhow to format the date in the footer: 01/01/1970 vs 1970-01-01.\n\nWe could keep using `--cut-header-footer`, but it would be nice if we\nhad a slightly smaller hammer `--cut-footer` that would be less likely\nto hide regressions. Rather than simply adding such an option, let's\nalso drop `--cut-header-footer`, i.e., rework it to lose the \"header\"\npart of its name and functionality.\n\n`--cut-header-footer` is just a developer tool and it probably has no\nmore than a handful of users, so we can afford to be aggressive.\n\nSigned-off-by: Martin Ågren \n---\n Documentation/doc-diff | 17 ++++++++---------\n 1 file changed, 8 insertions(+), 9 deletions(-)\n\ndiff --git a/Documentation/doc-diff b/Documentation/doc-diff\nindex 3355be4798..88a9b20168 100755\n--- a/Documentation/doc-diff\n+++ b/Documentation/doc-diff\n@@ -21,7 +21,7 @@ asciidoc\t\tuse asciidoc with both commits\n to-asciidoc\t\tuse asciidoc with the 'to'-commit\n to-asciidoctor\t\tuse asciidoctor with the 'to'-commit\n asciidoctor\t\tuse asciidoctor with both commits\n-cut-header-footer\tcut away header and footer\n+cut-footer\t\tcut away footer\n \"\n SUBDIRECTORY_OK=1\n . \"$(git --exec-path)/git-sh-setup\"\n@@ -31,7 +31,7 @@ force=\n clean=\n from_program=\n to_program=\n-cut_header_footer=\n+cut_footer=\n while test $# -gt 0\n do\n \tcase \"$1\" in\n@@ -55,8 +55,8 @@ do\n \t--asciidoc)\n \t\tfrom_program=-asciidoc\n \t\tto_program=-asciidoc ;;\n-\t--cut-header-footer)\n-\t\tcut_header_footer=-cut-header-footer ;;\n+\t--cut-footer)\n+\t\tcut_footer=-cut-footer ;;\n \t--)\n \t\tshift; break ;;\n \t*)\n@@ -118,8 +118,8 @@ construct_makemanflags () {\n from_makemanflags=$(construct_makemanflags \"$from_program\") &&\n to_makemanflags=$(construct_makemanflags \"$to_program\") &&\n \n-from_dir=$from_oid$from_program$cut_header_footer &&\n-to_dir=$to_oid$to_program$cut_header_footer &&\n+from_dir=$from_oid$from_program$cut_footer &&\n+to_dir=$to_oid$to_program$cut_footer &&\n \n # generate_render_makefile \n generate_render_makefile () {\n@@ -169,12 +169,11 @@ render_tree () {\n \t\tmake -j$parallel -f - &&\n \t\tmv \"$tmp/rendered/$dname+\" \"$tmp/rendered/$dname\"\n \n-\t\tif test \"$cut_header_footer\" = \"-cut-header-footer\"\n+\t\tif test \"$cut_footer\" = \"-cut-footer\"\n \t\tthen\n \t\t\tfor f in $(find \"$tmp/rendered/$dname\" -type f)\n \t\t\tdo\n-\t\t\t\ttail -n +3 \"$f\" | head -n -2 |\n-\t\t\t\tsed -e '1{/^$/d}' -e '${/^$/d}' >\"$f+\" &&\n+\t\t\t\thead -n -2 \"$f\" | sed -e '${/^$/d}' >\"$f+\" &&\n \t\t\t\tmv \"$f+\" \"$f\" ||\n \t\t\t\treturn 1\n \t\t\tdone\n-- \n2.23.0\n\n"}]}