patch, 4 partsDocumentation: implement linkgit macro for Asciidoctor

This series is designed to implement the changes necessary to build Git using Asciidoctor instead of AsciiDoc.

The first two patches are bug fixes. Asciidoctor is stricter about title underline lengths (± 1 character instead of 2) and requires matching delimiter lengths[0]. They're needed regardless of whether the other two patches are accepted because git-scm.com uses Asciidoctor to render the documentation, so we might as well render it correctly.

Even with these patches, Asciidoctor warns about everyday.txt and user-manual.txt. I'm not sending patches for these right now because I've seen recent series including those and don't want to cause a merge conflict.

The second two patches implement some basic support for building with Asciidoctor. The first of these moves some items into variables due to some differences between the AsciiDoc and Asciidoctor command lines. The user can then override these values when invoking make.

The final patch adds support for the linkgit macro. Asciidoctor uses Ruby extensions to implement macro support, unlike AsciiDoc, which uses a configuration file.

In order to build the documentation successfully, the patches from [1] are required. The current version of Asciidoctor uses the wrong elements in some cases and is lacking elements in others when generating manpage (refentry) output in DocBook format. The need for these patches is also the reason I didn't implement a configuration macro for Asciidoctor, as almost nobody will be able to use it at the moment.

With these patches and the ones for Asciidoctor, it's possible to build the documentation with the following command line:

  make doc ASCIIDOC=asciidoctor ASCIIDOC_DOCBOOK=docbook45 ASCIIDOC_HTML=html5 ASCIIDOC_CONF='-I. -rasciidoctor/extensions -rextensions'

brian m. carlson (4):
  Documentation: adjust document title underlining
  Documentation: fix mismatched delimiters in git-imap-send
  Documentation: move some AsciiDoc parameters into variables
  Documentation: implement linkgit macro for Asciidoctor

 Documentation/Makefile             | 18 +++++++++++-------
 Documentation/extensions.rb        | 39 ++++++++++++++++++++++++++++++++++++++
 Documentation/git-imap-send.txt    |  4 ++--
 Documentation/git-prune-packed.txt |  2 +-
 Documentation/git-quiltimport.txt  |  2 +-
 Documentation/git-stage.txt        |  2 +-
 6 files changed, 55 insertions(+), 12 deletions(-)
 create mode 100644 Documentation/extensions.rb

[0] http://asciidoctor.org/docs/asciidoc-asciidoctor-diffs/ [1] https://github.com/asciidoctor/asciidoctor/pull/1142

AsciiDoc specification states that in two-line titles, the title underline has to be the same length as the title text, plus or minus two characters. Asciidoctor, however, requires that this must be plus or minus one character. Adjust the underlines to be the same length as the title text to improve compatibility.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 Documentation/git-prune-packed.txt | 2 +-
 Documentation/git-quiltimport.txt  | 2 +-
 Documentation/git-stage.txt        | 2 +-
 3 files changed, 3 insertions(+), 3 deletions(-)

Thanks, obviously correct.

The documentation for git-imap-send uses block delimiters with mismatched lengths, which Asciidoctor doesn't support. As a result, the page is misrendered. Adjust the delimiters so that they are of the same length.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 Documentation/git-imap-send.txt | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

Asciidoctor takes slightly different arguments from AsciiDoc in some cases. It has a different name for the HTML backend and the "docbook" backend produces DocBook 5, not DocBook 4.5. Also, Asciidoctor does not accept the -f option. Move these values into variables so that they can be overridden by users wishing to use Asciidoctor instead of Asciidoc.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 Documentation/Makefile | 18 +++++++++++-------
 1 file changed, 11 insertions(+), 7 deletions(-)

"brian m. carlson" <sandals@crustytoothpaste.net> writes:

I think it makes sense to make these customizable, but I wonder if it makes the result easier to maintain if we make units of logical definitions larger, e.g.

	ASCIIDOC = asciidoc
        TXT_TO_MANHTML = $(ASCIIDOC) -b xhtml11 -d manpage $(ASCIIDOC_CONF)
        TXT_TO_ARTICLE = $(ASCIIDOC) -b docbook -d article
	...

ASCIIDOC_EXTRA may want to apply all of them, even though I see that we do not feed it to OBSOLETE_HTML right now. It may also be that $(ASCIIDOC_CONF) and -agit_version=$(GIT_VERSION) could be shared among the ones that currently do not have.

Then the above would become something like:

	ASCIIDOC = asciidoc
	ASCIIDOC_COMMON = $(ASCIIDOC) \
        	$(ASCIIDOC_EXTRA) $(ASCIIDOC_CONF) -agit_version=$(GIT_VERSION)
        TXT_TO_MANHTML = $(ASCIIDOC_COMMON) -b xhtml11 -d manpage
        ...

and would further simplify this part

into just

	$(TXT_TO_MANHTML) -o $@+ $<

After all, our output formats are fairly limited, I would think. Are there too many different variants and exceptions to make such an approach infeasible?

On Wed, Oct 15, 2014 at 01:43:49PM -0700, Junio C Hamano wrote:

Looking at the code, it seems the most reusable unit is something like the following:

  TXT_TO_HTML = $(ASCIIDOC) -b xhtml11 $(ASCIIDOC_CONF)
  TXT_TO_XML = $(ASCIIDOC) -b docbook $(ASCIIDOC_CONF)

that is, omitting the -d argument from the variable. Using -d would mean we'd have to have a variable for each of the seven locations we call $(ASCIIDOC).

I'm on board with the $(ASCIIDOC_COMMON) idea, but to minimize the number of variables, I think we should leave the -d out of the macro itself. I'll re-roll over the next couple of days.

AsciiDoc uses a configuration file to implement macros like linkgit, while Asciidoctor uses Ruby extensions. Implement a Ruby extension that implements the linkgit macro for Asciidoctor in the same way that asciidoc.conf does for AsciiDoc.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 Documentation/extensions.rb | 39 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 39 insertions(+)
 create mode 100644 Documentation/extensions.rb

"brian m. carlson" <sandals@crustytoothpaste.net> writes:

Sounds good.

What I do not understand is that 3/4 lets you drop inclusion of asciidoc.conf which contains a lot more than just linkgit: definition.

For now I'll queue only the first two, which unquestionably take us in the right direction.

Thanks.

On Mon, Oct 13, 2014 at 01:41:31PM -0700, Junio C Hamano wrote:

Asciidoctor just doesn't understand the -f argument, so trying to pass it is going to fail. For Asciidoctor, you're going to want to do something like "-I. -rasciidoctor/extensions -rextensions" there instead.

As for the rest of the asciidoc.conf file, the DocBook manpage header declarations are implemented automatically by Asciidoctor after my recent patches. The paragraph hacks do not appear to be necessary with Asciidoctor, so they've been omitted.

That leaves the attributes. All but litdd are built-in to Asciidoctor, and I can reroll with a modification to extensions.rb that implements that one.

brian m. carlson wrote:

Would it be possible to automatically convert asciidoc.conf file to Asciidoctor extension?

On Tue, Oct 14, 2014 at 12:07:51PM +0200, Jakub Narębski wrote:

It is in theory possible, but it's going to result in a lot of messy code. I'm also not sure that Junio wants more than the minimal amount of Ruby possible, since the goal has been to move away from scripting languages and to C.

On Sat, Oct 11, 2014 at 11:37:32PM +0000, brian m. carlson wrote:

Thanks. I had always taken the attitude that we wrote for the original Python AsciiDoc, and that using AsciiDoctor was a choice that git-scm.com made, and something they would have to deal with as far as compatibility (AFAIK, AsciiDoctor grew out of git-scm.com's home-grown asciidoc parser).

What's the status on AsciiDoc versus AsciiDoctor? The latter seems more actively developed these days, but perhaps that is just my perception. The incompatibilities seem fairly minimal (if those first two patches are the extent of it, I have no problem at all trying to remain compatible with both). Would it ever make sense to switch to AsciiDoctor as our official command-line build program? I know it is supposed to be much faster (though a lot of the slowness in our build chain is due to docbook, not asciidoc itself).

Specifically I'm not excited about getting into a state where we have to maintain both an asciidoc.conf file _and_ ruby extensions for asciidoctor. I don't mind if somebody wants to step up and keep the asciidoctor bits in sync with the asciidoc.conf, but I feel like one of them needs to be considered the "master".

-Peff

Jeff King <peff@peff.net> writes:

My so-far-unstated inclination, since seeing the patch to fix the unbalanced example block separators from Brian (which was outside and before this four-patch series), has been to keep our Makefile in Documentation/ aware only of AsciiDoc while maintaining *.txt files in a state so that AsciiDoctor could also be used to process them, if people want to futz with their copies of Documentation/Makefile.

I do not mind to have the machinery to run AsciiDoctor too much in my tree. It may make it easier for those who use it to spot places in *.txt that need (in)compatibility workarounds between the two formatters than keeping it outside.

But somebody needs to maintain that machinery and that will not be me.

Thanks.

On Tue, Oct 14, 2014 at 10:08:19AM -0700, Junio C Hamano wrote:

I'd be happy if you simply picked up patch 3 and left out patch 4. It gets us most of the way there, which is good enough for most things. It's even possible to handle the litdd attribute on the command line, so the only thing we'd really lose is the linkgit links.

Alternately, I'm happy to be responsible for maintaining the extensions.rb file. The asciidoc.conf file has not had a substantive (non-comment) change since 2012, and it has not had a change that would require an update to the extensions since 2010. I don't anticipate that keeping it up-to-date will require a significant amount of work. We can even drop it into contrib if you think that's a better place.

It's really up to you which you'd prefer.

"brian m. carlson" <sandals@crustytoothpaste.net> writes:

Let's see how well the patches 3 and 4 work for other people with AsciiDoctor and then decide to go in that direction.

I do not forsee that changes to allow our *.txt to be used with AsciiDoctor interfere with what GitHub folks do with their own documentation toolchain, but I am not sure how the AsciiDoctor specific alternative build infrastructure we would eventually ship would interact with them---maybe they are not affected at all, or maybe they can even take advantage of it.

Thanks.

Am 14.10.2014 um 11:51 schrieb Jeff King:

Just recently we added the AsciiDoc toolchain to our git-for-windows/sdk (formerly known as msysgit). So I'm not really fond of switching now to something different again.

Remaining compatible with both would therefore be my choice.

Thomas

On Wed, Oct 15, 2014 at 01:24:22PM +0200, Thomas Braun wrote:

I don't think there's a lot of benefit for us to switch, and I say that being a contributor to Asciidoctor. It's useful to be able to build Git with both simply to find incompatibilities that we're going to need to fix anyway, due to the fact that Asciidoctor is used for the website.

And yes, those first two patches are it, as far as I'm aware.

That's my goal. I simply wanted the ability to support both AsciiDoc and Asciidoctor without making major changes to the codebase. Hence, moving the calls into variables.

From: "brian m. carlson" <sandals@crustytoothpaste.net>

[..]

Does the new version for giteveryday.txt and everyday.txt which graduated to master, 1cb3324 (Merge branch 'po/everyday-doc', 2014-10-16) format OK?

i.e. does 'git help everyday' now correct the Asciidoctor warnings.

I don't have access to Asciidoctor (on MsysGit), but did make sure the header underlines were updated. -- Philip