WIP: asciidoc replacement

Hi,

I do not want to depend on more than necessary in msysGit, and therefore I started to write an asciidoc replacement.

So here it is: a perl script that does a good job on many .txt files in Documentation/, although for some it deviates from "make man"'s output, and for others it is outright broken. It is meant to be run in Documentation/.

My intention is not to fix the script for all cases, but to make patches to Documentation/*.txt themselves, so that they are more consistent (and incidentally nicer to the script).

Now, I hear you already moan: "But Dscho, you know you suck at Perl!"

Yeah, I know, but maybe instead of bashing on me (pun intended), you may want to enlighten me with tips how to make it nicer to read. (Yes, there are no comments; yes, I will gladly add them where appropriate; yes, html is just a stub.)

So here, without further ado, da script:

-- snip -- #!/usr/bin/perl

$conv = new man_page(); $conv->{manual} = 'Git Manual'; $conv->{git_version} = 'Git ' . `cat ../GIT-VERSION-FILE`; $conv->{git_version} =~ s/GIT_VERSION = //; $conv->{git_version} =~ s/-/\\-/; $conv->{git_version} =~ s/\n//; $conv->{date} = `date +%m/%d/%Y`; $conv->{date} =~ s/\n//;

$par = ''; handle_file($ARGV[0]); $conv->finish();

sub handle_text {
	if ($par =~ /^\. /s) {
		my @lines = split(/^\. /m, $par);
		shift @lines;
		$conv->enumeration(\@lines);
	} elsif ($par =~ /^\* /s) {
		my @lines = split(/^\* /m, $par);
		shift @lines;
		$conv->enumeration(\@lines, 'unnumbered');
	} elsif ($par =~ /^\[verse\]/) {
		$par =~ s/\[verse\] *\n?//;
		$conv->verse($par);
	} elsif ($par =~ /^(\t|  +)/s) {
		$par =~ s/^$1//mg;
		$par =~ s/^\+$//mg;
		$conv->indent($par);
	} elsif ($par =~ /^([^\n]*)::\n((\t|  +).*)$/s) {
		my ($first, $rest, $indent) = ($1, $2, $3);
		$rest =~ s/^\+$//mg;
		while ($rest =~ /^(.*?\n\n)--+\n(.*?\n)--+\n\n(.*)$/s) {
			my ($pre, $verb, $post) = ($1, $2, $3);

			$pre =~ s/^(\t|$indent)//mg;
			if ($first ne '') {
				$conv->begin_item($first, $pre);
				$first = '';
			} else {
				$conv->normal($pre);
			}

			$conv->verbatim($verb);
			$rest = $post;
		}
		$rest =~ s/^(\t|$indent)//mg;
		if ($first ne '') {
			$conv->begin_item($first, $rest);
		} else {
			$conv->normal($rest);
		}
		$conv->end_item();
	} elsif ($par =~ /^-+\n(.*\n)-+\n$/s) {
		$conv->verbatim($1);
	} else {
		$conv->normal($par);
	}
	$par = '';
}

sub handle_file {
	my $in;
	open($in, '<' . $_[0]);
	while (<$in>) {
		if (/^=+$/) {
			if ($par ne '' && length($_) >= length($par)) {
				$conv->header($par);
				$par = '';
				next;
			}
		} elsif (/^-+$/) {
			if ($par ne '' && length($_) >= length($par)) {
				$conv->section($par);
				$par = '';
				next;
			}
		} elsif (/^~+$/) {
			if ($par ne '' && length($_) >= length($par)) {
				$conv->subsection($par);
				$par = '';
				next;
			}
		} elsif (/^\[\[(.*)\]\]$/) {
			handle_text();
			$conv->anchor($1);
			next;
		} elsif (/^$/) {
			if ($par =~ /^-+\n.*[^-]\n$/s) {
				# fallthru; is verbatim, but needs more.
			} elsif ($par =~ /::\n$/s) {
				# is item, but needs more.
				next;
			} else {
				handle_text();
				next;
			}
		} elsif (/^include::(.*)\[\]$/) {
			handle_text();
			handle_file($1);
			next;
		}

		# convert "\--" to "--"
		s/\\--/--/g;
		# convert "\*" to "*"
		s/\\\*/*/g;

		# handle gitlink:
		s/gitlink:([^\[ ]*)\[(\d+)\]/sprintf "%s",
			$conv->get_link($1, $2)/ge;
		# handle link:
		s/link:([^\[ ]*)\[(.+)\]/sprintf "%s",
			$conv->get_link($1, $2, 'external')/ge;

		$par .= $_;
	}
	close($in);
	handle_text();
}

package man_page;

sub new {
	my ($class) = @_;
	my $self = {
		sep => '',
		links => [],
#		generator => 'Home grown git txt2man converter'
		generator => 'DocBook XSL Stylesheets v1.71.1 <http://docbook.sf.net/>'
	};
	bless $self, $class;
	return $self;
}

sub header {
	my ($self, $text) = @_;
	$text =~ s/-/\\-/g;

	if ($self->{preamble_shown} == undef) {
		$title = $text;
		$title =~ s/\(\d+\)$//;
		print '.\"     Title: ' . $title
			. '.\"    Author: ' . "\n"
			. '.\" Generator: ' . $self->{generator} . "\n"
			. '.\"      Date: ' . $self->{date} . "\n"
			. '.\"    Manual: ' . $self->{manual} . "\n"
			. '.\"    Source: ' . $self->{git_version} . "\n"
			. '.\"' . "\n";
	}

	$text =~ tr/a-z/A-Z/;
	my $suffix = "\"$self->{date}\" \"$self->{git_version}\""
		. " \"$self->{manual}\"";
	$text =~ s/^(.*)\((\d+)\)$/.TH "\1" "\2" $suffix/;
	print $text;

	if ($self->{preamble_shown} == undef) {
		print '.\" disable hyphenation' . "\n"
			. '.nh' . "\n"
			. '.\" disable justification (adjust text to left'
				. ' margin only)' . "\n"
			. '.ad l' . "\n";
		$self->{preamble_shown} = 1;
	}

	$self->{last_op} = 'header';
}

sub section {
	my ($self, $text) = @_;

	$text =~ tr/a-z/A-Z/;
	$text =~ s/^(.*)$/.SH "\1"/;

	print $text;

	$self->{last_op} = 'section';
}

sub subsection {
	my ($self, $text) = @_;

	$text =~ s/^(.*)$/.SS "\1"/;

	print $text;

	$self->{last_op} = 'subsection';
}

sub get_link {
	my ($self, $command, $section, $option) = @_;

	if ($option eq 'external') {
		my $links = $self->{links};
		push(@$links, $command);
		$command =~ s/\.html$//;
		$command =~ s/-/ /g;
		push(@$links, $command);
		return '\fI' . $command . '\fR\&[1]';
	} else {
		return '\fB' . $command . '\fR(' . $section . ')';
	}
}

sub common {
	my ($self, $text, $option) = @_;

	# escape backslashes, but not in "\n", "\&" or "\fB"
	$text =~ s/\\(?!n|f[A-Z]|&)/\\\\/g;
	# escape "-"
	$text =~ s/-/\\-/g;
	# handle ...
	$text =~ s/(\.\.\.)/\\&\1/g;
	# remove double space after full stop or comma
	$text =~ s/([\.,])  /\1 /g;

	if ($option ne 'no-markup') {
		# make 'italic'
		$text =~ s/'([^'\n]*)'/\\fI\1\\fR/g;
		# ignore `
		$text =~ s/`//g;
		# make *bold*
		$text =~ s/\*([^\*\n]*)\*/\\fB\1\\fR/g;
		# handle <<sections>
		$text =~ s/<<([^>]*)>>/the section called \\(lq\1\\(rq/g;
	}

	return $text;
}

sub normal {
	my ($self, $text) = @_;

	if ($text eq "") {
		return;
	}

	$text = $self->common($text);

	$text =~ s/ *\n(.)/ \1/g;

	if ($self->{last_op} eq 'normal') {
		print "\n";
	}

	print $text;

	$self->{last_op} = 'normal';
}

sub verse {
	my ($self, $text) = @_;

	$text = $self->common($text);
	$text =~ s/^\t/        /mg;

	print ".sp\n.RS 4\n.nf\n" . $text . ".fi\n.RE\n";

	$self->{last_op} = 'verse';
}

sub enumeration {
	my ($self, $text, $option) = @_;

	my $counter = 0;
	foreach $line (@$text) {
		$counter++;
		print ".TP 4\n"
			. ($option eq 'unnumbered' ? '\(bu' : $counter . '.')
			. "\n"
			. $self->common($line);
	}

	$self->{last_op} = 'enumeration';
}

sub begin_item {
	my ($self, $item, $text) = @_;

	$item = $self->common($item);
	$text = $self->common($text);

	$text =~ s/([^\n]) *\n([^\n])/\1 \2/g;

	print ".PP\n" . $item . "\n.RS 4\n" . $text;

	$self->{last_op} = 'item'; 
}

sub end_item {
	my ($self) = @_;

	print ".RE\n";

	$self->{last_op} = 'end_item';
}

sub indent {
	my ($self, $text) = @_;

	$text = $self->common($text, 'no-markup');
	$text =~ s/^\t/        /mg;

	if ($self->{last_op} eq 'normal') {
		print "\n";
	}

	print ".sp\n.RS 4\n.nf\n" . $text . ".fi\n.RE\n";

	$self->{last_op} = 'indent';
}

sub verbatim {
	my ($self, $text) = @_;

	$text = $self->common($text, 'no-markup');

	# convert tabs to spaces
	$text =~ s/^\t/        /mg;
	# remove trailing empty lines
	$text =~ s/\n\n*$/\n/;

	if ($self->{last_op} eq 'normal') {
		print "\n";
	}

	print ".sp\n.RS 4\n.nf\n.ft C\n" . $text . ".ft\n\n.fi\n.RE\n";

	$self->{last_op} = 'verbatim';
}

sub anchor {
	my ($self, $text) = @_;

	$self->{last_op} = 'anchor';
}

sub finish {
	my ($self) = @_;
	my $links = $self->{links};

	if ($#$links >= 0) {
		print '.SH "REFERENCES"' . "\n";
		my $i = 1;
		while ($#$links >= 0) {
			my $ref = shift(@$links);
			$ref =~ s/-/\\-/g;
			my $label = shift(@$links);
			printf (".IP \"% 2d.\" 4\n%s\n.RS 4\n\\%%%s\n.RE\n",
				$i++, $label, $ref);
		}
	} else {
		print "\n";
	}
}

package html_page;

sub new {
	my ($class) = @_;
	my $self = {};
	bless $self, $class;
	return $self;
}

-- snap --

Ciao, Dscho

P.S.: I need to catch some Zs, and do some real work, so do not be surprised if I do not respond within the next 24 hours.

Johannes Schindelin wrote:

It's pretty good, I certainly wouldn't have trouble reading or maintaining it, but I'll give you suggestions anyway.

nice work, replacing a massive XML/XSL/etc stack with a small Perl script ;-)

Sam.

Add -w for warnings, also use strict;

this function acts on globals; make them explicit arguments to the function.

uncuddle your elsif's; also consider making this a "tabular ternary" with the actions in separate functions.

ie

$result = ( $par =~ /^\. /s      ? $conv->do_enum($par)    :
            $par =~ /^\[verse\]/ ? $conv->do_verse($par)  :
            ... )

However I have a suspicion that your script is doing line-based parsing instead of recursive descent; I don't know whether that's the right thing for asciidoc. It's actually fairly easy to convert a grammar to code blocks using tricks from MJD's _Higher Order Perl_. Is it necessary for the asciidoc grammar?

These REs suffer from LTS (Leaning Toothpick Syndrome). Consider using s{foo}{bar} and adding the 'x' modifier to space out groups.

I'd consider a HERE-doc, or multi-line qq{ } more readable than this.

Use qq{} when making strings with lots of embedded double quotes and interpolation.

Using commas rather than "." will safe you a concat when printing to filehandles, but that's a very small nit to pick :)

Hmm, that regex would not match for <<foo > bar>>, if you care you'd need to write something like <<((?:[^>]+|>[^>])*)>>

"." is the same as [^\n] (without the 's' modifier).

just use if (@$links) and while (@$links)

Hi,

On Wed, 3 Oct 2007, Sam Vilain wrote:

Thank you very much!  (On both accounts...)

Uhm... It is less capable, though...

<dumb>What does "use strict;" imply?</dumb>

Actually, it resets the global $par.  Should I rather make it a class?

I'm sorry... What do you mean?

I do not like that way... is it Perl standard to code like that?

I wanted to keep it simple. So I'll try to stay away from any fancy grammar parsing, and stay with "read lines until you have something to process".

I guess you mean the forward slash. Alas, that's what I'm used to, and I'd rather not change it unless forced to... lest I stop understanding my own code!

(Besides, I did not find _any_ example showing why "x" should be useful.)

Can you give me an example of a HERE-doc? (What I tried to avoid is having ugly indentation-breaking tlobs.)

I'll try to find something about qq{} in the docs.

Does that also work with older perl? IIRC there was some strange problem with my perl when lots of code in git.git was changed to using commata.

I'd rather leave it as is -- this script is not meant to grok all kind of sh*t. It is meant to make translating the docs as fast and uncumbersome as possible. Which will involve making the documentation more consistent (in and of itself something I rather like).

So unless there comes a compelling reason, I'd rather leave it.

But I need the (implicit) 's' modifier, otherwise the "\n" in the middle is not interpreted correctly. This regsub is meant to unwrap the paragraph and put it into a very long line (but leaving \n\n alone).

Thanks.  I hoped there would be something like this.

Another thing: if I want to add some documentation, what would be the common way to do it? =pod...=cut?

Thank you for all your tips!

Ciao, Dscho

On Wed, Oct 03, 2007 at 05:23:35AM +0100, Johannes Schindelin wrote:

Try "perldoc strict" for details.

I think he means reformatting to

  if (condition) {
  }
  elsif (condition) {
  }

It's quite common if you have a dispatch function, but obviously not required.

print <<EOF; foo EOF

HERE-docs necessarily break indentation unless you strip it out manually (which is inefficient and ugly).

But two things that might make that look better are using qq// (to avoid having to escape quotes) and interpolating the variables:

  . qq/." Generator: $self->{generator}\n/

It's in perlop, but it's basically a fancy way of double-quoting, except that you get to choose the delimiter.

I think you might be confused about how the 's' modifier works. You are not using it, so '.' is the same as '[^\n]'. Perl will always match a newline if it's in your regex. If you specify 'm', then it will also allow '^' and '$' to match at line boundaries (instead of just at the beginning and end of the string).

-Peff

On Wed, Oct 03, 2007 at 05:23:35AM +0100, Johannes Schindelin wrote:

By the way, without having looked at your script to see it does, a couple things I know of that the user manual relies on that other docs may not as much are automatic table of contents generation, and cross references.

I'd be similarly inclined to work on improving asciidoc rather than inventing something new, but I guess it's at least interesting to see how far your approach gets us.

--b.

Johannes,

Given other people have answered some points, I'll answer the rest.

Johannes Schindelin wrote:

Three things;

1. variables must be declared with 'my', 'our' or 'use var' before they
are used, to catch typos

2. when subroutine calls are found, they are checked to exist otherwise
they throw a compile-time error

3. force all dereferences to follow real references and not allow symbol
table access (don't worry about that ;-))

Well, just channelling Dijkstra really. Functions should take all their input as formal arguments rather than globals.

ie

  sub handle_text {
      my $par = shift;
  }

If it really should be a global, it is perhaps best declared up front with "our" or "use vars". "use strict" will force you to do one of these.

It's in Perl Best Practices, but these are always suggestions and not hard and fast rules. It just means that you have a big table of regex -> function that you can quickly check rather than looking at a lot of spaced out 'elsif's

Before:

s/link:([^\[ ]*)\[(.+)\]/sprintf "%s",
 			$conv->get_link($1, $2, 'external')/ge;

After:

s{ link: ([^\[\040]*) \[(.+)\] }
 { sprintf "%s", $conv->get_link($1, $2, 'external') }gex;

That should go back all the way to perl 4, if not earlier. If you're assigning to a scalar, then you need to use concat. But very minor.

Sure, KISS.

That's right. If you're an emacs user, in cperl-mode with abbrevs on you type '=head1' and you'll get:

=head1 NAME

scriptname

=head1 SYNOPSIS

=head1 DESCRIPTION

=cut

See perlpod(3) for more!

Sam.

Hi,

On Thu, 4 Oct 2007, Sam Vilain wrote:

Thanks for your patient explanations!

Ciao, Dscho

El 3/10/2007, a las 3:56, Sam Vilain escribió:

I haven't looked at all of the asciidoc source for the Git documentation but I suspect that almost all of it is entirely regular, and if there is any nesting in it is is probably of a limited scope (ie. sections, subsections, subsubsections in the user manual) and so you can avoid the complexity of a full recursive descent parser. I'd also expect Johannes' approach to be faster (is it faster than the existing tool chain? I would expect so).

Cheers, Wincent

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

How you spend your time is up to you, but I need to wonder...

 - Is "man" format important for msysGit aka Windows
   environment?  I had an impression that their helpfile format
   were closer to "html" output.

 - Does it make sense in the longer term for us to maintain
   in-house documentation tools?  Can we afford it?

It appears that we heard about breakages for every minor docbook updates, and it is really appealing if we do not have to rely on xsl toolchain for manpage generation. But if patching the text means making it compatible with the in-house script _and_ incompatible with AsciiDoc, hmmm...

El 3/10/2007, a las 6:48, Junio C Hamano escribió:

Indeed, especially seeing as asciidoc and the xsl toolchain are the trickiest build dependencies to install. If all that could be replaced by a single simple script like this one then that would be awesome, and probably more maintainable in the long run seeing as it would eliminate those intermittent breakages caused by changes in third-party tools.

Cheers, Wincent

Wincent Colaiuta <win@wincent.com> writes:

What with output in print, HTML, info? The advantage of a toolchain in that it is flexible. I am the first to admit that getting the AsciiDoc/Docbook/Docbook2X toolchain to get it to do what one wants to is like baking cake in a lightless kitchen. But it is not like we go through that pain without any reason.

Personally, I think it might make sense to just step away from the AsciiDoc documentation to Docbook: plain text (without cutified formatting control like in AsciiDoc) can be generated _from_ Docbook.

And AsciiDoc keeps us from documenting the formatting: Docbook, which is a source format and looks it, can easily admit comments that won't get through to the formatted versions. Sure, the first version would likely be generated with AsciiDoc and thus basically uncommented.

El 3/10/2007, a las 10:12, David Kastrup escribió:

Yes, that's still a problem...

Yes, but editing DocBook (XML) is relatively painful compared to editing plain text. You either have to rely on a bloated XML- validating editor or instead ask your doc authors to manually write valid XML (and I totally agree with Terrence Parr that, "XML makes a lousy human interface "; see <http://www.ibm.com/developerworks/xml/library/x-sbxml.html> for his full take).

I know that Linus has argued for AsciiDoc because the source *is* the plain text documentation and is therefore easily readable, but for me the real benefit lies in the fact that *because* the source is plain text it is easily edited (ie. that the source is easily *writeable*), and things like documentation patches are very neat with AsciiDoc.

Cheers, Wincent

Wincent Colaiuta <win@wincent.com> writes:

The problem is that we are not editing plain text, but Docbook source masquerading as plain text.

But it is not all _all_ easily writeable the moment you try to do something with _structural_ impact. In fact, it is pretty much impossible for anybody except wizards to do that. And when the wizards do it, they can't actually document what they have been doing since that would mean cluttering the purported "plain text documentation" with formatting comments.

Maybe it would help to rename *.txt to *.asciidoc and generate *.txt in future. That would at least make it possible to document stuff in the AsciiDoc source, and also would make it possible to add indexing info and other stuff without cluttering up the plain text use case.

Hi David.

When I converted part of the kbuild documentation to ascii doc is was a strightforward excercise. The txt file was equally readable before and after, and the generated HTML looks OK.

Up until 3.6 in following link were properly converted: http://www.ravnborg.org/kbuild/makefiles.html

But I did not convince asciidoc to generate an index - is it this you refer to as magic?

Using the kbuild doc as my background I would say that with or without asciidoc there were a requirment for a consistent style used throughout the file. And that style was and are simple to use just by copying relevant examples.

What asciidoc gave me was a simple syntax chack of what I did. I found wrong references when playing with asciidoc as one example.

If the asciidoc replacement prove a success for git I would consider suggesting it for the kernel too. It would be good to generate some nicer loking online documentation and here an asii tool like thing would be a help.

	Sam

On Wed, Oct 03, 2007 at 12:25:44PM +0200, David Kastrup wrote:

I do a fair amount of editing of the asciidoc source, but 99% of it is done by just blind imitation of what's already there. I've never learned docbook (I've barely learned asciidoc, to be honest), and with a few (now forgotten) exceptions haven't tried to understand how the toolchain works.

Maybe my experience would be the same with Docbook--I have no idea, never having worked with it--but if you're suggesting that knowledge of Docbook is a prerequisite for working with asciidoc, that certainly hasn't been my experience.

I'm not sure what you're talking about here.  Example?

--b.

"J. Bruce Fields" <bfields@fieldses.org> writes:

But not everything is already there, and when something surprising happens, there is little chance to see how it came about.

"making use of" and "working with" are two different things.

Try including the manual pages as a (properly linked when man pages are referenced) appendix in the user manual, so that the printed form (or PDF) of the user manual is a single coherent document with all information inside. That's what I tried for about a week, digging into the various available (and unavailable) documentation and then postponing the project indefinitely because it both exceeded my current capability as well as demonstrating that there was no reasonably outlined path for acquiring the necessary skills.

In Texinfo, this takes few commands, all of which are well-documented and in a reasonable place in the Texinfo manual (which is all you need to consult in order to write Texinfo documents).

But with git's AsciiDoc information, not only is the required information scattered through half a dozen of different manuals all describing completely different systems, but the necessary other documentation is, at best, only mentioned in passing in every single relevant document. So while you may know where you want to start and end your journey, there is nothing which would tell you how to get from start to end. You have to randomly pick your road until you may or may not find something closer to the end.

Wincent Colaiuta <win@wincent.com> writes:

To give credits to where they are due, most of the structure of the initial documentation was done during the first week of May 2005 by David Greaves while Linus was vacationing, and the first person who brought up AsciiDoc was Bert Hubert.

    http://article.gmane.org/gmane.comp.version-control.git/2323

One thing Linus had to say about the issue from early on, and I still agree with, is the last paragraph in:

    http://article.gmane.org/gmane.comp.version-control.git/2298

There was another thread in the recent past

    http://article.gmane.org/gmane.comp.version-control.git/55059

I've seen markdown used elsewhere, and I regularly read pod. These are both used by other people (I _care_ about good external support and user community), are readable as straight text (I personally care more about this than prettyprinted output), and can be alternative candidates if we consider switching.

How good are HTML and manpage output support from these (or other candidates) formats these days? Output to help page format Windows folks use (I am assuming Mac people are happy as long as man is available) would be a definite plus.

Another alternative could be to build on AsciiDoc to get manpage and html output without relying too heavily on docbook toolchain, and considering the fact that we still seem to be one of the most important customers listed on its home page, we might be able to get help from Stuart himself to improve the situation (I am assuming that mingwgit folks do not have problem with Python, but I may be mistaken).

In short, although I do appreciate Johannes's and Sam's attempt, I would really prefer to see us pick some externally maintained alternative, instead of inventing a homebrew system that we need to maintain ourselves. It is rumored that git has much higher developer count vs loc count ratio than many other open source projects, doing the documentation format is not part of our project, and I'd rather see them spend time working on git, not building and maintaining AsciiDoc lookalike.

Hi Junio.

For the kernel I would like to see a tool that does:

o Based on nicely formatted ascii/utf-8 be able to:
 - Generate good looking and easy to read HTML
 - Possible generate other output formats too

And if the kernel folks do not like it then at least the possibility to run this on kbuild documentation locally so I can generate nice html docs for that part.

For a kernel integrated tool the dependencies shall be minimal which is where asciidoc fails today. If asciidoc people could address this issue for the simpler output format then the tool IMO would have a much stronger position.

	Sam

Hi,

On Wed, 3 Oct 2007, Sam Ravnborg wrote:

Is perl not too much already?

Ciao, Dscho

On Wed, Oct 03, 2007 at 07:57:48PM +0100, Johannes Schindelin wrote:

No - perl has not proved to be a problem. But like git we have had a smaller share of docbook incompatibilities.

We even require perl for a regular kernel build for some arch's these days.

	Sam

Junio,

great summary!

On 10/3/07, Junio C Hamano <gitster@pobox.com> wrote:

I'm in complete agreement here...

...

...

And something that leads to PDF (perhaps via latex).

The problem I see here -- and that I've bumped into several times in other projects -- is that readable & easy to edit text formats for the source are key, and those can do most of what we need for man/info/html outputs. But documentation formats that produce high quality print output usually need arcane formats _and_ a high-maintenance toolchain.

With AsciiDoc we've managed to avoid the arcane format, but we are still laden with a horrid toolchain. In that light, I actually like what Johannes is doing, even though it's a timesink.

Do the other text based alternatives these days have a workable high quality PDF/latex output format without pulling in brittle dependencies like XSLT?

cheers

martin

"Martin Langhoff" <martin.langhoff@gmail.com> writes:

Let's put this somewhat into perspective: the toolchain is horrid with regard to the complexity and documentation (well, AsciiDoc documentation itself is quite thorough and reasonably organized, but it does not buy you much without learning Docbook, and learning Docbook is such a chore that people would rather use AsciiDoc in order to avoid it), not horrid regarding the usability or flexibility of the results.

If we had a few people specializing in Docbook/AsciiDoc/XSLT available constantly on the team, we could probably get along fine.

The problem is not just that it is a timesink now, but that it will remain a timesink. There is a reason that we don't have so many formats around with multiple high-quality backends.

Texinfo produces good info and plain text, tolerable PDF and HTML and not-quite-usable Docbook. The output for HTML from the current git documentation toolchain certainly looks better. I don't see how to generate PDF right now, but there must be a way (xmlto complains about not seeing passivetex though I have it installed in TeXlive).

The source is uglier than AsciiDoc, but then there is no cleverly disguised information in it: every formatting detail is quite out in the open. The same is true for Docbook, but Docbook really eats the cake, platter and all concerning unreadability of the input. On the other hand, there are more special-purpose editors that know how to deal with Docbook/XML than there are for Texinfo.

Texinfo gives a reasonable subset of Linus-thinkalikes the cooties, and that pretty much rules it out: we need a format that people are willing to write in.

There are not really many options for versatile formats, I am afraid.

On 10/5/07, David Kastrup <dak@gnu.org> wrote:

Exactly. I'm not complaining about asciidoc itself. But the toolchain is very fragile, and not crossplatform. Git compiles and works on many unixen, win32, and some embedded posixy OSs if IIRC.

OTOH asciidoc can be pretty hard to get going even on modern linuxen.The asciidoc toolchain doesn't even work on Debian Sarge, which isn't *that* old, while I'm pretty sure git itself can be built and used on older linuxen. That's where a good old regex-insanity Perl-based parser beats anything else: no dependencies, works everywhere.

In that sense, this is close to being a rehash of the "let's use autoconf" argument...

cheers

martin

Hi,

On Tue, 2 Oct 2007, Junio C Hamano wrote:

I wanted something that can output both "man" and "html" output (and if some suck^Wlos^Wtexi-fan wants to provide it, also a "texi" or even "info" backend).

IMHO "man" needs a stricter framework in place, so I went with that.

In the long run, I expect only few bugs (and I will try hard to squash them when they crop up, _and_ make this beast more maintainable whenever somebody has an idea how to do that).

However, it should definitely help keeping the docs clean, as now nobody has an excuse to test doc changes a la "I do not have asciidoc, so I do not know if it works, so please test".

Exactly.

No, I do not want it _incompatible_. I want it _stricter_. For example, you can do this in asciidoc:

    This is some paragraph that is indented,
    but the funny thing is:

This paragraph: ---------------

	is indented all the same!

So one thing I absolutely detest here is that you are free to use one, two, three or more spaces, or tabs, and asciidoc does the DWIMery of handling them the same. But _not_ if there was any indentation before that with _less_ spaces and/or tabs!

Therefore I'd like to enforce strict rules here: Tab it is. One tab per indentation level. No spaces, no ambiguities.

Ciao, Dscho

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

And you have not even talked about print. The thing is that high quality output is a lot of work (including design and design decisions) for _every_ _single_ backend. If the only target were "man" and text, then one would be better off writing as groff source and be done.

We really can't afford another time sink. That nobody can actually solve structural problems ("how do we include the man pages as an appendix in the user manual?") also implies a waste of time, but at some point of time people just throw up their hands and give up in disgust.

You propose a larger time sink where people won't be forced to give up. But we really don't want to waste time reinventing the wheel. It would be better spent communicating with the wheelbuilders.