threads / discuss / 11417

generated HTML contains broken links

Subject: generated HTML contains broken links

## tl;dr

9 messages between Dec 28, 2007 and Jan 4, 2008.

replies: 8people: 5as markdown or json

Eric Hanchrow· Dec 28, 2007, 23:49 UTC · lore

I'm just starting to play with git, and have checked it out (with "git clone git://git.kernel.org/pub/scm/git/git.git"), and built the documentation (cd Documentation; make), on Cygwin. I notice that the generated HTML docs are full of broken links -- for example, my file C:/cygwin/usr/local/src/git/Documentation/git.html includes this:

        git<a href="git-instaweb">1</a>

but there is no document named "git-instaweb" on my disk; instead, it's named "git-instaweb.html".

I'm using asciidoc version 8.2.4, if it matters.
-- 
I write [from 5 AM to 7 AM] to discover what I think.  After
all, the bars aren't open that early.
        -- Daniel Boorstin, former Librarian of Congress
Dan McGee· Dec 29, 2007, 03:01 UTC · re: Eric Hanchrow · lore

Re: generated HTML contains broken links

On 12/28/2007 05:49 PM, Eric Hanchrow wrote:
Show 12 quoted lines
> I'm just starting to play with git, and have checked it out (with "git
> clone git://git.kernel.org/pub/scm/git/git.git"), and built the
> documentation (cd Documentation; make), on Cygwin.  I notice that the
> generated HTML docs are full of broken links -- for example, my file
> C:/cygwin/usr/local/src/git/Documentation/git.html includes this:
> 
>         git<a href="git-instaweb">1</a>
> 
> but there is no document named "git-instaweb" on my disk; instead,
> it's named "git-instaweb.html".
> 
> I'm using asciidoc version 8.2.4, if it matters.
We noticed this with the upgrade from Asciidoc 8.2.2 -> 8.2.3 on our project. It is broken in both the manpages and the HTML generated documentation. I've included an example below. So far, I haven't had luck tracking down the reason but I am looking into trying to fix this tonight. If anyone else is better with this stuff, it would be great if you could take a look.
Relevant email on the pacman-dev list [1]:
On Nov 9, 2007 4:05 PM, Andrew Fyfe <andrew@neptune-one.net> wrote:
Show 8 quoted lines
> Did little digging, the breakage/change is in now asciidoc converts from asciidoc to xml (so it's 
> not docbook-xsl). In 8.2.2 manlink:pacman.conf[5] expands to
> 
> <citerefentry><refentrytitle>pacman.conf</refentrytitle><manvolnum>5</manvolnum></citerefentry>,
> 
> in 8.2.3 it expands to
> 
> man<ulink url="pacman.conf">5</ulink>,

Note that manlink is basically just a renamed gitlink, and can be found here: <http://projects.archlinux.org/git/?p=pacman.git;a=blob;f=doc/asciidoc.conf>

-Dan
[1] <http://archlinux.org/pipermail/pacman-dev/2007-November/009937.html>
Generated man pages from Junio:
HOOKS
       This command can run commit-msg, pre-commit, and post-commit hooks. See
       [1]hooks for more information.
SEE ALSO
       git-add(1), git-rm(1), git-mv(1), git-merge(1), git-commit-tree(1)
AUTHOR
       Written by Linus Torvalds <torvalds@osdl.org> and Junio C Hamano
       <junkio@cox.net>
GIT
       Part of the git(7) suite
Man pages generated locally (with Asciidoc 8.2.3 or 8.2.5):
HOOKS
       This command can run commit-msg, pre-commit, and post-commit hooks. See
       hooks[5] for more information.
SEE ALSO
       git1[1], git1[2], git1[8], git1[6], git1[9]
AUTHOR
       Written by Linus Torvalds <torvalds@osdl.org> and Junio C Hamano
       <junkio@cox.net>
GIT
       Part of the git7[10] suite
Miklos Vajna· Dec 29, 2007, 15:57 UTC · re: Dan McGee · lore

Re: generated HTML contains broken links

On Fri, Dec 28, 2007 at 09:01:17PM -0600, Dan McGee <dpmcgee@gmail.com> wrote:
Show 29 quoted lines
> Generated man pages from Junio:
> HOOKS
>        This command can run commit-msg, pre-commit, and post-commit hooks. See
>        [1]hooks for more information.
> 
> SEE ALSO
>        git-add(1), git-rm(1), git-mv(1), git-merge(1), git-commit-tree(1)
> 
> AUTHOR
>        Written by Linus Torvalds <torvalds@osdl.org> and Junio C Hamano
>        <junkio@cox.net>
> 
> GIT
>        Part of the git(7) suite
> 
> Man pages generated locally (with Asciidoc 8.2.3 or 8.2.5):
> HOOKS
>        This command can run commit-msg, pre-commit, and post-commit hooks. See
>        hooks[5] for more information.
> 
> SEE ALSO
>        git1[1], git1[2], git1[8], git1[6], git1[9]
> 
> AUTHOR
>        Written by Linus Torvalds <torvalds@osdl.org> and Junio C Hamano
>        <junkio@cox.net>
> 
> GIT
>        Part of the git7[10] suite
http://code.toofishes.net/gitweb.cgi?p=pacman.git;a=commitdiff;h=b3c6bdda38f7e370e1f80f02a61f1d3f08c1b57d

here is the commit that fixed the problem for pacman. what do you think, should we rename gitlink to something else, too - or should we contact upstream to notify them about they caused a breakage?

thanks,
- VMiklos
Dan McGee· Dec 29, 2007, 16:24 UTC · re: Miklos Vajna · lore

Re: generated HTML contains broken links

On Dec 29, 2007 9:57 AM, Miklos Vajna <vmiklos@frugalware.org> wrote:
Show 36 quoted lines
> On Fri, Dec 28, 2007 at 09:01:17PM -0600, Dan McGee <dpmcgee@gmail.com> wrote:
> > Generated man pages from Junio:
> > HOOKS
> >        This command can run commit-msg, pre-commit, and post-commit hooks. See
> >        [1]hooks for more information.
> >
> > SEE ALSO
> >        git-add(1), git-rm(1), git-mv(1), git-merge(1), git-commit-tree(1)
> >
> > AUTHOR
> >        Written by Linus Torvalds <torvalds@osdl.org> and Junio C Hamano
> >        <junkio@cox.net>
> >
> > GIT
> >        Part of the git(7) suite
> >
> > Man pages generated locally (with Asciidoc 8.2.3 or 8.2.5):
> > HOOKS
> >        This command can run commit-msg, pre-commit, and post-commit hooks. See
> >        hooks[5] for more information.
> >
> > SEE ALSO
> >        git1[1], git1[2], git1[8], git1[6], git1[9]
> >
> > AUTHOR
> >        Written by Linus Torvalds <torvalds@osdl.org> and Junio C Hamano
> >        <junkio@cox.net>
> >
> > GIT
> >        Part of the git7[10] suite
>
> http://code.toofishes.net/gitweb.cgi?p=pacman.git;a=commitdiff;h=b3c6bdda38f7e370e1f80f02a61f1d3f08c1b57d
>
> here is the commit that fixed the problem for pacman. what do you think,
> should we rename gitlink to something else, too - or should we contact
> upstream to notify them about they caused a breakage?

I've tried twice now to send a patch to the list...please let me know if it isn't working, because I am getting CCed on the send, and the mailing list address is set in the sendemail.to variable. I did notice that send-email doesn't even prompt you for confirmation of the default "to" address if you have this variable set, so I'm not sure if something weird is happening there.

-Dan
Dan McGee· Dec 29, 2007, 16:34 UTC · re: Dan McGee · lore

[PATCH] Documentation: rename gitlink macro to linkgit

On 12/29/2007 10:24 AM, Dan McGee wrote:
Show 15 quoted lines
> On Dec 29, 2007 9:57 AM, Miklos Vajna <vmiklos@frugalware.org> wrote:
>> http://code.toofishes.net/gitweb.cgi?p=pacman.git;a=commitdiff;h=b3c6bdda38f7e370e1f80f02a61f1d3f08c1b57d
>>
>> here is the commit that fixed the problem for pacman. what do you think,
>> should we rename gitlink to something else, too - or should we contact
>> upstream to notify them about they caused a breakage?
> 
> I've tried twice now to send a patch to the list...please let me know
> if it isn't working, because I am getting CCed on the send, and the
> mailing list address is set in the sendemail.to variable. I did notice
> that send-email doesn't even prompt you for confirmation of the
> default "to" address if you have this variable set, so I'm not sure if
> something weird is happening there.
> 
> -Dan
>From 68bf426e810e732ff3f9f75ffcd69f777b538685 Mon Sep 17 00:00:00 2001
From: Dan McGee <dpmcgee@gmail.com>
Date: Sat, 29 Dec 2007 00:20:38 -0600
Subject: [PATCH] Documentation: rename gitlink macro to linkgit

Between AsciiDoc 8.2.2 and 8.2.3, the following change was made to the stock Asciidoc configuration:

@@ -149,7 +153,10 @@
 # Inline macros.
 # Backslash prefix required for escape processing.
 # (?s) re flag for line spanning.
-(?su)[\\]?(?P<name>\w(\w|-)*?):(?P<target>\S*?)(\[(?P<attrlist>.*?)\])=
+
+# Explicit so they can be nested.
+(?su)[\\]?(?P<name>(http|https|ftp|file|mailto|callto|image|link)):(?P<target>\S*?)(\[(?P<attrlist>.*?)\])=
+
 # Anchor: [[[id]]]. Bibliographic anchor.
 (?su)[\\]?\[\[\[(?P<attrlist>[\w][\w-]*?)\]\]\]=anchor3
 # Anchor: [[id,xreflabel]]

This default regex now matches explicit values, and unfortunately in this
case gitlink was being matched by just 'link', causing the wrong inline
macro template to be applied. By renaming the macro, we can avoid being
matched by the wrong regex.

Signed-off-by: Dan McGee <dpmcgee@gmail.com>
---
 Documentation/asciidoc.conf              |    8 +-
 Documentation/blame-options.txt          |    2 +-
 Documentation/cmd-list.perl              |    2 +-
 Documentation/config.txt                 |  126 +++++++++++++++---------------
 Documentation/cvs-migration.txt          |   10 +-
 Documentation/diff-options.txt           |    4 +-
 Documentation/everyday.txt               |   48 ++++++------
 Documentation/fetch-options.txt          |    2 +-
 Documentation/git-add.txt                |   16 ++--
 Documentation/git-am.txt                 |   16 ++--
 Documentation/git-annotate.txt           |    4 +-
 Documentation/git-apply.txt              |   12 ++--
 Documentation/git-archimport.txt         |    2 +-
 Documentation/git-archive.txt            |    2 +-
 Documentation/git-bisect.txt             |    2 +-
 Documentation/git-blame.txt              |    8 +-
 Documentation/git-branch.txt             |   12 ++--
 Documentation/git-bundle.txt             |   10 +-
 Documentation/git-cat-file.txt           |    4 +-
 Documentation/git-check-attr.txt         |    4 +-
 Documentation/git-check-ref-format.txt   |    6 +-
 Documentation/git-checkout-index.txt     |    2 +-
 Documentation/git-checkout.txt           |    4 +-
 Documentation/git-cherry-pick.txt        |    4 +-
 Documentation/git-cherry.txt             |    2 +-
 Documentation/git-citool.txt             |    6 +-
 Documentation/git-clean.txt              |    4 +-
 Documentation/git-clone.txt              |    2 +-
 Documentation/git-commit-tree.txt        |    6 +-
 Documentation/git-commit.txt             |   30 ++++----
 Documentation/git-config.txt             |    2 +-
 Documentation/git-count-objects.txt      |    2 +-
 Documentation/git-cvsexportcommit.txt    |    2 +-
 Documentation/git-cvsimport.txt          |    4 +-
 Documentation/git-cvsserver.txt          |    4 +-
 Documentation/git-daemon.txt             |    2 +-
 Documentation/git-describe.txt           |    2 +-
 Documentation/git-diff-files.txt         |    2 +-
 Documentation/git-diff-index.txt         |    2 +-
 Documentation/git-diff-tree.txt          |    2 +-
 Documentation/git-diff.txt               |    8 +-
 Documentation/git-fast-export.txt        |   12 ++--
 Documentation/git-fast-import.txt        |   18 ++--
 Documentation/git-fetch-pack.txt         |    4 +-
 Documentation/git-fetch.txt              |    4 +-
 Documentation/git-filter-branch.txt      |   16 ++--
 Documentation/git-fmt-merge-msg.txt      |    4 +-
 Documentation/git-format-patch.txt       |    8 +-
 Documentation/git-fsck-objects.txt       |    2 +-
 Documentation/git-fsck.txt               |    2 +-
 Documentation/git-gc.txt                 |   14 ++--
 Documentation/git-get-tar-commit-id.txt  |    4 +-
 Documentation/git-grep.txt               |    2 +-
 Documentation/git-gui.txt                |    4 +-
 Documentation/git-hash-object.txt        |    2 +-
 Documentation/git-help.txt               |    6 +-
 Documentation/git-http-fetch.txt         |    2 +-
 Documentation/git-http-push.txt          |    2 +-
 Documentation/git-imap-send.txt          |    2 +-
 Documentation/git-index-pack.txt         |   10 +-
 Documentation/git-init-db.txt            |    2 +-
 Documentation/git-init.txt               |    2 +-
 Documentation/git-instaweb.txt           |    2 +-
 Documentation/git-log.txt                |   10 +-
 Documentation/git-lost-found.txt         |    4 +-
 Documentation/git-ls-files.txt           |    8 +-
 Documentation/git-ls-remote.txt          |    4 +-
 Documentation/git-ls-tree.txt            |    2 +-
 Documentation/git-mailinfo.txt           |    4 +-
 Documentation/git-mailsplit.txt          |    2 +-
 Documentation/git-merge-base.txt         |    2 +-
 Documentation/git-merge-file.txt         |    4 +-
 Documentation/git-merge-index.txt        |    2 +-
 Documentation/git-merge-one-file.txt     |    2 +-
 Documentation/git-merge-tree.txt         |    2 +-
 Documentation/git-merge.txt              |    8 +-
 Documentation/git-mergetool.txt          |    4 +-
 Documentation/git-mktag.txt              |    2 +-
 Documentation/git-mktree.txt             |    2 +-
 Documentation/git-mv.txt                 |    2 +-
 Documentation/git-name-rev.txt           |    4 +-
 Documentation/git-pack-objects.txt       |   10 +-
 Documentation/git-pack-redundant.txt     |    8 +-
 Documentation/git-pack-refs.txt          |    2 +-
 Documentation/git-parse-remote.txt       |    2 +-
 Documentation/git-patch-id.txt           |    2 +-
 Documentation/git-peek-remote.txt        |    2 +-
 Documentation/git-prune-packed.txt       |    6 +-
 Documentation/git-prune.txt              |    2 +-
 Documentation/git-pull.txt               |   10 +-
 Documentation/git-push.txt               |    4 +-
 Documentation/git-quiltimport.txt        |    2 +-
 Documentation/git-read-tree.txt          |    8 +-
 Documentation/git-rebase.txt             |   10 +-
 Documentation/git-receive-pack.txt       |    4 +-
 Documentation/git-reflog.txt             |    8 +-
 Documentation/git-relink.txt             |    2 +-
 Documentation/git-remote.txt             |   12 ++--
 Documentation/git-repack.txt             |   14 ++--
 Documentation/git-repo-config.txt        |    2 +-
 Documentation/git-request-pull.txt       |    2 +-
 Documentation/git-rerere.txt             |    4 +-
 Documentation/git-reset.txt              |    8 +-
 Documentation/git-rev-list.txt           |   20 +++---
 Documentation/git-rev-parse.txt          |    2 +-
 Documentation/git-revert.txt             |    4 +-
 Documentation/git-rm.txt                 |    4 +-
 Documentation/git-send-email.txt         |    2 +-
 Documentation/git-send-pack.txt          |    6 +-
 Documentation/git-sh-setup.txt           |    2 +-
 Documentation/git-shell.txt              |    2 +-
 Documentation/git-shortlog.txt           |    2 +-
 Documentation/git-show-branch.txt        |    2 +-
 Documentation/git-show-index.txt         |    2 +-
 Documentation/git-show-ref.txt           |    6 +-
 Documentation/git-show.txt               |    8 +-
 Documentation/git-stash.txt              |   10 +-
 Documentation/git-status.txt             |    6 +-
 Documentation/git-stripspace.txt         |    2 +-
 Documentation/git-submodule.txt          |    6 +-
 Documentation/git-svn.txt                |   12 ++--
 Documentation/git-symbolic-ref.txt       |    2 +-
 Documentation/git-tag.txt                |    2 +-
 Documentation/git-tar-tree.txt           |    2 +-
 Documentation/git-unpack-file.txt        |    2 +-
 Documentation/git-unpack-objects.txt     |    2 +-
 Documentation/git-update-index.txt       |   12 ++--
 Documentation/git-update-ref.txt         |    2 +-
 Documentation/git-update-server-info.txt |    2 +-
 Documentation/git-upload-archive.txt     |    2 +-
 Documentation/git-upload-pack.txt        |    2 +-
 Documentation/git-var.txt                |    8 +-
 Documentation/git-verify-pack.txt        |    2 +-
 Documentation/git-verify-tag.txt         |    2 +-
 Documentation/git-whatchanged.txt        |    2 +-
 Documentation/git-write-tree.txt         |    2 +-
 Documentation/git.txt                    |   20 +++---
 Documentation/gitattributes.txt          |   10 +-
 Documentation/gitcli.txt                 |    2 +-
 Documentation/gitignore.txt              |    6 +-
 Documentation/gitk.txt                   |    6 +-
 Documentation/gitmodules.txt             |    6 +-
 Documentation/glossary.txt               |   18 ++--
 143 files changed, 460 insertions(+), 460 deletions(-)

Looks like the patch was bigger than I thought. Gzipped and attached if that is acceptable.
Junio C Hamano· Jan 3, 2008, 21:01 UTC · re: Dan McGee · lore

Re: [PATCH] Documentation: rename gitlink macro to linkgit

Dan McGee <dpmcgee@gmail.com> writes:
Show 25 quoted lines
>>From 68bf426e810e732ff3f9f75ffcd69f777b538685 Mon Sep 17 00:00:00 2001
> From: Dan McGee <dpmcgee@gmail.com>
> Date: Sat, 29 Dec 2007 00:20:38 -0600
> Subject: [PATCH] Documentation: rename gitlink macro to linkgit
>
> Between AsciiDoc 8.2.2 and 8.2.3, the following change was made to the stock
> Asciidoc configuration:
>
> @@ -149,7 +153,10 @@
>  # Inline macros.
>  # Backslash prefix required for escape processing.
>  # (?s) re flag for line spanning.
> -(?su)[\\]?(?P<name>\w(\w|-)*?):(?P<target>\S*?)(\[(?P<attrlist>.*?)\])=
> +
> +# Explicit so they can be nested.
> +(?su)[\\]?(?P<name>(http|https|ftp|file|mailto|callto|image|link)):(?P<target>\S*?)(\[(?P<attrlist>.*?)\])=
> +
>  # Anchor: [[[id]]]. Bibliographic anchor.
>  (?su)[\\]?\[\[\[(?P<attrlist>[\w][\w-]*?)\]\]\]=anchor3
>  # Anchor: [[id,xreflabel]]
>
> This default regex now matches explicit values, and unfortunately in this
> case gitlink was being matched by just 'link', causing the wrong inline
> macro template to be applied. By renaming the macro, we can avoid being
> matched by the wrong regex.

What's already tagged, released to the wild and picked up by distros cannot be taken back, so I'd most likely have to apply your patch anyway, but I have to say I am not very amused. I'd call this a regression on AsciiDoc's part.

I would have expected some courtesy to make sure that updates to AsciiDoc would not to break existing users, especially the ones that they use as the top advertising material in the "Projects using AsciiDoc" list at http://www.methods.co.nz/asciidoc/ ;-)

Stuart, is there anything we can help you to set up some automated tests to catch AsciiDoc regression, so we do not have to suffer like this again?

Yannick Gingras· Jan 4, 2008, 05:22 UTC · re: Junio C Hamano · lore

Re: [PATCH] Documentation: rename gitlink macro to linkgit

Junio C Hamano <gitster@pobox.com> writes:
> Stuart, is there anything we can help you to set up some automated
> tests to catch AsciiDoc regression, so we do not have to suffer like
> this again?

We considered adding a nose test suite. The upcoming v9.0 release involves quite a bit of code massaging and we will definitely need an extensive test suite. But the test suite can only catch obvious rendering failures so any help in eyeballing the output will be appreciated.

Should we setup a distributed proof reading system like Project Gutenberg? I hope we don't need it. But a few scripts to batch render all the doc of popular projects would be nice. We could upload such snapshot render jobs where everyone could take a quick glance and spot obvious rendering errors.

If you follow AsciiDoc's mailing list [1], we'll go through a series of release candidates before the final 9.0. "With enough eyeballs, all bugs are shallow."

[1]: asciidoc-discuss@lists.metaperl.com

I reply to two mailing lists; I think this particular problem belongs to asciidoc-discuss.

Best regards, 
-- 
Yannick Gingras
Junio C Hamano· Jan 4, 2008, 05:50 UTC · re: Yannick Gingras · lore

Re: [PATCH] Documentation: rename gitlink macro to linkgit

Yannick Gingras <ygingras@ygingras.net> writes:
Show 11 quoted lines
> Junio C Hamano <gitster@pobox.com> writes:
>
>> Stuart, is there anything we can help you to set up some automated
>> tests to catch AsciiDoc regression, so we do not have to suffer like
>> this again?
>
> We considered adding a nose test suite.  The upcoming v9.0 release
> involves quite a bit of code massaging and we will definitely need an
> extensive test suite.  But the test suite can only catch obvious
> rendering failures so any help in eyeballing the output will be
> appreciated.

You could go fancy like that, but I suspect that an automated test to compare the text dump (e.g. "links -dump doc.html") generated by before and after version, perhaps with minimum massaging, would go a long enough way. At least that would have caught the "gitlink" breakage, wouldn't it?

Yannick Gingras· Jan 4, 2008, 06:11 UTC · re: Junio C Hamano · lore

Re: [PATCH] Documentation: rename gitlink macro to linkgit

Junio C Hamano <gitster@pobox.com> writes:
Show 8 quoted lines
>> But the test suite can only catch obvious rendering failures so any
>> help in eyeballing the output will be appreciated.
>
> You could go fancy like that, but I suspect that an automated
> test to compare the text dump (e.g. "links -dump doc.html")
> generated by before and after version, perhaps with minimum
> massaging, would go a long enough way.  At least that would have
> caught the "gitlink" breakage, wouldn't it?

Yeah it would have caught it. I'll hack something like that before the 9.0 release.

-- 
Yannick Gingras

← back to recent threads