threads / patch / 42176

v2travis-ci: build documentation

Subject: [PATCH v2] travis-ci: build documentation

## tl;dr

53 messages between Apr 29, 2016 and May 13, 2016. Diffs are folded; open one to read it.

replies: 52people: 9as markdown or json

larsxschneider@gmail.com· Apr 29, 2016, 09:35 UTC · lore
From: Lars Schneider <larsxschneider@gmail.com>

Run "make doc" as separate Travis CI build job to check if all documentation can be built without errors.

Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
---
diff to v1:
* add quick sanity check to documentation results (thanks Matthieu)
* fix typo in commits message (thanks Stefan)
* move 'make doc' into seperate Travis CI build job (thanks Peff)
* started to move CI helper scripts to /ci (thanks Matthieu & Junio)

I really like the idea of the "/ci" directory. Over time I plan to migrate all non Travis-CI related code from .travis.yml to this directory. This would ease the transition to or parallel execution with another CI system (e.g. GitLab CI to test Git on Windows).

Thanks, Lars

 .travis.yml              | 15 +++++++++++++++
 ci/test-documentation.sh | 23 +++++++++++++++++++++++
 2 files changed, 38 insertions(+)
 create mode 100755 ci/test-documentation.sh
Show changes to 2 files +38 −1

.travis.yml, ci/test-documentation.sh

diff --git a/.travis.yml b/.travis.yml
index 78e433b..9f71d23 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -32,6 +32,21 @@ env:
     # t9816 occasionally fails with "TAP out of sequence errors" on Travis CI OS X
     - GIT_SKIP_TESTS="t9810 t9816"

+matrix:
+  include:
+    - env: Documentation
+      os: linux
+      compiler: clang
+      addons:
+        apt:
+          packages:
+          - asciidoc
+          - xmlto
+      before_install:
+      before_script: make doc
+      script: ci/test-documentation.sh
+      after_failure:
+
 before_install:
   - >
     case "${TRAVIS_OS_NAME:-linux}" in
diff --git a/ci/test-documentation.sh b/ci/test-documentation.sh
new file mode 100755
index 0000000..329ff4b
--- /dev/null
+++ b/ci/test-documentation.sh
@@ -0,0 +1,23 @@
+#!/bin/sh
+#
+# Perform a quick sanity check on documentation generated with 'make doc'.
+#
+
+set -e
+
+test_file_count () {
+    SUFFIX=$1
+    EXPECTED_COUNT=$2
+    ACTUAL_COUNT=$(find Documentation -type f -name "*.$SUFFIX" | wc -l)
+    echo "$ACTUAL_COUNT *.$SUFFIX files found. $EXPECTED_COUNT expected."
+    test $ACTUAL_COUNT -eq $EXPECTED_COUNT
+}
+
+test -s Documentation/git.html
+test -s Documentation/git.xml
+test -s Documentation/git.1
+
+# The follow numbers need to be adjusted when new documentation is added.
+test_file_count html 233
+test_file_count xml 171
+test_file_count 1 152
--
2.5.1
Jeff King· Apr 29, 2016, 12:14 UTC · re: larsxschneider@gmail.com · lore

Re: [PATCH v2] travis-ci: build documentation

On Fri, Apr 29, 2016 at 11:35:34AM +0200, larsxschneider@gmail.com wrote:
> +# The follow numbers need to be adjusted when new documentation is added.
> +test_file_count html 233
> +test_file_count xml 171
> +test_file_count 1 152

This seems like it will be really flaky and a pain in the future. I'm not really sure what it's accomplishing, either. The earlier steps would complain if something failed to render, wouldn't they? At some point we have to have some faith in "make doc".

-Peff
Matthieu Moy· Apr 29, 2016, 12:21 UTC · re: Jeff King · lore

Re: [PATCH v2] travis-ci: build documentation

Jeff King <peff@peff.net> writes:
Show 11 quoted lines
> On Fri, Apr 29, 2016 at 11:35:34AM +0200, larsxschneider@gmail.com wrote:
>
>> +# The follow numbers need to be adjusted when new documentation is added.
>> +test_file_count html 233
>> +test_file_count xml 171
>> +test_file_count 1 152
>
> This seems like it will be really flaky and a pain in the future. I'm
> not really sure what it's accomplishing, either. The earlier steps would
> complain if something failed to render, wouldn't they? At some point we
> have to have some faith in "make doc".

I agree. My proposal to check for a handful of generated files was just because this extra paranoia was almost free (just 3 lines of code that won't need particular maintenance).

In this case, I'm afraid the maintenance cost is much bigger than the expected benefits.

-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/
Lars Schneider· Apr 29, 2016, 14:22 UTC · re: Matthieu Moy · lore

Re: [PATCH v2] travis-ci: build documentation

Show 22 quoted lines
> On 29 Apr 2016, at 14:21, Matthieu Moy <Matthieu.Moy@grenoble-inp.fr> wrote:
> 
> Jeff King <peff@peff.net> writes:
> 
>> On Fri, Apr 29, 2016 at 11:35:34AM +0200, larsxschneider@gmail.com wrote:
>> 
>>> +# The follow numbers need to be adjusted when new documentation is added.
>>> +test_file_count html 233
>>> +test_file_count xml 171
>>> +test_file_count 1 152
>> 
>> This seems like it will be really flaky and a pain in the future. I'm
>> not really sure what it's accomplishing, either. The earlier steps would
>> complain if something failed to render, wouldn't they? At some point we
>> have to have some faith in "make doc".
> 
> I agree. My proposal to check for a handful of generated files was just
> because this extra paranoia was almost free (just 3 lines of code that
> won't need particular maintenance).
> 
> In this case, I'm afraid the maintenance cost is much bigger than the
> expected benefits.

I agree, too. I wasn't sure about this check. That's why I added the little comment above to point out the problem.

Should I reroll?

Thanks, Lars

Jeff King· Apr 29, 2016, 14:32 UTC · re: Lars Schneider · lore

Re: [PATCH v2] travis-ci: build documentation

On Fri, Apr 29, 2016 at 04:22:05PM +0200, Lars Schneider wrote:
Show 9 quoted lines
> >>> +# The follow numbers need to be adjusted when new documentation is added.
> >>> +test_file_count html 233
> >>> +test_file_count xml 171
> >>> +test_file_count 1 152
> [...]
> I agree, too. I wasn't sure about this check. That's why I added
> the little comment above to point out the problem.
> 
> Should I reroll?
IMHO, yes.
-Peff
Stefan Beller· Apr 29, 2016, 17:27 UTC · re: Matthieu Moy · lore

Re: [PATCH v2] travis-ci: build documentation

On Fri, Apr 29, 2016 at 5:21 AM, Matthieu Moy <Matthieu.Moy@grenoble-inp.fr> wrote:

Show 20 quoted lines
> Jeff King <peff@peff.net> writes:
>
>> On Fri, Apr 29, 2016 at 11:35:34AM +0200, larsxschneider@gmail.com wrote:
>>
>>> +# The follow numbers need to be adjusted when new documentation is added.
>>> +test_file_count html 233
>>> +test_file_count xml 171
>>> +test_file_count 1 152
>>
>> This seems like it will be really flaky and a pain in the future. I'm
>> not really sure what it's accomplishing, either. The earlier steps would
>> complain if something failed to render, wouldn't they? At some point we
>> have to have some faith in "make doc".
>
> I agree. My proposal to check for a handful of generated files was just
> because this extra paranoia was almost free (just 3 lines of code that
> won't need particular maintenance).
>
> In this case, I'm afraid the maintenance cost is much bigger than the
> expected benefits.
So you proposed to check a handful files for its exact content?

This could be less of maintenance if we'd check with a "larger as" operator such as

    test_file_count_more_than html 200
using an arbitrary slightly smaller number.
Show 8 quoted lines
>
> --
> Matthieu Moy
> http://www-verimag.imag.fr/~moy/
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
Matthieu Moy· Apr 29, 2016, 17:53 UTC · re: Stefan Beller · lore

Re: [PATCH v2] travis-ci: build documentation

Stefan Beller <sbeller@google.com> writes:
Show 24 quoted lines
> On Fri, Apr 29, 2016 at 5:21 AM, Matthieu Moy
> <Matthieu.Moy@grenoble-inp.fr> wrote:
>> Jeff King <peff@peff.net> writes:
>>
>>> On Fri, Apr 29, 2016 at 11:35:34AM +0200, larsxschneider@gmail.com wrote:
>>>
>>>> +# The follow numbers need to be adjusted when new documentation is added.
>>>> +test_file_count html 233
>>>> +test_file_count xml 171
>>>> +test_file_count 1 152
>>>
>>> This seems like it will be really flaky and a pain in the future. I'm
>>> not really sure what it's accomplishing, either. The earlier steps would
>>> complain if something failed to render, wouldn't they? At some point we
>>> have to have some faith in "make doc".
>>
>> I agree. My proposal to check for a handful of generated files was just
>> because this extra paranoia was almost free (just 3 lines of code that
>> won't need particular maintenance).
>>
>> In this case, I'm afraid the maintenance cost is much bigger than the
>> expected benefits.
>
> So you proposed to check a handful files for its exact content?

No, just to check that the files exist and are non-empty, i.e. the "test -s" part of Lars' patch.

-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/
Lars Schneider· Apr 29, 2016, 20:44 UTC · re: Stefan Beller · lore

Re: [PATCH v2] travis-ci: build documentation

On 29 Apr 2016, at 19:27, Stefan Beller <sbeller@google.com> wrote:
Show 31 quoted lines
> On Fri, Apr 29, 2016 at 5:21 AM, Matthieu Moy
> <Matthieu.Moy@grenoble-inp.fr> wrote:
>> Jeff King <peff@peff.net> writes:
>> 
>>> On Fri, Apr 29, 2016 at 11:35:34AM +0200, larsxschneider@gmail.com wrote:
>>> 
>>>> +# The follow numbers need to be adjusted when new documentation is added.
>>>> +test_file_count html 233
>>>> +test_file_count xml 171
>>>> +test_file_count 1 152
>>> 
>>> This seems like it will be really flaky and a pain in the future. I'm
>>> not really sure what it's accomplishing, either. The earlier steps would
>>> complain if something failed to render, wouldn't they? At some point we
>>> have to have some faith in "make doc".
>> 
>> I agree. My proposal to check for a handful of generated files was just
>> because this extra paranoia was almost free (just 3 lines of code that
>> won't need particular maintenance).
>> 
>> In this case, I'm afraid the maintenance cost is much bigger than the
>> expected benefits.
> 
> So you proposed to check a handful files for its exact content?
> 
> This could be less of maintenance if we'd check with a "larger as" operator
> such as
> 
>    test_file_count_more_than html 200
> 
> using an arbitrary slightly smaller number.

Well, I was thinking about testing against something like $(find . -type f -name "git*.txt" | wc -l) but it the end all of this is not really meaningful I think...

- Lars
Show 10 quoted lines
> 
> 
>> 
>> --
>> Matthieu Moy
>> http://www-verimag.imag.fr/~moy/
>> --
>> To unsubscribe from this list: send the line "unsubscribe git" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
Junio C Hamano· Apr 29, 2016, 21:05 UTC · re: Lars Schneider · lore

Re: [PATCH v2] travis-ci: build documentation

Lars Schneider <larsxschneider@gmail.com> writes:
Show 10 quoted lines
>> This could be less of maintenance if we'd check with a "larger as" operator
>> such as
>> 
>>    test_file_count_more_than html 200
>> 
>> using an arbitrary slightly smaller number.
>
> Well, I was thinking about testing against something like 
> $(find . -type f -name "git*.txt" | wc -l) but it the end
> all of this is not really meaningful I think...

Either is too much, I would say--I have too much faith in the exit status from "make doc", I guess.

What would _REALLY_ be nice is a check that lets us catch an error like this deliberate breakage:

    diff --git a/Documentation/diff-options.txt b/Documentation/diff-options.txt
    index 4b0318e..a684f2d 100644
    --- a/Documentation/diff-options.txt
    +++ b/Documentation/diff-options.txt
    @@ -560,4 +560,4 @@ endif::git-format-patch[]
            Do not show any source or destination prefix.
     For more detailed explanation on these common options, see also
    -linkgit:gitdiffcore[7].
    +linkgit:gitdiffnore[7].

We just get a dangling link in the result without an error exit from "make doc"; neither "test -s" nor "size is about what we expect" would catch such a breakage, though.

Other things that might be of interest are
    make check-builtins
    make check-docs

but I am not sure if the latter built target is up to date (it has a whiltelist that needs to stay current). We rarely add new commands these days, so it is easy to forget what these build targets try to check, which makes them good candidates to be thrown into the set of automated tests.

larsxschneider@gmail.com· May 2, 2016, 20:20 UTC · re: Junio C Hamano · lore

[PATCH v3 0/2] travis-ci: build documentation

From: Lars Schneider <larsxschneider@gmail.com>
diff to v2:
* remove file count check for generated documentation as it is too flaky
* run `make doc` in `test-documentation.sh` to reduce the code in
  .travis.yml and ease the execution in other CI systems
* add linkgit check
* fix existing linkgit errors
* add `make check-builtins` and `make check-docs`
Thanks to Jeff, Matthieu, Stefan, and Junio for the review!

Cheers, Lars

Lars Schneider (2):
  Documentation: fix linkgit references
  travis-ci: build documentation
 .travis.yml                         | 15 +++++++++++++++
 Documentation/config.txt            |  6 +++---
 Documentation/git-check-ignore.txt  |  2 +-
 Documentation/git-filter-branch.txt |  4 ++--
 Documentation/git-for-each-ref.txt  |  2 +-
 Documentation/git-help.txt          |  4 ++--
 Documentation/git-instaweb.txt      |  4 ++--
 Documentation/git-sh-i18n.txt       |  2 +-
 ci/test-documentation.sh            | 24 ++++++++++++++++++++++++
 9 files changed, 51 insertions(+), 12 deletions(-)
 create mode 100755 ci/test-documentation.sh

-- 2.5.1

larsxschneider@gmail.com· May 2, 2016, 20:20 UTC · re: larsxschneider@gmail.com · lore

[PATCH v3 1/2] Documentation: fix linkgit references

From: Lars Schneider <larsxschneider@gmail.com>
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
---
 Documentation/config.txt            | 6 +++---
 Documentation/git-check-ignore.txt  | 2 +-
 Documentation/git-filter-branch.txt | 4 ++--
 Documentation/git-for-each-ref.txt  | 2 +-
 Documentation/git-help.txt          | 4 ++--
 Documentation/git-instaweb.txt      | 4 ++--
 Documentation/git-sh-i18n.txt       | 2 +-
 7 files changed, 12 insertions(+), 12 deletions(-)
Show changes to 7 files +12 −12

Documentation/config.txt, Documentation/git-check-ignore.txt, Documentation/git-filter-branch.txt, Documentation/git-for-each-ref.txt, Documentation/git-help.txt, Documentation/git-instaweb.txt, Documentation/git-sh-i18n.txt

diff --git a/Documentation/config.txt b/Documentation/config.txt
index c7bbe98..c5f1d6b 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -894,7 +894,7 @@ branch.<name>.description::
 browser.<tool>.cmd::
 	Specify the command to invoke the specified browser. The
 	specified command is evaluated in shell with the URLs passed
-	as arguments. (See linkgit:git-web{litdd}browse[1].)
+	as arguments. (See linkgit:git-web--browse[1].)
 
 browser.<tool>.path::
 	Override the path for the given tool that may be used to
@@ -1494,7 +1494,7 @@ gui.diffContext::
 	made by the linkgit:git-gui[1]. The default is "5".
 
 gui.displayUntracked::
-	Determines if linkgit::git-gui[1] shows untracked files
+	Determines if linkgit:git-gui[1] shows untracked files
 	in the file list. The default is "true".
 
 gui.encoding::
@@ -1665,7 +1665,7 @@ http.cookieFile::
 	File containing previously stored cookie lines which should be used
 	in the Git http session, if they match the server. The file format
 	of the file to read cookies from should be plain HTTP headers or
-	the Netscape/Mozilla cookie file format (see linkgit:curl[1]).
+	the Netscape/Mozilla cookie file format (see `curl(1)`).
 	NOTE that the file specified with http.cookieFile is only used as
 	input unless http.saveCookies is set.
 
diff --git a/Documentation/git-check-ignore.txt b/Documentation/git-check-ignore.txt
index e94367a..9a85998 100644
--- a/Documentation/git-check-ignore.txt
+++ b/Documentation/git-check-ignore.txt
@@ -112,7 +112,7 @@ EXIT STATUS
 SEE ALSO
 --------
 linkgit:gitignore[5]
-linkgit:gitconfig[5]
+linkgit:git-config[5]
 linkgit:git-ls-files[1]
 
 GIT
diff --git a/Documentation/git-filter-branch.txt b/Documentation/git-filter-branch.txt
index 73fd9e8..6538cb1 100644
--- a/Documentation/git-filter-branch.txt
+++ b/Documentation/git-filter-branch.txt
@@ -205,8 +205,8 @@ to other tags will be rewritten to point to the underlying commit.
 Remap to ancestor
 ~~~~~~~~~~~~~~~~~
 
-By using linkgit:rev-list[1] arguments, e.g., path limiters, you can limit the
-set of revisions which get rewritten. However, positive refs on the command
+By using linkgit:git-rev-list[1] arguments, e.g., path limiters, you can limit
+the set of revisions which get rewritten. However, positive refs on the command
 line are distinguished: we don't let them be excluded by such limiters. For
 this purpose, they are instead rewritten to point at the nearest ancestor that
 was not excluded.
diff --git a/Documentation/git-for-each-ref.txt b/Documentation/git-for-each-ref.txt
index c52578b..d9d406d 100644
--- a/Documentation/git-for-each-ref.txt
+++ b/Documentation/git-for-each-ref.txt
@@ -179,7 +179,7 @@ returns an empty string instead.
 
 As a special case for the date-type fields, you may specify a format for
 the date by adding `:` followed by date format name (see the
-values the `--date` option to linkgit::git-rev-list[1] takes).
+values the `--date` option to linkgit:git-rev-list[1] takes).
 
 
 EXAMPLES
diff --git a/Documentation/git-help.txt b/Documentation/git-help.txt
index 3956525..7af18f9 100644
--- a/Documentation/git-help.txt
+++ b/Documentation/git-help.txt
@@ -72,7 +72,7 @@ The web browser can be specified using the configuration variable
 'help.browser', or 'web.browser' if the former is not set. If none of
 these config variables is set, the 'git web{litdd}browse' helper script
 (called by 'git help') will pick a suitable default. See
-linkgit:git-web{litdd}browse[1] for more information about this.
+linkgit:git-web--browse[1] for more information about this.
 
 CONFIGURATION VARIABLES
 -----------------------
@@ -95,7 +95,7 @@ help.browser, web.browser and browser.<tool>.path
 The 'help.browser', 'web.browser' and 'browser.<tool>.path' will also
 be checked if the 'web' format is chosen (either by command-line
 option or configuration variable). See '-w|--web' in the OPTIONS
-section above and linkgit:git-web{litdd}browse[1].
+section above and linkgit:git-web--browse[1].
 
 man.viewer
 ~~~~~~~~~~
diff --git a/Documentation/git-instaweb.txt b/Documentation/git-instaweb.txt
index cc75b25..b95cc15 100644
--- a/Documentation/git-instaweb.txt
+++ b/Documentation/git-instaweb.txt
@@ -46,7 +46,7 @@ OPTIONS
 	The web browser that should be used to view the gitweb
 	page. This will be passed to the 'git web{litdd}browse' helper
 	script along with the URL of the gitweb instance. See
-	linkgit:git-web{litdd}browse[1] for more information about this. If
+	linkgit:git-web--browse[1] for more information about this. If
 	the script fails, the URL will be printed to stdout.
 
 start::
@@ -82,7 +82,7 @@ You may specify configuration in your .git/config
 
 If the configuration variable 'instaweb.browser' is not set,
 'web.browser' will be used instead if it is defined. See
-linkgit:git-web{litdd}browse[1] for more information about this.
+linkgit:git-web--browse[1] for more information about this.
 
 SEE ALSO
 --------
diff --git a/Documentation/git-sh-i18n.txt b/Documentation/git-sh-i18n.txt
index 60cf49c..eafa55a 100644
--- a/Documentation/git-sh-i18n.txt
+++ b/Documentation/git-sh-i18n.txt
@@ -35,7 +35,7 @@ gettext::
 eval_gettext::
 	Currently a dummy fall-through function implemented as a wrapper
 	around `printf(1)` with variables expanded by the
-	linkgit:git-sh-i18n{litdd}envsubst[1] helper. Will be replaced by a
+	linkgit:git-sh-i18n--envsubst[1] helper. Will be replaced by a
 	real gettext implementation in a later version.
 
 GIT
-- 
2.5.1
Jeff King· May 2, 2016, 20:34 UTC · re: larsxschneider@gmail.com · lore

Re: [PATCH v3 1/2] Documentation: fix linkgit references

On Mon, May 02, 2016 at 10:20:04PM +0200, larsxschneider@gmail.com wrote:
> From: Lars Schneider <larsxschneider@gmail.com>
> 
> Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
> ---

Fix how? Your commit message doesn't say why this is a good idea. Since this is v3, I'm guessing that reasoning is on the list, but it needs to be summarized here in the commit message.

Show 10 quoted lines
> diff --git a/Documentation/config.txt b/Documentation/config.txt
> index c7bbe98..c5f1d6b 100644
> --- a/Documentation/config.txt
> +++ b/Documentation/config.txt
> @@ -894,7 +894,7 @@ branch.<name>.description::
>  browser.<tool>.cmd::
>  	Specify the command to invoke the specified browser. The
>  	specified command is evaluated in shell with the URLs passed
> -	as arguments. (See linkgit:git-web{litdd}browse[1].)
> +	as arguments. (See linkgit:git-web--browse[1].)

The existing code renders fine for me with "make git-config.1". But with your patch, I get a unicode emdash, which is wrong:

--- old 2016-05-02 16:27:53.242050262 -0400 +++ new 2016-05-02 16:27:57.742050360 -0400

Show changes to diff +2 −3
@@ -978,7 +978,7 @@
 
        browser.<tool>.cmd
            Specify the command to invoke the specified browser. The specified command is evaluated in shell with the
-           URLs passed as arguments. (See git-web--browse(1).)
+           URLs passed as arguments. (See git-web—browse(1).)
 
        browser.<tool>.path
            Override the path for the given tool that may be used to browse HTML help (see -w option in git-help(1))

In case it's hard to see with your font, the generated roff looks like
this:

-\fBgit-web--browse\fR(1)\&.)
+\fBgit-web\(embrowse\fR(1)\&.)

So I think that's a step backwards. I did check the asciidoctor
rendering on git-scm.com, though, and it gets the {litdd} case wrong. So
I think it does need fixing, but we need a solution that looks correct
in both cases. Maybe linkgit:`git-web--browse`[1] would work; it seems
OK with my version of asciidoc, but I have a feeling it will run into
the same problem with asciidoctor (if it's not respecting {litdd} in
that context, it's probably also not respecting backticks).

-Peff
Lars Schneider· May 3, 2016, 08:30 UTC · re: Jeff King · lore

Re: [PATCH v3 1/2] Documentation: fix linkgit references

On 02 May 2016, at 22:34, Jeff King <peff@peff.net> wrote:
Show 10 quoted lines
> On Mon, May 02, 2016 at 10:20:04PM +0200, larsxschneider@gmail.com wrote:
> 
>> From: Lars Schneider <larsxschneider@gmail.com>
>> 
>> Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
>> ---
> 
> Fix how? Your commit message doesn't say why this is a good idea. Since
> this is v3, I'm guessing that reasoning is on the list, but it needs to
> be summarized here in the commit message.

You are right, I should have explained my thinking a bit more detailed. A few of the fixed linkgit references are just typos, e.g.:

-linkgit:gitconfig[5] +linkgit:git-config[5]

-values the `--date` option to linkgit::git-rev-list[1] takes). +values the `--date` option to linkgit:git-rev-list[1] takes).

- the Netscape/Mozilla cookie file format (see linkgit:curl[1]). + the Netscape/Mozilla cookie file format (see `curl(1)`).

I mistakenly assumed the "{litdd}" was a typo/bad search replace, too. I checked this website and thought my change would fix it, too:

https://git-scm.com/docs/git-config

There it is rendered as "(See git-web{litdd}browse[1].)" and the link is broken.

Show 31 quoted lines
>> diff --git a/Documentation/config.txt b/Documentation/config.txt
>> index c7bbe98..c5f1d6b 100644
>> --- a/Documentation/config.txt
>> +++ b/Documentation/config.txt
>> @@ -894,7 +894,7 @@ branch.<name>.description::
>> browser.<tool>.cmd::
>> 	Specify the command to invoke the specified browser. The
>> 	specified command is evaluated in shell with the URLs passed
>> -	as arguments. (See linkgit:git-web{litdd}browse[1].)
>> +	as arguments. (See linkgit:git-web--browse[1].)
> 
> The existing code renders fine for me with "make git-config.1". But with
> your patch, I get a unicode emdash, which is wrong:
> 
> --- old	2016-05-02 16:27:53.242050262 -0400
> +++ new	2016-05-02 16:27:57.742050360 -0400
> @@ -978,7 +978,7 @@
> 
>        browser.<tool>.cmd
>            Specify the command to invoke the specified browser. The specified command is evaluated in shell with the
> -           URLs passed as arguments. (See git-web--browse(1).)
> +           URLs passed as arguments. (See git-web—browse(1).)
> 
>        browser.<tool>.path
>            Override the path for the given tool that may be used to browse HTML help (see -w option in git-help(1))
> 
> In case it's hard to see with your font, the generated roff looks like
> this:
> 
> -\fBgit-web--browse\fR(1)\&.)
> +\fBgit-web\(embrowse\fR(1)\&.)
I can confirm. Sorry, I indeed missed that.
Show 7 quoted lines
> So I think that's a step backwards. I did check the asciidoctor
> rendering on git-scm.com, though, and it gets the {litdd} case wrong. So
> I think it does need fixing, but we need a solution that looks correct
> in both cases. Maybe linkgit:`git-web--browse`[1] would work; it seems
> OK with my version of asciidoc, but I have a feeling it will run into
> the same problem with asciidoctor (if it's not respecting {litdd} in
> that context, it's probably also not respecting backticks).

I will play with this to find a solution. Would it be an option to replace "--" with "-"? Why do we need two dashes if they cause trouble?

Thanks for the review, Lars

Junio C Hamano· May 3, 2016, 15:41 UTC · re: Lars Schneider · lore

Re: [PATCH v3 1/2] Documentation: fix linkgit references

Lars Schneider <larsxschneider@gmail.com> writes:
Show 17 quoted lines
> On 02 May 2016, at 22:34, Jeff King <peff@peff.net> wrote:
>
>> On Mon, May 02, 2016 at 10:20:04PM +0200, larsxschneider@gmail.com wrote:
>> 
>>> From: Lars Schneider <larsxschneider@gmail.com>
>>> 
>>> Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
>>> ---
>> 
>> Fix how? Your commit message doesn't say why this is a good idea. Since
>> this is v3, I'm guessing that reasoning is on the list, but it needs to
>> be summarized here in the commit message.
> You are right, I should have explained my thinking a bit more detailed. 
> A few of the fixed linkgit references are just typos, e.g.:
>
> -linkgit:gitconfig[5]
> +linkgit:git-config[5]

I didn't run "git blame" yet, so it might be that we used to have it in section 5, but "git config --help" puts the latter in section 1.

Even though you obviously were fooled by AsciiDoctor regarding literal double dash {litdd}, other parts of the changes did look good typofixes. Thanks for starting this.

Jeff King· May 3, 2016, 17:59 UTC · re: Junio C Hamano · lore

Re: [PATCH v3 1/2] Documentation: fix linkgit references

On Tue, May 03, 2016 at 08:41:37AM -0700, Junio C Hamano wrote:
> Even though you obviously were fooled by AsciiDoctor regarding
> literal double dash {litdd}, other parts of the changes did look
> good typofixes.  Thanks for starting this.

Yeah, I stopped reading after seeing the first hunk, but I agree the rest look trivial.

-Peff
Jeff King· May 3, 2016, 17:58 UTC · re: Lars Schneider · lore

Re: [PATCH v3 1/2] Documentation: fix linkgit references

On Tue, May 03, 2016 at 10:30:09AM +0200, Lars Schneider wrote:
Show 9 quoted lines
> > So I think that's a step backwards. I did check the asciidoctor
> > rendering on git-scm.com, though, and it gets the {litdd} case wrong. So
> > I think it does need fixing, but we need a solution that looks correct
> > in both cases. Maybe linkgit:`git-web--browse`[1] would work; it seems
> > OK with my version of asciidoc, but I have a feeling it will run into
> > the same problem with asciidoctor (if it's not respecting {litdd} in
> > that context, it's probably also not respecting backticks).
> I will play with this to find a solution. Would it be an option to
> replace "--" with "-"? Why do we need two dashes if they cause trouble?

We use two dashes to signify "internal" programs that users should not rely on. So "git-web-browse" would be something we'd expect to support forever, but "git-web--browse" is an implementation detail of one of our scripts (that just happens to require an extra program).

So I don't think we want to switch away from that convention just to make the documentation work.

AFAICT, the {litdd} is working fine with asciidoc; it's only asciidoctor that is the problem. So the first step may be talking with asciidoctor folks to see if it's a bug, or if they have a recommended workaround.

-Peff
larsxschneider@gmail.com· May 4, 2016, 08:38 UTC · re: Lars Schneider · lore

[PATCH v4 0/2] travis-ci: build documentation

From: Lars Schneider <larsxschneider@gmail.com>
diff to v3:
* Revert the change from "{litdd}" to "--" in the documentation.
  "{litdd}" is rendered wrong in some HTML output [1], but "--"
  breaks the roff output ... I will investigate this and try to fix
  it in a future patch.
* I removed the doc link checker for now. I will try to reintroduce
  an improved version of the link checker as part of `make check-docs`
  in a future patch.

Thanks Peff and Junio for the review, Lars

[1] https://git-scm.com/docs/git-config
Lars Schneider (2):
  Documentation: fix linkgit references
  travis-ci: build documentation
 .travis.yml                         | 15 +++++++++++++++
 Documentation/config.txt            |  4 ++--
 Documentation/git-check-ignore.txt  |  2 +-
 Documentation/git-filter-branch.txt |  4 ++--
 Documentation/git-for-each-ref.txt  |  2 +-
 ci/test-documentation.sh            | 14 ++++++++++++++
 6 files changed, 35 insertions(+), 6 deletions(-)
 create mode 100755 ci/test-documentation.sh

-- 2.5.1

larsxschneider@gmail.com· May 4, 2016, 08:38 UTC · re: larsxschneider@gmail.com · lore

[PATCH v4 1/2] Documentation: fix linkgit references

From: Lars Schneider <larsxschneider@gmail.com>
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
---
 Documentation/config.txt            | 4 ++--
 Documentation/git-check-ignore.txt  | 2 +-
 Documentation/git-filter-branch.txt | 4 ++--
 Documentation/git-for-each-ref.txt  | 2 +-
 4 files changed, 6 insertions(+), 6 deletions(-)
Show changes to 4 files +6 −6

Documentation/config.txt, Documentation/git-check-ignore.txt, Documentation/git-filter-branch.txt, Documentation/git-for-each-ref.txt

diff --git a/Documentation/config.txt b/Documentation/config.txt
index c7bbe98..5683400 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -1494,7 +1494,7 @@ gui.diffContext::
 	made by the linkgit:git-gui[1]. The default is "5".
 
 gui.displayUntracked::
-	Determines if linkgit::git-gui[1] shows untracked files
+	Determines if linkgit:git-gui[1] shows untracked files
 	in the file list. The default is "true".
 
 gui.encoding::
@@ -1665,7 +1665,7 @@ http.cookieFile::
 	File containing previously stored cookie lines which should be used
 	in the Git http session, if they match the server. The file format
 	of the file to read cookies from should be plain HTTP headers or
-	the Netscape/Mozilla cookie file format (see linkgit:curl[1]).
+	the Netscape/Mozilla cookie file format (see `curl(1)`).
 	NOTE that the file specified with http.cookieFile is only used as
 	input unless http.saveCookies is set.
 
diff --git a/Documentation/git-check-ignore.txt b/Documentation/git-check-ignore.txt
index e94367a..9a85998 100644
--- a/Documentation/git-check-ignore.txt
+++ b/Documentation/git-check-ignore.txt
@@ -112,7 +112,7 @@ EXIT STATUS
 SEE ALSO
 --------
 linkgit:gitignore[5]
-linkgit:gitconfig[5]
+linkgit:git-config[5]
 linkgit:git-ls-files[1]
 
 GIT
diff --git a/Documentation/git-filter-branch.txt b/Documentation/git-filter-branch.txt
index 73fd9e8..6538cb1 100644
--- a/Documentation/git-filter-branch.txt
+++ b/Documentation/git-filter-branch.txt
@@ -205,8 +205,8 @@ to other tags will be rewritten to point to the underlying commit.
 Remap to ancestor
 ~~~~~~~~~~~~~~~~~
 
-By using linkgit:rev-list[1] arguments, e.g., path limiters, you can limit the
-set of revisions which get rewritten. However, positive refs on the command
+By using linkgit:git-rev-list[1] arguments, e.g., path limiters, you can limit
+the set of revisions which get rewritten. However, positive refs on the command
 line are distinguished: we don't let them be excluded by such limiters. For
 this purpose, they are instead rewritten to point at the nearest ancestor that
 was not excluded.
diff --git a/Documentation/git-for-each-ref.txt b/Documentation/git-for-each-ref.txt
index c52578b..d9d406d 100644
--- a/Documentation/git-for-each-ref.txt
+++ b/Documentation/git-for-each-ref.txt
@@ -179,7 +179,7 @@ returns an empty string instead.
 
 As a special case for the date-type fields, you may specify a format for
 the date by adding `:` followed by date format name (see the
-values the `--date` option to linkgit::git-rev-list[1] takes).
+values the `--date` option to linkgit:git-rev-list[1] takes).
 
 
 EXAMPLES
-- 
2.5.1
Lars Schneider· May 4, 2016, 08:43 UTC · re: larsxschneider@gmail.com · lore

Re: [PATCH v4 1/2] Documentation: fix linkgit references

On 04 May 2016, at 10:38, larsxschneider@gmail.com wrote:
Show 57 quoted lines
> From: Lars Schneider <larsxschneider@gmail.com>
> 
> Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
> ---
> Documentation/config.txt            | 4 ++--
> Documentation/git-check-ignore.txt  | 2 +-
> Documentation/git-filter-branch.txt | 4 ++--
> Documentation/git-for-each-ref.txt  | 2 +-
> 4 files changed, 6 insertions(+), 6 deletions(-)
> 
> diff --git a/Documentation/config.txt b/Documentation/config.txt
> index c7bbe98..5683400 100644
> --- a/Documentation/config.txt
> +++ b/Documentation/config.txt
> @@ -1494,7 +1494,7 @@ gui.diffContext::
> 	made by the linkgit:git-gui[1]. The default is "5".
> 
> gui.displayUntracked::
> -	Determines if linkgit::git-gui[1] shows untracked files
> +	Determines if linkgit:git-gui[1] shows untracked files
> 	in the file list. The default is "true".
> 
> gui.encoding::
> @@ -1665,7 +1665,7 @@ http.cookieFile::
> 	File containing previously stored cookie lines which should be used
> 	in the Git http session, if they match the server. The file format
> 	of the file to read cookies from should be plain HTTP headers or
> -	the Netscape/Mozilla cookie file format (see linkgit:curl[1]).
> +	the Netscape/Mozilla cookie file format (see `curl(1)`).
> 	NOTE that the file specified with http.cookieFile is only used as
> 	input unless http.saveCookies is set.
> 
> diff --git a/Documentation/git-check-ignore.txt b/Documentation/git-check-ignore.txt
> index e94367a..9a85998 100644
> --- a/Documentation/git-check-ignore.txt
> +++ b/Documentation/git-check-ignore.txt
> @@ -112,7 +112,7 @@ EXIT STATUS
> SEE ALSO
> --------
> linkgit:gitignore[5]
> -linkgit:gitconfig[5]
> +linkgit:git-config[5]
> linkgit:git-ls-files[1]
> 
> GIT
> diff --git a/Documentation/git-filter-branch.txt b/Documentation/git-filter-branch.txt
> index 73fd9e8..6538cb1 100644
> --- a/Documentation/git-filter-branch.txt
> +++ b/Documentation/git-filter-branch.txt
> @@ -205,8 +205,8 @@ to other tags will be rewritten to point to the underlying commit.
> Remap to ancestor
> ~~~~~~~~~~~~~~~~~
> 
> -By using linkgit:rev-list[1] arguments, e.g., path limiters, you can limit the
> -set of revisions which get rewritten. However, positive refs on the command
> +By using linkgit:git-rev-list[1] arguments, e.g., path limiters, you can limit
> +the set of revisions which get rewritten. However, positive refs on the command
All other linkgit fixes seem legimiate to me although I am not sure of this case
-linkgit:rev-list[1] 
+linkgit:git-rev-list[1]
  
"rev-list" works but I think "git-rev-list" would be the canonical form?
See: https://git-scm.com/docs/git-filter-branch
Show 19 quoted lines
> line are distinguished: we don't let them be excluded by such limiters. For
> this purpose, they are instead rewritten to point at the nearest ancestor that
> was not excluded.
> diff --git a/Documentation/git-for-each-ref.txt b/Documentation/git-for-each-ref.txt
> index c52578b..d9d406d 100644
> --- a/Documentation/git-for-each-ref.txt
> +++ b/Documentation/git-for-each-ref.txt
> @@ -179,7 +179,7 @@ returns an empty string instead.
> 
> As a special case for the date-type fields, you may specify a format for
> the date by adding `:` followed by date format name (see the
> -values the `--date` option to linkgit::git-rev-list[1] takes).
> +values the `--date` option to linkgit:git-rev-list[1] takes).
> 
> 
> EXAMPLES
> -- 
> 2.5.1
> 
Jeff King· May 4, 2016, 08:57 UTC · re: Lars Schneider · lore

Re: [PATCH v4 1/2] Documentation: fix linkgit references

On Wed, May 04, 2016 at 10:43:04AM +0200, Lars Schneider wrote:
Show 20 quoted lines
> > diff --git a/Documentation/git-filter-branch.txt b/Documentation/git-filter-branch.txt
> > index 73fd9e8..6538cb1 100644
> > --- a/Documentation/git-filter-branch.txt
> > +++ b/Documentation/git-filter-branch.txt
> > @@ -205,8 +205,8 @@ to other tags will be rewritten to point to the underlying commit.
> > Remap to ancestor
> > ~~~~~~~~~~~~~~~~~
> > 
> > -By using linkgit:rev-list[1] arguments, e.g., path limiters, you can limit the
> > -set of revisions which get rewritten. However, positive refs on the command
> > +By using linkgit:git-rev-list[1] arguments, e.g., path limiters, you can limit
> > +the set of revisions which get rewritten. However, positive refs on the command
> 
> All other linkgit fixes seem legimiate to me although I am not sure of this case
> 
> -linkgit:rev-list[1] 
> +linkgit:git-rev-list[1]
>   
> "rev-list" works but I think "git-rev-list" would be the canonical form?
> See: https://git-scm.com/docs/git-filter-branch

It should definitely be "git-rev-list". The "linkgit" macro will format whatever text you feed it. For the manpages, that doesn't matter, because they don't actually hyperlink. But for other formats (like HTML), using just "rev-list" will generate a broken link.

-Peff
Ramsay Jones· May 4, 2016, 11:38 UTC · re: Lars Schneider · lore

Re: [PATCH v4 1/2] Documentation: fix linkgit references

On 04/05/16 09:43, Lars Schneider wrote:
Show 45 quoted lines
> 
> On 04 May 2016, at 10:38, larsxschneider@gmail.com wrote:
> 
>> From: Lars Schneider <larsxschneider@gmail.com>
>>
>> Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
>> ---
>> Documentation/config.txt            | 4 ++--
>> Documentation/git-check-ignore.txt  | 2 +-
>> Documentation/git-filter-branch.txt | 4 ++--
>> Documentation/git-for-each-ref.txt  | 2 +-
>> 4 files changed, 6 insertions(+), 6 deletions(-)
>>
>> diff --git a/Documentation/config.txt b/Documentation/config.txt
>> index c7bbe98..5683400 100644
>> --- a/Documentation/config.txt
>> +++ b/Documentation/config.txt
>> @@ -1494,7 +1494,7 @@ gui.diffContext::
>> 	made by the linkgit:git-gui[1]. The default is "5".
>>
>> gui.displayUntracked::
>> -	Determines if linkgit::git-gui[1] shows untracked files
>> +	Determines if linkgit:git-gui[1] shows untracked files
>> 	in the file list. The default is "true".
>>
>> gui.encoding::
>> @@ -1665,7 +1665,7 @@ http.cookieFile::
>> 	File containing previously stored cookie lines which should be used
>> 	in the Git http session, if they match the server. The file format
>> 	of the file to read cookies from should be plain HTTP headers or
>> -	the Netscape/Mozilla cookie file format (see linkgit:curl[1]).
>> +	the Netscape/Mozilla cookie file format (see `curl(1)`).
>> 	NOTE that the file specified with http.cookieFile is only used as
>> 	input unless http.saveCookies is set.
>>
>> diff --git a/Documentation/git-check-ignore.txt b/Documentation/git-check-ignore.txt
>> index e94367a..9a85998 100644
>> --- a/Documentation/git-check-ignore.txt
>> +++ b/Documentation/git-check-ignore.txt
>> @@ -112,7 +112,7 @@ EXIT STATUS
>> SEE ALSO
>> --------
>> linkgit:gitignore[5]
>> -linkgit:gitconfig[5]
>> +linkgit:git-config[5]
I think Junio already noted, git-config is in section 1 not 5.

ATB, Ramsay Jones

Junio C Hamano· May 4, 2016, 17:28 UTC · re: Ramsay Jones · lore

Re: [PATCH v4 1/2] Documentation: fix linkgit references

Ramsay Jones <ramsay@ramsayjones.plus.com> writes:
Show 12 quoted lines
>>> diff --git a/Documentation/git-check-ignore.txt b/Documentation/git-check-ignore.txt
>>> index e94367a..9a85998 100644
>>> --- a/Documentation/git-check-ignore.txt
>>> +++ b/Documentation/git-check-ignore.txt
>>> @@ -112,7 +112,7 @@ EXIT STATUS
>>> SEE ALSO
>>> --------
>>> linkgit:gitignore[5]
>>> -linkgit:gitconfig[5]
>>> +linkgit:git-config[5]
>
> I think Junio already noted, git-config is in section 1 not 5.

Not just that, I am afraid. This came from 368aa529 (add git-check-ignore sub-command, 2013-01-06) and it added these:

+linkgit:gitignore[5] +linkgit:gitconfig[5] +linkgit:git-ls-files[5]

The last one was later corrected, but who knows what other mistakes there are?

So I used the script attached at the bottom to audit the whole thing, and the result is here.

-- >8 -- Documentation/config.txt:1497: nongit link: :git-gui[1] Documentation/config.txt:1662: nongit link: curl[1] Documentation/diff-options.txt:274: wrong section (should be 5): gitattributes[1] Documentation/git-check-ignore.txt:115: no such source: gitconfig[5] Documentation/git-filter-branch.txt:208: nongit link: rev-list[1] Documentation/git-for-each-ref.txt:182: nongit link: :git-rev-list[1] Documentation/git-notes.txt:405: wrong section (should be 1): git[7] Documentation/technical/api-credentials.txt:246: wrong section (should be 1): git-credential[7] Documentation/technical/api-credentials.txt:271: wrong section (should be 1): git-config[5] -- 8< --

I do not think there is any false positive above, so perhaps the checker script below can be used as the link checker we discussed?

-- >8 -- #!/bin/sh

git grep -l linkgit: Documentation/ |
while read path
do
	perl -e '
	sub report {
		my ($where, $what, $error) = @_;
		print "$where: $error: $what\n";
	}
	sub grab_section {
		my ($page) = @_;
		open my $fh, "<", "Documentation/$page.txt";
		my $firstline = <$fh>;
		chomp $firstline;
		close $fh;
		my ($section) = ($firstline =~ /.*\((\d)\)$/);
		return $section;
	}
	while (<>) {
		my $where = "$ARGV:$.";
		while (s/linkgit:((.*?)\[(\d)\])//) {
			my ($target, $page, $section) = ($1, $2, $3);
			# De-AsciiDoc
			$page =~ s/{litdd}/--/g;
			if ($page !~ /^git/) {
				report($where, $target, "nongit link");
				next;
			}
			if (! -f "Documentation/$page.txt") {
				report($where, $target, "no such source");
				next;
			}
			$real_section = grab_section($page); 
			if ($real_section != $section) {
				report($where, $target,
					"wrong section (should be $real_section)");
				next;
			}
		}
	}
        ' "$path"
done
Junio C Hamano· May 4, 2016, 17:36 UTC · re: Junio C Hamano · lore

Re: [PATCH v4 1/2] Documentation: fix linkgit references

Junio C Hamano <gitster@pobox.com> writes:
> So I used the script attached at the bottom to audit the whole
> thing, and the result is here.
> ...

While I was at it, I just did this to make the documentation set lint clean.

-- >8 --

There are a handful of incorrect "linkgit:<page>[<section>]" instances in our documentation set.

 * Some have an extra colon after "linkgit:"; fix them by removing
   the extra colon;
 * Some refer to a page outside the Git suite, namely curl(1); fix
   them by using the `curl(1)` that already appears on the same page
   for the same purpose of referring the readers to its manual page.
 * Some spell the name of the page incorrectly, e.g. "rev-list" when
   they mean "git-rev-list"; fix them.
 * Some list the manual section incorrectly; fix them to make sure
   they match what is at the top of the target of the link.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 Documentation/config.txt                    | 4 ++--
 Documentation/diff-options.txt              | 2 +-
 Documentation/everyday.txto                 | 2 +-
 Documentation/git-check-ignore.txt          | 2 +-
 Documentation/git-filter-branch.txt         | 2 +-
 Documentation/git-for-each-ref.txt          | 2 +-
 Documentation/git-notes.txt                 | 2 +-
 Documentation/technical/api-credentials.txt | 4 ++--
 8 files changed, 10 insertions(+), 10 deletions(-)
Show changes to 8 files +10 −10

Documentation/config.txt, Documentation/diff-options.txt, Documentation/everyday.txto, Documentation/git-check-ignore.txt, Documentation/git-filter-branch.txt, Documentation/git-for-each-ref.txt, Documentation/git-notes.txt, Documentation/technical/api-credentials.txt

diff --git a/Documentation/config.txt b/Documentation/config.txt
index 42d2b50..f37e16b 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -1494,7 +1494,7 @@ gui.diffContext::
 	made by the linkgit:git-gui[1]. The default is "5".
 
 gui.displayUntracked::
-	Determines if linkgit::git-gui[1] shows untracked files
+	Determines if linkgit:git-gui[1] shows untracked files
 	in the file list. The default is "true".
 
 gui.encoding::
@@ -1659,7 +1659,7 @@ http.cookieFile::
 	File containing previously stored cookie lines which should be used
 	in the Git http session, if they match the server. The file format
 	of the file to read cookies from should be plain HTTP headers or
-	the Netscape/Mozilla cookie file format (see linkgit:curl[1]).
+	the Netscape/Mozilla cookie file format (see `curl(1)`).
 	NOTE that the file specified with http.cookieFile is only used as
 	input unless http.saveCookies is set.
 
diff --git a/Documentation/diff-options.txt b/Documentation/diff-options.txt
index 4b0318e..3cb3015 100644
--- a/Documentation/diff-options.txt
+++ b/Documentation/diff-options.txt
@@ -271,7 +271,7 @@ For example, `--word-diff-regex=.` will treat each character as a word
 and, correspondingly, show differences character by character.
 +
 The regex can also be set via a diff driver or configuration option, see
-linkgit:gitattributes[1] or linkgit:git-config[1].  Giving it explicitly
+linkgit:gitattributes[5] or linkgit:git-config[1].  Giving it explicitly
 overrides any diff driver or configuration setting.  Diff drivers
 override configuration settings.
 
diff --git a/Documentation/everyday.txto b/Documentation/everyday.txto
index c5047d8..ae555bd 100644
--- a/Documentation/everyday.txto
+++ b/Documentation/everyday.txto
@@ -1,7 +1,7 @@
 Everyday Git With 20 Commands Or So
 ===================================
 
-This document has been moved to linkgit:giteveryday[1].
+This document has been moved to linkgit:giteveryday[7].
 
 Please let the owners of the referring site know so that they can update the
 link you clicked to get here.
diff --git a/Documentation/git-check-ignore.txt b/Documentation/git-check-ignore.txt
index e94367a..611754f 100644
--- a/Documentation/git-check-ignore.txt
+++ b/Documentation/git-check-ignore.txt
@@ -112,7 +112,7 @@ EXIT STATUS
 SEE ALSO
 --------
 linkgit:gitignore[5]
-linkgit:gitconfig[5]
+linkgit:git-config[1]
 linkgit:git-ls-files[1]
 
 GIT
diff --git a/Documentation/git-filter-branch.txt b/Documentation/git-filter-branch.txt
index 73fd9e8..003731f 100644
--- a/Documentation/git-filter-branch.txt
+++ b/Documentation/git-filter-branch.txt
@@ -205,7 +205,7 @@ to other tags will be rewritten to point to the underlying commit.
 Remap to ancestor
 ~~~~~~~~~~~~~~~~~
 
-By using linkgit:rev-list[1] arguments, e.g., path limiters, you can limit the
+By using linkgit:git-rev-list[1] arguments, e.g., path limiters, you can limit the
 set of revisions which get rewritten. However, positive refs on the command
 line are distinguished: we don't let them be excluded by such limiters. For
 this purpose, they are instead rewritten to point at the nearest ancestor that
diff --git a/Documentation/git-for-each-ref.txt b/Documentation/git-for-each-ref.txt
index c52578b..d9d406d 100644
--- a/Documentation/git-for-each-ref.txt
+++ b/Documentation/git-for-each-ref.txt
@@ -179,7 +179,7 @@ returns an empty string instead.
 
 As a special case for the date-type fields, you may specify a format for
 the date by adding `:` followed by date format name (see the
-values the `--date` option to linkgit::git-rev-list[1] takes).
+values the `--date` option to linkgit:git-rev-list[1] takes).
 
 
 EXAMPLES
diff --git a/Documentation/git-notes.txt b/Documentation/git-notes.txt
index 8de3499..9c4fd68 100644
--- a/Documentation/git-notes.txt
+++ b/Documentation/git-notes.txt
@@ -402,4 +402,4 @@ on the `notes.rewrite.<command>` and `notes.rewriteRef` settings.
 
 GIT
 ---
-Part of the linkgit:git[7] suite
+Part of the linkgit:git[1] suite
diff --git a/Documentation/technical/api-credentials.txt b/Documentation/technical/api-credentials.txt
index e44426d..75368f2 100644
--- a/Documentation/technical/api-credentials.txt
+++ b/Documentation/technical/api-credentials.txt
@@ -243,7 +243,7 @@ appended to its command line, which is one of:
 The details of the credential will be provided on the helper's stdin
 stream. The exact format is the same as the input/output format of the
 `git credential` plumbing command (see the section `INPUT/OUTPUT
-FORMAT` in linkgit:git-credential[7] for a detailed specification).
+FORMAT` in linkgit:git-credential[1] for a detailed specification).
 
 For a `get` operation, the helper should produce a list of attributes
 on stdout in the same format. A helper is free to produce a subset, or
@@ -268,4 +268,4 @@ See also
 
 linkgit:gitcredentials[7]
 
-linkgit:git-config[5] (See configuration variables `credential.*`)
+linkgit:git-config[1] (See configuration variables `credential.*`)
Junio C Hamano· May 4, 2016, 18:52 UTC · re: Junio C Hamano · lore

Re: [PATCH v4 1/2] Documentation: fix linkgit references

Junio C Hamano <gitster@pobox.com> writes:
> I do not think there is any false positive above, so perhaps the
> checker script below can be used as the link checker we discussed?
-- >8 --
Subject: [PATCH] ci: validate "gitlink:" in documentation

It is easy to add incorrect "linkgit:<page>[<section>]" references to our documentation suite. Catch these common classes of errors:

 * Referring to Documentation/<page>.txt that does not exist.
 * Referring to a <page> outside the Git suite.  In general, <page>
   must begin with "git".
 * Listing the manual <section> incorrectly.  The first line of the
   Documentation/<page>.txt must end with "(<section>)".
with a new script ci/lint-gitlink.sh.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 ci/lint-gitlink.sh | 47 +++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 47 insertions(+)
 create mode 100755 ci/lint-gitlink.sh
Show changes to ci/lint-gitlink.sh +47 −0
diff --git a/ci/lint-gitlink.sh b/ci/lint-gitlink.sh
new file mode 100755
index 0000000..2379626
--- /dev/null
+++ b/ci/lint-gitlink.sh
@@ -0,0 +1,47 @@
+#!/bin/sh
+
+git grep -l linkgit: Documentation/ |
+while read path
+do
+	perl -e '
+	sub report {
+		my ($where, $what, $error) = @_;
+		print "$where: $error: $what\n";
+	}
+
+	sub grab_section {
+		my ($page) = @_;
+		open my $fh, "<", "Documentation/$page.txt";
+		my $firstline = <$fh>;
+		chomp $firstline;
+		close $fh;
+		my ($section) = ($firstline =~ /.*\((\d)\)$/);
+		return $section;
+	}
+
+	while (<>) {
+		my $where = "$ARGV:$.";
+		while (s/linkgit:((.*?)\[(\d)\])//) {
+			my ($target, $page, $section) = ($1, $2, $3);
+
+			# De-AsciiDoc
+			$page =~ s/{litdd}/--/g;
+
+			if ($page !~ /^git/) {
+				report($where, $target, "nongit link");
+				next;
+			}
+			if (! -f "Documentation/$page.txt") {
+				report($where, $target, "no such source");
+				next;
+			}
+			$real_section = grab_section($page);
+			if ($real_section != $section) {
+				report($where, $target,
+					"wrong section (should be $real_section)");
+				next;
+			}
+		}
+	}
+	' "$path"
+done
-- 
2.8.2-498-g6350fe8
Jeff King· May 4, 2016, 19:25 UTC · re: Junio C Hamano · lore

Re: [PATCH v4 1/2] Documentation: fix linkgit references

On Wed, May 04, 2016 at 11:52:52AM -0700, Junio C Hamano wrote:
Show 18 quoted lines
> > I do not think there is any false positive above, so perhaps the
> > checker script below can be used as the link checker we discussed?
> 
> -- >8 --
> Subject: [PATCH] ci: validate "gitlink:" in documentation
> 
> It is easy to add incorrect "linkgit:<page>[<section>]" references
> to our documentation suite.  Catch these common classes of errors:
> 
>  * Referring to Documentation/<page>.txt that does not exist.
> 
>  * Referring to a <page> outside the Git suite.  In general, <page>
>    must begin with "git".
> 
>  * Listing the manual <section> incorrectly.  The first line of the
>    Documentation/<page>.txt must end with "(<section>)".
> 
> with a new script ci/lint-gitlink.sh.

It seems like this is something we _should_ be able to do while asciidoc is actually expanding the macro, rather than as a separate step. But:

  1. I don't think stock asciidoc has this much flexibility in its
     macros.
  2. There are some ordering questions (e.g., while building "foo.1",
     "bar.1" may not be built yet, so we still have to massage requests
     for "bar.1" into "bar.txt".

Likewise, I think we could build the whole HTML source and then actually just look for broken links in it. But that script would probably end up looking similar to this one, with s/linkgit/href/. But it does more directly measure what we want, which is that the rendered doc is usable; it would catch something like using "--" instead of "{litdd}".

Show 6 quoted lines
> +#!/bin/sh
> +
> +git grep -l linkgit: Documentation/ |
> +while read path
> +do
> +	perl -e '

Is it worth just making this a perl script, rather than a shell script with a giant inline perl script? Perl is actually really good at doing that "grep" as it reads the file. :)

Besides being slightly more efficient (reading the files only once rather than filtering once via grep and then re-reading via perl). But more importantly, it avoids a layer of quoting, which makes it less likely to make a mistake by using single-quote in the perl script (I do not see any errors in what you wrote, though).

-Peff
Junio C Hamano· May 4, 2016, 19:57 UTC · re: Jeff King · lore

Re: [PATCH v4 1/2] Documentation: fix linkgit references

Jeff King <peff@peff.net> writes:
> Likewise, I think we could build the whole HTML source and then actually
> just look for broken links in it. But that script would probably end up
> looking similar to this one, with s/linkgit/href/. But it does more
> directly measure what we want, which is that the rendered doc is usable;

I debated about this myself, but chose to inspect the source material, as that approach is easier to give actionable lint output to the user that points out the file:lineno to be corrected.

> it would catch something like using "--" instead of "{litdd}".

That is true indeed. With the "source" approach, that would indeed be harder.

Show 10 quoted lines
>> +#!/bin/sh
>> +
>> +git grep -l linkgit: Documentation/ |
>> +while read path
>> +do
>> +	perl -e '
>
> Is it worth just making this a perl script, rather than a shell script
> with a giant inline perl script? Perl is actually really good at doing
> that "grep" as it reads the file. :)
OK.
-- >8 --
From: Junio C Hamano <gitster@pobox.com>
Date: Wed, 4 May 2016 11:48:06 -0700
Subject: [PATCH v2] ci: validate "gitlink:" in documentation

It is easy to add incorrect "linkgit:<page>[<section>]" references to our documentation suite. Catch these common classes of errors:

 * Referring to Documentation/<page>.txt that does not exist.
 * Referring to a <page> outside the Git suite.  In general, <page>
   must begin with "git".
 * Listing the manual <section> incorrectly.  The first line of the
   Documentation/<page>.txt must end with "(<section>)".
with a new script "ci/lint-gitlink".
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 ci/lint-gitlink | 60 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 60 insertions(+)
 create mode 100755 ci/lint-gitlink
Show changes to ci/lint-gitlink +60 −0
diff --git a/ci/lint-gitlink b/ci/lint-gitlink
new file mode 100755
index 0000000..bb73e89
--- /dev/null
+++ b/ci/lint-gitlink
@@ -0,0 +1,60 @@
+#!/usr/bin/perl
+
+use File::Find;
+
+my $found_errors = 0;
+
+sub report {
+	my ($where, $what, $error) = @_;
+	print "$where: $error: $what\n";
+	$found_errors = 1;
+}
+
+sub grab_section {
+	my ($page) = @_;
+	open my $fh, "<", "Documentation/$page.txt";
+	my $firstline = <$fh>;
+	chomp $firstline;
+	close $fh;
+	my ($section) = ($firstline =~ /.*\((\d)\)$/);
+	return $section;
+}
+
+sub lint {
+	my ($file) = @_;
+	open my $fh, "<", $file
+		or return;
+	while (<$fh>) {
+		my $where = "$file:$.";
+		while (s/linkgit:((.*?)\[(\d)\])//) {
+			my ($target, $page, $section) = ($1, $2, $3);
+
+			# De-AsciiDoc
+			$page =~ s/{litdd}/--/g;
+
+			if ($page !~ /^git/) {
+				report($where, $target, "nongit link");
+				next;
+			}
+			if (! -f "Documentation/$page.txt") {
+				report($where, $target, "no such source");
+				next;
+			}
+			$real_section = grab_section($page);
+			if ($real_section != $section) {
+				report($where, $target,
+					"wrong section (should be $real_section)");
+				next;
+			}
+		}
+	}
+	close $fh;
+}
+
+sub lint_it {
+	lint($File::Find::name) if -f;
+}
+
+find({ wanted => \&lint_it, no_chdir => 1 }, "Documentation");
+
+exit $found_errors;
-- 
2.8.2-498-g6350fe8
Jeff King· May 4, 2016, 20:06 UTC · re: Junio C Hamano · lore

Re: [PATCH v4 1/2] Documentation: fix linkgit references

On Wed, May 04, 2016 at 12:57:31PM -0700, Junio C Hamano wrote:
Show 5 quoted lines
> > Is it worth just making this a perl script, rather than a shell script
> > with a giant inline perl script? Perl is actually really good at doing
> > that "grep" as it reads the file. :)
> 
> OK.
Hmm. This new version uses File::Find:
Show 5 quoted lines
> +sub lint_it {
> +	lint($File::Find::name) if -f;
> +}
> +
> +find({ wanted => \&lint_it, no_chdir => 1 }, "Documentation");
That will inspect non-source files, too.
Would:
  open(my $files, '-|', qw(git ls-files));
  while (<$files>) {
    chomp;
    ...
  }
make sense? Or a simpler but non-streaming spelling:
  my @files = map { chomp; $_ } `git ls-files`;

Or just taking the list of files on the command line as your original did, and feeding `ls-files` from the caller. That also lets you do "link-gitlink git-foo.txt", etc.

-Peff
Junio C Hamano· May 4, 2016, 21:15 UTC · re: Jeff King · lore

Re: [PATCH v4 1/2] Documentation: fix linkgit references

Jeff King <peff@peff.net> writes:
Show 29 quoted lines
> On Wed, May 04, 2016 at 12:57:31PM -0700, Junio C Hamano wrote:
>
>> > Is it worth just making this a perl script, rather than a shell script
>> > with a giant inline perl script? Perl is actually really good at doing
>> > that "grep" as it reads the file. :)
>> 
>> OK.
>
> Hmm. This new version uses File::Find:
>
>> +sub lint_it {
>> +	lint($File::Find::name) if -f;
>> +}
>> +
>> +find({ wanted => \&lint_it, no_chdir => 1 }, "Documentation");
>
> That will inspect non-source files, too.
>
> Would:
>
>   open(my $files, '-|', qw(git ls-files));
>   while (<$files>) {
>     chomp;
>     ...
>   }
>
> make sense? Or a simpler but non-streaming spelling:
>
>   my @files = map { chomp; $_ } `git ls-files`;

I forgot to say that I wanted not to rely on "git" (i.e. OK to use this on tarball extract).

> Or just taking the list of files on the command line as your original
> did, and feeding `ls-files` from the caller. That also lets you do
> "link-gitlink git-foo.txt", etc.
Yes, I think that is the most sensible.
Jeff King· May 4, 2016, 21:31 UTC · re: Junio C Hamano · lore

Re: [PATCH v4 1/2] Documentation: fix linkgit references

On Wed, May 04, 2016 at 02:15:39PM -0700, Junio C Hamano wrote:
Show 6 quoted lines
> > make sense? Or a simpler but non-streaming spelling:
> >
> >   my @files = map { chomp; $_ } `git ls-files`;
> 
> I forgot to say that I wanted not to rely on "git" (i.e. OK to use
> this on tarball extract).
Oh, that's a good idea.
Show 5 quoted lines
> > Or just taking the list of files on the command line as your original
> > did, and feeding `ls-files` from the caller. That also lets you do
> > "link-gitlink git-foo.txt", etc.
> 
> Yes, I think that is the most sensible.

Yeah, and then the Makefile can drive it from $(MAN_TXT), etc, without requiring git (which I think is what you were getting at, but just spelling it out for myself and the list).

-Peff
Junio C Hamano· May 4, 2016, 21:34 UTC · re: Jeff King · lore

Re: [PATCH v4 1/2] Documentation: fix linkgit references

Third time's a charm, perhaps?
 
-- >8 --
Subject: [PATCH] ci: validate "gitlink:" in documentation

It is easy to add incorrect "linkgit:<page>[<section>]" references to our documentation suite. Catch these common classes of errors:

 * Referring to Documentation/<page>.txt that does not exist.
 * Referring to a <page> outside the Git suite.  In general, <page>
   must begin with "git".
 * Listing the manual <section> incorrectly.  The first line of the
   Documentation/<page>.txt must end with "(<section>)".
with a new script "ci/lint-gitlink", and drive it from "make check-docs".
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 Documentation/Makefile |  5 +++++
 Makefile               |  1 +
 ci/lint-gitlink        | 56 ++++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 62 insertions(+)
 create mode 100755 ci/lint-gitlink
Show changes to 3 files +62 −0

Documentation/Makefile, Makefile, ci/lint-gitlink

diff --git a/Documentation/Makefile b/Documentation/Makefile
index 3e39e28..e9cd43d 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -204,6 +204,7 @@ ifndef V
 	QUIET_DBLATEX	= @echo '   ' DBLATEX $@;
 	QUIET_XSLTPROC	= @echo '   ' XSLTPROC $@;
 	QUIET_GEN	= @echo '   ' GEN $@;
+	QUIET_LINT	= @echo '   ' LINT $@;
 	QUIET_STDERR	= 2> /dev/null
 	QUIET_SUBDIR0	= +@subdir=
 	QUIET_SUBDIR1	= ;$(NO_SUBDIR) echo '   ' SUBDIR $$subdir; \
@@ -427,4 +428,8 @@ quick-install-html: require-htmlrepo
 print-man1:
 	@for i in $(MAN1_TXT); do echo $$i; done
 
+lint-docs::
+	$(QUIET_LINT)$(foreach txt,$(patsubst %.html,%.txt,$(DOC_HTML)), \
+	../ci/lint-gitlink $(txt))
+
 .PHONY: FORCE
diff --git a/Makefile b/Makefile
index 2742a69..61bd0ab 100644
--- a/Makefile
+++ b/Makefile
@@ -2496,6 +2496,7 @@ ALL_COMMANDS += git-gui git-citool
 
 .PHONY: check-docs
 check-docs::
+	$(MAKE) -C Documentation lint-docs
 	@(for v in $(ALL_COMMANDS); \
 	do \
 		case "$$v" in \
diff --git a/ci/lint-gitlink b/ci/lint-gitlink
new file mode 100755
index 0000000..6b6bf91
--- /dev/null
+++ b/ci/lint-gitlink
@@ -0,0 +1,56 @@
+#!/usr/bin/perl
+
+my $found_errors = 0;
+
+sub report {
+	my ($where, $what, $error) = @_;
+	print "$where: $error: $what\n";
+	$found_errors = 1;
+}
+
+sub grab_section {
+	my ($page) = @_;
+	open my $fh, "<", "$page.txt";
+	my $firstline = <$fh>;
+	chomp $firstline;
+	close $fh;
+	my ($section) = ($firstline =~ /.*\((\d)\)$/);
+	return $section;
+}
+
+sub lint {
+	my ($file) = @_;
+	open my $fh, "<", $file
+		or return;
+	while (<$fh>) {
+		my $where = "$file:$.";
+		while (s/linkgit:((.*?)\[(\d)\])//) {
+			my ($target, $page, $section) = ($1, $2, $3);
+
+			# De-AsciiDoc
+			$page =~ s/{litdd}/--/g;
+
+			if ($page !~ /^git/) {
+				report($where, $target, "nongit link");
+				next;
+			}
+			if (! -f "$page.txt") {
+				report($where, $target, "no such source");
+				next;
+			}
+			$real_section = grab_section($page);
+			if ($real_section != $section) {
+				report($where, $target,
+					"wrong section (should be $real_section)");
+				next;
+			}
+		}
+	}
+	close $fh;
+}
+
+for (@ARGV) {
+	lint($_);
+}
+
+exit $found_errors;
-- 
2.8.2-498-g6350fe8
Jeff King· May 4, 2016, 21:44 UTC · re: Junio C Hamano · lore

Re: [PATCH v4 1/2] Documentation: fix linkgit references

On Wed, May 04, 2016 at 02:34:23PM -0700, Junio C Hamano wrote:
Show 20 quoted lines
> Third time's a charm, perhaps?
>  
> -- >8 --
> Subject: [PATCH] ci: validate "gitlink:" in documentation
> 
> It is easy to add incorrect "linkgit:<page>[<section>]" references
> to our documentation suite.  Catch these common classes of errors:
> 
>  * Referring to Documentation/<page>.txt that does not exist.
> 
>  * Referring to a <page> outside the Git suite.  In general, <page>
>    must begin with "git".
> 
>  * Listing the manual <section> incorrectly.  The first line of the
>    Documentation/<page>.txt must end with "(<section>)".
> 
> with a new script "ci/lint-gitlink", and drive it from "make check-docs".
> 
> Signed-off-by: Junio C Hamano <gitster@pobox.com>
> ---
This looks good to me. Two minor nits that may not be worth addressing:
> +lint-docs::
> +	$(QUIET_LINT)$(foreach txt,$(patsubst %.html,%.txt,$(DOC_HTML)), \
> +	../ci/lint-gitlink $(txt))
> +
This dependency feels funny. Wouldn't CI want to invoke this as:
  make -C Documentation lint-docs

IOW, Documentation owns the script (just like t/ owns its own lint scripts like check-non-portable-shell.pl), and CI always just triggers the make-driven checks, just as a normal developer would?

Show 12 quoted lines
> +sub lint {
> +	my ($file) = @_;
> +	open my $fh, "<", $file
> +		or return;
> +	while (<$fh>) {
> +		my $where = "$file:$.";
> [... actual linting of line ...]
> +}
> +
> +for (@ARGV) {
> +	lint($_);
> +}
The usual perl way here would be:
  while(<>) {
	my $where = "$ARGV:$.";
	... actual linting of line ...
  }

where "<>" automagically reads from files on the command-line or stdin if none were specified (and sets $ARGV as appropriate). But maybe you prefer not to handle the stdin case (it is a benefit sometimes, but an annoyance if you accidentally end up with an empty file list).

-Peff
Junio C Hamano· May 4, 2016, 21:52 UTC · re: Jeff King · lore

Re: [PATCH v4 1/2] Documentation: fix linkgit references

Jeff King <peff@peff.net> writes:
> This dependency feels funny. Wouldn't CI want to invoke this as:
>
>   make -C Documentation lint-docs
I expected CI to do this instead
	make check-docs
Jeff King· May 4, 2016, 22:09 UTC · re: Junio C Hamano · lore

Re: [PATCH v4 1/2] Documentation: fix linkgit references

On Wed, May 04, 2016 at 02:52:27PM -0700, Junio C Hamano wrote:
Show 9 quoted lines
> Jeff King <peff@peff.net> writes:
> 
> > This dependency feels funny. Wouldn't CI want to invoke this as:
> >
> >   make -C Documentation lint-docs
> 
> I expected CI to do this instead
> 
> 	make check-docs

Ah, sure, that makes even more sense. But I think the point remains, which is that your perl script is an implementation detail of the Makefile process. I thought the "ci" directory was supposed to be for ci-specific scripts that would be driven directly by Travis, etc.

-Peff
Junio C Hamano· May 4, 2016, 22:17 UTC · re: Jeff King · lore

Re: [PATCH v4 1/2] Documentation: fix linkgit references

Jeff King <peff@peff.net> writes:
> But I think the point remains,
> which is that your perl script is an implementation detail of the
> Makefile process. I thought the "ci" directory was supposed to be for
> ci-specific scripts that would be driven directly by Travis, etc.

True. Documentation/ has tools like built-docdep.perl, howto-index.sh, etc., so it can live next to it.

Thanks.
Junio C Hamano· May 4, 2016, 22:27 UTC · re: Junio C Hamano · lore

Re: [PATCH v4 1/2] Documentation: fix linkgit references

Junio C Hamano <gitster@pobox.com> writes:
Show 11 quoted lines
> Jeff King <peff@peff.net> writes:
>
>> But I think the point remains,
>> which is that your perl script is an implementation detail of the
>> Makefile process. I thought the "ci" directory was supposed to be for
>> ci-specific scripts that would be driven directly by Travis, etc.
>
> True.  Documentation/ has tools like built-docdep.perl,
> howto-index.sh, etc., so it can live next to it.
>
> Thanks.

Gaah, the Makefile part of the final patch is wrong; we do not check the included sources at all if we only passed the top-level targets' sources.

Junio C Hamano· May 4, 2016, 23:28 UTC · re: Junio C Hamano · lore

Re: [PATCH v4 1/2] Documentation: fix linkgit references

Junio C Hamano <gitster@pobox.com> writes:
> Gaah, the Makefile part of the final patch is wrong; we do not check
> the included sources at all if we only passed the top-level targets'
> sources.

I ended up queuing an enhanced version of File::Find based one on 'pu', but I won't be posting it here today.

Jeff King· May 4, 2016, 23:41 UTC · re: Junio C Hamano · lore

Re: [PATCH v4 1/2] Documentation: fix linkgit references

On Wed, May 04, 2016 at 04:28:31PM -0700, Junio C Hamano wrote:
Show 8 quoted lines
> Junio C Hamano <gitster@pobox.com> writes:
> 
> > Gaah, the Makefile part of the final patch is wrong; we do not check
> > the included sources at all if we only passed the top-level targets'
> > sources.
> 
> I ended up queuing an enhanced version of File::Find based one on
> 'pu', but I won't be posting it here today.

Hmm. It seems like we should still be able to drive it from the Makefile using the dependencies generated by build-docdep.

-Peff
Jeff King· May 4, 2016, 23:53 UTC · re: Jeff King · lore

Re: [PATCH v4 1/2] Documentation: fix linkgit references

On Wed, May 04, 2016 at 07:41:53PM -0400, Jeff King wrote:
Show 13 quoted lines
> On Wed, May 04, 2016 at 04:28:31PM -0700, Junio C Hamano wrote:
> 
> > Junio C Hamano <gitster@pobox.com> writes:
> > 
> > > Gaah, the Makefile part of the final patch is wrong; we do not check
> > > the included sources at all if we only passed the top-level targets'
> > > sources.
> > 
> > I ended up queuing an enhanced version of File::Find based one on
> > 'pu', but I won't be posting it here today.
> 
> Hmm. It seems like we should still be able to drive it from the Makefile
> using the dependencies generated by build-docdep.

Having just read that script, it is blindly looking at "*.txt", so...as long as that is what your script is doing (and it looks like the version in pu is), I don't think there's any reason to get more complicated. The only difference is that build-docdep does not even bother looking in subdirectories (maybe it should, since we now build stuff in technical, I think).

-Peff
Ævar Arnfjörð Bjarmason· May 4, 2016, 21:54 UTC · re: Jeff King · lore

Re: [PATCH v4 1/2] Documentation: fix linkgit references

On Wed, May 4, 2016 at 10:06 PM, Jeff King <peff@peff.net> wrote:
Show 11 quoted lines
> Would:
>
>   open(my $files, '-|', qw(git ls-files));
>   while (<$files>) {
>     chomp;
>     ...
>   }
>
> make sense? Or a simpler but non-streaming spelling:
>
>   my @files = map { chomp; $_ } `git ls-files`;
As a minor sidenote you can equivalently write that as:
    chomp(my @files = `git ls-files`);

I.e. chomp itself when given a list will chomp each item of the list. So no need for a map.

Jeff King· May 4, 2016, 22:11 UTC · re: Ævar Arnfjörð Bjarmason · lore

Re: [PATCH v4 1/2] Documentation: fix linkgit references

On Wed, May 04, 2016 at 11:54:52PM +0200, Ævar Arnfjörð Bjarmason wrote:
Show 8 quoted lines
> >   my @files = map { chomp; $_ } `git ls-files`;
> 
> As a minor sidenote you can equivalently write that as:
> 
>     chomp(my @files = `git ls-files`);
> 
> I.e. chomp itself when given a list will chomp each item of the list.
> So no need for a map.

Right, thanks, that is more readable. I use the map form reflexively because I am often doing stuff like:

  my @list = do {
    open(my $fh, '<', $fn);
    map { chomp; $_ }
  };

(or replacing "do" with an actual function which is returning a grotesque pipeline of grep/map, etc).

-Peff
larsxschneider@gmail.com· May 4, 2016, 08:38 UTC · re: larsxschneider@gmail.com · lore

[PATCH v4 2/2] travis-ci: build documentation

From: Lars Schneider <larsxschneider@gmail.com>

Build documentation as separate Travis CI job to check for documentation errors.

Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
---
 .travis.yml              | 15 +++++++++++++++
 ci/test-documentation.sh | 14 ++++++++++++++
 2 files changed, 29 insertions(+)
 create mode 100755 ci/test-documentation.sh
Show changes to 2 files +29 −0

.travis.yml, ci/test-documentation.sh

diff --git a/.travis.yml b/.travis.yml
index 78e433b..55299bd 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -32,6 +32,21 @@ env:
     # t9816 occasionally fails with "TAP out of sequence errors" on Travis CI OS X
     - GIT_SKIP_TESTS="t9810 t9816"
 
+matrix:
+  include:
+    - env: Documentation
+      os: linux
+      compiler: clang
+      addons:
+        apt:
+          packages:
+          - asciidoc
+          - xmlto
+      before_install:
+      before_script:
+      script: ci/test-documentation.sh
+      after_failure:
+
 before_install:
   - >
     case "${TRAVIS_OS_NAME:-linux}" in
diff --git a/ci/test-documentation.sh b/ci/test-documentation.sh
new file mode 100755
index 0000000..579d540
--- /dev/null
+++ b/ci/test-documentation.sh
@@ -0,0 +1,14 @@
+#!/bin/sh
+#
+# Perform sanity checks on documentation and build it.
+#
+
+set -e
+
+make check-builtins
+make check-docs
+make doc
+
+test -s Documentation/git.html
+test -s Documentation/git.xml
+test -s Documentation/git.1
-- 
2.5.1
Junio C Hamano· May 10, 2016, 17:12 UTC · re: larsxschneider@gmail.com · lore

Re: [PATCH v4 2/2] travis-ci: build documentation

larsxschneider@gmail.com writes:
Show 19 quoted lines
> From: Lars Schneider <larsxschneider@gmail.com>
>
> Build documentation as separate Travis CI job to check for
> documentation errors.
>
> Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
> ---
>  .travis.yml              | 15 +++++++++++++++
>  ci/test-documentation.sh | 14 ++++++++++++++
>  2 files changed, 29 insertions(+)
>  create mode 100755 ci/test-documentation.sh
>
> diff --git a/.travis.yml b/.travis.yml
> index 78e433b..55299bd 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -32,6 +32,21 @@ env:
>      # t9816 occasionally fails with "TAP out of sequence errors" on Travis CI OS X
>      - GIT_SKIP_TESTS="t9810 t9816"

Completely offtopic, but this looks like this is made to apply to all archs, not limited to OSX? It of course would be ideal to see why they fail only on OSX and fix them, but shouldn't the blacklist at least limited to the platform with the problem?

Lars Schneider· May 13, 2016, 06:26 UTC · re: Junio C Hamano · lore

Re: [PATCH v4 2/2] travis-ci: build documentation

Show 28 quoted lines
> On 10 May 2016, at 19:12, Junio C Hamano <gitster@pobox.com> wrote:
> 
> larsxschneider@gmail.com writes:
> 
>> From: Lars Schneider <larsxschneider@gmail.com>
>> 
>> Build documentation as separate Travis CI job to check for
>> documentation errors.
>> 
>> Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
>> ---
>> .travis.yml              | 15 +++++++++++++++
>> ci/test-documentation.sh | 14 ++++++++++++++
>> 2 files changed, 29 insertions(+)
>> create mode 100755 ci/test-documentation.sh
>> 
>> diff --git a/.travis.yml b/.travis.yml
>> index 78e433b..55299bd 100644
>> --- a/.travis.yml
>> +++ b/.travis.yml
>> @@ -32,6 +32,21 @@ env:
>>     # t9816 occasionally fails with "TAP out of sequence errors" on Travis CI OS X
>>     - GIT_SKIP_TESTS="t9810 t9816"
> 
> Completely offtopic, but this looks like this is made to apply to
> all archs, not limited to OSX?  It of course would be ideal to see
> why they fail only on OSX and fix them, but shouldn't the blacklist
> at least limited to the platform with the problem?

As far as I remember the test was flaky on Linux and OSX. The problem was not Git. The problem was the Perforce Server "p4d" p4 daemon which would refuse to die sometimes.

I run a bigger number of Travis tests and it looks like the problem is gone. Out of 60 runs only just one failed and t9810 / t9816 passed: https://travis-ci.org/larsxschneider/git/jobs/129383851 (BTW: does anyone know how I can make prove show me the running tests? I would like to know which test died there..)

It looks like as Perforce solved the t9810/t9816 problem with the update from 15.2 to 16.1 that we made in 31f3c86 "travis-ci: update Git-LFS and P4 to the latest version". I will post a patch to enable these tests, again.

Thanks, Lars

larsxschneider@gmail.com· May 2, 2016, 20:20 UTC · re: larsxschneider@gmail.com · lore

[PATCH v3 2/2] travis-ci: build documentation

From: Lars Schneider <larsxschneider@gmail.com>

Build documentation as separate Travis CI job to check for documentation errors.

Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
---
 .travis.yml              | 15 +++++++++++++++
 ci/test-documentation.sh | 24 ++++++++++++++++++++++++
 2 files changed, 39 insertions(+)
 create mode 100755 ci/test-documentation.sh
Show changes to 2 files +39 −0

.travis.yml, ci/test-documentation.sh

diff --git a/.travis.yml b/.travis.yml
index 78e433b..55299bd 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -32,6 +32,21 @@ env:
     # t9816 occasionally fails with "TAP out of sequence errors" on Travis CI OS X
     - GIT_SKIP_TESTS="t9810 t9816"
 
+matrix:
+  include:
+    - env: Documentation
+      os: linux
+      compiler: clang
+      addons:
+        apt:
+          packages:
+          - asciidoc
+          - xmlto
+      before_install:
+      before_script:
+      script: ci/test-documentation.sh
+      after_failure:
+
 before_install:
   - >
     case "${TRAVIS_OS_NAME:-linux}" in
diff --git a/ci/test-documentation.sh b/ci/test-documentation.sh
new file mode 100755
index 0000000..889e6fd
--- /dev/null
+++ b/ci/test-documentation.sh
@@ -0,0 +1,24 @@
+#!/bin/sh
+#
+# Perform sanity checks on documentation and build it.
+#
+
+set -e
+
+LINKS=$(grep --recursive --only-matching --no-filename --perl-regexp \
+    '(?<=linkgit:).*?(?=\[\d+\])' Documentation/* \
+    | sort -u \
+)
+
+for LINK in $LINKS; do
+    echo "Checking linkgit:$LINK..."
+    test -s Documentation/$LINK.txt
+done
+
+make check-builtins
+make check-docs
+make doc
+
+test -s Documentation/git.html
+test -s Documentation/git.xml
+test -s Documentation/git.1
-- 
2.5.1
Junio C Hamano· May 2, 2016, 20:45 UTC · re: larsxschneider@gmail.com · lore

Re: [PATCH v3 2/2] travis-ci: build documentation

larsxschneider@gmail.com writes:
Show 11 quoted lines
> +set -e
> +
> +LINKS=$(grep --recursive --only-matching --no-filename --perl-regexp \
> +    '(?<=linkgit:).*?(?=\[\d+\])' Documentation/* \
> +    | sort -u \
> +)
> +
> +for LINK in $LINKS; do
> +    echo "Checking linkgit:$LINK..."
> +    test -s Documentation/$LINK.txt
> +done

Please separate the above link check out of this step and do so separately after the move of test body to a separate script settles.

When you reintroduce the tests, please make sure the shell script follow the coding style of other scripts. E.g. I do not think the last one in the $(...) needs a backslash continuation at all. I am assuming that you are doing this only on Linux, in which case use of GNUism with grep may be fine.

Show 7 quoted lines
> +make check-builtins
> +make check-docs
> +make doc
> +
> +test -s Documentation/git.html
> +test -s Documentation/git.xml
> +test -s Documentation/git.1
Lars Schneider· May 3, 2016, 08:12 UTC · re: Junio C Hamano · lore

Re: [PATCH v3 2/2] travis-ci: build documentation

On 02 May 2016, at 22:45, Junio C Hamano <gitster@pobox.com> wrote:
Show 17 quoted lines
> larsxschneider@gmail.com writes:
> 
>> +set -e
>> +
>> +LINKS=$(grep --recursive --only-matching --no-filename --perl-regexp \
>> +    '(?<=linkgit:).*?(?=\[\d+\])' Documentation/* \
>> +    | sort -u \
>> +)
>> +
>> +for LINK in $LINKS; do
>> +    echo "Checking linkgit:$LINK..."
>> +    test -s Documentation/$LINK.txt
>> +done
> 
> Please separate the above link check out of this step and do so
> separately after the move of test body to a separate script
> settles.

OK. I also wonder if the link check should rather go to the "check-docs" Makefile target?

Show 5 quoted lines
> When you reintroduce the tests, please make sure the shell script
> follow the coding style of other scripts.  E.g. I do not think the
> last one in the $(...) needs a backslash continuation at all.  I am
> assuming that you are doing this only on Linux, in which case use of
> GNUism with grep may be fine.
OK.

Thanks for the review, Lars

Show 8 quoted lines
> 
>> +make check-builtins
>> +make check-docs
>> +make doc
>> +
>> +test -s Documentation/git.html
>> +test -s Documentation/git.xml
>> +test -s Documentation/git.1
Junio C Hamano· May 3, 2016, 15:43 UTC · re: Lars Schneider · lore

Re: [PATCH v3 2/2] travis-ci: build documentation

Lars Schneider <larsxschneider@gmail.com> writes:
Show 22 quoted lines
> On 02 May 2016, at 22:45, Junio C Hamano <gitster@pobox.com> wrote:
>
>> larsxschneider@gmail.com writes:
>> 
>>> +set -e
>>> +
>>> +LINKS=$(grep --recursive --only-matching --no-filename --perl-regexp \
>>> +    '(?<=linkgit:).*?(?=\[\d+\])' Documentation/* \
>>> +    | sort -u \
>>> +)
>>> +
>>> +for LINK in $LINKS; do
>>> +    echo "Checking linkgit:$LINK..."
>>> +    test -s Documentation/$LINK.txt
>>> +done
>> 
>> Please separate the above link check out of this step and do so
>> separately after the move of test body to a separate script
>> settles.
>
> OK. I also wonder if the link check should rather go to the 
> "check-docs" Makefile target?
That sounds like a good direction.

Which in turn means that people on all platforms are welcome to run it, which in turn means that the script must be even more portable, with avoiding GNUism and bash-isms etc.

Lars Schneider· May 4, 2016, 08:04 UTC · re: Junio C Hamano · lore

Re: [PATCH v3 2/2] travis-ci: build documentation

On 03 May 2016, at 17:43, Junio C Hamano <gitster@pobox.com> wrote:
Show 30 quoted lines
> Lars Schneider <larsxschneider@gmail.com> writes:
> 
>> On 02 May 2016, at 22:45, Junio C Hamano <gitster@pobox.com> wrote:
>> 
>>> larsxschneider@gmail.com writes:
>>> 
>>>> +set -e
>>>> +
>>>> +LINKS=$(grep --recursive --only-matching --no-filename --perl-regexp \
>>>> +    '(?<=linkgit:).*?(?=\[\d+\])' Documentation/* \
>>>> +    | sort -u \
>>>> +)
>>>> +
>>>> +for LINK in $LINKS; do
>>>> +    echo "Checking linkgit:$LINK..."
>>>> +    test -s Documentation/$LINK.txt
>>>> +done
>>> 
>>> Please separate the above link check out of this step and do so
>>> separately after the move of test body to a separate script
>>> settles.
>> 
>> OK. I also wonder if the link check should rather go to the 
>> "check-docs" Makefile target?
> 
> That sounds like a good direction.
> 
> Which in turn means that people on all platforms are welcome to run
> it, which in turn means that the script must be even more portable,
> with avoiding GNUism and bash-isms etc.

OK. I am not that experienced with shell scripting and therefore it is hard for me to distinguish between the different shell features. Do you know/can you recommend the most basic shell to test/work with? A quick Google search told me that "dash" from Ubuntu seems to be a good baseline as it aims to support pretty much only POSIX [1].

Thanks, Lars

[1] http://www.cyberciti.biz/faq/debian-ubuntu-linux-binbash-vs-bindash-vs-binshshell/
Eric Wong· May 4, 2016, 08:18 UTC · re: Lars Schneider · lore

Re: [PATCH v3 2/2] travis-ci: build documentation

Lars Schneider <larsxschneider@gmail.com> wrote:
Show 5 quoted lines
> OK. I am not that experienced with shell scripting and therefore it
> is hard for me to distinguish between the different shell features.
> Do you know/can you recommend the most basic shell to test/work 
> with? A quick Google search told me that "dash" from Ubuntu seems
> to be a good baseline as it aims to support pretty much only POSIX [1].

Yes, I recommend dash as your /bin/sh if you're using a Debian-based system ("Debian Almquist shell") such as Ubuntu.

Also, the Debian "devscripts" package has a tool called "checkbashisms" which I have also found very useful.

Maybe ksh(93) or posh could be good, too, but it's been a while...

I also rely on checking the manpages-posix and manpages-posix-dev in the "non-free" section of Debian ("multiverse" in Ubuntu?)

Junio C Hamano· May 4, 2016, 08:19 UTC · re: Lars Schneider · lore

Re: [PATCH v3 2/2] travis-ci: build documentation

Lars Schneider <larsxschneider@gmail.com> writes:
> A quick Google search told me that "dash" from Ubuntu seems
> to be a good baseline as it aims to support pretty much only POSIX [1].
Yeah that is a good and safe starting point.
Christian Couder· May 4, 2016, 09:11 UTC · re: Lars Schneider · lore

Re: [PATCH v3 2/2] travis-ci: build documentation

On Wed, May 4, 2016 at 10:04 AM, Lars Schneider <larsxschneider@gmail.com> wrote:

Show 39 quoted lines
>
> On 03 May 2016, at 17:43, Junio C Hamano <gitster@pobox.com> wrote:
>
>> Lars Schneider <larsxschneider@gmail.com> writes:
>>
>>> On 02 May 2016, at 22:45, Junio C Hamano <gitster@pobox.com> wrote:
>>>
>>>> larsxschneider@gmail.com writes:
>>>>
>>>>> +set -e
>>>>> +
>>>>> +LINKS=$(grep --recursive --only-matching --no-filename --perl-regexp \
>>>>> +    '(?<=linkgit:).*?(?=\[\d+\])' Documentation/* \
>>>>> +    | sort -u \
>>>>> +)
>>>>> +
>>>>> +for LINK in $LINKS; do
>>>>> +    echo "Checking linkgit:$LINK..."
>>>>> +    test -s Documentation/$LINK.txt
>>>>> +done
>>>>
>>>> Please separate the above link check out of this step and do so
>>>> separately after the move of test body to a separate script
>>>> settles.
>>>
>>> OK. I also wonder if the link check should rather go to the
>>> "check-docs" Makefile target?
>>
>> That sounds like a good direction.
>>
>> Which in turn means that people on all platforms are welcome to run
>> it, which in turn means that the script must be even more portable,
>> with avoiding GNUism and bash-isms etc.
>
> OK. I am not that experienced with shell scripting and therefore it
> is hard for me to distinguish between the different shell features.
> Do you know/can you recommend the most basic shell to test/work
> with? A quick Google search told me that "dash" from Ubuntu seems
> to be a good baseline as it aims to support pretty much only POSIX [1].
You might also want to check:
http://www.shellcheck.net/
Matthieu Moy· May 4, 2016, 11:27 UTC · re: Christian Couder · lore

Re: [PATCH v3 2/2] travis-ci: build documentation

Christian Couder <christian.couder@gmail.com> writes:
> You might also want to check:
>
> http://www.shellcheck.net/

Indeed, it's also an excellent tool to check for common mistakes in shell scripts (there are many, and shellcheck is good at pointing them out and explaining them).

http://www.shellcheck.net/
(Available online and as a command-line tool)
-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/
Matthieu Moy· Apr 29, 2016, 14:40 UTC · re: larsxschneider@gmail.com · lore

Re: [PATCH v2] travis-ci: build documentation

larsxschneider@gmail.com writes:
> +      before_install:
> +      before_script: make doc
> +      script: ci/test-documentation.sh

If you are to re-roll, I think before_script and script should be merged, i.e. just write

script: ci/test-documentation.sh
and have ci/test-documentation.sh be:
#!/bin/sh
set -e

make doc test -s Documentation/git.html test -s Documentation/git.xml test -s Documentation/git.1

In the perspective of using another CI tool, everything within ci/ is potentially shared between systems so I'd tend to minimize the content of .travis.yml in favor of ci/*

-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/

← back to recent threads