git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v3 6/6] ci: fix AsciiDoc/Asciidoctor stderr check in the documentation build job

From
SZEDER Gábor <szeder.dev@gmail.com>
Date
Mar 29, 2019, 12:35 UTC
Message-ID
<20190329123520.27549-7-szeder.dev@gmail.com>
In-Reply-To
<20190329123520.27549-1-szeder.dev@gmail.com>

In 'ci/test-documentation.sh' we save the standard error of 'make doc', and, in an attempt to make sure that neither AsciiDoc nor Asciidoctor printed any warnings, we check the emptiness of the resulting file with '! test -s stderr.log'. This check has never actually worked, because in our 'ci/*' build scripts we rely on 'set -e' aborting the build job when a command exits with error, and, unfortunately, the combination of the two doesn't work as intended. According to POSIX [1]:

  "The -e setting shall be ignored when executing [...] a pipeline
  beginning with the ! reserved word" [2]
Watch and learn:
  $ echo unexpected >file
  $ ( set -e; ! test -s file ; echo "should not reach this" ) ; echo $?
  should not reach this
  0

This is why we haven't noticed the warnings from Asciidoctor that were fixed in the first patches of this patch series, though some of them were already there in the build of v2.18.0-rc0 [3].

Check the emptiness of that file with 'test ! -s' instead, which works properly with 'set -e':

  $ ( set -e; test ! -s file ; echo "should not reach this" ) ; echo $?
  1

Furthermore, dump the contents of that file to the log for our convenience, so if it were to unexpectedly end up being non-empty, then we wouldn't have to scroll through all that long build log looking for warnings, but could see them right away near the end of the log.

Note that we are only really interested in the standard error of AsciiDoc and Asciidoctor, but by saving the stderr of 'make doc' we also save any error output from the make rules. Currently there is only one such line: we build the docs with Asciidoctor right after a 'make clean', meaning that 'make USE_ASCIIDOCTOR=1 doc' always starts with running 'GIT-VERSION-GEN', which in turn prints the version to stderr. A 'sed' command was supposed to remove this version line to prevent it from triggering that (previously defunct) emptiness check, but, unfortunately, this command doesn't work as intended, either, because it leaves the file to be checked intact, but that defunct emptiness check hid this issue, too... Furthermore, in the near future there will be an other line on stderr, because commit 9a71722b4d (Doc: auto-detect changed build flags, 2019-03-17) in the currently cooking branch 'ma/doc-diff-doc-vs-doctor-comparison' will print "* new asciidoc flags" at the beginning of both 'make doc' invokations.

Extend that 'sed' command to remove this line, too, wrap it in a helper function so the output of both 'make doc' is filtered the same way, and change its invokation to actually write the logfile to be checked.

[1] http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#set
[2] POSIX doesn't discuss the meaning of '! cmd' in case of simple
    commands, but it defines that "A pipeline is a sequence of one or
    more commands separated by the control operator '|'", so
    apparently a simple command is considered as pipeline as well.
    http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_09_02
[3] https://travis-ci.org/git/git/jobs/385932007#L1463
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
---
 ci/test-documentation.sh | 23 ++++++++++++++++-------
 1 file changed, 16 insertions(+), 7 deletions(-)
diff --git a/ci/test-documentation.sh b/ci/test-documentation.sh
index 8f91f48c81..d49089832d 100755
--- a/ci/test-documentation.sh
+++ b/ci/test-documentation.sh
@@ -5,29 +5,38 @@
 
 . ${0%/*}/lib.sh
 
+filter_log () {
+	sed -e '/^GIT_VERSION = /d' \
+	    -e '/^    \* new asciidoc flags$/d' \
+	    "$1"
+}
+
 make check-builtins
 make check-docs
 
 # Build docs with AsciiDoc
-make doc > >(tee stdout.log) 2> >(tee stderr.log >&2)
-! test -s stderr.log
+make doc > >(tee stdout.log) 2> >(tee stderr.raw >&2)
+cat stderr.raw
+filter_log stderr.raw >stderr.log
+test ! -s stderr.log
 test -s Documentation/git.html
 test -s Documentation/git.xml
 test -s Documentation/git.1
 grep '<meta name="generator" content="AsciiDoc ' Documentation/git.html
 
-rm -f stdout.log stderr.log
+rm -f stdout.log stderr.log stderr.raw
 check_unignored_build_artifacts
 
 # Build docs with AsciiDoctor
 make clean
-make USE_ASCIIDOCTOR=1 doc > >(tee stdout.log) 2> >(tee stderr.log >&2)
-sed '/^GIT_VERSION = / d' stderr.log
-! test -s stderr.log
+make USE_ASCIIDOCTOR=1 doc > >(tee stdout.log) 2> >(tee stderr.raw >&2)
+cat stderr.raw
+filter_log stderr.raw >stderr.log
+test ! -s stderr.log
 test -s Documentation/git.html
 grep '<meta name="generator" content="Asciidoctor ' Documentation/git.html
 
-rm -f stdout.log stderr.log
+rm -f stdout.log stderr.log stderr.raw
 check_unignored_build_artifacts
 
 save_good_tree
-- 
2.21.0.539.g07239c3a71.dirty
Previous: Martin ÅgrenNext: Johannes Schindelin
Message 24 of 25 in “Documentation/git-diff-tree.txt: fix formatting”
  1. 1/3 Documentation/git-diff-tree.txt: fix formattingSZEDER Gábor, Mar 24, 2019
  2. 2/3 Documentation/technical/api-config.txt: fix formattingSZEDER Gábor, Mar 24, 2019
  3. 3/3 Documentation/technical/protocol-v2.txt: fix formattingSZEDER Gábor, Mar 24, 2019
  4. SZEDER GáborMar 24, 2019
  5. 0/6 Asciidoctor-related formatting and CI fixesSZEDER Gábor, Mar 24, 2019
  6. 1/6 Documentation/git-diff-tree.txt: fix formattingSZEDER Gábor, Mar 24, 2019
  7. 2/6 Documentation/technical/api-config.txt: fix formattingSZEDER Gábor, Mar 24, 2019
  8. 3/6 Documentation/technical/protocol-v2.txt: fix formattingSZEDER Gábor, Mar 24, 2019
  9. 4/6 ci: install Asciidoctor in 'ci/install-dependencies.sh'SZEDER Gábor, Mar 24, 2019
  10. Johannes SchindelinMar 25, 2019
  11. SZEDER GáborMar 25, 2019
  12. Johannes SchindelinMar 26, 2019
  13. 6/6 ci: fix AsciiDoc/Asciidoctor stderr check in the documentation build jobSZEDER Gábor, Mar 24, 2019
  14. 5/6 ci: stick with Asciidoctor v1.5.8 for nowSZEDER Gábor, Mar 24, 2019
  15. Jeff KingMar 26, 2019
  16. 0/6 Asciidoctor-related formatting and CI fixesSZEDER Gábor, Mar 29, 2019
  17. 1/6 Documentation/git-diff-tree.txt: fix formattingSZEDER Gábor, Mar 29, 2019
  18. 2/6 Documentation/technical/api-config.txt: fix formattingSZEDER Gábor, Mar 29, 2019
  19. 4/6 ci: install Asciidoctor in 'ci/install-dependencies.sh'SZEDER Gábor, Mar 29, 2019
  20. 3/6 Documentation/technical/protocol-v2.txt: fix formattingSZEDER Gábor, Mar 29, 2019
  21. 5/6 ci: stick with Asciidoctor v1.5.8 for nowSZEDER Gábor, Mar 29, 2019
  22. 5/6 ci: stick with Asciidoctor v1.5.8 for nowSZEDER Gábor, Mar 29, 2019
  23. Martin ÅgrenApr 3, 2019
  24. 6/6 ci: fix AsciiDoc/Asciidoctor stderr check in the documentation build jobSZEDER Gábor, Mar 29, 2019
  25. Johannes SchindelinMar 29, 2019

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.