git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v8 09/15] bugreport: generate config safelist based on docs

From
Emily Shaffer <emilyshaffer@google.com>
Date
Feb 20, 2020, 01:58 UTC
Message-ID
<20200220015858.181086-10-emilyshaffer@google.com>
In-Reply-To
<20200220015858.181086-1-emilyshaffer@google.com>

Add a new step to the build to generate a safelist of git-config variables which are appropriate to include in the output of git-bugreport. New variables can be added to the safelist by annotating their documentation in Documentation/config with the "annotate" macro, which is a no-op in AsciiDoc and AsciiDoctor.

Some configs are private in nature, and can contain remote URLs, passwords, or other sensitive information. In the event that a user doesn't notice their information while reviewing a bugreport, that user may leak their credentials to other individuals, mailing lists, or bug tracking tools inadvertently. Heuristic blocklisting of configuration keys is imperfect and prone to false negatives; given the nature of the information which can be leaked, a safelist is more reliable.

However, it's possible that in some situations, an organization may be less concerned with privacy of things like remote URLs and branch names, and more concerned with ease of diagnosis for their support staff. In those cases, it may make more sense for that organization to modify the code to use a blocklist. To that end, we should try to mark configs which are definitely safe, and configs which are definitely unsafe, and leave blank configs which are somewhere in between. To mark a config as safe, add "annotate:bugreport[include]" to the corresponding line in the config documentation; to mark it as unsafe, add "annotate:bugreport[exclude]" instead.

Generating bugreport-config-safelist.h at build time by grepping the documentation for this new macro helps us prevent staleness. The macro itself is a no-op and should not alter the appearance of the documentation in either AsciiDoc or AsciiDoctor, confirmable by running:

  cd Documentation
  ./doc-diff --asciidoctor HEAD^ HEAD
  ./doc-diff --asciidoc HEAD^ HEAD

Diffing the rendered HTML shows that only inline comments were added, which shouldn't be a problem.

Additionally, add annotations to the sendemail config documentation in order to demonstrate a proof of concept.

Helped-by: Martin Ågren <martin.agren@gmail.com>
Helped-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 .gitignore                              |  1 +
 Documentation/asciidoc.conf             |  9 ++++
 Documentation/asciidoctor-extensions.rb |  7 ++++
 Documentation/config/sendemail.txt      | 56 ++++++++++++-------------
 Makefile                                |  7 ++++
 generate-bugreport-config-safelist.sh   | 18 ++++++++
 6 files changed, 70 insertions(+), 28 deletions(-)
 create mode 100755 generate-bugreport-config-safelist.sh
diff --git a/.gitignore b/.gitignore
index d89bf9e11e..bd2f49b996 100644
--- a/.gitignore
+++ b/.gitignore
@@ -192,6 +192,7 @@
 /gitweb/static/gitweb.min.*
 /config-list.h
 /command-list.h
+/bugreport-config-safelist.h
 *.tar.gz
 *.dsc
 *.deb
diff --git a/Documentation/asciidoc.conf b/Documentation/asciidoc.conf
index 8fc4b67081..663e06481f 100644
--- a/Documentation/asciidoc.conf
+++ b/Documentation/asciidoc.conf
@@ -6,9 +6,14 @@
 #
 # Show Git link as: <command>(<section>); if section is defined, else just show
 # the command.
+#
+# The annotate macro does nothing as far as rendering is
+# concerned -- we just grep for it in the sources to populate
+# things like the bugreport safelist.
 
 [macros]
 (?su)[\\]?(?P<name>linkgit):(?P<target>\S*?)\[(?P<attrlist>.*?)\]=
+(?su)[\\]?(?P<name>annotate):(?P<target>\S*?)\[(?P<attrlist>.*?)\]=
 
 [attributes]
 asterisk=&#42;
@@ -28,6 +33,8 @@ ifdef::backend-docbook[]
 {0#<citerefentry>}
 {0#<refentrytitle>{target}</refentrytitle><manvolnum>{0}</manvolnum>}
 {0#</citerefentry>}
+[annotate-inlinemacro]
+{0#}
 endif::backend-docbook[]
 
 ifdef::backend-docbook[]
@@ -94,4 +101,6 @@ ifdef::backend-xhtml11[]
 git-relative-html-prefix=
 [linkgit-inlinemacro]
 <a href="{git-relative-html-prefix}{target}.html">{target}{0?({0})}</a>
+[annotate-inlinemacro]
+<!-- -->
 endif::backend-xhtml11[]
diff --git a/Documentation/asciidoctor-extensions.rb b/Documentation/asciidoctor-extensions.rb
index d906a00803..03c80af0e5 100644
--- a/Documentation/asciidoctor-extensions.rb
+++ b/Documentation/asciidoctor-extensions.rb
@@ -39,10 +39,17 @@ module Git
         output
       end
     end
+
+    class AnnotateProcessor < Asciidoctor::Extensions::InlineMacroProcessor
+      def process(parent, target, attrs)
+        ""
+      end
+    end
   end
 end
 
 Asciidoctor::Extensions.register do
   inline_macro Git::Documentation::LinkGitProcessor, :linkgit
   postprocessor Git::Documentation::DocumentPostProcessor
+  inline_macro Git::Documentation::AnnotateProcessor, :annotate
 end
diff --git a/Documentation/config/sendemail.txt b/Documentation/config/sendemail.txt
index 0006faf800..fe27473e44 100644
--- a/Documentation/config/sendemail.txt
+++ b/Documentation/config/sendemail.txt
@@ -4,7 +4,7 @@ sendemail.identity::
 	values in the 'sendemail' section. The default identity is
 	the value of `sendemail.identity`.
 
-sendemail.smtpEncryption::
+sendemail.smtpEncryption annotate:bugreport[include] ::
 	See linkgit:git-send-email[1] for description.  Note that this
 	setting is not subject to the 'identity' mechanism.
 
@@ -15,7 +15,7 @@ sendemail.smtpsslcertpath::
 	Path to ca-certificates (either a directory or a single file).
 	Set it to an empty string to disable certificate verification.
 
-sendemail.<identity>.*::
+sendemail.<identity>.* annotate:bugreport[exclude] ::
 	Identity-specific versions of the 'sendemail.*' parameters
 	found below, taking precedence over those when this
 	identity is selected, through either the command-line or
@@ -23,41 +23,41 @@ sendemail.<identity>.*::
 
 sendemail.aliasesFile::
 sendemail.aliasFileType::
-sendemail.annotate::
-sendemail.bcc::
-sendemail.cc::
-sendemail.ccCmd::
-sendemail.chainReplyTo::
-sendemail.confirm::
-sendemail.envelopeSender::
-sendemail.from::
-sendemail.multiEdit::
-sendemail.signedoffbycc::
-sendemail.smtpPass::
-sendemail.suppresscc::
-sendemail.suppressFrom::
-sendemail.to::
-sendemail.tocmd::
-sendemail.smtpDomain::
-sendemail.smtpServer::
-sendemail.smtpServerPort::
-sendemail.smtpServerOption::
-sendemail.smtpUser::
-sendemail.thread::
-sendemail.transferEncoding::
-sendemail.validate::
-sendemail.xmailer::
+sendemail.annotate annotate:bugreport[include] ::
+sendemail.bcc annotate:bugreport[include] ::
+sendemail.cc annotate:bugreport[include] ::
+sendemail.ccCmd annotate:bugreport[include] ::
+sendemail.chainReplyTo annotate:bugreport[include] ::
+sendemail.confirm annotate:bugreport[include] ::
+sendemail.envelopeSender annotate:bugreport[include] ::
+sendemail.from annotate:bugreport[include] ::
+sendemail.multiEdit annotate:bugreport[include] ::
+sendemail.signedoffbycc annotate:bugreport[include] ::
+sendemail.smtpPass annotate:bugreport[exclude] ::
+sendemail.suppresscc annotate:bugreport[include] ::
+sendemail.suppressFrom annotate:bugreport[include] ::
+sendemail.to annotate:bugreport[include] ::
+sendemail.tocmd annotate:bugreport[include] ::
+sendemail.smtpDomain annotate:bugreport[include] ::
+sendemail.smtpServer annotate:bugreport[include] ::
+sendemail.smtpServerPort annotate:bugreport[include] ::
+sendemail.smtpServerOption annotate:bugreport[include] ::
+sendemail.smtpUser annotate:bugreport[exclude] ::
+sendemail.thread annotate:bugreport[include] ::
+sendemail.transferEncoding annotate:bugreport[include] ::
+sendemail.validate annotate:bugreport[include] ::
+sendemail.xmailer annotate:bugreport[include] ::
 	See linkgit:git-send-email[1] for description.
 
 sendemail.signedoffcc (deprecated)::
 	Deprecated alias for `sendemail.signedoffbycc`.
 
-sendemail.smtpBatchSize::
+sendemail.smtpBatchSize annotate:bugreport[include] ::
 	Number of messages to be sent per connection, after that a relogin
 	will happen.  If the value is 0 or undefined, send all messages in
 	one connection.
 	See also the `--batch-size` option of linkgit:git-send-email[1].
 
-sendemail.smtpReloginDelay::
+sendemail.smtpReloginDelay annotate:bugreport[include] ::
 	Seconds wait before reconnecting to smtp server.
 	See also the `--relogin-delay` option of linkgit:git-send-email[1].
diff --git a/Makefile b/Makefile
index 9e6705061d..6bdd3b9337 100644
--- a/Makefile
+++ b/Makefile
@@ -818,6 +818,7 @@ VCSSVN_LIB = vcs-svn/lib.a
 
 GENERATED_H += config-list.h
 GENERATED_H += command-list.h
+GENERATED_H += bugreport-config-safelist.h
 
 LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
 	$(FIND) . \
@@ -2167,6 +2168,12 @@ command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Doc
 		$(patsubst %,--exclude-program %,$(EXCLUDED_PROGRAMS)) \
 		command-list.txt >$@+ && mv $@+ $@
 
+bugreport-config-safelist.h: generate-bugreport-config-safelist.sh
+
+bugreport-config-safelist.h: Documentation/config/*.txt
+	$(QUIET_GEN)$(SHELL_PATH) ./generate-bugreport-config-safelist.sh \
+		>$@+ && mv $@+ $@
+
 SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):$(GIT_VERSION):\
 	$(localedir_SQ):$(NO_CURL):$(USE_GETTEXT_SCHEME):$(SANE_TOOL_PATH_SQ):\
 	$(gitwebdir_SQ):$(PERL_PATH_SQ):$(SANE_TEXT_GREP):$(PAGER_ENV):\
diff --git a/generate-bugreport-config-safelist.sh b/generate-bugreport-config-safelist.sh
new file mode 100755
index 0000000000..a9e5b6b2a0
--- /dev/null
+++ b/generate-bugreport-config-safelist.sh
@@ -0,0 +1,18 @@
+#!/bin/sh
+
+cat <<EOF
+/* Automatically generated by bugreport-generate-config-safelist.sh */
+
+
+static const char *bugreport_config_safelist[] = {
+EOF
+
+# cat all regular files in Documentation/config
+find Documentation/config -type f -exec cat {} \; |
+# print the command name which matches the annotate-bugreport macro
+sed -n 's/^\([^ ]*\)  *annotate:bugreport\[include\].* ::$/  "\1",/p' \
+	| sort
+
+cat <<EOF
+};
+EOF

-- 
2.25.0.265.gbab2e86ba0-goog
Previous: Emily ShafferNext: Junio C Hamano
Message 32 of 87 in “add git-bugreport tool”
  1. 00/15 add git-bugreport toolEmily Shaffer, Feb 20, 2020
  2. 02/15 help: add shell-path to --build-optionsEmily Shaffer, Feb 20, 2020
  3. Junio C HamanoFeb 20, 2020
  4. Emily ShafferFeb 20, 2020
  5. 01/15 help: move list_config_help to builtin/helpEmily Shaffer, Feb 20, 2020
  6. 03/15 bugreport: add tool to generate debugging infoEmily Shaffer, Feb 20, 2020
  7. Junio C HamanoFeb 20, 2020
  8. Emily ShafferFeb 20, 2020
  9. Johannes SchindelinFeb 26, 2020
  10. 04/15 bugreport: gather git version and build infoEmily Shaffer, Feb 20, 2020
  11. Junio C HamanoFeb 20, 2020
  12. Emily ShafferFeb 20, 2020
  13. Junio C HamanoFeb 20, 2020
  14. 05/15 bugreport: add uname infoEmily Shaffer, Feb 20, 2020
  15. Junio C HamanoFeb 20, 2020
  16. Emily ShafferFeb 20, 2020
  17. 06/15 bugreport: add compiler infoEmily Shaffer, Feb 20, 2020
  18. Danh DoanFeb 20, 2020
  19. Emily ShafferFeb 20, 2020
  20. Junio C HamanoFeb 20, 2020
  21. Junio C HamanoFeb 21, 2020
  22. 07/15 bugreport: add git-remote-https versionEmily Shaffer, Feb 20, 2020
  23. Junio C HamanoFeb 20, 2020
  24. Emily ShafferFeb 20, 2020
  25. Junio C HamanoFeb 21, 2020
  26. Emily ShafferFeb 25, 2020
  27. Junio C HamanoFeb 25, 2020
  28. Emily ShafferFeb 25, 2020
  29. Junio C HamanoFeb 25, 2020
  30. Emily ShafferFeb 25, 2020
  31. 08/15 bugreport: include user interactive shellEmily Shaffer, Feb 20, 2020
  32. 09/15 bugreport: generate config safelist based on docsEmily Shaffer, Feb 20, 2020
  33. Junio C HamanoFeb 20, 2020
  34. Johannes SchindelinFeb 26, 2020
  35. Junio C HamanoFeb 26, 2020
  36. 10/15 bugreport: add config values from safelistEmily Shaffer, Feb 20, 2020
  37. Junio C HamanoFeb 20, 2020
  38. 11/15 bugreport: collect list of populated hooksEmily Shaffer, Feb 20, 2020
  39. Junio C HamanoFeb 20, 2020
  40. Emily ShafferFeb 25, 2020
  41. 12/15 bugreport: count loose objectsEmily Shaffer, Feb 20, 2020
  42. Junio C HamanoFeb 20, 2020
  43. Emily ShafferFeb 25, 2020
  44. Emily ShafferFeb 25, 2020
  45. 13/15 bugreport: add packed object summaryEmily Shaffer, Feb 20, 2020
  46. Junio C HamanoFeb 20, 2020
  47. Emily ShafferFeb 25, 2020
  48. 14/15 bugreport: list contents of $OBJDIR/infoEmily Shaffer, Feb 20, 2020
  49. Junio C HamanoFeb 20, 2020
  50. 15/15 bugreport: summarize contents of alternates fileEmily Shaffer, Feb 20, 2020
  51. Johannes SchindelinFeb 20, 2020
  52. Junio C HamanoFeb 20, 2020
  53. 0/5 add git-bugreport toolEmily Shaffer, Mar 2, 2020
  54. 1/5 help: move list_config_help to builtin/helpEmily Shaffer, Mar 2, 2020
  55. 2/5 bugreport: add tool to generate debugging infoEmily Shaffer, Mar 2, 2020
  56. Johannes SchindelinMar 3, 2020
  57. Johannes SchindelinMar 4, 2020
  58. Jeff HostetlerMar 5, 2020
  59. Johannes SchindelinMar 6, 2020
  60. Junio C HamanoMar 6, 2020
  61. Junio C HamanoMar 6, 2020
  62. Jeff HostetlerMar 6, 2020
  63. Johannes SchindelinMar 8, 2020
  64. Junio C HamanoMar 9, 2020
  65. Johannes SchindelinMar 9, 2020
  66. Junio C HamanoMar 9, 2020
  67. Johannes SchindelinMar 10, 2020
  68. Junio C HamanoMar 10, 2020
  69. Johannes SchindelinMar 10, 2020
  70. Emily ShafferMar 19, 2020
  71. Junio C HamanoMar 20, 2020
  72. Johannes SchindelinMar 20, 2020
  73. Emily ShafferMar 23, 2020
  74. Johannes SchindelinMar 20, 2020
  75. Emily ShafferMar 23, 2020
  76. Junio C HamanoMar 20, 2020
  77. Johannes SchindelinMar 20, 2020
  78. Junio C HamanoMar 20, 2020
  79. Johannes SchindelinMar 21, 2020
  80. 3/5 bugreport: gather git version and build infoEmily Shaffer, Mar 2, 2020
  81. Junio C HamanoMar 23, 2020
  82. 4/5 bugreport: add uname infoEmily Shaffer, Mar 2, 2020
  83. 5/5 bugreport: add compiler infoEmily Shaffer, Mar 2, 2020
  84. Danh DoanMar 3, 2020
  85. Junio C HamanoMar 3, 2020
  86. Johannes SchindelinMar 4, 2020
  87. Emily ShafferMar 23, 2020

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.