threads / patch / 52442

v4, 15 partsadd git-bugreport tool

Subject: [PATCH v4 00/15] add git-bugreport tool

## tl;dr

186 messages between Dec 13, 2019 and Feb 24, 2020. Diffs are folded; open one to read it.

replies: 185people: 10as markdown or json

Emily Shaffer· Dec 13, 2019, 00:42 UTC · lore

This topic branch depends on the patch mailed in lore.kernel.org/git/20191211233820.185153-1-emilyshaffer@google.com in order to display scopes for configs gathered during "bugreport: add config values from safelist".

Quite a few changes since v3. I'll try to summarize the big picture stuff, plus highlight new changes and difference in patches inline in the commit list. I'll also stick an interdiff since v3 at the bottom of this cover letter.

Some overall changes:
- The entire command now exists in one file, bugreport.c, which is
  compiled as a standalone.
- static-ness of local functions has been cleaned up.
- strbuf lifetime management has been cleaned up.
- Mention of "whitelist" and "blacklist" has been changed to "safelist"
  and "blocklist" while dealing with Git-approved bugreportable configs.
  Thanks Dscho for pointing this out; "safelist" makes sense to me as
  these are configs we have deemed safe for public inspection.
- Inquiry into both Git's $SHELL_PATH and the user's $SHELL is new to
  v4.
- The 'man git bugreport' page which was lost from the script version of
  git-bugreport has been recovered, as have the "tests" (which only test
  whether it runs and doesn't crash).
Emily Shaffer (15):
  bugreport: add tool to generate debugging info
Moved from library + builtin to a standalone git-bugreport binary.
  help: move list_config_help to builtin/help
Resolves a circular dependency where using help.o required all builtins
to exist. This also moves generation of config-list.h into its own
script (it was previously roommates with command-list.h).

The following 4 commits were previously a single commit which used run-command.h to call different processes; this has been removed except for the cURL version.

  bugreport: gather git version and build info
Additionally capture the "--build-info" output from 'git version' where it
was not previously captured.
  help: add shell-path to --build-options
Adding $SHELL_PATH to 'git --version --build-options' gets it in the
bugreport for free. (This is the shell Git uses, determined at build
time.)
  bugreport: add uname info
Capture uname from the library instead of invoking a different process.
  bugreport: add glibc version
Capture glibc version info from the library instead of invoking the
'ldd' process.
  bugreport: add curl version
Capture the cURL version from git-http-fetch, and add a -V flag to that
binary for this purpose.
  bugreport: include user interactive shell
A new feature; check what $SHELL the user is using (at runtime)
  bugreport: generate config safelist based on docs
This commit squashes in suggestions both from Johannes Schindelin and
Martin Ågren, hopefully resulting in Asciidoc(tor) not showing the
safelist macros, and a more portable safelist generation script. It also
puts the safelist, sorted, directly into a header instead of into a text
file, saving us a file read at runtime.
  bugreport: add config values from safelist
At Dscho's suggestion, now we put these into a map (based on khash and
oidset.h's example) for constant-time lookup during the config parse.
  bugreport: collect list of populated hooks
Only a minor change - now this works if 'git bugreport' is invoked
outside of a Git repo.
  bugreport: count loose objects
  bugreport: add packed object summary
  bugreport: list contents of $OBJDIR/info
For the above 3, the entire 'string_list' has been refactored out; now
the obj_info buffer is written to directly during the directory
traversal. Also paid more attention to when to free strbuf used for paths.
  bugreport: summarize contents of alternates file
Rather than printing the contents of the alternates file, now the number
of specified alternate stores are listed, and they are tested to check
whether they exist. This way we stop looking at user filepaths
(sensitive) and start learning whether they have a broken alternate (new
and useful info).
 .gitignore                              |   3 +
 Documentation/asciidoc.conf             |   8 +
 Documentation/asciidoctor-extensions.rb |   7 +
 Documentation/config/sendemail.txt      |  68 ++--
 Documentation/git-bugreport.txt         |  43 +++
 Documentation/git-http-fetch.txt        |   5 +
 Makefile                                |  31 +-
 bugreport.c                             | 434 ++++++++++++++++++++++++
 builtin/help.c                          |  86 +++++
 generate-bugreport-config-safelist.sh   |  22 ++
 generate-cmdlist.sh                     |  19 --
 generate-configlist.sh                  |  24 ++
 help.c                                  | 131 ++-----
 help.h                                  |   2 +-
 http-fetch.c                            |  13 +-
 t/t0091-bugreport.sh                    |  41 +++
 16 files changed, 774 insertions(+), 163 deletions(-)
 create mode 100644 Documentation/git-bugreport.txt
 create mode 100644 bugreport.c
 create mode 100755 generate-bugreport-config-safelist.sh
 create mode 100755 generate-configlist.sh
 create mode 100755 t/t0091-bugreport.sh
Show changes to 23 files +668 −503

.gitignore, Documentation/asciidoc.conf, Documentation/asciidoctor-extensions.rb, Documentation/config/sendemail.txt, Documentation/git-bugreport.txt, Documentation/git-http-fetch.txt, Makefile, bugreport-generate-config-whitelist.sh, bugreport.c, bugreport.h, builtin.h, builtin/bugreport.c, builtin/help.c, config.c, config.h, generate-bugreport-config-safelist.sh, generate-cmdlist.sh, generate-configlist.sh, git.c, help.c, help.h, http-fetch.c, t/t0091-bugreport.sh

diff --git a/.gitignore b/.gitignore
index 055a84c4a8..30935621d9 100644
--- a/.gitignore
+++ b/.gitignore
@@ -189,7 +189,9 @@
 /gitweb/gitweb.cgi
 /gitweb/static/gitweb.js
 /gitweb/static/gitweb.min.*
+/config-list.h
 /command-list.h
+/bugreport-config-safelist.h
 *.tar.gz
 *.dsc
 *.deb
diff --git a/Documentation/asciidoc.conf b/Documentation/asciidoc.conf
index 8fc4b67081..5d5359fcf9 100644
--- a/Documentation/asciidoc.conf
+++ b/Documentation/asciidoc.conf
@@ -6,9 +6,13 @@
 #
 # Show Git link as: <command>(<section>); if section is defined, else just show
 # the command.
+#
+# The bugreport macro does nothing as far as rendering is
+# concerned -- we just grep for it in the sources.
 
 [macros]
 (?su)[\\]?(?P<name>linkgit):(?P<target>\S*?)\[(?P<attrlist>.*?)\]=
+(?su)[\\]?(?P<name>bugreport):(?P<action>\S*?)\[(?P<attrlist>.*?)\]=
 
 [attributes]
 asterisk=&#42;
@@ -28,6 +32,8 @@ ifdef::backend-docbook[]
 {0#<citerefentry>}
 {0#<refentrytitle>{target}</refentrytitle><manvolnum>{0}</manvolnum>}
 {0#</citerefentry>}
+[bugreport-inlinemacro]
+{0#}
 endif::backend-docbook[]
 
 ifdef::backend-docbook[]
@@ -94,4 +100,6 @@ ifdef::backend-xhtml11[]
 git-relative-html-prefix=
 [linkgit-inlinemacro]
 <a href="{git-relative-html-prefix}{target}.html">{target}{0?({0})}</a>
+[bugreport-inlinemacro]
+<!-- -->
 endif::backend-xhtml11[]
diff --git a/Documentation/asciidoctor-extensions.rb b/Documentation/asciidoctor-extensions.rb
index d906a00803..750bdff9af 100644
--- a/Documentation/asciidoctor-extensions.rb
+++ b/Documentation/asciidoctor-extensions.rb
@@ -37,6 +37,10 @@ module Git
           output = output.sub(/<\/refmeta>/, new_tags + "</refmeta>")
         end
         output
+
+    class BugReportProcessor < Asciidoctor::Extensions::InlineMacroProcessor
+      def process(parent, action, attrs)
+        ""
       end
     end
   end
@@ -45,4 +49,7 @@ end
 Asciidoctor::Extensions.register do
   inline_macro Git::Documentation::LinkGitProcessor, :linkgit
   postprocessor Git::Documentation::DocumentPostProcessor
+  # The bugreport macro does nothing as far as rendering is
+  # concerned -- we just grep for it in the sources.
+  inline_macro Git::Documentation::BugReportProcessor, :bugreport
 end
diff --git a/Documentation/config/sendemail.txt b/Documentation/config/sendemail.txt
index 69f3e4f219..92f5082013 100644
--- a/Documentation/config/sendemail.txt
+++ b/Documentation/config/sendemail.txt
@@ -1,63 +1,63 @@
-sendemail.identity:: // bugreport-exclude
+sendemail.identity bugreport:exclude[x] ::
 	A configuration identity. When given, causes values in the
 	'sendemail.<identity>' subsection to take precedence over
 	values in the 'sendemail' section. The default identity is
 	the value of `sendemail.identity`.
 
-sendemail.smtpEncryption:: // bugreport-include
+sendemail.smtpEncryption bugreport:include[x] ::
 	See linkgit:git-send-email[1] for description.  Note that this
 	setting is not subject to the 'identity' mechanism.
 
-sendemail.smtpssl (deprecated):: // bugreport-exclude
+sendemail.smtpssl (deprecated) bugreport:exclude[x] ::
 	Deprecated alias for 'sendemail.smtpEncryption = ssl'.
 
-sendemail.smtpsslcertpath:: // bugreport-exclude
+sendemail.smtpsslcertpath bugreport:exclude[x] ::
 	Path to ca-certificates (either a directory or a single file).
 	Set it to an empty string to disable certificate verification.
 
-sendemail.<identity>.*:: // bugreport-exclude
+sendemail.<identity>.* bugreport:exclude[x] ::
 	Identity-specific versions of the 'sendemail.*' parameters
 	found below, taking precedence over those when this
 	identity is selected, through either the command-line or
 	`sendemail.identity`.
 
-sendemail.aliasesFile:: // bugreport-exclude
-sendemail.aliasFileType:: // bugreport-exclude
-sendemail.annotate:: // bugreport-include
-sendemail.bcc:: // bugreport-include
-sendemail.cc:: // bugreport-include
-sendemail.ccCmd:: // bugreport-include
-sendemail.chainReplyTo:: // bugreport-include
-sendemail.confirm:: // bugreport-include
-sendemail.envelopeSender:: // bugreport-include
-sendemail.from:: // bugreport-include
-sendemail.multiEdit:: // bugreport-include
-sendemail.signedoffbycc:: // bugreport-include
-sendemail.smtpPass:: // bugreport-exclude
-sendemail.suppresscc:: // bugreport-include
-sendemail.suppressFrom:: // bugreport-include
-sendemail.to:: // bugreport-include
-sendemail.tocmd:: // bugreport-include
-sendemail.smtpDomain:: // bugreport-include
-sendemail.smtpServer:: // bugreport-include
-sendemail.smtpServerPort:: // bugreport-include
-sendemail.smtpServerOption:: // bugreport-include
-sendemail.smtpUser:: // bugreport-exclude
-sendemail.thread:: // bugreport-include
-sendemail.transferEncoding:: // bugreport-include
-sendemail.validate:: // bugreport-include
-sendemail.xmailer:: // bugreport-include
+sendemail.aliasesFile bugreport:exclude[x] ::
+sendemail.aliasFileType bugreport:exclude[x] ::
+sendemail.annotate bugreport:include[x] ::
+sendemail.bcc bugreport:include[x] ::
+sendemail.cc bugreport:include[x] ::
+sendemail.ccCmd bugreport:include[x] ::
+sendemail.chainReplyTo bugreport:include[x] ::
+sendemail.confirm bugreport:include[x] ::
+sendemail.envelopeSender bugreport:include[x] ::
+sendemail.from bugreport:include[x] ::
+sendemail.multiEdit bugreport:include[x] ::
+sendemail.signedoffbycc bugreport:include[x] ::
+sendemail.smtpPass bugreport:exclude[x] ::
+sendemail.suppresscc bugreport:include[x] ::
+sendemail.suppressFrom bugreport:include[x] ::
+sendemail.to bugreport:include[x] ::
+sendemail.tocmd bugreport:include[x] ::
+sendemail.smtpDomain bugreport:include[x] ::
+sendemail.smtpServer bugreport:include[x] ::
+sendemail.smtpServerPort bugreport:include[x] ::
+sendemail.smtpServerOption bugreport:include[x] ::
+sendemail.smtpUser bugreport:exclude[x] ::
+sendemail.thread bugreport:include[x] ::
+sendemail.transferEncoding bugreport:include[x] ::
+sendemail.validate bugreport:include[x] ::
+sendemail.xmailer bugreport:include[x] ::
 	See linkgit:git-send-email[1] for description.
 
-sendemail.signedoffcc (deprecated):: // bugreport-exclude
+sendemail.signedoffcc (deprecated) bugreport:exclude[x] ::
 	Deprecated alias for `sendemail.signedoffbycc`.
 
-sendemail.smtpBatchSize:: // bugreport-include
+sendemail.smtpBatchSize bugreport:include[x] ::
 	Number of messages to be sent per connection, after that a relogin
 	will happen.  If the value is 0 or undefined, send all messages in
 	one connection.
 	See also the `--batch-size` option of linkgit:git-send-email[1].
 
-sendemail.smtpReloginDelay:: // bugreport-include
+sendemail.smtpReloginDelay bugreport:include[x] ::
 	Seconds wait before reconnecting to smtp server.
 	See also the `--relogin-delay` option of linkgit:git-send-email[1].
diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
new file mode 100644
index 0000000000..75f0c80acf
--- /dev/null
+++ b/Documentation/git-bugreport.txt
@@ -0,0 +1,43 @@
+git-bugreport(1)
+================
+
+NAME
+----
+git-bugreport - Collect information for user to file a bug report
+
+SYNOPSIS
+--------
+[verse]
+'git bugreport' [-o | --output <path>]
+
+DESCRIPTION
+-----------
+Captures information about the user's machine, Git client, and repository state,
+as well as a form requesting information about the behavior the user observed,
+into a single text file which the user can then share, for example to the Git
+mailing list, in order to report an observed bug.
+
+The following information is requested from the user:
+
+ - Reproduction steps
+ - Expected behavior
+ - Actual behavior
+
+The following information is captured automatically:
+
+ - Git version (`git version --build-options`)
+ - Machine information (`uname -a`)
+ - Versions of various dependencies
+ - Git config contents (`git config --show-origin --list`)
+ - The names of all configured git-hooks in `.git/hooks/`
+
+OPTIONS
+-------
+-o [<path>]::
+--output [<path>]::
+	Place the resulting bug report file in <path> instead of the root of the
+	Git repository.
+
+GIT
+---
+Part of the linkgit:git[1] suite
diff --git a/Documentation/git-http-fetch.txt b/Documentation/git-http-fetch.txt
index 666b042679..2894c5e82b 100644
--- a/Documentation/git-http-fetch.txt
+++ b/Documentation/git-http-fetch.txt
@@ -10,6 +10,7 @@ SYNOPSIS
 --------
 [verse]
 'git http-fetch' [-c] [-t] [-a] [-d] [-v] [-w filename] [--recover] [--stdin] <commit> <url>
+'git http-fetch' [-V]
 
 DESCRIPTION
 -----------
@@ -30,6 +31,10 @@ commit-id::
 -v::
 	Report what is downloaded.
 
+-V::
+	Report information about the version of git-http-fetch, including the
+	versions of its dependencies.
+
 -w <filename>::
         Writes the commit-id into the filename under $GIT_DIR/refs/<filename> on
         the local end after the transfer is complete.
diff --git a/Makefile b/Makefile
index 6e361eed64..76dc51e2b1 100644
--- a/Makefile
+++ b/Makefile
@@ -634,10 +634,6 @@ SCRIPT_PYTHON += git-p4.py
 SCRIPT_SH_GEN = $(patsubst %.sh,%,$(SCRIPT_SH))
 SCRIPT_PERL_GEN = $(patsubst %.perl,%,$(SCRIPT_PERL))
 SCRIPT_PYTHON_GEN = $(patsubst %.py,%,$(SCRIPT_PYTHON))
-SCRIPT_DEPENDENCIES = git-bugreport-config-whitelist
-
-$(SCRIPT_DEPENDENCIES): Documentation/config/*.txt
-	sh bugreport-generate-config-whitelist.sh
 
 # Individual rules to allow e.g.
 # "make -C ../.. SCRIPT_PERL=contrib/foo/bar.perl build-perl-script"
@@ -662,13 +658,10 @@ clean-perl-script:
 	$(RM) $(SCRIPT_PERL_GEN)
 clean-python-script:
 	$(RM) $(SCRIPT_PYTHON_GEN)
-clean-script-dependencies:
-	$(RM) $(SCRIPT_DEPENDENCIES)
 
 SCRIPTS = $(SCRIPT_SH_GEN) \
 	  $(SCRIPT_PERL_GEN) \
 	  $(SCRIPT_PYTHON_GEN) \
-	  $(SCRIPT_DEPENDENCIES) \
 	  git-instaweb
 
 ETAGS_TARGET = TAGS
@@ -688,6 +681,7 @@ EXTRA_PROGRAMS =
 # ... and all the rest that could be moved out of bindir to gitexecdir
 PROGRAMS += $(EXTRA_PROGRAMS)
 
+PROGRAM_OBJS += bugreport.o
 PROGRAM_OBJS += credential-store.o
 PROGRAM_OBJS += daemon.o
 PROGRAM_OBJS += fast-import.o
@@ -821,7 +815,9 @@ LIB_FILE = libgit.a
 XDIFF_LIB = xdiff/lib.a
 VCSSVN_LIB = vcs-svn/lib.a
 
+GENERATED_H += config-list.h
 GENERATED_H += command-list.h
+GENERATED_H += bugreport-config-safelist.h
 
 LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
 	$(FIND) . \
@@ -846,7 +842,6 @@ LIB_OBJS += bisect.o
 LIB_OBJS += blame.o
 LIB_OBJS += blob.o
 LIB_OBJS += branch.o
-LIB_OBJS += bugreport.o
 LIB_OBJS += bulk-checkin.o
 LIB_OBJS += bundle.o
 LIB_OBJS += cache-tree.o
@@ -1049,7 +1044,6 @@ BUILTIN_OBJS += builtin/archive.o
 BUILTIN_OBJS += builtin/bisect--helper.o
 BUILTIN_OBJS += builtin/blame.o
 BUILTIN_OBJS += builtin/branch.o
-BUILTIN_OBJS += builtin/bugreport.o
 BUILTIN_OBJS += builtin/bundle.o
 BUILTIN_OBJS += builtin/cat-file.o
 BUILTIN_OBJS += builtin/check-attr.o
@@ -2135,7 +2129,7 @@ git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS)
 
 help.sp help.s help.o: command-list.h
 
-builtin/help.sp builtin/help.s builtin/help.o: command-list.h GIT-PREFIX
+builtin/help.sp builtin/help.s builtin/help.o: config-list.h GIT-PREFIX
 builtin/help.sp builtin/help.s builtin/help.o: EXTRA_CPPFLAGS = \
 	'-DGIT_HTML_PATH="$(htmldir_relative_SQ)"' \
 	'-DGIT_MAN_PATH="$(mandir_relative_SQ)"' \
@@ -2155,6 +2149,12 @@ $(BUILT_INS): git$X
 	ln -s $< $@ 2>/dev/null || \
 	cp $< $@
 
+config-list.h: generate-configlist.sh
+
+config-list.h:
+	$(QUIET_GEN)$(SHELL_PATH) ./generate-configlist.sh \
+		>$@+ && mv $@+ $@
+
 command-list.h: generate-cmdlist.sh command-list.txt
 
 command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Documentation/config/*.txt
@@ -2162,6 +2162,12 @@ command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Doc
 		$(patsubst %,--exclude-program %,$(EXCLUDED_PROGRAMS)) \
 		command-list.txt >$@+ && mv $@+ $@
 
+bugreport-config-safelist.h: generate-bugreport-config-safelist.sh
+
+bugreport-config-safelist.h: Documentation/config/*.txt
+	$(QUIET_GEN)$(SHELL_PATH) ./generate-bugreport-config-safelist.sh \
+		>$@+ && mv $@+ $@
+
 SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):$(GIT_VERSION):\
 	$(localedir_SQ):$(NO_CURL):$(USE_GETTEXT_SCHEME):$(SANE_TOOL_PATH_SQ):\
 	$(gitwebdir_SQ):$(PERL_PATH_SQ):$(SANE_TEXT_GREP):$(PAGER_ENV):\
@@ -2457,6 +2463,10 @@ endif
 git-%$X: %.o GIT-LDFLAGS $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) $(LIBS)
 
+git-bugreport$X: bugreport.o GIT-LDFLAGS $(GITLIBS)
+	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
+		$(LIBS)
+
 git-imap-send$X: imap-send.o $(IMAP_SEND_BUILDDEPS) GIT-LDFLAGS $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
 		$(IMAP_SEND_LDFLAGS) $(LIBS)
@@ -2788,7 +2798,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE
 .PHONY: sparse $(SP_OBJ)
 sparse: $(SP_OBJ)
 
-GEN_HDRS := command-list.h unicode-width.h
+GEN_HDRS := config-list.h command-list.h unicode-width.h bugreport-config-safelist.h
 EXCEPT_HDRS := $(GEN_HDRS) compat/% xdiff/%
 ifndef GCRYPT_SHA256
 	EXCEPT_HDRS += sha256/gcrypt.h
@@ -2811,7 +2821,7 @@ hdr-check: $(HCO)
 style:
 	git clang-format --style file --diff --extensions c,h
 
-check: command-list.h
+check: config-list.h command-list.h
 	@if sparse; \
 	then \
 		echo >&2 "Use 'make sparse' instead"; \
@@ -3114,7 +3124,9 @@ clean: profile-clean coverage-clean cocciclean
 	$(RM) $(HCC)
 	$(RM) -r bin-wrappers $(dep_dirs)
 	$(RM) -r po/build/
-	$(RM) *.pyc *.pyo */*.pyc */*.pyo command-list.h $(ETAGS_TARGET) tags cscope*
+	$(RM) *.pyc *.pyo */*.pyc */*.pyo
+	$(RM) config-list.h command-list.h bugreport-config-safelist.h
+	$(RM) $(ETAGS_TARGET) tags cscope*
 	$(RM) -r $(GIT_TARNAME) .doc-tmp-dir
 	$(RM) $(GIT_TARNAME).tar.gz git-core_$(GIT_VERSION)-*.tar.gz
 	$(RM) $(htmldocs).tar.gz $(manpages).tar.gz
diff --git a/bugreport-generate-config-whitelist.sh b/bugreport-generate-config-whitelist.sh
deleted file mode 100755
index ca6b232024..0000000000
--- a/bugreport-generate-config-whitelist.sh
+++ /dev/null
@@ -1,4 +0,0 @@
-#!/bin/sh
-
-grep -RhPo ".*(?=:: \/\/ bugreport-include)" Documentation/config \
-  >git-bugreport-config-whitelist
diff --git a/bugreport.c b/bugreport.c
index a7bdc72b7f..d2a3fb1f67 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -1,114 +1,128 @@
 #include "cache.h"
-
-#include "bugreport.h"
-#include "config.h"
-#include "exec-cmd.h"
+#include "parse-options.h"
+#include "stdio.h"
+#include "strbuf.h"
+#include "time.h"
 #include "help.h"
+#include <gnu/libc-version.h>
 #include "run-command.h"
-#include "strbuf.h"
-#include "string-list.h"
-#include "version.h"
-
-#include "dirent.h"
-
-/**
- * A sorted list of config options which we will add to the bugreport. Managed
- * by 'gather_whitelist(...)'.
- */
-struct string_list whitelist = STRING_LIST_INIT_DUP;
-struct strbuf configs_and_values = STRBUF_INIT;
-
-// git version --build-options
-// uname -a
-// curl-config --version
-// ldd --version
-// echo $SHELL
-void get_system_info(struct strbuf *sys_info)
+#include "config.h"
+#include "bugreport-config-safelist.h"
+#include "khash.h"
+#include "run-command.h"
+
+static void get_http_version_info(struct strbuf *http_info)
 {
 	struct child_process cp = CHILD_PROCESS_INIT;
-	struct strbuf std_out = STRBUF_INIT;
 
-	strbuf_reset(sys_info);
+	argv_array_push(&cp.args, "git");
+	argv_array_push(&cp.args, "http-fetch");
+	argv_array_push(&cp.args, "-V");
+	if (capture_command(&cp, http_info, 0))
+	    strbuf_addstr(http_info, "'git-http-fetch -V' not supported\n");
+}
 
-	// get git version from native cmd
-	strbuf_addstr(sys_info, "git version: ");
-	strbuf_addstr(sys_info, git_version_string);
-	strbuf_complete_line(sys_info);
+KHASH_INIT(cfg_set, const char*, int, 0, kh_str_hash_func, kh_str_hash_equal);
 
-	// system call for other version info
-	argv_array_push(&cp.args, "uname");
-	argv_array_push(&cp.args, "-a");
-	capture_command(&cp, &std_out, 0);
+struct cfgset {
+	kh_cfg_set_t set;
+};
 
-	strbuf_addstr(sys_info, "uname -a: ");
-	strbuf_addbuf(sys_info, &std_out);
-	strbuf_complete_line(sys_info);
-
-	argv_array_clear(&cp.args);
-	strbuf_reset(&std_out);
+struct cfgset safelist;
 
+static void cfgset_init(struct cfgset *set, size_t initial_size)
+{
+	memset(&set->set, 0, sizeof(set->set));
+	if (initial_size)
+		kh_resize_cfg_set(&set->set, initial_size);
+}
 
-	argv_array_push(&cp.args, "curl-config");
-	argv_array_push(&cp.args, "--version");
-	capture_command(&cp, &std_out, 0);
+static int cfgset_insert(struct cfgset *set, const char *cfg_key)
+{
+	int added;
+	kh_put_cfg_set(&set->set, cfg_key, &added);
+	printf("ESS: added %s\n", cfg_key);
+	return !added;
+}
 
-	strbuf_addstr(sys_info, "curl-config --version: ");
-	strbuf_addbuf(sys_info, &std_out);
-	strbuf_complete_line(sys_info);
+static int cfgset_contains(struct cfgset *set, const char *cfg_key)
+{
+	khiter_t pos = kh_get_cfg_set(&set->set, cfg_key);
+	return pos != kh_end(&set->set);
+}
 
-	argv_array_clear(&cp.args);
-	strbuf_reset(&std_out);
+static void cfgset_clear(struct cfgset *set)
+{
+	kh_release_cfg_set(&set->set);
+	cfgset_init(set, 0);
+}
 
+static void get_system_info(struct strbuf *sys_info)
+{
+	struct strbuf version_info = STRBUF_INIT;
+	struct utsname uname_info;
 
-	argv_array_push(&cp.args, "ldd");
-	argv_array_push(&cp.args, "--version");
-	capture_command(&cp, &std_out, 0);
+	/* get git version from native cmd */
+	strbuf_addstr(sys_info, "git version:\n");
+	list_version_info(&version_info, 1);
+	strbuf_addbuf(sys_info, &version_info);
+	strbuf_complete_line(sys_info);
 
-	strbuf_addstr(sys_info, "ldd --version: ");
-	strbuf_addbuf(sys_info, &std_out);
+	/* system call for other version info */
+	strbuf_addstr(sys_info, "uname -a: ");
+	if (uname(&uname_info))
+		strbuf_addf(sys_info, "uname() failed with code %d\n", errno);
+	else
+		strbuf_addf(sys_info, "%s %s %s %s %s\n",
+			    uname_info.sysname,
+			    uname_info.nodename,
+			    uname_info.release,
+			    uname_info.version,
+			    uname_info.machine);
+
+	strbuf_addstr(sys_info, "glibc version: ");
+	strbuf_addstr(sys_info, gnu_get_libc_version());
 	strbuf_complete_line(sys_info);
 
-	argv_array_clear(&cp.args);
-	strbuf_reset(&std_out);
-}
+	strbuf_addf(sys_info, "$SHELL (typically, interactive shell): %s\n",
+		    getenv("SHELL"));
 
-void gather_whitelist(struct strbuf *path)
-{
-	struct strbuf tmp = STRBUF_INIT;
-	strbuf_read_file(&tmp, path->buf, 0);
-	string_list_init(&whitelist, 1);
-	string_list_split(&whitelist, tmp.buf, '\n', -1);
-	string_list_sort(&whitelist);
+	strbuf_addstr(sys_info, "git-http-fetch -V:\n");
+	get_http_version_info(sys_info);
+	strbuf_complete_line(sys_info);
 }
 
-int git_config_bugreport(const char *var, const char *value, void *cb)
+static void gather_safelist()
 {
-	if (string_list_has_string(&whitelist, var)) {
-		strbuf_addf(&configs_and_values,
-			    "%s : %s\n",
-			    var, value);
-	}
+	int index;
+	int safelist_len = sizeof(bugreport_config_safelist) / sizeof(const char *);
+	cfgset_init(&safelist, safelist_len);
+	for (index = 0; index < safelist_len; index++)
+		cfgset_insert(&safelist, bugreport_config_safelist[index]);
 
-	return 0;
 }
 
-void get_whitelisted_config(struct strbuf *config_info)
+static int git_config_bugreport(const char *var, const char *value, void *cb)
 {
-	struct strbuf path = STRBUF_INIT;
+	struct strbuf *config_info = (struct strbuf *)cb;
 
-	strbuf_addstr(&path, git_exec_path());
-	strbuf_addstr(&path, "/git-bugreport-config-whitelist");
+	if (cfgset_contains(&safelist, var))
+		strbuf_addf(config_info,
+			    "%s (%s) : %s\n",
+			    var, config_scope_to_string(current_config_scope()),
+			    value);
 
-	gather_whitelist(&path);
-	strbuf_init(&configs_and_values, whitelist.nr);
-
-	git_config(git_config_bugreport, NULL);
+	return 0;
+}
 
-	strbuf_reset(config_info);
-	strbuf_addbuf(config_info, &configs_and_values);
+static void get_safelisted_config(struct strbuf *config_info)
+{
+	gather_safelist();
+	git_config(git_config_bugreport, config_info);
+	cfgset_clear(&safelist);
 }
 
-void get_populated_hooks(struct strbuf *hook_info)
+static void get_populated_hooks(struct strbuf *hook_info)
 {
 	/*
 	 * Doesn't look like there is a list of all possible hooks; so below is
@@ -139,8 +153,15 @@ void get_populated_hooks(struct strbuf *hook_info)
 			    "post-index-changex";
 	struct string_list hooks_list = STRING_LIST_INIT_DUP;
 	struct string_list_item *iter = NULL;
+	int nongit_ok;
+
+	setup_git_directory_gently(&nongit_ok);
 
-	strbuf_reset(hook_info);
+	if (nongit_ok) {
+		strbuf_addstr(hook_info,
+			"not run from a git repository - no hooks to show\n");
+		return;
+	}
 
 	string_list_split(&hooks_list, hooks, ',', -1);
 
@@ -152,163 +173,262 @@ void get_populated_hooks(struct strbuf *hook_info)
 	}
 }
 
-/**
- * Fill 'contents' with the contents of the dir at 'dirpath'.
- * If 'filter' is nonzero, the contents are filtered on d_type as 'type' - see
- * 'man readdir'. opendir() doesn't take string length as an arg, so don't
- * bother passing it in.
- */
-void list_contents_of_dir(struct string_list *contents, struct strbuf *dirpath,
-			  int filter, unsigned char type)
+static int is_hex(const char *string, size_t count)
 {
-	struct dirent *dir = NULL;
-	DIR *dh = NULL;
-
-	dh = opendir(dirpath->buf);
-	while (dh && (dir = readdir(dh))) {
-		if (!filter || type == dir->d_type) {
-			string_list_append(contents, dir->d_name);
-		}
+	for (; count; string++, count--) {
+		if (!isxdigit(*string))
+			return 0;
 	}
+	return 1;
 }
 
-/**
- * Fills 'contents' with a list of all directories within the provided
- * directory, recursing into each directory.
- */
-void list_contents_of_dir_recursively(struct string_list *contents,
-				      struct strbuf *dirpath)
-{
-	struct string_list current_contents = STRING_LIST_INIT_DUP;
-	struct string_list current_subdirs = STRING_LIST_INIT_DUP;
-	struct string_list_item *it;
-	struct strbuf buf = STRBUF_INIT;
-
-	list_contents_of_dir(&current_contents, dirpath, 0, 0);
-	for_each_string_list_item(it, &current_contents) {
-		strbuf_reset(&buf);
-		strbuf_addbuf(&buf, dirpath);
-		strbuf_complete(&buf, '/');
-		strbuf_addstr(&buf, it->string);
-
-		string_list_append(contents, buf.buf);
-	}
+static void get_loose_object_summary(struct strbuf *obj_info) {
+	struct dirent *d = NULL;
+	DIR *dir, *subdir = NULL;
+	size_t dir_len;
+	struct strbuf dirpath = STRBUF_INIT;
 
-	list_contents_of_dir(&current_subdirs, dirpath, 1, DT_DIR);
-	for_each_string_list_item(it, &current_subdirs) {
-		if (strcmp(it->string, ".") == 0
-		    || strcmp(it->string, "..") == 0)
-			continue;
-		strbuf_reset(&buf);
-		strbuf_addbuf(&buf, dirpath);
-		strbuf_complete(&buf, '/');
-		strbuf_addstr(&buf, it->string);
+	strbuf_addstr(&dirpath, get_object_directory());
+	strbuf_complete(&dirpath, '/');
 
-		list_contents_of_dir_recursively(contents, &buf);
+	dir = opendir(dirpath.buf);
+	if (!dir) {
+		strbuf_addf(obj_info, "could not open object directory '%s'\n",
+			    dirpath.buf);
+		strbuf_release(&dirpath);
+		return;
 	}
-}
 
-void get_object_counts(struct strbuf *obj_info)
-{
-	struct child_process cp = CHILD_PROCESS_INIT;
-	struct strbuf std_out = STRBUF_INIT;
-
-	argv_array_push(&cp.args, "count-objects");
-	argv_array_push(&cp.args, "-vH");
-	cp.git_cmd = 1;
-	capture_command(&cp, &std_out, 0);
+	dir_len = dirpath.len;
 
-	strbuf_reset(obj_info);
-	strbuf_addstr(obj_info, "git-count-objects -vH:\n");
-	strbuf_addbuf(obj_info, &std_out);
-}
+	while ((d = readdir(dir))) {
+		int object_count = 0;
+		char subdir_name[3];
 
-void get_loose_object_summary(struct strbuf *obj_info)
-{
-	struct strbuf dirpath = STRBUF_INIT;
-	struct string_list subdirs = STRING_LIST_INIT_DUP;
-	struct string_list_item *subdir;
-
-	strbuf_reset(obj_info);
+		if (d->d_type != DT_DIR)
+			continue;
 
-	strbuf_addstr(&dirpath, get_object_directory());
-	strbuf_complete(&dirpath, '/');
+		if ((strlen(d->d_name) != 2) || (!is_hex(d->d_name, 2)))
+			continue;
 
-	list_contents_of_dir(&subdirs, &dirpath, 1, DT_DIR);
+		/* copy directory name + \0 */
+		memcpy(subdir_name, d->d_name, 3);
 
-	for_each_string_list_item(subdir, &subdirs)
-	{
-		struct strbuf subdir_buf = STRBUF_INIT;
-		struct string_list objects = STRING_LIST_INIT_DUP;
+		strbuf_setlen(&dirpath, dir_len);
+		strbuf_addstr(&dirpath, d->d_name);
 
-		/*
-		 * Only interested in loose objects - so dirs named with the
-		 * first byte of the object ID
-		 */
-		if (strlen(subdir->string) != 2 || !strcmp(subdir->string, ".."))
+		subdir = opendir(dirpath.buf);
+		if (!subdir)
 			continue;
+		while ((d = readdir(subdir)))
+			if (d->d_type == DT_REG)
+				object_count++;
+
+		closedir(subdir);
 
-		strbuf_addbuf(&subdir_buf, &dirpath);
-		strbuf_addstr(&subdir_buf, subdir->string);
-		list_contents_of_dir(&objects, &subdir_buf, 0, 0);
-		strbuf_addf(obj_info, "%s: %d objects\n", subdir->string,
-			    objects.nr);
+		strbuf_addf(obj_info, "%s: %d\n", subdir_name, object_count);
 	}
+
+
+	closedir(dir);
+	strbuf_release(&dirpath);
 }
 
-void get_packed_object_summary(struct strbuf *obj_info)
+static void get_packed_object_summary(struct strbuf *obj_info)
 {
 	struct strbuf dirpath = STRBUF_INIT;
-	struct string_list contents = STRING_LIST_INIT_DUP;
-	struct string_list_item *entry;
-
-	strbuf_reset(obj_info);
+	struct dirent *d;
+	DIR *dir = NULL;
 
 	strbuf_addstr(&dirpath, get_object_directory());
 	strbuf_complete(&dirpath, '/');
 	strbuf_addstr(&dirpath, "pack/");
-	list_contents_of_dir(&contents, &dirpath, 0, 0);
 
-	// list full contents of $GIT_OBJECT_DIRECTORY/pack/
-	for_each_string_list_item(entry, &contents) {
+	dir = opendir(dirpath.buf);
+	if (!dir) {
+		strbuf_addf(obj_info, "could not open packed object directory '%s'\n",
+			    dirpath.buf);
+		strbuf_release(&dirpath);
+		return;
+	}
+
+	while ((d = readdir(dir))) {
 		strbuf_addbuf(obj_info, &dirpath);
-		strbuf_addstr(obj_info, entry->string);
+		strbuf_addstr(obj_info, d->d_name);
 		strbuf_complete_line(obj_info);
 	}
+
+	closedir(dir);
+	strbuf_release(&dirpath);
 }
 
-void get_object_info_summary(struct strbuf *obj_info)
+static void list_contents_of_dir_recursively(struct strbuf *contents,
+				      	     struct strbuf *dirpath)
 {
-	// strbuf += GITDIR/info/:
-	// recursively list contents of $GIT_OBJECT_DIRECTORY/info
-	struct strbuf dirpath = STRBUF_INIT;
-	struct string_list contents = STRING_LIST_INIT_DUP;
-	struct string_list_item *entry;
+	struct dirent *d;
+	DIR *dir;
+	size_t path_len;
 
-	strbuf_reset(obj_info);
+	dir = opendir(dirpath->buf);
+	if (!dir)
+		return;
+
+	strbuf_complete(dirpath, '/');
+	path_len = dirpath->len;
+
+	while ((d = readdir(dir))) {
+		if (!strcmp(d->d_name, ".") || !strcmp(d->d_name, ".."))
+			continue;
+
+		strbuf_addbuf(contents, dirpath);
+		strbuf_addstr(contents, d->d_name);
+		strbuf_complete_line(contents);
+
+		if (d->d_type == DT_DIR) {
+			strbuf_addstr(dirpath, d->d_name);
+			list_contents_of_dir_recursively(contents, dirpath);
+		}
+		strbuf_setlen(dirpath, path_len);
+	}
+
+	closedir(dir);
+}
+
+static void get_object_info_summary(struct strbuf *obj_info)
+{
+	struct strbuf dirpath = STRBUF_INIT;
 
 	strbuf_addstr(&dirpath, get_object_directory());
 	strbuf_complete(&dirpath, '/');
 	strbuf_addstr(&dirpath, "info/");
 
-	list_contents_of_dir_recursively(&contents, &dirpath);
+	list_contents_of_dir_recursively(obj_info, &dirpath);
 
-	for_each_string_list_item(entry, &contents) {
-		strbuf_addstr(obj_info, entry->string);
-		strbuf_complete_line(obj_info);
-	}
+	strbuf_release(&dirpath);
 }
 
-void get_alternates_file(struct strbuf *alternates_info)
+static void get_alternates_summary(struct strbuf *alternates_info)
 {
 	struct strbuf alternates_path = STRBUF_INIT;
+	struct strbuf alternate = STRBUF_INIT;
+	FILE *file;
+	size_t exists = 0, broken = 0;
 
 	strbuf_addstr(&alternates_path, get_object_directory());
 	strbuf_complete(&alternates_path, '/');
 	strbuf_addstr(&alternates_path, "info/alternates");
 
-	strbuf_reset(alternates_info);
-	strbuf_addbuf(alternates_info, &alternates_path);
-	strbuf_complete_line(alternates_info);
-	strbuf_read_file(alternates_info, alternates_path.buf, 0);
+	file = fopen(alternates_path.buf, "r");
+	if (!file) {
+		strbuf_addstr(alternates_info, "No alternates file found.\n");
+		strbuf_release(&alternates_path);
+		return;
+	}
+
+	while (strbuf_getline(&alternate, file) != EOF) {
+		if (!access(alternate.buf, F_OK))
+			exists++;
+		else
+			broken++;
+	}
+
+	strbuf_addf(alternates_info,
+		    "%zd alternates found (%zd working, %zd broken)\n",
+		    exists + broken,
+		    exists,
+		    broken);
+
+	fclose(file);
+	strbuf_release(&alternate);
+	strbuf_release(&alternates_path);
+}
+
+static const char * const bugreport_usage[] = {
+	N_("git bugreport [-o|--output <file>]"),
+	NULL
+};
+
+static int get_bug_template(struct strbuf *template)
+{
+	const char template_text[] = N_(
+"Thank you for filling out a Git bug report!\n"
+"Please answer the following questions to help us understand your issue.\n"
+"\n"
+"What did you do before the bug happened? (Steps to reproduce your issue)\n"
+"\n"
+"What did you expect to happen? (Expected behavior)\n"
+"\n"
+"What happened instead? (Actual behavior)\n"
+"\n"
+"What's different between what you expected and what actually happened?\n"
+"\n"
+"Anything else you want to add:\n"
+"\n"
+"Please review the rest of the bug report below.\n"
+"You can delete any lines you don't wish to send.\n");
+
+	strbuf_addstr(template, template_text);
+	return 0;
+}
+
+static void get_header(struct strbuf *buf, const char *title)
+{
+	strbuf_addf(buf, "\n\n[%s]\n", title);
+}
+
+int cmd_main(int argc, const char **argv)
+{
+	struct strbuf buffer = STRBUF_INIT;
+	struct strbuf report_path = STRBUF_INIT;
+	FILE *report;
+	time_t now = time(NULL);
+	char *option_output = NULL;
+
+	const struct option bugreport_options[] = {
+		OPT_STRING('o', "output", &option_output, N_("path"),
+			   N_("specify a destination for the bugreport file")),
+		OPT_END()
+	};
+	argc = parse_options(argc, argv, "", bugreport_options,
+			     bugreport_usage, 0);
+
+	if (option_output) {
+		strbuf_addstr(&report_path, option_output);
+		strbuf_complete(&report_path, '/');
+	}
+
+	strbuf_addstr(&report_path, "git-bugreport-");
+	strbuf_addftime(&report_path, "%F", gmtime(&now), 0, 0);
+	strbuf_addstr(&report_path, ".txt");
+
+
+	get_bug_template(&buffer);
+
+	get_header(&buffer, "System Info");
+	get_system_info(&buffer);
+
+	get_header(&buffer, "Safelisted Config Info");
+	get_safelisted_config(&buffer);
+
+	get_header(&buffer, "Configured Hooks");
+	get_populated_hooks(&buffer);
+
+	get_header(&buffer, "Loose Object Counts");
+	get_loose_object_summary(&buffer);
+
+	get_header(&buffer, "Packed Object Summary");
+	get_packed_object_summary(&buffer);
+
+	get_header(&buffer, "Object Info Summary");
+	get_object_info_summary(&buffer);
+
+	get_header(&buffer, "Alternates");
+	get_alternates_summary(&buffer);
+
+	report = fopen_for_writing(report_path.buf);
+	strbuf_write(&buffer, report);
+	fclose(report);
+
+	launch_editor(report_path.buf, NULL, NULL);
+	return 0;
 }
diff --git a/bugreport.h b/bugreport.h
deleted file mode 100644
index 74d1f79960..0000000000
--- a/bugreport.h
+++ /dev/null
@@ -1,44 +0,0 @@
-#include "strbuf.h"
-
-/**
- * Adds the Git version, `uname -a`, and `curl-config --version` to sys_info.
- * The previous contents of sys_info will be discarded.
- */
-void get_system_info(struct strbuf *sys_info);
-
-/**
- * Adds the values of the config items listed in
- * 'git-bugreport-config-whitelist' to config_info. The previous contents of
- * config_info will be discarded.
- */
-void get_whitelisted_config(struct strbuf *sys_info);
-
-/**
- * Adds the paths to all configured hooks (but not their contents). The previous
- * contents of hook_info will be discarded.
- */
-void get_populated_hooks(struct strbuf *hook_info);
-
-/**
- * Adds the output of `git count-object -vH`. The previous contents of hook_info
- * will be discarded.
- */
-void get_loose_object_summary(struct strbuf *obj_info);
-
-/**
- * Adds a list of the contents of '.git/objects/pack'. The previous contents of
- * hook_info will be discarded.
- */
-void get_packed_object_summary(struct strbuf *obj_info);
-
-/**
- * Adds a list of all contents (recursively) of '.git/objects/info'. The
- * previous contents of hook_info will be discarded.
- */
-void get_object_info_summary(struct strbuf *obj_info);
-
-/**
- * Adds the contents of '.git/info/alternates'. The previous contents of
- * alternates_info will be discarded.
- */
-void get_alternates_file(struct strbuf *alt_info);
diff --git a/builtin.h b/builtin.h
index c6373d3289..5cf5df69f7 100644
--- a/builtin.h
+++ b/builtin.h
@@ -135,7 +135,6 @@ int cmd_archive(int argc, const char **argv, const char *prefix);
 int cmd_bisect__helper(int argc, const char **argv, const char *prefix);
 int cmd_blame(int argc, const char **argv, const char *prefix);
 int cmd_branch(int argc, const char **argv, const char *prefix);
-int cmd_bugreport(int argc, const char **argv, const char *prefix);
 int cmd_bundle(int argc, const char **argv, const char *prefix);
 int cmd_cat_file(int argc, const char **argv, const char *prefix);
 int cmd_checkout(int argc, const char **argv, const char *prefix);
diff --git a/builtin/bugreport.c b/builtin/bugreport.c
deleted file mode 100644
index 0784bdc42a..0000000000
--- a/builtin/bugreport.c
+++ /dev/null
@@ -1,90 +0,0 @@
-#include "builtin.h"
-#include "bugreport.h"
-#include "stdio.h"
-#include "strbuf.h"
-#include "time.h"
-
-int get_bug_template(struct strbuf *template)
-{
-	const char template_text[] =
-"Thank you for filling out a Git bug report!\n"
-"Please answer the following questions to help us understand your issue.\n"
-"\n"
-"What did you do before the bug happened? (Steps to reproduce your issue)\n"
-"\n"
-"What did you expect to happen? (Expected behavior)\n"
-"\n"
-"What happened instead? (Actual behavior)\n"
-"\n"
-"What's different between what you expected and what actually happened?\n"
-"\n"
-"Anything else you want to add:\n"
-"\n"
-"Please review the rest of the bug report below.\n"
-"You can delete any lines you don't wish to send.\n";
-
-	strbuf_reset(template);
-	strbuf_add(template, template_text, strlen(template_text));
-	return 0;
-}
-
-void add_header(FILE *report, const char *title)
-{
-	struct strbuf buffer = STRBUF_INIT;
-	strbuf_addf(&buffer, "\n\n[%s]\n", title);
-	strbuf_write(&buffer, report);
-}
-
-int cmd_bugreport(int argc, const char **argv, const char *prefix)
-{
-	struct strbuf buffer = STRBUF_INIT;
-	struct strbuf report_path = STRBUF_INIT;
-	FILE *report;
-	time_t now = time(NULL);
-
-	strbuf_addstr(&report_path, "git-bugreport-");
-	strbuf_addftime(&report_path, "%F", gmtime(&now), 0, 0);
-	strbuf_addstr(&report_path, ".txt");
-
-	report = fopen_for_writing(report_path.buf);
-
-	get_bug_template(&buffer);
-	strbuf_write(&buffer, report);
-
-	// add other contents
-	add_header(report, "System Info");
-	get_system_info(&buffer);
-	strbuf_write(&buffer, report);
-
-	add_header(report, "Whitelisted Config");
-	get_whitelisted_config(&buffer);
-	strbuf_write(&buffer, report);
-
-	add_header(report, "Populated Hooks");
-	get_populated_hooks(&buffer);
-	strbuf_write(&buffer, report);
-
-	add_header(report, "Object Counts");
-	get_loose_object_summary(&buffer);
-	strbuf_write(&buffer, report);
-
-	add_header(report, "Packed Object Summary");
-	get_packed_object_summary(&buffer);
-	strbuf_write(&buffer, report);
-
-	add_header(report, "Object Info Data");
-	get_object_info_summary(&buffer);
-	strbuf_write(&buffer, report);
-
-	add_header(report, "Alternates File");
-	get_alternates_file(&buffer);
-	strbuf_write(&buffer, report);
-
-	// Close file
-	// open file in editor
-	launch_editor(report_path, NULL, NULL);
-	fclose(report);
-
-	launch_editor(report_path.buf, NULL, NULL);
-	return 0;
-}
diff --git a/builtin/help.c b/builtin/help.c
index e5590d7787..1c5f2b9255 100644
--- a/builtin/help.c
+++ b/builtin/help.c
@@ -8,6 +8,7 @@
 #include "parse-options.h"
 #include "run-command.h"
 #include "column.h"
+#include "config-list.h"
 #include "help.h"
 #include "alias.h"
 
@@ -62,6 +63,91 @@ static const char * const builtin_help_usage[] = {
 	NULL
 };
 
+struct slot_expansion {
+	const char *prefix;
+	const char *placeholder;
+	void (*fn)(struct string_list *list, const char *prefix);
+	int found;
+};
+
+static void list_config_help(int for_human)
+{
+	struct slot_expansion slot_expansions[] = {
+		{ "advice", "*", list_config_advices },
+		{ "color.branch", "<slot>", list_config_color_branch_slots },
+		{ "color.decorate", "<slot>", list_config_color_decorate_slots },
+		{ "color.diff", "<slot>", list_config_color_diff_slots },
+		{ "color.grep", "<slot>", list_config_color_grep_slots },
+		{ "color.interactive", "<slot>", list_config_color_interactive_slots },
+		{ "color.remote", "<slot>", list_config_color_sideband_slots },
+		{ "color.status", "<slot>", list_config_color_status_slots },
+		{ "fsck", "<msg-id>", list_config_fsck_msg_ids },
+		{ "receive.fsck", "<msg-id>", list_config_fsck_msg_ids },
+		{ NULL, NULL, NULL }
+	};
+	const char **p;
+	struct slot_expansion *e;
+	struct string_list keys = STRING_LIST_INIT_DUP;
+	int i;
+
+	for (p = config_name_list; *p; p++) {
+		const char *var = *p;
+		struct strbuf sb = STRBUF_INIT;
+
+		for (e = slot_expansions; e->prefix; e++) {
+
+			strbuf_reset(&sb);
+			strbuf_addf(&sb, "%s.%s", e->prefix, e->placeholder);
+			if (!strcasecmp(var, sb.buf)) {
+				e->fn(&keys, e->prefix);
+				e->found++;
+				break;
+			}
+		}
+		strbuf_release(&sb);
+		if (!e->prefix)
+			string_list_append(&keys, var);
+	}
+
+	for (e = slot_expansions; e->prefix; e++)
+		if (!e->found)
+			BUG("slot_expansion %s.%s is not used",
+			    e->prefix, e->placeholder);
+
+	string_list_sort(&keys);
+	for (i = 0; i < keys.nr; i++) {
+		const char *var = keys.items[i].string;
+		const char *wildcard, *tag, *cut;
+
+		if (for_human) {
+			puts(var);
+			continue;
+		}
+
+		wildcard = strchr(var, '*');
+		tag = strchr(var, '<');
+
+		if (!wildcard && !tag) {
+			puts(var);
+			continue;
+		}
+
+		if (wildcard && !tag)
+			cut = wildcard;
+		else if (!wildcard && tag)
+			cut = tag;
+		else
+			cut = wildcard < tag ? wildcard : tag;
+
+		/*
+		 * We may produce duplicates, but that's up to
+		 * git-completion.bash to handle
+		 */
+		printf("%.*s\n", (int)(cut - var), var);
+	}
+	string_list_clear(&keys, 0);
+}
+
 static enum help_format parse_help_format(const char *format)
 {
 	if (!strcmp(format, "man"))
diff --git a/config.c b/config.c
index e7052b3977..baab4a916e 100644
--- a/config.c
+++ b/config.c
@@ -3312,6 +3312,23 @@ enum config_scope current_config_scope(void)
 		return current_parsing_scope;
 }
 
+const char *config_scope_to_string(enum config_scope scope)
+{
+	switch (scope) {
+	case CONFIG_SCOPE_SYSTEM:
+		return "system";
+	case CONFIG_SCOPE_GLOBAL:
+		return "global";
+	case CONFIG_SCOPE_REPO:
+		return "repo";
+	case CONFIG_SCOPE_CMDLINE:
+		return "cmdline";
+	case CONFIG_SCOPE_UNKNOWN:
+	default:
+		return "unknown";
+	}
+}
+
 int lookup_config(const char **mapping, int nr_mapping, const char *var)
 {
 	int i;
diff --git a/config.h b/config.h
index 91fd4c5e96..c8bf296dcc 100644
--- a/config.h
+++ b/config.h
@@ -303,6 +303,7 @@ enum config_scope {
 };
 
 enum config_scope current_config_scope(void);
+const char *config_scope_to_string(enum config_scope);
 const char *current_config_origin_type(void);
 const char *current_config_name(void);
 
diff --git a/generate-bugreport-config-safelist.sh b/generate-bugreport-config-safelist.sh
new file mode 100755
index 0000000000..06b8e0c3c4
--- /dev/null
+++ b/generate-bugreport-config-safelist.sh
@@ -0,0 +1,22 @@
+#!/bin/sh
+
+cat <<EOF
+/* Automatically generated by bugreport-generate-config-safelist.sh */
+
+
+static const char *bugreport_config_safelist[] = {
+EOF
+
+# cat all regular files in Documentation/config
+find Documentation/config -type f -exec cat {} \; |
+# print the command name which matches the bugreport-include macro
+sed -n 's/^\(.*\) \+bugreport:include.* ::$/\1/p' |
+sort |
+while read line
+do
+	echo "	\"$line\","
+done
+
+cat <<EOF
+};
+EOF
diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
index 71158f7d8b..45fecf8bdf 100755
--- a/generate-cmdlist.sh
+++ b/generate-cmdlist.sh
@@ -76,23 +76,6 @@ print_command_list () {
 	echo "};"
 }
 
-print_config_list () {
-	cat <<EOF
-static const char *config_name_list[] = {
-EOF
-	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
-	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
-	sort |
-	while read line
-	do
-		echo "	\"$line\","
-	done
-	cat <<EOF
-	NULL,
-};
-EOF
-}
-
 exclude_programs=
 while test "--exclude-program" = "$1"
 do
@@ -113,5 +96,3 @@ echo
 define_category_names "$1"
 echo
 print_command_list "$1"
-echo
-print_config_list
diff --git a/generate-configlist.sh b/generate-configlist.sh
new file mode 100755
index 0000000000..eca6a00c30
--- /dev/null
+++ b/generate-configlist.sh
@@ -0,0 +1,24 @@
+#!/bin/sh
+
+echo "/* Automatically generated by generate-configlist.sh */"
+echo
+
+print_config_list () {
+	cat <<EOF
+static const char *config_name_list[] = {
+EOF
+	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
+	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
+	sort |
+	while read line
+	do
+		echo "	\"$line\","
+	done
+	cat <<EOF
+	NULL,
+};
+EOF
+}
+
+echo
+print_config_list
diff --git a/git.c b/git.c
index 2d6a64f019..ce6ab0ece2 100644
--- a/git.c
+++ b/git.c
@@ -473,7 +473,6 @@ static struct cmd_struct commands[] = {
 	{ "bisect--helper", cmd_bisect__helper, RUN_SETUP },
 	{ "blame", cmd_blame, RUN_SETUP },
 	{ "branch", cmd_branch, RUN_SETUP | DELAY_PAGER_CONFIG },
-	{ "bugreport", cmd_bugreport, RUN_SETUP },
 	{ "bundle", cmd_bundle, RUN_SETUP_GENTLY | NO_PARSEOPT },
 	{ "cat-file", cmd_cat_file, RUN_SETUP },
 	{ "check-attr", cmd_check_attr, RUN_SETUP },
diff --git a/help.c b/help.c
index cf67624a94..6d1bd17971 100644
--- a/help.c
+++ b/help.c
@@ -407,91 +407,6 @@ void list_common_guides_help(void)
 	putchar('\n');
 }
 
-struct slot_expansion {
-	const char *prefix;
-	const char *placeholder;
-	void (*fn)(struct string_list *list, const char *prefix);
-	int found;
-};
-
-void list_config_help(int for_human)
-{
-	struct slot_expansion slot_expansions[] = {
-		{ "advice", "*", list_config_advices },
-		{ "color.branch", "<slot>", list_config_color_branch_slots },
-		{ "color.decorate", "<slot>", list_config_color_decorate_slots },
-		{ "color.diff", "<slot>", list_config_color_diff_slots },
-		{ "color.grep", "<slot>", list_config_color_grep_slots },
-		{ "color.interactive", "<slot>", list_config_color_interactive_slots },
-		{ "color.remote", "<slot>", list_config_color_sideband_slots },
-		{ "color.status", "<slot>", list_config_color_status_slots },
-		{ "fsck", "<msg-id>", list_config_fsck_msg_ids },
-		{ "receive.fsck", "<msg-id>", list_config_fsck_msg_ids },
-		{ NULL, NULL, NULL }
-	};
-	const char **p;
-	struct slot_expansion *e;
-	struct string_list keys = STRING_LIST_INIT_DUP;
-	int i;
-
-	for (p = config_name_list; *p; p++) {
-		const char *var = *p;
-		struct strbuf sb = STRBUF_INIT;
-
-		for (e = slot_expansions; e->prefix; e++) {
-
-			strbuf_reset(&sb);
-			strbuf_addf(&sb, "%s.%s", e->prefix, e->placeholder);
-			if (!strcasecmp(var, sb.buf)) {
-				e->fn(&keys, e->prefix);
-				e->found++;
-				break;
-			}
-		}
-		strbuf_release(&sb);
-		if (!e->prefix)
-			string_list_append(&keys, var);
-	}
-
-	for (e = slot_expansions; e->prefix; e++)
-		if (!e->found)
-			BUG("slot_expansion %s.%s is not used",
-			    e->prefix, e->placeholder);
-
-	string_list_sort(&keys);
-	for (i = 0; i < keys.nr; i++) {
-		const char *var = keys.items[i].string;
-		const char *wildcard, *tag, *cut;
-
-		if (for_human) {
-			puts(var);
-			continue;
-		}
-
-		wildcard = strchr(var, '*');
-		tag = strchr(var, '<');
-
-		if (!wildcard && !tag) {
-			puts(var);
-			continue;
-		}
-
-		if (wildcard && !tag)
-			cut = wildcard;
-		else if (!wildcard && tag)
-			cut = tag;
-		else
-			cut = wildcard < tag ? wildcard : tag;
-
-		/*
-		 * We may produce duplicates, but that's up to
-		 * git-completion.bash to handle
-		 */
-		printf("%.*s\n", (int)(cut - var), var);
-	}
-	string_list_clear(&keys, 0);
-}
-
 static int get_alias(const char *var, const char *value, void *data)
 {
 	struct string_list *list = data;
@@ -707,8 +622,34 @@ const char *help_unknown_cmd(const char *cmd)
 	exit(1);
 }
 
+void list_version_info(struct strbuf *buf, int build_options)
+{
+	strbuf_reset(buf);
+	/*
+	 * The format of this string should be kept stable for compatibility
+	 * with external projects that rely on the output of "git version".
+	 *
+	 * Always show the version, even if other options are given.
+	 */
+	strbuf_addf(buf, "git version %s\n", git_version_string);
+
+	if (build_options) {
+		strbuf_addf(buf, "cpu: %s\n", GIT_HOST_CPU);
+		if (git_built_from_commit_string[0])
+			strbuf_addf(buf, "built from commit: %s\n",
+			       git_built_from_commit_string);
+		else
+			strbuf_addf(buf, "no commit associated with this build\n");
+		strbuf_addf(buf, "sizeof-long: %d\n", (int)sizeof(long));
+		strbuf_addf(buf, "sizeof-size_t: %d\n", (int)sizeof(size_t));
+		strbuf_addf(buf, "shell-path: %s\n", SHELL_PATH);
+		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
+	}
+}
+
 int cmd_version(int argc, const char **argv, const char *prefix)
 {
+	struct strbuf buf = STRBUF_INIT;
 	int build_options = 0;
 	const char * const usage[] = {
 		N_("git version [<options>]"),
@@ -722,25 +663,9 @@ int cmd_version(int argc, const char **argv, const char *prefix)
 
 	argc = parse_options(argc, argv, prefix, options, usage, 0);
 
-	/*
-	 * The format of this string should be kept stable for compatibility
-	 * with external projects that rely on the output of "git version".
-	 *
-	 * Always show the version, even if other options are given.
-	 */
-	printf("git version %s\n", git_version_string);
+	list_version_info(&buf, build_options);
+	printf("%s", buf.buf);
 
-	if (build_options) {
-		printf("cpu: %s\n", GIT_HOST_CPU);
-		if (git_built_from_commit_string[0])
-			printf("built from commit: %s\n",
-			       git_built_from_commit_string);
-		else
-			printf("no commit associated with this build\n");
-		printf("sizeof-long: %d\n", (int)sizeof(long));
-		printf("sizeof-size_t: %d\n", (int)sizeof(size_t));
-		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
-	}
 	return 0;
 }
 
diff --git a/help.h b/help.h
index 7a455beeb7..54f6b5f793 100644
--- a/help.h
+++ b/help.h
@@ -22,7 +22,6 @@ static inline void mput_char(char c, unsigned int num)
 void list_common_cmds_help(void);
 void list_all_cmds_help(void);
 void list_common_guides_help(void);
-void list_config_help(int for_human);
 
 void list_all_main_cmds(struct string_list *list);
 void list_all_other_cmds(struct string_list *list);
@@ -38,6 +37,7 @@ void add_cmdname(struct cmdnames *cmds, const char *name, int len);
 void exclude_cmds(struct cmdnames *cmds, struct cmdnames *excludes);
 int is_in_cmdlist(struct cmdnames *cmds, const char *name);
 void list_commands(unsigned int colopts, struct cmdnames *main_cmds, struct cmdnames *other_cmds);
+void list_version_info(struct strbuf *buf, int build_options);
 
 /*
  * call this to die(), when it is suspected that the user mistyped a
diff --git a/http-fetch.c b/http-fetch.c
index a32ac118d9..31844812a1 100644
--- a/http-fetch.c
+++ b/http-fetch.c
@@ -3,9 +3,18 @@
 #include "exec-cmd.h"
 #include "http.h"
 #include "walker.h"
+#include "version.h"
 
 static const char http_fetch_usage[] = "git http-fetch "
-"[-c] [-t] [-a] [-v] [--recover] [-w ref] [--stdin] commit-id url";
+"[-c] [-t] [-a] [-v] [-V] [--recover] [-w ref] [--stdin] commit-id url";
+
+void NORETURN version_info()
+{
+	printf("git-http-fetch version: %s\n", git_version_string);
+	printf("built from commit: %s\n", git_built_from_commit_string);
+	printf("curl version: %s\n", curl_version());
+	exit(0);
+}
 
 int cmd_main(int argc, const char **argv)
 {
@@ -26,6 +35,8 @@ int cmd_main(int argc, const char **argv)
 		} else if (argv[arg][1] == 'a') {
 		} else if (argv[arg][1] == 'v') {
 			get_verbosely = 1;
+		} else if (argv[arg][1] == 'V') {
+			version_info();
 		} else if (argv[arg][1] == 'w') {
 			write_ref = &argv[arg + 1];
 			arg++;
diff --git a/t/t0091-bugreport.sh b/t/t0091-bugreport.sh
new file mode 100755
index 0000000000..6eb2ee4f66
--- /dev/null
+++ b/t/t0091-bugreport.sh
@@ -0,0 +1,41 @@
+#!/bin/bash
+
+test_description='git bugreport'
+
+. ./test-lib.sh
+
+# Headers "[System Info]" will be followed by a non-empty line if we put some
+# information there; we can make sure all our headers were followed by some
+# information to check if the command was successful.
+HEADER_PATTERN="^\[.*\]$"
+check_all_headers_populated() {
+	while read -r line; do
+		if [$(grep $HEADER_PATTERN $line)]; then
+			read -r nextline
+			if [-z $nextline]; then
+				return 1;
+			fi
+		fi
+	done
+}
+
+test_expect_success 'creates a report with content in the right places' '
+	git bugreport &&
+	check_all_headers_populated <git-bugreport-* &&
+	rm git-bugreport-*
+'
+
+test_expect_success '--output puts the report in the provided dir' '
+	mkdir foo/ &&
+	git bugreport -o foo/ &&
+	test -f foo/git-bugreport-* &&
+	rm -fr foo/
+'
+
+test_expect_success 'incorrect arguments abort with usage' '
+	test_must_fail git bugreport --false 2>output &&
+	grep usage output &&
+	test ! -f git-bugreport-*
+'
+
+test_done
-- 
2.24.1.735.g03f4e72817-goog
Emily Shaffer· Dec 13, 2019, 00:42 UTC · re: Emily Shaffer · lore

[PATCH v4 02/15] help: move list_config_help to builtin/help

Starting in 3ac68a93fd2, help.o began to depend on builtin/branch.o, builtin/clean.o, and builtin/config.o. This meant that help.o was unusable outside of the context of the main Git executable.

To make help.o usable by other commands again, move list_config_help() into builtin/help.c (where it makes sense to assume other builtin libraries are present).

When command-list.h is included but a member is not used, we start to hear a compiler warning. Since the config list is generated in a fairly different way than the command list, and since commands and config options are semantically different, move the config list into its own header and move the generator into its own script and build rule.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 .gitignore             |  1 +
 Makefile               | 16 ++++++--
 builtin/help.c         | 86 ++++++++++++++++++++++++++++++++++++++++++
 generate-cmdlist.sh    | 19 ----------
 generate-configlist.sh | 24 ++++++++++++
 help.c                 | 85 -----------------------------------------
 help.h                 |  1 -
 7 files changed, 123 insertions(+), 109 deletions(-)
 create mode 100755 generate-configlist.sh
Show changes to 7 files +123 −109

.gitignore, Makefile, builtin/help.c, generate-cmdlist.sh, generate-configlist.sh, help.c, help.h

diff --git a/.gitignore b/.gitignore
index 055a84c4a8..5dde2cc4c8 100644
--- a/.gitignore
+++ b/.gitignore
@@ -189,6 +189,7 @@
 /gitweb/gitweb.cgi
 /gitweb/static/gitweb.js
 /gitweb/static/gitweb.min.*
+/config-list.h
 /command-list.h
 *.tar.gz
 *.dsc
diff --git a/Makefile b/Makefile
index 9dff91436e..c49f55a521 100644
--- a/Makefile
+++ b/Makefile
@@ -815,6 +815,7 @@ LIB_FILE = libgit.a
 XDIFF_LIB = xdiff/lib.a
 VCSSVN_LIB = vcs-svn/lib.a
 
+GENERATED_H += config-list.h
 GENERATED_H += command-list.h
 
 LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
@@ -2127,7 +2128,7 @@ git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS)
 
 help.sp help.s help.o: command-list.h
 
-builtin/help.sp builtin/help.s builtin/help.o: command-list.h GIT-PREFIX
+builtin/help.sp builtin/help.s builtin/help.o: config-list.h GIT-PREFIX
 builtin/help.sp builtin/help.s builtin/help.o: EXTRA_CPPFLAGS = \
 	'-DGIT_HTML_PATH="$(htmldir_relative_SQ)"' \
 	'-DGIT_MAN_PATH="$(mandir_relative_SQ)"' \
@@ -2147,6 +2148,12 @@ $(BUILT_INS): git$X
 	ln -s $< $@ 2>/dev/null || \
 	cp $< $@
 
+config-list.h: generate-configlist.sh
+
+config-list.h:
+	$(QUIET_GEN)$(SHELL_PATH) ./generate-configlist.sh \
+		>$@+ && mv $@+ $@
+
 command-list.h: generate-cmdlist.sh command-list.txt
 
 command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Documentation/config/*.txt
@@ -2784,7 +2791,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE
 .PHONY: sparse $(SP_OBJ)
 sparse: $(SP_OBJ)
 
-GEN_HDRS := command-list.h unicode-width.h
+GEN_HDRS := config-list.h command-list.h unicode-width.h
 EXCEPT_HDRS := $(GEN_HDRS) compat/% xdiff/%
 ifndef GCRYPT_SHA256
 	EXCEPT_HDRS += sha256/gcrypt.h
@@ -2807,7 +2814,7 @@ hdr-check: $(HCO)
 style:
 	git clang-format --style file --diff --extensions c,h
 
-check: command-list.h
+check: config-list.h command-list.h
 	@if sparse; \
 	then \
 		echo >&2 "Use 'make sparse' instead"; \
@@ -3110,7 +3117,8 @@ clean: profile-clean coverage-clean cocciclean
 	$(RM) $(HCC)
 	$(RM) -r bin-wrappers $(dep_dirs)
 	$(RM) -r po/build/
-	$(RM) *.pyc *.pyo */*.pyc */*.pyo command-list.h $(ETAGS_TARGET) tags cscope*
+	$(RM) *.pyc *.pyo */*.pyc */*.pyo config-list.h command-list.h
+	$(RM) $(ETAGS_TARGET) tags cscope*
 	$(RM) -r $(GIT_TARNAME) .doc-tmp-dir
 	$(RM) $(GIT_TARNAME).tar.gz git-core_$(GIT_VERSION)-*.tar.gz
 	$(RM) $(htmldocs).tar.gz $(manpages).tar.gz
diff --git a/builtin/help.c b/builtin/help.c
index e5590d7787..1c5f2b9255 100644
--- a/builtin/help.c
+++ b/builtin/help.c
@@ -8,6 +8,7 @@
 #include "parse-options.h"
 #include "run-command.h"
 #include "column.h"
+#include "config-list.h"
 #include "help.h"
 #include "alias.h"
 
@@ -62,6 +63,91 @@ static const char * const builtin_help_usage[] = {
 	NULL
 };
 
+struct slot_expansion {
+	const char *prefix;
+	const char *placeholder;
+	void (*fn)(struct string_list *list, const char *prefix);
+	int found;
+};
+
+static void list_config_help(int for_human)
+{
+	struct slot_expansion slot_expansions[] = {
+		{ "advice", "*", list_config_advices },
+		{ "color.branch", "<slot>", list_config_color_branch_slots },
+		{ "color.decorate", "<slot>", list_config_color_decorate_slots },
+		{ "color.diff", "<slot>", list_config_color_diff_slots },
+		{ "color.grep", "<slot>", list_config_color_grep_slots },
+		{ "color.interactive", "<slot>", list_config_color_interactive_slots },
+		{ "color.remote", "<slot>", list_config_color_sideband_slots },
+		{ "color.status", "<slot>", list_config_color_status_slots },
+		{ "fsck", "<msg-id>", list_config_fsck_msg_ids },
+		{ "receive.fsck", "<msg-id>", list_config_fsck_msg_ids },
+		{ NULL, NULL, NULL }
+	};
+	const char **p;
+	struct slot_expansion *e;
+	struct string_list keys = STRING_LIST_INIT_DUP;
+	int i;
+
+	for (p = config_name_list; *p; p++) {
+		const char *var = *p;
+		struct strbuf sb = STRBUF_INIT;
+
+		for (e = slot_expansions; e->prefix; e++) {
+
+			strbuf_reset(&sb);
+			strbuf_addf(&sb, "%s.%s", e->prefix, e->placeholder);
+			if (!strcasecmp(var, sb.buf)) {
+				e->fn(&keys, e->prefix);
+				e->found++;
+				break;
+			}
+		}
+		strbuf_release(&sb);
+		if (!e->prefix)
+			string_list_append(&keys, var);
+	}
+
+	for (e = slot_expansions; e->prefix; e++)
+		if (!e->found)
+			BUG("slot_expansion %s.%s is not used",
+			    e->prefix, e->placeholder);
+
+	string_list_sort(&keys);
+	for (i = 0; i < keys.nr; i++) {
+		const char *var = keys.items[i].string;
+		const char *wildcard, *tag, *cut;
+
+		if (for_human) {
+			puts(var);
+			continue;
+		}
+
+		wildcard = strchr(var, '*');
+		tag = strchr(var, '<');
+
+		if (!wildcard && !tag) {
+			puts(var);
+			continue;
+		}
+
+		if (wildcard && !tag)
+			cut = wildcard;
+		else if (!wildcard && tag)
+			cut = tag;
+		else
+			cut = wildcard < tag ? wildcard : tag;
+
+		/*
+		 * We may produce duplicates, but that's up to
+		 * git-completion.bash to handle
+		 */
+		printf("%.*s\n", (int)(cut - var), var);
+	}
+	string_list_clear(&keys, 0);
+}
+
 static enum help_format parse_help_format(const char *format)
 {
 	if (!strcmp(format, "man"))
diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
index 71158f7d8b..45fecf8bdf 100755
--- a/generate-cmdlist.sh
+++ b/generate-cmdlist.sh
@@ -76,23 +76,6 @@ print_command_list () {
 	echo "};"
 }
 
-print_config_list () {
-	cat <<EOF
-static const char *config_name_list[] = {
-EOF
-	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
-	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
-	sort |
-	while read line
-	do
-		echo "	\"$line\","
-	done
-	cat <<EOF
-	NULL,
-};
-EOF
-}
-
 exclude_programs=
 while test "--exclude-program" = "$1"
 do
@@ -113,5 +96,3 @@ echo
 define_category_names "$1"
 echo
 print_command_list "$1"
-echo
-print_config_list
diff --git a/generate-configlist.sh b/generate-configlist.sh
new file mode 100755
index 0000000000..eca6a00c30
--- /dev/null
+++ b/generate-configlist.sh
@@ -0,0 +1,24 @@
+#!/bin/sh
+
+echo "/* Automatically generated by generate-configlist.sh */"
+echo
+
+print_config_list () {
+	cat <<EOF
+static const char *config_name_list[] = {
+EOF
+	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
+	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
+	sort |
+	while read line
+	do
+		echo "	\"$line\","
+	done
+	cat <<EOF
+	NULL,
+};
+EOF
+}
+
+echo
+print_config_list
diff --git a/help.c b/help.c
index cf67624a94..a21487db77 100644
--- a/help.c
+++ b/help.c
@@ -407,91 +407,6 @@ void list_common_guides_help(void)
 	putchar('\n');
 }
 
-struct slot_expansion {
-	const char *prefix;
-	const char *placeholder;
-	void (*fn)(struct string_list *list, const char *prefix);
-	int found;
-};
-
-void list_config_help(int for_human)
-{
-	struct slot_expansion slot_expansions[] = {
-		{ "advice", "*", list_config_advices },
-		{ "color.branch", "<slot>", list_config_color_branch_slots },
-		{ "color.decorate", "<slot>", list_config_color_decorate_slots },
-		{ "color.diff", "<slot>", list_config_color_diff_slots },
-		{ "color.grep", "<slot>", list_config_color_grep_slots },
-		{ "color.interactive", "<slot>", list_config_color_interactive_slots },
-		{ "color.remote", "<slot>", list_config_color_sideband_slots },
-		{ "color.status", "<slot>", list_config_color_status_slots },
-		{ "fsck", "<msg-id>", list_config_fsck_msg_ids },
-		{ "receive.fsck", "<msg-id>", list_config_fsck_msg_ids },
-		{ NULL, NULL, NULL }
-	};
-	const char **p;
-	struct slot_expansion *e;
-	struct string_list keys = STRING_LIST_INIT_DUP;
-	int i;
-
-	for (p = config_name_list; *p; p++) {
-		const char *var = *p;
-		struct strbuf sb = STRBUF_INIT;
-
-		for (e = slot_expansions; e->prefix; e++) {
-
-			strbuf_reset(&sb);
-			strbuf_addf(&sb, "%s.%s", e->prefix, e->placeholder);
-			if (!strcasecmp(var, sb.buf)) {
-				e->fn(&keys, e->prefix);
-				e->found++;
-				break;
-			}
-		}
-		strbuf_release(&sb);
-		if (!e->prefix)
-			string_list_append(&keys, var);
-	}
-
-	for (e = slot_expansions; e->prefix; e++)
-		if (!e->found)
-			BUG("slot_expansion %s.%s is not used",
-			    e->prefix, e->placeholder);
-
-	string_list_sort(&keys);
-	for (i = 0; i < keys.nr; i++) {
-		const char *var = keys.items[i].string;
-		const char *wildcard, *tag, *cut;
-
-		if (for_human) {
-			puts(var);
-			continue;
-		}
-
-		wildcard = strchr(var, '*');
-		tag = strchr(var, '<');
-
-		if (!wildcard && !tag) {
-			puts(var);
-			continue;
-		}
-
-		if (wildcard && !tag)
-			cut = wildcard;
-		else if (!wildcard && tag)
-			cut = tag;
-		else
-			cut = wildcard < tag ? wildcard : tag;
-
-		/*
-		 * We may produce duplicates, but that's up to
-		 * git-completion.bash to handle
-		 */
-		printf("%.*s\n", (int)(cut - var), var);
-	}
-	string_list_clear(&keys, 0);
-}
-
 static int get_alias(const char *var, const char *value, void *data)
 {
 	struct string_list *list = data;
diff --git a/help.h b/help.h
index 7a455beeb7..9071894e8c 100644
--- a/help.h
+++ b/help.h
@@ -22,7 +22,6 @@ static inline void mput_char(char c, unsigned int num)
 void list_common_cmds_help(void);
 void list_all_cmds_help(void);
 void list_common_guides_help(void);
-void list_config_help(int for_human);
 
 void list_all_main_cmds(struct string_list *list);
 void list_all_other_cmds(struct string_list *list);
-- 
2.24.1.735.g03f4e72817-goog
Junio C Hamano· Dec 13, 2019, 20:51 UTC · re: Emily Shaffer · lore

Re: [PATCH v4 02/15] help: move list_config_help to builtin/help

Emily Shaffer <emilyshaffer@google.com> writes:
Show 16 quoted lines
> Starting in 3ac68a93fd2, help.o began to depend on builtin/branch.o,
> builtin/clean.o, and builtin/config.o. This meant that help.o was
> unusable outside of the context of the main Git executable.
>
> To make help.o usable by other commands again, move list_config_help()
> into builtin/help.c (where it makes sense to assume other builtin libraries
> are present).
>
> When command-list.h is included but a member is not used, we start to
> hear a compiler warning. Since the config list is generated in a fairly
> different way than the command list, and since commands and config
> options are semantically different, move the config list into its own
> header and move the generator into its own script and build rule.
>
> Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
> ---

OK. Looks like a clean-up that is very worthy even without the remainder of the topic.

Thanks.
Show 360 quoted lines
>  .gitignore             |  1 +
>  Makefile               | 16 ++++++--
>  builtin/help.c         | 86 ++++++++++++++++++++++++++++++++++++++++++
>  generate-cmdlist.sh    | 19 ----------
>  generate-configlist.sh | 24 ++++++++++++
>  help.c                 | 85 -----------------------------------------
>  help.h                 |  1 -
>  7 files changed, 123 insertions(+), 109 deletions(-)
>  create mode 100755 generate-configlist.sh
>
> diff --git a/.gitignore b/.gitignore
> index 055a84c4a8..5dde2cc4c8 100644
> --- a/.gitignore
> +++ b/.gitignore
> @@ -189,6 +189,7 @@
>  /gitweb/gitweb.cgi
>  /gitweb/static/gitweb.js
>  /gitweb/static/gitweb.min.*
> +/config-list.h
>  /command-list.h
>  *.tar.gz
>  *.dsc
> diff --git a/Makefile b/Makefile
> index 9dff91436e..c49f55a521 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -815,6 +815,7 @@ LIB_FILE = libgit.a
>  XDIFF_LIB = xdiff/lib.a
>  VCSSVN_LIB = vcs-svn/lib.a
>  
> +GENERATED_H += config-list.h
>  GENERATED_H += command-list.h
>  
>  LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
> @@ -2127,7 +2128,7 @@ git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS)
>  
>  help.sp help.s help.o: command-list.h
>  
> -builtin/help.sp builtin/help.s builtin/help.o: command-list.h GIT-PREFIX
> +builtin/help.sp builtin/help.s builtin/help.o: config-list.h GIT-PREFIX
>  builtin/help.sp builtin/help.s builtin/help.o: EXTRA_CPPFLAGS = \
>  	'-DGIT_HTML_PATH="$(htmldir_relative_SQ)"' \
>  	'-DGIT_MAN_PATH="$(mandir_relative_SQ)"' \
> @@ -2147,6 +2148,12 @@ $(BUILT_INS): git$X
>  	ln -s $< $@ 2>/dev/null || \
>  	cp $< $@
>  
> +config-list.h: generate-configlist.sh
> +
> +config-list.h:
> +	$(QUIET_GEN)$(SHELL_PATH) ./generate-configlist.sh \
> +		>$@+ && mv $@+ $@
> +
>  command-list.h: generate-cmdlist.sh command-list.txt
>  
>  command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Documentation/config/*.txt
> @@ -2784,7 +2791,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE
>  .PHONY: sparse $(SP_OBJ)
>  sparse: $(SP_OBJ)
>  
> -GEN_HDRS := command-list.h unicode-width.h
> +GEN_HDRS := config-list.h command-list.h unicode-width.h
>  EXCEPT_HDRS := $(GEN_HDRS) compat/% xdiff/%
>  ifndef GCRYPT_SHA256
>  	EXCEPT_HDRS += sha256/gcrypt.h
> @@ -2807,7 +2814,7 @@ hdr-check: $(HCO)
>  style:
>  	git clang-format --style file --diff --extensions c,h
>  
> -check: command-list.h
> +check: config-list.h command-list.h
>  	@if sparse; \
>  	then \
>  		echo >&2 "Use 'make sparse' instead"; \
> @@ -3110,7 +3117,8 @@ clean: profile-clean coverage-clean cocciclean
>  	$(RM) $(HCC)
>  	$(RM) -r bin-wrappers $(dep_dirs)
>  	$(RM) -r po/build/
> -	$(RM) *.pyc *.pyo */*.pyc */*.pyo command-list.h $(ETAGS_TARGET) tags cscope*
> +	$(RM) *.pyc *.pyo */*.pyc */*.pyo config-list.h command-list.h
> +	$(RM) $(ETAGS_TARGET) tags cscope*
>  	$(RM) -r $(GIT_TARNAME) .doc-tmp-dir
>  	$(RM) $(GIT_TARNAME).tar.gz git-core_$(GIT_VERSION)-*.tar.gz
>  	$(RM) $(htmldocs).tar.gz $(manpages).tar.gz
> diff --git a/builtin/help.c b/builtin/help.c
> index e5590d7787..1c5f2b9255 100644
> --- a/builtin/help.c
> +++ b/builtin/help.c
> @@ -8,6 +8,7 @@
>  #include "parse-options.h"
>  #include "run-command.h"
>  #include "column.h"
> +#include "config-list.h"
>  #include "help.h"
>  #include "alias.h"
>  
> @@ -62,6 +63,91 @@ static const char * const builtin_help_usage[] = {
>  	NULL
>  };
>  
> +struct slot_expansion {
> +	const char *prefix;
> +	const char *placeholder;
> +	void (*fn)(struct string_list *list, const char *prefix);
> +	int found;
> +};
> +
> +static void list_config_help(int for_human)
> +{
> +	struct slot_expansion slot_expansions[] = {
> +		{ "advice", "*", list_config_advices },
> +		{ "color.branch", "<slot>", list_config_color_branch_slots },
> +		{ "color.decorate", "<slot>", list_config_color_decorate_slots },
> +		{ "color.diff", "<slot>", list_config_color_diff_slots },
> +		{ "color.grep", "<slot>", list_config_color_grep_slots },
> +		{ "color.interactive", "<slot>", list_config_color_interactive_slots },
> +		{ "color.remote", "<slot>", list_config_color_sideband_slots },
> +		{ "color.status", "<slot>", list_config_color_status_slots },
> +		{ "fsck", "<msg-id>", list_config_fsck_msg_ids },
> +		{ "receive.fsck", "<msg-id>", list_config_fsck_msg_ids },
> +		{ NULL, NULL, NULL }
> +	};
> +	const char **p;
> +	struct slot_expansion *e;
> +	struct string_list keys = STRING_LIST_INIT_DUP;
> +	int i;
> +
> +	for (p = config_name_list; *p; p++) {
> +		const char *var = *p;
> +		struct strbuf sb = STRBUF_INIT;
> +
> +		for (e = slot_expansions; e->prefix; e++) {
> +
> +			strbuf_reset(&sb);
> +			strbuf_addf(&sb, "%s.%s", e->prefix, e->placeholder);
> +			if (!strcasecmp(var, sb.buf)) {
> +				e->fn(&keys, e->prefix);
> +				e->found++;
> +				break;
> +			}
> +		}
> +		strbuf_release(&sb);
> +		if (!e->prefix)
> +			string_list_append(&keys, var);
> +	}
> +
> +	for (e = slot_expansions; e->prefix; e++)
> +		if (!e->found)
> +			BUG("slot_expansion %s.%s is not used",
> +			    e->prefix, e->placeholder);
> +
> +	string_list_sort(&keys);
> +	for (i = 0; i < keys.nr; i++) {
> +		const char *var = keys.items[i].string;
> +		const char *wildcard, *tag, *cut;
> +
> +		if (for_human) {
> +			puts(var);
> +			continue;
> +		}
> +
> +		wildcard = strchr(var, '*');
> +		tag = strchr(var, '<');
> +
> +		if (!wildcard && !tag) {
> +			puts(var);
> +			continue;
> +		}
> +
> +		if (wildcard && !tag)
> +			cut = wildcard;
> +		else if (!wildcard && tag)
> +			cut = tag;
> +		else
> +			cut = wildcard < tag ? wildcard : tag;
> +
> +		/*
> +		 * We may produce duplicates, but that's up to
> +		 * git-completion.bash to handle
> +		 */
> +		printf("%.*s\n", (int)(cut - var), var);
> +	}
> +	string_list_clear(&keys, 0);
> +}
> +
>  static enum help_format parse_help_format(const char *format)
>  {
>  	if (!strcmp(format, "man"))
> diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
> index 71158f7d8b..45fecf8bdf 100755
> --- a/generate-cmdlist.sh
> +++ b/generate-cmdlist.sh
> @@ -76,23 +76,6 @@ print_command_list () {
>  	echo "};"
>  }
>  
> -print_config_list () {
> -	cat <<EOF
> -static const char *config_name_list[] = {
> -EOF
> -	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
> -	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
> -	sort |
> -	while read line
> -	do
> -		echo "	\"$line\","
> -	done
> -	cat <<EOF
> -	NULL,
> -};
> -EOF
> -}
> -
>  exclude_programs=
>  while test "--exclude-program" = "$1"
>  do
> @@ -113,5 +96,3 @@ echo
>  define_category_names "$1"
>  echo
>  print_command_list "$1"
> -echo
> -print_config_list
> diff --git a/generate-configlist.sh b/generate-configlist.sh
> new file mode 100755
> index 0000000000..eca6a00c30
> --- /dev/null
> +++ b/generate-configlist.sh
> @@ -0,0 +1,24 @@
> +#!/bin/sh
> +
> +echo "/* Automatically generated by generate-configlist.sh */"
> +echo
> +
> +print_config_list () {
> +	cat <<EOF
> +static const char *config_name_list[] = {
> +EOF
> +	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
> +	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
> +	sort |
> +	while read line
> +	do
> +		echo "	\"$line\","
> +	done
> +	cat <<EOF
> +	NULL,
> +};
> +EOF
> +}
> +
> +echo
> +print_config_list
> diff --git a/help.c b/help.c
> index cf67624a94..a21487db77 100644
> --- a/help.c
> +++ b/help.c
> @@ -407,91 +407,6 @@ void list_common_guides_help(void)
>  	putchar('\n');
>  }
>  
> -struct slot_expansion {
> -	const char *prefix;
> -	const char *placeholder;
> -	void (*fn)(struct string_list *list, const char *prefix);
> -	int found;
> -};
> -
> -void list_config_help(int for_human)
> -{
> -	struct slot_expansion slot_expansions[] = {
> -		{ "advice", "*", list_config_advices },
> -		{ "color.branch", "<slot>", list_config_color_branch_slots },
> -		{ "color.decorate", "<slot>", list_config_color_decorate_slots },
> -		{ "color.diff", "<slot>", list_config_color_diff_slots },
> -		{ "color.grep", "<slot>", list_config_color_grep_slots },
> -		{ "color.interactive", "<slot>", list_config_color_interactive_slots },
> -		{ "color.remote", "<slot>", list_config_color_sideband_slots },
> -		{ "color.status", "<slot>", list_config_color_status_slots },
> -		{ "fsck", "<msg-id>", list_config_fsck_msg_ids },
> -		{ "receive.fsck", "<msg-id>", list_config_fsck_msg_ids },
> -		{ NULL, NULL, NULL }
> -	};
> -	const char **p;
> -	struct slot_expansion *e;
> -	struct string_list keys = STRING_LIST_INIT_DUP;
> -	int i;
> -
> -	for (p = config_name_list; *p; p++) {
> -		const char *var = *p;
> -		struct strbuf sb = STRBUF_INIT;
> -
> -		for (e = slot_expansions; e->prefix; e++) {
> -
> -			strbuf_reset(&sb);
> -			strbuf_addf(&sb, "%s.%s", e->prefix, e->placeholder);
> -			if (!strcasecmp(var, sb.buf)) {
> -				e->fn(&keys, e->prefix);
> -				e->found++;
> -				break;
> -			}
> -		}
> -		strbuf_release(&sb);
> -		if (!e->prefix)
> -			string_list_append(&keys, var);
> -	}
> -
> -	for (e = slot_expansions; e->prefix; e++)
> -		if (!e->found)
> -			BUG("slot_expansion %s.%s is not used",
> -			    e->prefix, e->placeholder);
> -
> -	string_list_sort(&keys);
> -	for (i = 0; i < keys.nr; i++) {
> -		const char *var = keys.items[i].string;
> -		const char *wildcard, *tag, *cut;
> -
> -		if (for_human) {
> -			puts(var);
> -			continue;
> -		}
> -
> -		wildcard = strchr(var, '*');
> -		tag = strchr(var, '<');
> -
> -		if (!wildcard && !tag) {
> -			puts(var);
> -			continue;
> -		}
> -
> -		if (wildcard && !tag)
> -			cut = wildcard;
> -		else if (!wildcard && tag)
> -			cut = tag;
> -		else
> -			cut = wildcard < tag ? wildcard : tag;
> -
> -		/*
> -		 * We may produce duplicates, but that's up to
> -		 * git-completion.bash to handle
> -		 */
> -		printf("%.*s\n", (int)(cut - var), var);
> -	}
> -	string_list_clear(&keys, 0);
> -}
> -
>  static int get_alias(const char *var, const char *value, void *data)
>  {
>  	struct string_list *list = data;
> diff --git a/help.h b/help.h
> index 7a455beeb7..9071894e8c 100644
> --- a/help.h
> +++ b/help.h
> @@ -22,7 +22,6 @@ static inline void mput_char(char c, unsigned int num)
>  void list_common_cmds_help(void);
>  void list_all_cmds_help(void);
>  void list_common_guides_help(void);
> -void list_config_help(int for_human);
>  
>  void list_all_main_cmds(struct string_list *list);
>  void list_all_other_cmds(struct string_list *list);
Emily Shaffer· Dec 16, 2019, 21:36 UTC · re: Junio C Hamano · lore

Re: [PATCH v4 02/15] help: move list_config_help to builtin/help

On Fri, Dec 13, 2019 at 12:51:58PM -0800, Junio C Hamano wrote:
Show 23 quoted lines
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> > Starting in 3ac68a93fd2, help.o began to depend on builtin/branch.o,
> > builtin/clean.o, and builtin/config.o. This meant that help.o was
> > unusable outside of the context of the main Git executable.
> >
> > To make help.o usable by other commands again, move list_config_help()
> > into builtin/help.c (where it makes sense to assume other builtin libraries
> > are present).
> >
> > When command-list.h is included but a member is not used, we start to
> > hear a compiler warning. Since the config list is generated in a fairly
> > different way than the command list, and since commands and config
> > options are semantically different, move the config list into its own
> > header and move the generator into its own script and build rule.
> >
> > Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
> > ---
> 
> OK.  Looks like a clean-up that is very worthy even without the
> remainder of the topic.
> 
> Thanks.

Ok. Are you going to take it on its own? If so, I'll be relying on two branches in pu/ - what's the best way to let you know I need them both if/when I send another version of this chain?

Show 361 quoted lines
> 
> >  .gitignore             |  1 +
> >  Makefile               | 16 ++++++--
> >  builtin/help.c         | 86 ++++++++++++++++++++++++++++++++++++++++++
> >  generate-cmdlist.sh    | 19 ----------
> >  generate-configlist.sh | 24 ++++++++++++
> >  help.c                 | 85 -----------------------------------------
> >  help.h                 |  1 -
> >  7 files changed, 123 insertions(+), 109 deletions(-)
> >  create mode 100755 generate-configlist.sh
> >
> > diff --git a/.gitignore b/.gitignore
> > index 055a84c4a8..5dde2cc4c8 100644
> > --- a/.gitignore
> > +++ b/.gitignore
> > @@ -189,6 +189,7 @@
> >  /gitweb/gitweb.cgi
> >  /gitweb/static/gitweb.js
> >  /gitweb/static/gitweb.min.*
> > +/config-list.h
> >  /command-list.h
> >  *.tar.gz
> >  *.dsc
> > diff --git a/Makefile b/Makefile
> > index 9dff91436e..c49f55a521 100644
> > --- a/Makefile
> > +++ b/Makefile
> > @@ -815,6 +815,7 @@ LIB_FILE = libgit.a
> >  XDIFF_LIB = xdiff/lib.a
> >  VCSSVN_LIB = vcs-svn/lib.a
> >  
> > +GENERATED_H += config-list.h
> >  GENERATED_H += command-list.h
> >  
> >  LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
> > @@ -2127,7 +2128,7 @@ git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS)
> >  
> >  help.sp help.s help.o: command-list.h
> >  
> > -builtin/help.sp builtin/help.s builtin/help.o: command-list.h GIT-PREFIX
> > +builtin/help.sp builtin/help.s builtin/help.o: config-list.h GIT-PREFIX
> >  builtin/help.sp builtin/help.s builtin/help.o: EXTRA_CPPFLAGS = \
> >  	'-DGIT_HTML_PATH="$(htmldir_relative_SQ)"' \
> >  	'-DGIT_MAN_PATH="$(mandir_relative_SQ)"' \
> > @@ -2147,6 +2148,12 @@ $(BUILT_INS): git$X
> >  	ln -s $< $@ 2>/dev/null || \
> >  	cp $< $@
> >  
> > +config-list.h: generate-configlist.sh
> > +
> > +config-list.h:
> > +	$(QUIET_GEN)$(SHELL_PATH) ./generate-configlist.sh \
> > +		>$@+ && mv $@+ $@
> > +
> >  command-list.h: generate-cmdlist.sh command-list.txt
> >  
> >  command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Documentation/config/*.txt
> > @@ -2784,7 +2791,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE
> >  .PHONY: sparse $(SP_OBJ)
> >  sparse: $(SP_OBJ)
> >  
> > -GEN_HDRS := command-list.h unicode-width.h
> > +GEN_HDRS := config-list.h command-list.h unicode-width.h
> >  EXCEPT_HDRS := $(GEN_HDRS) compat/% xdiff/%
> >  ifndef GCRYPT_SHA256
> >  	EXCEPT_HDRS += sha256/gcrypt.h
> > @@ -2807,7 +2814,7 @@ hdr-check: $(HCO)
> >  style:
> >  	git clang-format --style file --diff --extensions c,h
> >  
> > -check: command-list.h
> > +check: config-list.h command-list.h
> >  	@if sparse; \
> >  	then \
> >  		echo >&2 "Use 'make sparse' instead"; \
> > @@ -3110,7 +3117,8 @@ clean: profile-clean coverage-clean cocciclean
> >  	$(RM) $(HCC)
> >  	$(RM) -r bin-wrappers $(dep_dirs)
> >  	$(RM) -r po/build/
> > -	$(RM) *.pyc *.pyo */*.pyc */*.pyo command-list.h $(ETAGS_TARGET) tags cscope*
> > +	$(RM) *.pyc *.pyo */*.pyc */*.pyo config-list.h command-list.h
> > +	$(RM) $(ETAGS_TARGET) tags cscope*
> >  	$(RM) -r $(GIT_TARNAME) .doc-tmp-dir
> >  	$(RM) $(GIT_TARNAME).tar.gz git-core_$(GIT_VERSION)-*.tar.gz
> >  	$(RM) $(htmldocs).tar.gz $(manpages).tar.gz
> > diff --git a/builtin/help.c b/builtin/help.c
> > index e5590d7787..1c5f2b9255 100644
> > --- a/builtin/help.c
> > +++ b/builtin/help.c
> > @@ -8,6 +8,7 @@
> >  #include "parse-options.h"
> >  #include "run-command.h"
> >  #include "column.h"
> > +#include "config-list.h"
> >  #include "help.h"
> >  #include "alias.h"
> >  
> > @@ -62,6 +63,91 @@ static const char * const builtin_help_usage[] = {
> >  	NULL
> >  };
> >  
> > +struct slot_expansion {
> > +	const char *prefix;
> > +	const char *placeholder;
> > +	void (*fn)(struct string_list *list, const char *prefix);
> > +	int found;
> > +};
> > +
> > +static void list_config_help(int for_human)
> > +{
> > +	struct slot_expansion slot_expansions[] = {
> > +		{ "advice", "*", list_config_advices },
> > +		{ "color.branch", "<slot>", list_config_color_branch_slots },
> > +		{ "color.decorate", "<slot>", list_config_color_decorate_slots },
> > +		{ "color.diff", "<slot>", list_config_color_diff_slots },
> > +		{ "color.grep", "<slot>", list_config_color_grep_slots },
> > +		{ "color.interactive", "<slot>", list_config_color_interactive_slots },
> > +		{ "color.remote", "<slot>", list_config_color_sideband_slots },
> > +		{ "color.status", "<slot>", list_config_color_status_slots },
> > +		{ "fsck", "<msg-id>", list_config_fsck_msg_ids },
> > +		{ "receive.fsck", "<msg-id>", list_config_fsck_msg_ids },
> > +		{ NULL, NULL, NULL }
> > +	};
> > +	const char **p;
> > +	struct slot_expansion *e;
> > +	struct string_list keys = STRING_LIST_INIT_DUP;
> > +	int i;
> > +
> > +	for (p = config_name_list; *p; p++) {
> > +		const char *var = *p;
> > +		struct strbuf sb = STRBUF_INIT;
> > +
> > +		for (e = slot_expansions; e->prefix; e++) {
> > +
> > +			strbuf_reset(&sb);
> > +			strbuf_addf(&sb, "%s.%s", e->prefix, e->placeholder);
> > +			if (!strcasecmp(var, sb.buf)) {
> > +				e->fn(&keys, e->prefix);
> > +				e->found++;
> > +				break;
> > +			}
> > +		}
> > +		strbuf_release(&sb);
> > +		if (!e->prefix)
> > +			string_list_append(&keys, var);
> > +	}
> > +
> > +	for (e = slot_expansions; e->prefix; e++)
> > +		if (!e->found)
> > +			BUG("slot_expansion %s.%s is not used",
> > +			    e->prefix, e->placeholder);
> > +
> > +	string_list_sort(&keys);
> > +	for (i = 0; i < keys.nr; i++) {
> > +		const char *var = keys.items[i].string;
> > +		const char *wildcard, *tag, *cut;
> > +
> > +		if (for_human) {
> > +			puts(var);
> > +			continue;
> > +		}
> > +
> > +		wildcard = strchr(var, '*');
> > +		tag = strchr(var, '<');
> > +
> > +		if (!wildcard && !tag) {
> > +			puts(var);
> > +			continue;
> > +		}
> > +
> > +		if (wildcard && !tag)
> > +			cut = wildcard;
> > +		else if (!wildcard && tag)
> > +			cut = tag;
> > +		else
> > +			cut = wildcard < tag ? wildcard : tag;
> > +
> > +		/*
> > +		 * We may produce duplicates, but that's up to
> > +		 * git-completion.bash to handle
> > +		 */
> > +		printf("%.*s\n", (int)(cut - var), var);
> > +	}
> > +	string_list_clear(&keys, 0);
> > +}
> > +
> >  static enum help_format parse_help_format(const char *format)
> >  {
> >  	if (!strcmp(format, "man"))
> > diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
> > index 71158f7d8b..45fecf8bdf 100755
> > --- a/generate-cmdlist.sh
> > +++ b/generate-cmdlist.sh
> > @@ -76,23 +76,6 @@ print_command_list () {
> >  	echo "};"
> >  }
> >  
> > -print_config_list () {
> > -	cat <<EOF
> > -static const char *config_name_list[] = {
> > -EOF
> > -	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
> > -	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
> > -	sort |
> > -	while read line
> > -	do
> > -		echo "	\"$line\","
> > -	done
> > -	cat <<EOF
> > -	NULL,
> > -};
> > -EOF
> > -}
> > -
> >  exclude_programs=
> >  while test "--exclude-program" = "$1"
> >  do
> > @@ -113,5 +96,3 @@ echo
> >  define_category_names "$1"
> >  echo
> >  print_command_list "$1"
> > -echo
> > -print_config_list
> > diff --git a/generate-configlist.sh b/generate-configlist.sh
> > new file mode 100755
> > index 0000000000..eca6a00c30
> > --- /dev/null
> > +++ b/generate-configlist.sh
> > @@ -0,0 +1,24 @@
> > +#!/bin/sh
> > +
> > +echo "/* Automatically generated by generate-configlist.sh */"
> > +echo
> > +
> > +print_config_list () {
> > +	cat <<EOF
> > +static const char *config_name_list[] = {
> > +EOF
> > +	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
> > +	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
> > +	sort |
> > +	while read line
> > +	do
> > +		echo "	\"$line\","
> > +	done
> > +	cat <<EOF
> > +	NULL,
> > +};
> > +EOF
> > +}
> > +
> > +echo
> > +print_config_list
> > diff --git a/help.c b/help.c
> > index cf67624a94..a21487db77 100644
> > --- a/help.c
> > +++ b/help.c
> > @@ -407,91 +407,6 @@ void list_common_guides_help(void)
> >  	putchar('\n');
> >  }
> >  
> > -struct slot_expansion {
> > -	const char *prefix;
> > -	const char *placeholder;
> > -	void (*fn)(struct string_list *list, const char *prefix);
> > -	int found;
> > -};
> > -
> > -void list_config_help(int for_human)
> > -{
> > -	struct slot_expansion slot_expansions[] = {
> > -		{ "advice", "*", list_config_advices },
> > -		{ "color.branch", "<slot>", list_config_color_branch_slots },
> > -		{ "color.decorate", "<slot>", list_config_color_decorate_slots },
> > -		{ "color.diff", "<slot>", list_config_color_diff_slots },
> > -		{ "color.grep", "<slot>", list_config_color_grep_slots },
> > -		{ "color.interactive", "<slot>", list_config_color_interactive_slots },
> > -		{ "color.remote", "<slot>", list_config_color_sideband_slots },
> > -		{ "color.status", "<slot>", list_config_color_status_slots },
> > -		{ "fsck", "<msg-id>", list_config_fsck_msg_ids },
> > -		{ "receive.fsck", "<msg-id>", list_config_fsck_msg_ids },
> > -		{ NULL, NULL, NULL }
> > -	};
> > -	const char **p;
> > -	struct slot_expansion *e;
> > -	struct string_list keys = STRING_LIST_INIT_DUP;
> > -	int i;
> > -
> > -	for (p = config_name_list; *p; p++) {
> > -		const char *var = *p;
> > -		struct strbuf sb = STRBUF_INIT;
> > -
> > -		for (e = slot_expansions; e->prefix; e++) {
> > -
> > -			strbuf_reset(&sb);
> > -			strbuf_addf(&sb, "%s.%s", e->prefix, e->placeholder);
> > -			if (!strcasecmp(var, sb.buf)) {
> > -				e->fn(&keys, e->prefix);
> > -				e->found++;
> > -				break;
> > -			}
> > -		}
> > -		strbuf_release(&sb);
> > -		if (!e->prefix)
> > -			string_list_append(&keys, var);
> > -	}
> > -
> > -	for (e = slot_expansions; e->prefix; e++)
> > -		if (!e->found)
> > -			BUG("slot_expansion %s.%s is not used",
> > -			    e->prefix, e->placeholder);
> > -
> > -	string_list_sort(&keys);
> > -	for (i = 0; i < keys.nr; i++) {
> > -		const char *var = keys.items[i].string;
> > -		const char *wildcard, *tag, *cut;
> > -
> > -		if (for_human) {
> > -			puts(var);
> > -			continue;
> > -		}
> > -
> > -		wildcard = strchr(var, '*');
> > -		tag = strchr(var, '<');
> > -
> > -		if (!wildcard && !tag) {
> > -			puts(var);
> > -			continue;
> > -		}
> > -
> > -		if (wildcard && !tag)
> > -			cut = wildcard;
> > -		else if (!wildcard && tag)
> > -			cut = tag;
> > -		else
> > -			cut = wildcard < tag ? wildcard : tag;
> > -
> > -		/*
> > -		 * We may produce duplicates, but that's up to
> > -		 * git-completion.bash to handle
> > -		 */
> > -		printf("%.*s\n", (int)(cut - var), var);
> > -	}
> > -	string_list_clear(&keys, 0);
> > -}
> > -
> >  static int get_alias(const char *var, const char *value, void *data)
> >  {
> >  	struct string_list *list = data;
> > diff --git a/help.h b/help.h
> > index 7a455beeb7..9071894e8c 100644
> > --- a/help.h
> > +++ b/help.h
> > @@ -22,7 +22,6 @@ static inline void mput_char(char c, unsigned int num)
> >  void list_common_cmds_help(void);
> >  void list_all_cmds_help(void);
> >  void list_common_guides_help(void);
> > -void list_config_help(int for_human);
> >  
> >  void list_all_main_cmds(struct string_list *list);
> >  void list_all_other_cmds(struct string_list *list);
Junio C Hamano· Dec 16, 2019, 22:19 UTC · re: Emily Shaffer · lore

Re: [PATCH v4 02/15] help: move list_config_help to builtin/help

Emily Shaffer <emilyshaffer@google.com> writes:
Show 6 quoted lines
>> OK.  Looks like a clean-up that is very worthy even without the
>> remainder of the topic.
>> 
>> Thanks.
>
> Ok. Are you going to take it on its own?

Not really. Unless you abandon the rest of the topic, in which case it may be worth resurrecting this one on its own, that is. But that is not what you plan to do anyway, so I expect to see it as part of an updated series ;-)

Emily Shaffer· Dec 16, 2019, 22:34 UTC · re: Junio C Hamano · lore

Re: [PATCH v4 02/15] help: move list_config_help to builtin/help

On Mon, Dec 16, 2019 at 02:19:48PM -0800, Junio C Hamano wrote:
Show 13 quoted lines
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> >> OK.  Looks like a clean-up that is very worthy even without the
> >> remainder of the topic.
> >> 
> >> Thanks.
> >
> > Ok. Are you going to take it on its own?
> 
> Not really.  Unless you abandon the rest of the topic, in which case
> it may be worth resurrecting this one on its own, that is.  But that
> is not what you plan to do anyway, so I expect to see it as part of
> an updated series ;-)
Ok! That's fine and makes my life easier. Just making sure. :)
 - Emily
Emily Shaffer· Dec 13, 2019, 00:42 UTC · re: Emily Shaffer · lore

[PATCH v4 01/15] bugreport: add tool to generate debugging info

Teach Git how to prompt the user for a good bug report: reproduction steps, expected behavior, and actual behavior. Later, Git can learn how to collect some diagnostic information from the repository.

If users can send us a well-written bug report which contains diagnostic information we would otherwise need to ask the user for, we can reduce the number of question-and-answer round trips between the reporter and the Git contributor.

Users may also wish to send a report like this to their local "Git expert" if they have put their repository into a state they are confused by.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 .gitignore                      |  1 +
 Documentation/git-bugreport.txt | 43 ++++++++++++++++++++
 Makefile                        |  5 +++
 bugreport.c                     | 69 +++++++++++++++++++++++++++++++++
 t/t0091-bugreport.sh            | 41 ++++++++++++++++++++
 5 files changed, 159 insertions(+)
 create mode 100644 Documentation/git-bugreport.txt
 create mode 100644 bugreport.c
 create mode 100755 t/t0091-bugreport.sh
Show changes to 5 files +159 −0

.gitignore, Documentation/git-bugreport.txt, Makefile, bugreport.c, t/t0091-bugreport.sh

diff --git a/.gitignore b/.gitignore
index 89b3b79c1a..055a84c4a8 100644
--- a/.gitignore
+++ b/.gitignore
@@ -25,6 +25,7 @@
 /git-bisect--helper
 /git-blame
 /git-branch
+/git-bugreport
 /git-bundle
 /git-cat-file
 /git-check-attr
diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
new file mode 100644
index 0000000000..75f0c80acf
--- /dev/null
+++ b/Documentation/git-bugreport.txt
@@ -0,0 +1,43 @@
+git-bugreport(1)
+================
+
+NAME
+----
+git-bugreport - Collect information for user to file a bug report
+
+SYNOPSIS
+--------
+[verse]
+'git bugreport' [-o | --output <path>]
+
+DESCRIPTION
+-----------
+Captures information about the user's machine, Git client, and repository state,
+as well as a form requesting information about the behavior the user observed,
+into a single text file which the user can then share, for example to the Git
+mailing list, in order to report an observed bug.
+
+The following information is requested from the user:
+
+ - Reproduction steps
+ - Expected behavior
+ - Actual behavior
+
+The following information is captured automatically:
+
+ - Git version (`git version --build-options`)
+ - Machine information (`uname -a`)
+ - Versions of various dependencies
+ - Git config contents (`git config --show-origin --list`)
+ - The names of all configured git-hooks in `.git/hooks/`
+
+OPTIONS
+-------
+-o [<path>]::
+--output [<path>]::
+	Place the resulting bug report file in <path> instead of the root of the
+	Git repository.
+
+GIT
+---
+Part of the linkgit:git[1] suite
diff --git a/Makefile b/Makefile
index b7d7374dac..9dff91436e 100644
--- a/Makefile
+++ b/Makefile
@@ -681,6 +681,7 @@ EXTRA_PROGRAMS =
 # ... and all the rest that could be moved out of bindir to gitexecdir
 PROGRAMS += $(EXTRA_PROGRAMS)
 
+PROGRAM_OBJS += bugreport.o
 PROGRAM_OBJS += credential-store.o
 PROGRAM_OBJS += daemon.o
 PROGRAM_OBJS += fast-import.o
@@ -2448,6 +2449,10 @@ endif
 git-%$X: %.o GIT-LDFLAGS $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) $(LIBS)
 
+git-bugreport$X: bugreport.o GIT-LDFLAGS $(GITLIBS)
+	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
+		$(LIBS)
+
 git-imap-send$X: imap-send.o $(IMAP_SEND_BUILDDEPS) GIT-LDFLAGS $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
 		$(IMAP_SEND_LDFLAGS) $(LIBS)
diff --git a/bugreport.c b/bugreport.c
new file mode 100644
index 0000000000..5495b31674
--- /dev/null
+++ b/bugreport.c
@@ -0,0 +1,69 @@
+#include "builtin.h"
+#include "parse-options.h"
+#include "stdio.h"
+#include "strbuf.h"
+#include "time.h"
+
+static const char * const bugreport_usage[] = {
+	N_("git bugreport [-o|--output <file>]"),
+	NULL
+};
+
+static int get_bug_template(struct strbuf *template)
+{
+	const char template_text[] = N_(
+"Thank you for filling out a Git bug report!\n"
+"Please answer the following questions to help us understand your issue.\n"
+"\n"
+"What did you do before the bug happened? (Steps to reproduce your issue)\n"
+"\n"
+"What did you expect to happen? (Expected behavior)\n"
+"\n"
+"What happened instead? (Actual behavior)\n"
+"\n"
+"What's different between what you expected and what actually happened?\n"
+"\n"
+"Anything else you want to add:\n"
+"\n"
+"Please review the rest of the bug report below.\n"
+"You can delete any lines you don't wish to send.\n");
+
+	strbuf_addstr(template, template_text);
+	return 0;
+}
+
+int cmd_main(int argc, const char **argv)
+{
+	struct strbuf buffer = STRBUF_INIT;
+	struct strbuf report_path = STRBUF_INIT;
+	FILE *report;
+	time_t now = time(NULL);
+	char *option_output = NULL;
+
+	const struct option bugreport_options[] = {
+		OPT_STRING('o', "output", &option_output, N_("path"),
+			   N_("specify a destination for the bugreport file")),
+		OPT_END()
+	};
+	argc = parse_options(argc, argv, "", bugreport_options,
+			     bugreport_usage, 0);
+
+	if (option_output) {
+		strbuf_addstr(&report_path, option_output);
+		strbuf_complete(&report_path, '/');
+	}
+
+	strbuf_addstr(&report_path, "git-bugreport-");
+	strbuf_addftime(&report_path, "%F", gmtime(&now), 0, 0);
+	strbuf_addstr(&report_path, ".txt");
+
+
+	get_bug_template(&buffer);
+
+	report = fopen_for_writing(report_path.buf);
+	strbuf_write(&buffer, report);
+	fclose(report);
+
+	launch_editor(report_path.buf, NULL, NULL);
+	return 0;
+}
diff --git a/t/t0091-bugreport.sh b/t/t0091-bugreport.sh
new file mode 100755
index 0000000000..6eb2ee4f66
--- /dev/null
+++ b/t/t0091-bugreport.sh
@@ -0,0 +1,41 @@
+#!/bin/bash
+
+test_description='git bugreport'
+
+. ./test-lib.sh
+
+# Headers "[System Info]" will be followed by a non-empty line if we put some
+# information there; we can make sure all our headers were followed by some
+# information to check if the command was successful.
+HEADER_PATTERN="^\[.*\]$"
+check_all_headers_populated() {
+	while read -r line; do
+		if [$(grep $HEADER_PATTERN $line)]; then
+			read -r nextline
+			if [-z $nextline]; then
+				return 1;
+			fi
+		fi
+	done
+}
+
+test_expect_success 'creates a report with content in the right places' '
+	git bugreport &&
+	check_all_headers_populated <git-bugreport-* &&
+	rm git-bugreport-*
+'
+
+test_expect_success '--output puts the report in the provided dir' '
+	mkdir foo/ &&
+	git bugreport -o foo/ &&
+	test -f foo/git-bugreport-* &&
+	rm -fr foo/
+'
+
+test_expect_success 'incorrect arguments abort with usage' '
+	test_must_fail git bugreport --false 2>output &&
+	grep usage output &&
+	test ! -f git-bugreport-*
+'
+
+test_done
-- 
2.24.1.735.g03f4e72817-goog
Emily Shaffer· Dec 13, 2019, 00:43 UTC · re: Emily Shaffer · lore

[PATCH v4 03/15] bugreport: gather git version and build info

Knowing which version of Git a user has and how it was built allows us to more precisely pin down the circumstances when a certain issue occurs, so teach bugreport how to tell us the same output as 'git version --build-options'.

It's not ideal to directly call 'git version --build-options' because that output goes to stdout. Instead, wrap the version string in a helper within help.[ch] library, and call that helper from within the bugreport library.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 bugreport.c | 23 ++++++++++++++++++++++-
 help.c      | 45 +++++++++++++++++++++++++++------------------
 help.h      |  1 +
 3 files changed, 50 insertions(+), 19 deletions(-)
Show changes to 3 files +50 −19

bugreport.c, help.c, help.h

diff --git a/bugreport.c b/bugreport.c
index 5495b31674..59d8b5a3af 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -1,8 +1,20 @@
-#include "builtin.h"
+#include "cache.h"
 #include "parse-options.h"
 #include "stdio.h"
 #include "strbuf.h"
 #include "time.h"
+#include "help.h"
+
+static void get_system_info(struct strbuf *sys_info)
+{
+	struct strbuf version_info = STRBUF_INIT;
+
+	/* get git version from native cmd */
+	strbuf_addstr(sys_info, "git version:\n");
+	list_version_info(&version_info, 1);
+	strbuf_addbuf(sys_info, &version_info);
+	strbuf_complete_line(sys_info);
+}
 
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output <file>]"),
@@ -32,6 +44,11 @@ static int get_bug_template(struct strbuf *template)
 	return 0;
 }
 
+static void get_header(struct strbuf *buf, const char *title)
+{
+	strbuf_addf(buf, "\n\n[%s]\n", title);
+}
+
 int cmd_main(int argc, const char **argv)
 {
 	struct strbuf buffer = STRBUF_INIT;
@@ -60,6 +77,10 @@ int cmd_main(int argc, const char **argv)
 
 	get_bug_template(&buffer);
 
+	// add other contents
+	get_header(&buffer, "System Info");
+	get_system_info(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 	strbuf_write(&buffer, report);
 	fclose(report);
diff --git a/help.c b/help.c
index a21487db77..a43693fca5 100644
--- a/help.c
+++ b/help.c
@@ -622,8 +622,33 @@ const char *help_unknown_cmd(const char *cmd)
 	exit(1);
 }
 
+void list_version_info(struct strbuf *buf, int build_options)
+{
+	strbuf_reset(buf);
+	/*
+	 * The format of this string should be kept stable for compatibility
+	 * with external projects that rely on the output of "git version".
+	 *
+	 * Always show the version, even if other options are given.
+	 */
+	strbuf_addf(buf, "git version %s\n", git_version_string);
+
+	if (build_options) {
+		strbuf_addf(buf, "cpu: %s\n", GIT_HOST_CPU);
+		if (git_built_from_commit_string[0])
+			strbuf_addf(buf, "built from commit: %s\n",
+			       git_built_from_commit_string);
+		else
+			strbuf_addf(buf, "no commit associated with this build\n");
+		strbuf_addf(buf, "sizeof-long: %d\n", (int)sizeof(long));
+		strbuf_addf(buf, "sizeof-size_t: %d\n", (int)sizeof(size_t));
+		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
+	}
+}
+
 int cmd_version(int argc, const char **argv, const char *prefix)
 {
+	struct strbuf buf = STRBUF_INIT;
 	int build_options = 0;
 	const char * const usage[] = {
 		N_("git version [<options>]"),
@@ -637,25 +662,9 @@ int cmd_version(int argc, const char **argv, const char *prefix)
 
 	argc = parse_options(argc, argv, prefix, options, usage, 0);
 
-	/*
-	 * The format of this string should be kept stable for compatibility
-	 * with external projects that rely on the output of "git version".
-	 *
-	 * Always show the version, even if other options are given.
-	 */
-	printf("git version %s\n", git_version_string);
+	list_version_info(&buf, build_options);
+	printf("%s", buf.buf);
 
-	if (build_options) {
-		printf("cpu: %s\n", GIT_HOST_CPU);
-		if (git_built_from_commit_string[0])
-			printf("built from commit: %s\n",
-			       git_built_from_commit_string);
-		else
-			printf("no commit associated with this build\n");
-		printf("sizeof-long: %d\n", (int)sizeof(long));
-		printf("sizeof-size_t: %d\n", (int)sizeof(size_t));
-		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
-	}
 	return 0;
 }
 
diff --git a/help.h b/help.h
index 9071894e8c..54f6b5f793 100644
--- a/help.h
+++ b/help.h
@@ -37,6 +37,7 @@ void add_cmdname(struct cmdnames *cmds, const char *name, int len);
 void exclude_cmds(struct cmdnames *cmds, struct cmdnames *excludes);
 int is_in_cmdlist(struct cmdnames *cmds, const char *name);
 void list_commands(unsigned int colopts, struct cmdnames *main_cmds, struct cmdnames *other_cmds);
+void list_version_info(struct strbuf *buf, int build_options);
 
 /*
  * call this to die(), when it is suspected that the user mistyped a
-- 
2.24.1.735.g03f4e72817-goog
Junio C Hamano· Dec 13, 2019, 21:06 UTC · re: Emily Shaffer · lore

Re: [PATCH v4 03/15] bugreport: gather git version and build info

Emily Shaffer <emilyshaffer@google.com> writes:
Show 9 quoted lines
> Knowing which version of Git a user has and how it was built allows us
> to more precisely pin down the circumstances when a certain issue
> occurs, so teach bugreport how to tell us the same output as 'git
> version --build-options'.
>
> It's not ideal to directly call 'git version --build-options' because
> that output goes to stdout. Instead, wrap the version string in a helper
> within help.[ch] library, and call that helper from within the bugreport
> library.
Move to strbuf() from stdio makes sense.  
> +	// add other contents
Style.
Show 7 quoted lines
> diff --git a/help.h b/help.h
> index 9071894e8c..54f6b5f793 100644
> --- a/help.h
> +++ b/help.h
> @@ -37,6 +37,7 @@ void add_cmdname(struct cmdnames *cmds, const char *name, int len);
>  void exclude_cmds(struct cmdnames *cmds, struct cmdnames *excludes);
>  int is_in_cmdlist(struct cmdnames *cmds, const char *name);

Many new helpers are called get_frotz(), but only one among these is called list_version_info(). I do not think the naming of the get_*() ones, that are private to the bugreport tool, matters that much, but unlike "list_commands()" whose purpose is to list the commands ;-), the new function does not list versions---it just gives information about a single version which is the one that is being run, so perhaps it is a misnomer.

>  void list_commands(unsigned int colopts, struct cmdnames *main_cmds, struct cmdnames *other_cmds);
> +void list_version_info(struct strbuf *buf, int build_options);

It is not clear to the readers build_options is a boolean that tells the function to include (or not to include) build options. Perhaps rename it to "int show_build_options" or something?

Emily Shaffer· Dec 20, 2019, 01:46 UTC · re: Junio C Hamano · lore

Re: [PATCH v4 03/15] bugreport: gather git version and build info

On Fri, Dec 13, 2019 at 01:06:29PM -0800, Junio C Hamano wrote:
Show 17 quoted lines
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> > Knowing which version of Git a user has and how it was built allows us
> > to more precisely pin down the circumstances when a certain issue
> > occurs, so teach bugreport how to tell us the same output as 'git
> > version --build-options'.
> >
> > It's not ideal to directly call 'git version --build-options' because
> > that output goes to stdout. Instead, wrap the version string in a helper
> > within help.[ch] library, and call that helper from within the bugreport
> > library.
> 
> Move to strbuf() from stdio makes sense.  
> 
> > +	// add other contents
> 
> Style.

Sure, dropped this entirely. I think with the helpers the code is self-documenting there.

Show 17 quoted lines
> 
> > diff --git a/help.h b/help.h
> > index 9071894e8c..54f6b5f793 100644
> > --- a/help.h
> > +++ b/help.h
> > @@ -37,6 +37,7 @@ void add_cmdname(struct cmdnames *cmds, const char *name, int len);
> >  void exclude_cmds(struct cmdnames *cmds, struct cmdnames *excludes);
> >  int is_in_cmdlist(struct cmdnames *cmds, const char *name);
> 
> 
> Many new helpers are called get_frotz(), but only one among these is
> called list_version_info().  I do not think the naming of the
> get_*() ones, that are private to the bugreport tool, matters that
> much, but unlike "list_commands()" whose purpose is to list the
> commands ;-), the new function does not list versions---it just
> gives information about a single version which is the one that is
> being run, so perhaps it is a misnomer.

Hm, sure. I renamed it to get_version_info(); I had named it list_* because all the other helpers in help.h are named list_*, and it does print more than one piece of info. But I do see your point (all the info is about the same version) so I've renamed it.

Show 7 quoted lines
> 
> >  void list_commands(unsigned int colopts, struct cmdnames *main_cmds, struct cmdnames *other_cmds);
> > +void list_version_info(struct strbuf *buf, int build_options);
> 
> It is not clear to the readers build_options is a boolean that
> tells the function to include (or not to include) build options.
> Perhaps rename it to "int show_build_options" or something?
Agree, done.
Johannes Schindelin· Dec 17, 2019, 18:45 UTC · re: Emily Shaffer · lore

Re: [PATCH v4 03/15] bugreport: gather git version and build info

Hi Emily,
On Thu, 12 Dec 2019, Emily Shaffer wrote:
Show 26 quoted lines
> diff --git a/help.c b/help.c
> index a21487db77..a43693fca5 100644
> --- a/help.c
> +++ b/help.c
> @@ -622,8 +622,33 @@ const char *help_unknown_cmd(const char *cmd)
>  	exit(1);
>  }
>
> +void list_version_info(struct strbuf *buf, int build_options)
> +{
> +	strbuf_reset(buf);
> +	/*
> +	 * The format of this string should be kept stable for compatibility
> +	 * with external projects that rely on the output of "git version".
> +	 *
> +	 * Always show the version, even if other options are given.
> +	 */
> +	strbuf_addf(buf, "git version %s\n", git_version_string);
> +
> +	if (build_options) {
> +		strbuf_addf(buf, "cpu: %s\n", GIT_HOST_CPU);
> +		if (git_built_from_commit_string[0])
> +			strbuf_addf(buf, "built from commit: %s\n",
> +			       git_built_from_commit_string);
> +		else
> +			strbuf_addf(buf, "no commit associated with this build\n");

The "StaticAnalysis" job of the Azure Pipeline is not happy with this, claiming that this should be an `strbuf_addstr()` call instead. For details, see:

https://dev.azure.com/gitgitgadget/8fc4a374-71dc-4558-a5ea-dd1c081ea621/_apis/build/builds/23830/logs/68

Ciao, Dscho

Show 57 quoted lines
> +		strbuf_addf(buf, "sizeof-long: %d\n", (int)sizeof(long));
> +		strbuf_addf(buf, "sizeof-size_t: %d\n", (int)sizeof(size_t));
> +		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
> +	}
> +}
> +
>  int cmd_version(int argc, const char **argv, const char *prefix)
>  {
> +	struct strbuf buf = STRBUF_INIT;
>  	int build_options = 0;
>  	const char * const usage[] = {
>  		N_("git version [<options>]"),
> @@ -637,25 +662,9 @@ int cmd_version(int argc, const char **argv, const char *prefix)
>
>  	argc = parse_options(argc, argv, prefix, options, usage, 0);
>
> -	/*
> -	 * The format of this string should be kept stable for compatibility
> -	 * with external projects that rely on the output of "git version".
> -	 *
> -	 * Always show the version, even if other options are given.
> -	 */
> -	printf("git version %s\n", git_version_string);
> +	list_version_info(&buf, build_options);
> +	printf("%s", buf.buf);
>
> -	if (build_options) {
> -		printf("cpu: %s\n", GIT_HOST_CPU);
> -		if (git_built_from_commit_string[0])
> -			printf("built from commit: %s\n",
> -			       git_built_from_commit_string);
> -		else
> -			printf("no commit associated with this build\n");
> -		printf("sizeof-long: %d\n", (int)sizeof(long));
> -		printf("sizeof-size_t: %d\n", (int)sizeof(size_t));
> -		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
> -	}
>  	return 0;
>  }
>
> diff --git a/help.h b/help.h
> index 9071894e8c..54f6b5f793 100644
> --- a/help.h
> +++ b/help.h
> @@ -37,6 +37,7 @@ void add_cmdname(struct cmdnames *cmds, const char *name, int len);
>  void exclude_cmds(struct cmdnames *cmds, struct cmdnames *excludes);
>  int is_in_cmdlist(struct cmdnames *cmds, const char *name);
>  void list_commands(unsigned int colopts, struct cmdnames *main_cmds, struct cmdnames *other_cmds);
> +void list_version_info(struct strbuf *buf, int build_options);
>
>  /*
>   * call this to die(), when it is suspected that the user mistyped a
> --
> 2.24.1.735.g03f4e72817-goog
>
>
>
Junio C Hamano· Dec 17, 2019, 20:34 UTC · re: Johannes Schindelin · lore

Re: [PATCH v4 03/15] bugreport: gather git version and build info

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
Show 10 quoted lines
>> +	if (build_options) {
>> +		strbuf_addf(buf, "cpu: %s\n", GIT_HOST_CPU);
>> +		if (git_built_from_commit_string[0])
>> +			strbuf_addf(buf, "built from commit: %s\n",
>> +			       git_built_from_commit_string);
>> +		else
>> +			strbuf_addf(buf, "no commit associated with this build\n");
>
> The "StaticAnalysis" job of the Azure Pipeline is not happy with this,
> claiming that this should be an `strbuf_addstr()` call instead.
You mean the "else" clause, right?  That feels similar to say
	printf("Hello world\n");
should better be written as
	fputs("Hello world\n", stdout);

which I do not agree with at all. IOW, I view the distinction more like "once it is written one way or the other way, it is not worth spending bits and braincycles to see if it is worth changing it" kind of minor stylistic preference.

Emily Shaffer· Dec 20, 2019, 01:25 UTC · re: Junio C Hamano · lore

Re: [PATCH v4 03/15] bugreport: gather git version and build info

On Tue, Dec 17, 2019 at 12:34:53PM -0800, Junio C Hamano wrote:
Show 25 quoted lines
> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
> >> +	if (build_options) {
> >> +		strbuf_addf(buf, "cpu: %s\n", GIT_HOST_CPU);
> >> +		if (git_built_from_commit_string[0])
> >> +			strbuf_addf(buf, "built from commit: %s\n",
> >> +			       git_built_from_commit_string);
> >> +		else
> >> +			strbuf_addf(buf, "no commit associated with this build\n");
> >
> > The "StaticAnalysis" job of the Azure Pipeline is not happy with this,
> > claiming that this should be an `strbuf_addstr()` call instead.
> 
> You mean the "else" clause, right?  That feels similar to say
> 
> 	printf("Hello world\n");
> 
> should better be written as
> 
> 	fputs("Hello world\n", stdout);
> 
> which I do not agree with at all.  IOW, I view the distinction more
> like "once it is written one way or the other way, it is not worth
> spending bits and braincycles to see if it is worth changing it"
> kind of minor stylistic preference.

I think I side with Junio here, although it's true that when strbuf_addstr() exists it doesn't make that much sense to use strbuf_addf(). Since there's some other comments, though, I'll change this too to make your CI shut up. :)

 - Emily
Emily Shaffer· Dec 13, 2019, 00:43 UTC · re: Emily Shaffer · lore

[PATCH v4 04/15] help: add shell-path to --build-options

It may be useful to know which shell Git was built to try to point to, in the event that shell-based Git commands are failing. $SHELL_PATH is set during the build and used to launch the manpage viewer, as well as by git-compat-util.h, and it's used during tests. 'git version --build-options' is encouraged for use in bug reports, so it makes sense to include this information there.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 help.c | 1 +
 1 file changed, 1 insertion(+)
Show changes to help.c +1 −0
diff --git a/help.c b/help.c
index a43693fca5..6d1bd17971 100644
--- a/help.c
+++ b/help.c
@@ -642,6 +642,7 @@ void list_version_info(struct strbuf *buf, int build_options)
 			strbuf_addf(buf, "no commit associated with this build\n");
 		strbuf_addf(buf, "sizeof-long: %d\n", (int)sizeof(long));
 		strbuf_addf(buf, "sizeof-size_t: %d\n", (int)sizeof(size_t));
+		strbuf_addf(buf, "shell-path: %s\n", SHELL_PATH);
 		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
 	}
 }
-- 
2.24.1.735.g03f4e72817-goog
Emily Shaffer· Dec 13, 2019, 00:43 UTC · re: Emily Shaffer · lore

[PATCH v4 05/15] bugreport: add uname info

The contents of uname() can give us some insight into what sort of system the user is running on, and help us replicate their setup if need be. The domainname field is not guaranteed to be available, so don't collect it.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 bugreport.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)
Show changes to bugreport.c +13 −0
diff --git a/bugreport.c b/bugreport.c
index 59d8b5a3af..9c69e3fa34 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -8,12 +8,25 @@
 static void get_system_info(struct strbuf *sys_info)
 {
 	struct strbuf version_info = STRBUF_INIT;
+	struct utsname uname_info;
 
 	/* get git version from native cmd */
 	strbuf_addstr(sys_info, "git version:\n");
 	list_version_info(&version_info, 1);
 	strbuf_addbuf(sys_info, &version_info);
 	strbuf_complete_line(sys_info);
+
+	/* system call for other version info */
+	strbuf_addstr(sys_info, "uname -a: ");
+	if (uname(&uname_info))
+		strbuf_addf(sys_info, "uname() failed with code %d\n", errno);
+	else
+		strbuf_addf(sys_info, "%s %s %s %s %s\n",
+			    uname_info.sysname,
+			    uname_info.nodename,
+			    uname_info.release,
+			    uname_info.version,
+			    uname_info.machine);
 }
 
 static const char * const bugreport_usage[] = {
-- 
2.24.1.735.g03f4e72817-goog
Junio C Hamano· Dec 13, 2019, 21:12 UTC · re: Emily Shaffer · lore

Re: [PATCH v4 05/15] bugreport: add uname info

Emily Shaffer <emilyshaffer@google.com> writes:
> The contents of uname() can give us some insight into what sort of
> system the user is running on, and help us replicate their setup if need
> be. The domainname field is not guaranteed to be available, so don't
> collect it.

It was surprising to me that we do use "struct utsname" somewhere in the system ;-) ... so at least we know this is just as portable as the remainder of Git, which is good.

Show 35 quoted lines
> Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
> ---
>  bugreport.c | 13 +++++++++++++
>  1 file changed, 13 insertions(+)
>
> diff --git a/bugreport.c b/bugreport.c
> index 59d8b5a3af..9c69e3fa34 100644
> --- a/bugreport.c
> +++ b/bugreport.c
> @@ -8,12 +8,25 @@
>  static void get_system_info(struct strbuf *sys_info)
>  {
>  	struct strbuf version_info = STRBUF_INIT;
> +	struct utsname uname_info;
>  
>  	/* get git version from native cmd */
>  	strbuf_addstr(sys_info, "git version:\n");
>  	list_version_info(&version_info, 1);
>  	strbuf_addbuf(sys_info, &version_info);
>  	strbuf_complete_line(sys_info);
> +
> +	/* system call for other version info */
> +	strbuf_addstr(sys_info, "uname -a: ");
> +	if (uname(&uname_info))
> +		strbuf_addf(sys_info, "uname() failed with code %d\n", errno);
> +	else
> +		strbuf_addf(sys_info, "%s %s %s %s %s\n",
> +			    uname_info.sysname,
> +			    uname_info.nodename,
> +			    uname_info.release,
> +			    uname_info.version,
> +			    uname_info.machine);
>  }
>  
>  static const char * const bugreport_usage[] = {
Aaron Schrab· Jan 10, 2020, 02:05 UTC · re: Emily Shaffer · lore

Re: [PATCH v4 05/15] bugreport: add uname info

At 16:43 -0800 12 Dec 2019, Emily Shaffer <emilyshaffer@google.com> wrote:
>The contents of uname() can give us some insight into what sort of
>system the user is running on, and help us replicate their setup if need
>be. The domainname field is not guaranteed to be available, so don't
>collect it.

The manpage on Linux says that it's a GNU extension; on Mac OS it isn't mentioned. So it would be more accurate to say that it's known not to be available on many systems rather than just not being guaranteed.

Besides that I think in some cases it may be considered sensitive information, so another reason to not include it. Perhaps even worthy of being mentioned as the primary reason.

>+			    uname_info.nodename,

I think in some cases this could also be considered as sensitive information, and is unlikely to help in diagnosing problems. So I'd move to exclude this as well.

Emily Shaffer· Dec 13, 2019, 00:43 UTC · re: Emily Shaffer · lore

[PATCH v4 06/15] bugreport: add glibc version

To help pinpoint the source of a regression, it is useful to know the version of libc which the user's Git client was built with. Let's include it alongside the other versioning information, which is used to identify how the client was built.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 bugreport.c | 5 +++++
 1 file changed, 5 insertions(+)
Show changes to bugreport.c +5 −0
diff --git a/bugreport.c b/bugreport.c
index 9c69e3fa34..af715dc157 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -4,6 +4,7 @@
 #include "strbuf.h"
 #include "time.h"
 #include "help.h"
+#include <gnu/libc-version.h>
 
 static void get_system_info(struct strbuf *sys_info)
 {
@@ -27,6 +28,10 @@ static void get_system_info(struct strbuf *sys_info)
 			    uname_info.release,
 			    uname_info.version,
 			    uname_info.machine);
+
+	strbuf_addstr(sys_info, "glibc version: ");
+	strbuf_addstr(sys_info, gnu_get_libc_version());
+	strbuf_complete_line(sys_info);
 }
 
 static const char * const bugreport_usage[] = {
-- 
2.24.1.735.g03f4e72817-goog
Junio C Hamano· Dec 13, 2019, 21:18 UTC · re: Emily Shaffer · lore

Re: [PATCH v4 06/15] bugreport: add glibc version

Emily Shaffer <emilyshaffer@google.com> writes:
Show 9 quoted lines
> To help pinpoint the source of a regression, it is useful to know the
> version of libc which the user's Git client was built with. Let's
> include it alongside the other versioning information, which is used to
> identify how the client was built.
>
> Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
> ---
>  bugreport.c | 5 +++++
>  1 file changed, 5 insertions(+)

This is the first iffy one in the series. It is unreasonable to assume that we can dictate that Git can be built only on glibc systems, no?

Making this conditional, perhaps make "bugreport.c" depend on "extern void get_compiler_info(struct strbuf *sys_info)" and require the function to be defined in compat/; the glibc variant that uses gnu_get_libc_version() would become just one of them.

Show 23 quoted lines
> diff --git a/bugreport.c b/bugreport.c
> index 9c69e3fa34..af715dc157 100644
> --- a/bugreport.c
> +++ b/bugreport.c
> @@ -4,6 +4,7 @@
>  #include "strbuf.h"
>  #include "time.h"
>  #include "help.h"
> +#include <gnu/libc-version.h>
>  
>  static void get_system_info(struct strbuf *sys_info)
>  {
> @@ -27,6 +28,10 @@ static void get_system_info(struct strbuf *sys_info)
>  			    uname_info.release,
>  			    uname_info.version,
>  			    uname_info.machine);
> +
> +	strbuf_addstr(sys_info, "glibc version: ");
> +	strbuf_addstr(sys_info, gnu_get_libc_version());
> +	strbuf_complete_line(sys_info);
>  }
>  
>  static const char * const bugreport_usage[] = {
Emily Shaffer· Dec 16, 2019, 22:39 UTC · re: Junio C Hamano · lore

Re: [PATCH v4 06/15] bugreport: add glibc version

On Fri, Dec 13, 2019 at 01:18:44PM -0800, Junio C Hamano wrote:
Show 20 quoted lines
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> > To help pinpoint the source of a regression, it is useful to know the
> > version of libc which the user's Git client was built with. Let's
> > include it alongside the other versioning information, which is used to
> > identify how the client was built.
> >
> > Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
> > ---
> >  bugreport.c | 5 +++++
> >  1 file changed, 5 insertions(+)
> 
> This is the first iffy one in the series.  It is unreasonable to
> assume that we can dictate that Git can be built only on glibc
> systems, no?
> 
> Making this conditional, perhaps make "bugreport.c" depend on
> "extern void get_compiler_info(struct strbuf *sys_info)" and require
> the function to be defined in compat/; the glibc variant that uses
> gnu_get_libc_version() would become just one of them.

Interesting. This sounds like a good approach - thanks, I'll look into it.

 - Emily
Emily Shaffer· Dec 13, 2019, 00:43 UTC · re: Emily Shaffer · lore

[PATCH v4 08/15] bugreport: include user interactive shell

It's possible a user may complain about the way that Git interacts with their interactive shell, e.g. autocompletion or shell prompt. In that case, it's useful for us to know which shell they're using interactively.

$SHELL isn't set by every shell, but getenv() returns NULL in the event the variable isn't found, so we'll see a line like "$SHELL: (null)" to tell us that variable wasn't set.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 bugreport.c | 3 +++
 1 file changed, 3 insertions(+)
Show changes to bugreport.c +3 −0
diff --git a/bugreport.c b/bugreport.c
index f5598513d9..759cc0b0f8 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -45,6 +45,9 @@ static void get_system_info(struct strbuf *sys_info)
 	strbuf_addstr(sys_info, gnu_get_libc_version());
 	strbuf_complete_line(sys_info);
 
+	strbuf_addf(sys_info, "$SHELL (typically, interactive shell): %s\n",
+		    getenv("SHELL"));
+
 	strbuf_addstr(sys_info, "git-http-fetch -V:\n");
 	get_http_version_info(sys_info);
 	strbuf_complete_line(sys_info);
-- 
2.24.1.735.g03f4e72817-goog
Junio C Hamano· Dec 13, 2019, 21:38 UTC · re: Emily Shaffer · lore

Re: [PATCH v4 08/15] bugreport: include user interactive shell

Emily Shaffer <emilyshaffer@google.com> writes:
Show 8 quoted lines
> It's possible a user may complain about the way that Git interacts with
> their interactive shell, e.g. autocompletion or shell prompt. In that
> case, it's useful for us to know which shell they're using
> interactively.
>
> $SHELL isn't set by every shell, but getenv() returns NULL in the
> event the variable isn't found, so we'll see a line like "$SHELL:
> (null)" to tell us that variable wasn't set.

Do not depend on vsnprintf() from glibc that accepts NULL and shows "(null)". You can segfault on systems without glibc that way.

Show 19 quoted lines
> Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
> ---
>  bugreport.c | 3 +++
>  1 file changed, 3 insertions(+)
>
> diff --git a/bugreport.c b/bugreport.c
> index f5598513d9..759cc0b0f8 100644
> --- a/bugreport.c
> +++ b/bugreport.c
> @@ -45,6 +45,9 @@ static void get_system_info(struct strbuf *sys_info)
>  	strbuf_addstr(sys_info, gnu_get_libc_version());
>  	strbuf_complete_line(sys_info);
>  
> +	strbuf_addf(sys_info, "$SHELL (typically, interactive shell): %s\n",
> +		    getenv("SHELL"));
> +
>  	strbuf_addstr(sys_info, "git-http-fetch -V:\n");
>  	get_http_version_info(sys_info);
>  	strbuf_complete_line(sys_info);
Emily Shaffer· Dec 13, 2019, 00:43 UTC · re: Emily Shaffer · lore

[PATCH v4 07/15] bugreport: add curl version

It's possible for git-http* to be built separately from git; in that case we want to know what version of cURL is used by git-http*, not necessarily which version was present at git-bugreport's build time. So instead, ask git-http-fetch for the version information it knows about.

git-http-fetch was chosen as git-http-backend was described as a server-side implementation, and as an accidental fetch in case of problems was considered less harmful than an accidental push.

Since it could have been built at a different time, also report the version and built-from commit of git-http-fetch alongside the cURL info.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-http-fetch.txt |  5 +++++
 bugreport.c                      | 16 ++++++++++++++++
 http-fetch.c                     | 13 ++++++++++++-
 3 files changed, 33 insertions(+), 1 deletion(-)
Show changes to 3 files +33 −1

Documentation/git-http-fetch.txt, bugreport.c, http-fetch.c

diff --git a/Documentation/git-http-fetch.txt b/Documentation/git-http-fetch.txt
index 666b042679..2894c5e82b 100644
--- a/Documentation/git-http-fetch.txt
+++ b/Documentation/git-http-fetch.txt
@@ -10,6 +10,7 @@ SYNOPSIS
 --------
 [verse]
 'git http-fetch' [-c] [-t] [-a] [-d] [-v] [-w filename] [--recover] [--stdin] <commit> <url>
+'git http-fetch' [-V]
 
 DESCRIPTION
 -----------
@@ -30,6 +31,10 @@ commit-id::
 -v::
 	Report what is downloaded.
 
+-V::
+	Report information about the version of git-http-fetch, including the
+	versions of its dependencies.
+
 -w <filename>::
         Writes the commit-id into the filename under $GIT_DIR/refs/<filename> on
         the local end after the transfer is complete.
diff --git a/bugreport.c b/bugreport.c
index af715dc157..f5598513d9 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -5,6 +5,18 @@
 #include "time.h"
 #include "help.h"
 #include <gnu/libc-version.h>
+#include "run-command.h"
+
+static void get_http_version_info(struct strbuf *http_info)
+{
+	struct child_process cp = CHILD_PROCESS_INIT;
+
+	argv_array_push(&cp.args, "git");
+	argv_array_push(&cp.args, "http-fetch");
+	argv_array_push(&cp.args, "-V");
+	if (capture_command(&cp, http_info, 0))
+	    strbuf_addstr(http_info, "'git-http-fetch -V' not supported\n");
+}
 
 static void get_system_info(struct strbuf *sys_info)
 {
@@ -32,6 +44,10 @@ static void get_system_info(struct strbuf *sys_info)
 	strbuf_addstr(sys_info, "glibc version: ");
 	strbuf_addstr(sys_info, gnu_get_libc_version());
 	strbuf_complete_line(sys_info);
+
+	strbuf_addstr(sys_info, "git-http-fetch -V:\n");
+	get_http_version_info(sys_info);
+	strbuf_complete_line(sys_info);
 }
 
 static const char * const bugreport_usage[] = {
diff --git a/http-fetch.c b/http-fetch.c
index a32ac118d9..31844812a1 100644
--- a/http-fetch.c
+++ b/http-fetch.c
@@ -3,9 +3,18 @@
 #include "exec-cmd.h"
 #include "http.h"
 #include "walker.h"
+#include "version.h"
 
 static const char http_fetch_usage[] = "git http-fetch "
-"[-c] [-t] [-a] [-v] [--recover] [-w ref] [--stdin] commit-id url";
+"[-c] [-t] [-a] [-v] [-V] [--recover] [-w ref] [--stdin] commit-id url";
+
+void NORETURN version_info()
+{
+	printf("git-http-fetch version: %s\n", git_version_string);
+	printf("built from commit: %s\n", git_built_from_commit_string);
+	printf("curl version: %s\n", curl_version());
+	exit(0);
+}
 
 int cmd_main(int argc, const char **argv)
 {
@@ -26,6 +35,8 @@ int cmd_main(int argc, const char **argv)
 		} else if (argv[arg][1] == 'a') {
 		} else if (argv[arg][1] == 'v') {
 			get_verbosely = 1;
+		} else if (argv[arg][1] == 'V') {
+			version_info();
 		} else if (argv[arg][1] == 'w') {
 			write_ref = &argv[arg + 1];
 			arg++;
-- 
2.24.1.735.g03f4e72817-goog
Junio C Hamano· Dec 13, 2019, 21:27 UTC · re: Emily Shaffer · lore

Re: [PATCH v4 07/15] bugreport: add curl version

Emily Shaffer <emilyshaffer@google.com> writes:
Show 12 quoted lines
> It's possible for git-http* to be built separately from git; in that
> case we want to know what version of cURL is used by git-http*, not
> necessarily which version was present at git-bugreport's build time.
> So instead, ask git-http-fetch for the version information it knows
> about.
>
> git-http-fetch was chosen as git-http-backend was described as a
> server-side implementation, and as an accidental fetch in case of
> problems was considered less harmful than an accidental push.
>
> Since it could have been built at a different time, also report the
> version and built-from commit of git-http-fetch alongside the cURL info.

One possible issue I have is that I was hoping that eventually we can discard "git http-fetch" altogether sometime in the future. Does anybody still use the dumb HTTP transport seriously?

And the first move in that direction would be to allow the system be built without http-fetch, even if git-remote-curl (and its aliases) would still be built to access smart-http transports.

So, I am not sure. This is just the matter of adding an out-of-line hidden option used only for environment inspection, so if it can be done to git-remote-curl, that would probably be much more future proof.

Show 19 quoted lines
> diff --git a/bugreport.c b/bugreport.c
> index af715dc157..f5598513d9 100644
> --- a/bugreport.c
> +++ b/bugreport.c
> @@ -5,6 +5,18 @@
>  #include "time.h"
>  #include "help.h"
>  #include <gnu/libc-version.h>
> +#include "run-command.h"
> +
> +static void get_http_version_info(struct strbuf *http_info)
> +{
> +	struct child_process cp = CHILD_PROCESS_INIT;
> +
> +	argv_array_push(&cp.args, "git");
> +	argv_array_push(&cp.args, "http-fetch");
> +	argv_array_push(&cp.args, "-V");
> +	if (capture_command(&cp, http_info, 0))
> +	    strbuf_addstr(http_info, "'git-http-fetch -V' not supported\n");

OK. We probably can also take the compile-time "NO_CURL" into account, so that we can tell a misconfigured installation that wanted to have CURL but failed to install a usable http-fetch and an installation that deliberately omitted anything cURL?

Show 28 quoted lines
>  static void get_system_info(struct strbuf *sys_info)
>  {
> @@ -32,6 +44,10 @@ static void get_system_info(struct strbuf *sys_info)
>  	strbuf_addstr(sys_info, "glibc version: ");
>  	strbuf_addstr(sys_info, gnu_get_libc_version());
>  	strbuf_complete_line(sys_info);
> +
> +	strbuf_addstr(sys_info, "git-http-fetch -V:\n");
> +	get_http_version_info(sys_info);
> +	strbuf_complete_line(sys_info);
>  }
>  
>  static const char * const bugreport_usage[] = {
> diff --git a/http-fetch.c b/http-fetch.c
> index a32ac118d9..31844812a1 100644
> --- a/http-fetch.c
> +++ b/http-fetch.c
> @@ -3,9 +3,18 @@
>  #include "exec-cmd.h"
>  #include "http.h"
>  #include "walker.h"
> +#include "version.h"
>  
>  static const char http_fetch_usage[] = "git http-fetch "
> -"[-c] [-t] [-a] [-v] [--recover] [-w ref] [--stdin] commit-id url";
> +"[-c] [-t] [-a] [-v] [-V] [--recover] [-w ref] [--stdin] commit-id url";
> +
> +void NORETURN version_info()
void NORETURN version_info(void)
Show 18 quoted lines
> +{
> +	printf("git-http-fetch version: %s\n", git_version_string);
> +	printf("built from commit: %s\n", git_built_from_commit_string);
> +	printf("curl version: %s\n", curl_version());
> +	exit(0);
> +}
>  
>  int cmd_main(int argc, const char **argv)
>  {
> @@ -26,6 +35,8 @@ int cmd_main(int argc, const char **argv)
>  		} else if (argv[arg][1] == 'a') {
>  		} else if (argv[arg][1] == 'v') {
>  			get_verbosely = 1;
> +		} else if (argv[arg][1] == 'V') {
> +			version_info();
>  		} else if (argv[arg][1] == 'w') {
>  			write_ref = &argv[arg + 1];
>  			arg++;
Emily Shaffer· Dec 16, 2019, 22:49 UTC · re: Junio C Hamano · lore

Re: [PATCH v4 07/15] bugreport: add curl version

On Fri, Dec 13, 2019 at 01:27:31PM -0800, Junio C Hamano wrote:
Show 18 quoted lines
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> > It's possible for git-http* to be built separately from git; in that
> > case we want to know what version of cURL is used by git-http*, not
> > necessarily which version was present at git-bugreport's build time.
> > So instead, ask git-http-fetch for the version information it knows
> > about.
> >
> > git-http-fetch was chosen as git-http-backend was described as a
> > server-side implementation, and as an accidental fetch in case of
> > problems was considered less harmful than an accidental push.
> >
> > Since it could have been built at a different time, also report the
> > version and built-from commit of git-http-fetch alongside the cURL info.
> 
> One possible issue I have is that I was hoping that eventually we
> can discard "git http-fetch" altogether sometime in the future.
> Does anybody still use the dumb HTTP transport seriously?  

Oh, interesting. I was about to say, "I still use it to fetch, when I don't really care" - but that isn't even true, as I fetch via https (and have so much muscle memory to type https that I don't even notice).`

Show 9 quoted lines
> 
> And the first move in that direction would be to allow the system be
> built without http-fetch, even if git-remote-curl (and its aliases)
> would still be built to access smart-http transports.
> 
> So, I am not sure.  This is just the matter of adding an out-of-line
> hidden option used only for environment inspection, so if it can be
> done to git-remote-curl, that would probably be much more future
> proof.
Ok. I'll move it.
Show 25 quoted lines
> 
> > diff --git a/bugreport.c b/bugreport.c
> > index af715dc157..f5598513d9 100644
> > --- a/bugreport.c
> > +++ b/bugreport.c
> > @@ -5,6 +5,18 @@
> >  #include "time.h"
> >  #include "help.h"
> >  #include <gnu/libc-version.h>
> > +#include "run-command.h"
> > +
> > +static void get_http_version_info(struct strbuf *http_info)
> > +{
> > +	struct child_process cp = CHILD_PROCESS_INIT;
> > +
> > +	argv_array_push(&cp.args, "git");
> > +	argv_array_push(&cp.args, "http-fetch");
> > +	argv_array_push(&cp.args, "-V");
> > +	if (capture_command(&cp, http_info, 0))
> > +	    strbuf_addstr(http_info, "'git-http-fetch -V' not supported\n");
> 
> OK.  We probably can also take the compile-time "NO_CURL" into account,
> so that we can tell a misconfigured installation that wanted to have
> CURL but failed to install a usable http-fetch and an installation
> that deliberately omitted anything cURL?
Oh, interesting idea! I'll add that.
Show 51 quoted lines
> 
> >  static void get_system_info(struct strbuf *sys_info)
> >  {
> > @@ -32,6 +44,10 @@ static void get_system_info(struct strbuf *sys_info)
> >  	strbuf_addstr(sys_info, "glibc version: ");
> >  	strbuf_addstr(sys_info, gnu_get_libc_version());
> >  	strbuf_complete_line(sys_info);
> > +
> > +	strbuf_addstr(sys_info, "git-http-fetch -V:\n");
> > +	get_http_version_info(sys_info);
> > +	strbuf_complete_line(sys_info);
> >  }
> >  
> >  static const char * const bugreport_usage[] = {
> > diff --git a/http-fetch.c b/http-fetch.c
> > index a32ac118d9..31844812a1 100644
> > --- a/http-fetch.c
> > +++ b/http-fetch.c
> > @@ -3,9 +3,18 @@
> >  #include "exec-cmd.h"
> >  #include "http.h"
> >  #include "walker.h"
> > +#include "version.h"
> >  
> >  static const char http_fetch_usage[] = "git http-fetch "
> > -"[-c] [-t] [-a] [-v] [--recover] [-w ref] [--stdin] commit-id url";
> > +"[-c] [-t] [-a] [-v] [-V] [--recover] [-w ref] [--stdin] commit-id url";
> > +
> > +void NORETURN version_info()
> 
> void NORETURN version_info(void)
> 
> 
> > +{
> > +	printf("git-http-fetch version: %s\n", git_version_string);
> > +	printf("built from commit: %s\n", git_built_from_commit_string);
> > +	printf("curl version: %s\n", curl_version());
> > +	exit(0);
> > +}
> >  
> >  int cmd_main(int argc, const char **argv)
> >  {
> > @@ -26,6 +35,8 @@ int cmd_main(int argc, const char **argv)
> >  		} else if (argv[arg][1] == 'a') {
> >  		} else if (argv[arg][1] == 'v') {
> >  			get_verbosely = 1;
> > +		} else if (argv[arg][1] == 'V') {
> > +			version_info();
> >  		} else if (argv[arg][1] == 'w') {
> >  			write_ref = &argv[arg + 1];
> >  			arg++;
Johannes Schindelin· Dec 17, 2019, 18:47 UTC · re: Emily Shaffer · lore

Re: [PATCH v4 07/15] bugreport: add curl version

Hi Emily,
On Thu, 12 Dec 2019, Emily Shaffer wrote:
Show 15 quoted lines
> diff --git a/http-fetch.c b/http-fetch.c
> index a32ac118d9..31844812a1 100644
> --- a/http-fetch.c
> +++ b/http-fetch.c
> @@ -3,9 +3,18 @@
>  #include "exec-cmd.h"
>  #include "http.h"
>  #include "walker.h"
> +#include "version.h"
>
>  static const char http_fetch_usage[] = "git http-fetch "
> -"[-c] [-t] [-a] [-v] [--recover] [-w ref] [--stdin] commit-id url";
> +"[-c] [-t] [-a] [-v] [-V] [--recover] [-w ref] [--stdin] commit-id url";
> +
> +void NORETURN version_info()

Pretty much all the builds in https://dev.azure.com/gitgitgadget/git/_build/results?buildId=23830&view=logs&jobId=253e5128-1058-5bd4-fdf1-9b556d3207f8&j=fd490c07-0b22-5182-fac9-6d67fe1e939b&t=ce91d5d6-0c55-50f5-8ab9-6695c03ab102 fail because this function definition needs `(void)` instead of `()`.

Ciao, Dscho

Show 23 quoted lines
> +{
> +	printf("git-http-fetch version: %s\n", git_version_string);
> +	printf("built from commit: %s\n", git_built_from_commit_string);
> +	printf("curl version: %s\n", curl_version());
> +	exit(0);
> +}
>
>  int cmd_main(int argc, const char **argv)
>  {
> @@ -26,6 +35,8 @@ int cmd_main(int argc, const char **argv)
>  		} else if (argv[arg][1] == 'a') {
>  		} else if (argv[arg][1] == 'v') {
>  			get_verbosely = 1;
> +		} else if (argv[arg][1] == 'V') {
> +			version_info();
>  		} else if (argv[arg][1] == 'w') {
>  			write_ref = &argv[arg + 1];
>  			arg++;
> --
> 2.24.1.735.g03f4e72817-goog
>
>
>
Emily Shaffer· Dec 13, 2019, 00:43 UTC · re: Emily Shaffer · lore

[PATCH v4 09/15] bugreport: generate config safelist based on docs

Add a new step to the build to generate a safelist of git-config variables which are appropriate to include in the output of git-bugreport. New variables can be added to the safelist by annotating their documentation in Documentation/config with the "bugreport" macro, which is recognized by AsciiDoc and AsciiDoctor.

Some configs are private in nature, and can contain remote URLs, passwords, or other sensitive information. In the event that a user doesn't notice their information while reviewing a bugreport, that user may leak their credentials to other individuals, mailing lists, or bug tracking tools inadvertently. Heuristic blocklisting of configuration keys is imperfect and prone to false negatives; given the nature of the information which can be leaked, a safelist is more reliable.

In order to prevent staleness of the safelist, add a mechanism to generate the safelist from annotations in the config documentation, where contributors are already used to documenting their new config keys.

Implement a new no-op "bugreport" macro for use as "bugreport:include[x]" to annotate the config keys that should be included in the automatically generated safelist. Use "exclude" for the others.

With Asciidoctor, it's ok to say "bugreport:include[]", but AsciiDoc seems to want something between the brackets. A bit unfortunate, but not a huge problem -- we'll just provide an "x".

"doc-diff" reports that this macro doesn't render at all. That is, these are both empty after this commit:

  cd Documentation
  ./doc-diff --asciidoctor :/"bugreport: add tool" HEAD
  ./doc-diff --asciidoc    :/"bugreport: add tool" HEAD

Diffing the rendered HTML shows that there is some small amount of whitespace and comments added. That shouldn't be a problem.

We could perhaps let the implementation verify that the "action" is one of "include" and "exclude". For the Asciidoctor implementation that should be straightforward, but for AsciiDoc I don't immediately know how to do it. Anyway, if someone stumbles on the keyboard and writes "bugreport:icndule", they'll "only" miss out on the config key being included in the safelist. If this were a blocklist, the consequences of a misspelled target could be a lot more severe.

Additionally, add annotations to the sendemail config documentation in order to demonstrate a proof of concept.

Helped-by: Martin Ågren <martin.agren@gmail.com>
Helped-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 .gitignore                              |  1 +
 Documentation/asciidoc.conf             |  8 +++
 Documentation/asciidoctor-extensions.rb |  7 +++
 Documentation/config/sendemail.txt      | 68 ++++++++++++-------------
 Makefile                                | 14 +++--
 generate-bugreport-config-safelist.sh   | 22 ++++++++
 6 files changed, 83 insertions(+), 37 deletions(-)
 create mode 100755 generate-bugreport-config-safelist.sh
Show changes to 6 files +83 −37

.gitignore, Documentation/asciidoc.conf, Documentation/asciidoctor-extensions.rb, Documentation/config/sendemail.txt, Makefile, generate-bugreport-config-safelist.sh

diff --git a/.gitignore b/.gitignore
index 5dde2cc4c8..30935621d9 100644
--- a/.gitignore
+++ b/.gitignore
@@ -191,6 +191,7 @@
 /gitweb/static/gitweb.min.*
 /config-list.h
 /command-list.h
+/bugreport-config-safelist.h
 *.tar.gz
 *.dsc
 *.deb
diff --git a/Documentation/asciidoc.conf b/Documentation/asciidoc.conf
index 8fc4b67081..5d5359fcf9 100644
--- a/Documentation/asciidoc.conf
+++ b/Documentation/asciidoc.conf
@@ -6,9 +6,13 @@
 #
 # Show Git link as: <command>(<section>); if section is defined, else just show
 # the command.
+#
+# The bugreport macro does nothing as far as rendering is
+# concerned -- we just grep for it in the sources.
 
 [macros]
 (?su)[\\]?(?P<name>linkgit):(?P<target>\S*?)\[(?P<attrlist>.*?)\]=
+(?su)[\\]?(?P<name>bugreport):(?P<action>\S*?)\[(?P<attrlist>.*?)\]=
 
 [attributes]
 asterisk=&#42;
@@ -28,6 +32,8 @@ ifdef::backend-docbook[]
 {0#<citerefentry>}
 {0#<refentrytitle>{target}</refentrytitle><manvolnum>{0}</manvolnum>}
 {0#</citerefentry>}
+[bugreport-inlinemacro]
+{0#}
 endif::backend-docbook[]
 
 ifdef::backend-docbook[]
@@ -94,4 +100,6 @@ ifdef::backend-xhtml11[]
 git-relative-html-prefix=
 [linkgit-inlinemacro]
 <a href="{git-relative-html-prefix}{target}.html">{target}{0?({0})}</a>
+[bugreport-inlinemacro]
+<!-- -->
 endif::backend-xhtml11[]
diff --git a/Documentation/asciidoctor-extensions.rb b/Documentation/asciidoctor-extensions.rb
index d906a00803..750bdff9af 100644
--- a/Documentation/asciidoctor-extensions.rb
+++ b/Documentation/asciidoctor-extensions.rb
@@ -37,6 +37,10 @@ module Git
           output = output.sub(/<\/refmeta>/, new_tags + "</refmeta>")
         end
         output
+
+    class BugReportProcessor < Asciidoctor::Extensions::InlineMacroProcessor
+      def process(parent, action, attrs)
+        ""
       end
     end
   end
@@ -45,4 +49,7 @@ end
 Asciidoctor::Extensions.register do
   inline_macro Git::Documentation::LinkGitProcessor, :linkgit
   postprocessor Git::Documentation::DocumentPostProcessor
+  # The bugreport macro does nothing as far as rendering is
+  # concerned -- we just grep for it in the sources.
+  inline_macro Git::Documentation::BugReportProcessor, :bugreport
 end
diff --git a/Documentation/config/sendemail.txt b/Documentation/config/sendemail.txt
index 0006faf800..92f5082013 100644
--- a/Documentation/config/sendemail.txt
+++ b/Documentation/config/sendemail.txt
@@ -1,63 +1,63 @@
-sendemail.identity::
+sendemail.identity bugreport:exclude[x] ::
 	A configuration identity. When given, causes values in the
 	'sendemail.<identity>' subsection to take precedence over
 	values in the 'sendemail' section. The default identity is
 	the value of `sendemail.identity`.
 
-sendemail.smtpEncryption::
+sendemail.smtpEncryption bugreport:include[x] ::
 	See linkgit:git-send-email[1] for description.  Note that this
 	setting is not subject to the 'identity' mechanism.
 
-sendemail.smtpssl (deprecated)::
+sendemail.smtpssl (deprecated) bugreport:exclude[x] ::
 	Deprecated alias for 'sendemail.smtpEncryption = ssl'.
 
-sendemail.smtpsslcertpath::
+sendemail.smtpsslcertpath bugreport:exclude[x] ::
 	Path to ca-certificates (either a directory or a single file).
 	Set it to an empty string to disable certificate verification.
 
-sendemail.<identity>.*::
+sendemail.<identity>.* bugreport:exclude[x] ::
 	Identity-specific versions of the 'sendemail.*' parameters
 	found below, taking precedence over those when this
 	identity is selected, through either the command-line or
 	`sendemail.identity`.
 
-sendemail.aliasesFile::
-sendemail.aliasFileType::
-sendemail.annotate::
-sendemail.bcc::
-sendemail.cc::
-sendemail.ccCmd::
-sendemail.chainReplyTo::
-sendemail.confirm::
-sendemail.envelopeSender::
-sendemail.from::
-sendemail.multiEdit::
-sendemail.signedoffbycc::
-sendemail.smtpPass::
-sendemail.suppresscc::
-sendemail.suppressFrom::
-sendemail.to::
-sendemail.tocmd::
-sendemail.smtpDomain::
-sendemail.smtpServer::
-sendemail.smtpServerPort::
-sendemail.smtpServerOption::
-sendemail.smtpUser::
-sendemail.thread::
-sendemail.transferEncoding::
-sendemail.validate::
-sendemail.xmailer::
+sendemail.aliasesFile bugreport:exclude[x] ::
+sendemail.aliasFileType bugreport:exclude[x] ::
+sendemail.annotate bugreport:include[x] ::
+sendemail.bcc bugreport:include[x] ::
+sendemail.cc bugreport:include[x] ::
+sendemail.ccCmd bugreport:include[x] ::
+sendemail.chainReplyTo bugreport:include[x] ::
+sendemail.confirm bugreport:include[x] ::
+sendemail.envelopeSender bugreport:include[x] ::
+sendemail.from bugreport:include[x] ::
+sendemail.multiEdit bugreport:include[x] ::
+sendemail.signedoffbycc bugreport:include[x] ::
+sendemail.smtpPass bugreport:exclude[x] ::
+sendemail.suppresscc bugreport:include[x] ::
+sendemail.suppressFrom bugreport:include[x] ::
+sendemail.to bugreport:include[x] ::
+sendemail.tocmd bugreport:include[x] ::
+sendemail.smtpDomain bugreport:include[x] ::
+sendemail.smtpServer bugreport:include[x] ::
+sendemail.smtpServerPort bugreport:include[x] ::
+sendemail.smtpServerOption bugreport:include[x] ::
+sendemail.smtpUser bugreport:exclude[x] ::
+sendemail.thread bugreport:include[x] ::
+sendemail.transferEncoding bugreport:include[x] ::
+sendemail.validate bugreport:include[x] ::
+sendemail.xmailer bugreport:include[x] ::
 	See linkgit:git-send-email[1] for description.
 
-sendemail.signedoffcc (deprecated)::
+sendemail.signedoffcc (deprecated) bugreport:exclude[x] ::
 	Deprecated alias for `sendemail.signedoffbycc`.
 
-sendemail.smtpBatchSize::
+sendemail.smtpBatchSize bugreport:include[x] ::
 	Number of messages to be sent per connection, after that a relogin
 	will happen.  If the value is 0 or undefined, send all messages in
 	one connection.
 	See also the `--batch-size` option of linkgit:git-send-email[1].
 
-sendemail.smtpReloginDelay::
+sendemail.smtpReloginDelay bugreport:include[x] ::
 	Seconds wait before reconnecting to smtp server.
 	See also the `--relogin-delay` option of linkgit:git-send-email[1].
diff --git a/Makefile b/Makefile
index c49f55a521..76dc51e2b1 100644
--- a/Makefile
+++ b/Makefile
@@ -651,7 +651,7 @@ install-perl-script: $(SCRIPT_PERL_GEN)
 install-python-script: $(SCRIPT_PYTHON_GEN)
 	$(INSTALL) $^ '$(DESTDIR_SQ)$(gitexec_instdir_SQ)'
 
-.PHONY: clean-perl-script clean-sh-script clean-python-script
+.PHONY: clean-perl-script clean-sh-script clean-python-script clean-script-dependencies
 clean-sh-script:
 	$(RM) $(SCRIPT_SH_GEN)
 clean-perl-script:
@@ -817,6 +817,7 @@ VCSSVN_LIB = vcs-svn/lib.a
 
 GENERATED_H += config-list.h
 GENERATED_H += command-list.h
+GENERATED_H += bugreport-config-safelist.h
 
 LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
 	$(FIND) . \
@@ -2161,6 +2162,12 @@ command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Doc
 		$(patsubst %,--exclude-program %,$(EXCLUDED_PROGRAMS)) \
 		command-list.txt >$@+ && mv $@+ $@
 
+bugreport-config-safelist.h: generate-bugreport-config-safelist.sh
+
+bugreport-config-safelist.h: Documentation/config/*.txt
+	$(QUIET_GEN)$(SHELL_PATH) ./generate-bugreport-config-safelist.sh \
+		>$@+ && mv $@+ $@
+
 SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):$(GIT_VERSION):\
 	$(localedir_SQ):$(NO_CURL):$(USE_GETTEXT_SCHEME):$(SANE_TOOL_PATH_SQ):\
 	$(gitwebdir_SQ):$(PERL_PATH_SQ):$(SANE_TEXT_GREP):$(PAGER_ENV):\
@@ -2791,7 +2798,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE
 .PHONY: sparse $(SP_OBJ)
 sparse: $(SP_OBJ)
 
-GEN_HDRS := config-list.h command-list.h unicode-width.h
+GEN_HDRS := config-list.h command-list.h unicode-width.h bugreport-config-safelist.h
 EXCEPT_HDRS := $(GEN_HDRS) compat/% xdiff/%
 ifndef GCRYPT_SHA256
 	EXCEPT_HDRS += sha256/gcrypt.h
@@ -3117,7 +3124,8 @@ clean: profile-clean coverage-clean cocciclean
 	$(RM) $(HCC)
 	$(RM) -r bin-wrappers $(dep_dirs)
 	$(RM) -r po/build/
-	$(RM) *.pyc *.pyo */*.pyc */*.pyo config-list.h command-list.h
+	$(RM) *.pyc *.pyo */*.pyc */*.pyo
+	$(RM) config-list.h command-list.h bugreport-config-safelist.h
 	$(RM) $(ETAGS_TARGET) tags cscope*
 	$(RM) -r $(GIT_TARNAME) .doc-tmp-dir
 	$(RM) $(GIT_TARNAME).tar.gz git-core_$(GIT_VERSION)-*.tar.gz
diff --git a/generate-bugreport-config-safelist.sh b/generate-bugreport-config-safelist.sh
new file mode 100755
index 0000000000..06b8e0c3c4
--- /dev/null
+++ b/generate-bugreport-config-safelist.sh
@@ -0,0 +1,22 @@
+#!/bin/sh
+
+cat <<EOF
+/* Automatically generated by bugreport-generate-config-safelist.sh */
+
+
+static const char *bugreport_config_safelist[] = {
+EOF
+
+# cat all regular files in Documentation/config
+find Documentation/config -type f -exec cat {} \; |
+# print the command name which matches the bugreport-include macro
+sed -n 's/^\(.*\) \+bugreport:include.* ::$/\1/p' |
+sort |
+while read line
+do
+	echo "	\"$line\","
+done
+
+cat <<EOF
+};
+EOF
-- 
2.24.1.735.g03f4e72817-goog
Junio C Hamano· Dec 13, 2019, 22:57 UTC · re: Emily Shaffer · lore

Re: [PATCH v4 09/15] bugreport: generate config safelist based on docs

Emily Shaffer <emilyshaffer@google.com> writes:
Show 18 quoted lines
> diff --git a/Makefile b/Makefile
> index c49f55a521..76dc51e2b1 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -651,7 +651,7 @@ install-perl-script: $(SCRIPT_PERL_GEN)
>  install-python-script: $(SCRIPT_PYTHON_GEN)
>  	$(INSTALL) $^ '$(DESTDIR_SQ)$(gitexec_instdir_SQ)'
>  
> -.PHONY: clean-perl-script clean-sh-script clean-python-script
> +.PHONY: clean-perl-script clean-sh-script clean-python-script clean-script-dependencies
>  clean-sh-script:
>  	$(RM) $(SCRIPT_SH_GEN)
>  clean-perl-script:
> @@ -817,6 +817,7 @@ VCSSVN_LIB = vcs-svn/lib.a
>  
>  GENERATED_H += config-list.h
>  GENERATED_H += command-list.h
> +GENERATED_H += bugreport-config-safelist.h
OK.
Show 11 quoted lines
>  LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
>  	$(FIND) . \
> @@ -2161,6 +2162,12 @@ command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Doc
>  		$(patsubst %,--exclude-program %,$(EXCLUDED_PROGRAMS)) \
>  		command-list.txt >$@+ && mv $@+ $@
>  
> +bugreport-config-safelist.h: generate-bugreport-config-safelist.sh
> +
> +bugreport-config-safelist.h: Documentation/config/*.txt
> +	$(QUIET_GEN)$(SHELL_PATH) ./generate-bugreport-config-safelist.sh \
> +		>$@+ && mv $@+ $@
OK.
>  SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):$(GIT_VERSION):\
>  	$(localedir_SQ):$(NO_CURL):$(USE_GETTEXT_SCHEME):$(SANE_TOOL_PATH_SQ):\
>  	$(gitwebdir_SQ):$(PERL_PATH_SQ):$(SANE_TEXT_GREP):$(PAGER_ENV):\

But bugreport.o needs this *.h file generated before a compiler attempts to produce it out of bugreport.c; that dependency is missing from this update to the Makefile.

Show 16 quoted lines
> @@ -2791,7 +2798,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE
>  .PHONY: sparse $(SP_OBJ)
>  sparse: $(SP_OBJ)
>  
> -GEN_HDRS := config-list.h command-list.h unicode-width.h
> +GEN_HDRS := config-list.h command-list.h unicode-width.h bugreport-config-safelist.h
>  EXCEPT_HDRS := $(GEN_HDRS) compat/% xdiff/%
>  ifndef GCRYPT_SHA256
>  	EXCEPT_HDRS += sha256/gcrypt.h
> @@ -3117,7 +3124,8 @@ clean: profile-clean coverage-clean cocciclean
>  	$(RM) $(HCC)
>  	$(RM) -r bin-wrappers $(dep_dirs)
>  	$(RM) -r po/build/
> -	$(RM) *.pyc *.pyo */*.pyc */*.pyo config-list.h command-list.h
> +	$(RM) *.pyc *.pyo */*.pyc */*.pyo
> +	$(RM) config-list.h command-list.h bugreport-config-safelist.h

It is kind of sad that GEN_HDRS defines the list of build artifact that we should be able to clean, and then we manually list them to be removed here independently. Can we fix it up too?

We probably clean up the build/update procedure around unicode-width.h before we can do so, probably. It may be generatable using contrib/ script, but as far as our normal build is concerned, it is a tracked source and not part of the build byproducts, so we probably would want to remove it from GEN_HDRS. When that happens, we can $(RM) all of the $(GEN_HDRS) in the "clean" target.

Emily Shaffer· Dec 16, 2019, 23:01 UTC · re: Junio C Hamano · lore

Re: [PATCH v4 09/15] bugreport: generate config safelist based on docs

On Fri, Dec 13, 2019 at 02:57:17PM -0800, Junio C Hamano wrote:
Show 44 quoted lines
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> > diff --git a/Makefile b/Makefile
> > index c49f55a521..76dc51e2b1 100644
> > --- a/Makefile
> > +++ b/Makefile
> > @@ -651,7 +651,7 @@ install-perl-script: $(SCRIPT_PERL_GEN)
> >  install-python-script: $(SCRIPT_PYTHON_GEN)
> >  	$(INSTALL) $^ '$(DESTDIR_SQ)$(gitexec_instdir_SQ)'
> >  
> > -.PHONY: clean-perl-script clean-sh-script clean-python-script
> > +.PHONY: clean-perl-script clean-sh-script clean-python-script clean-script-dependencies
> >  clean-sh-script:
> >  	$(RM) $(SCRIPT_SH_GEN)
> >  clean-perl-script:
> > @@ -817,6 +817,7 @@ VCSSVN_LIB = vcs-svn/lib.a
> >  
> >  GENERATED_H += config-list.h
> >  GENERATED_H += command-list.h
> > +GENERATED_H += bugreport-config-safelist.h
> 
> OK.
> 
> >  LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
> >  	$(FIND) . \
> > @@ -2161,6 +2162,12 @@ command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Doc
> >  		$(patsubst %,--exclude-program %,$(EXCLUDED_PROGRAMS)) \
> >  		command-list.txt >$@+ && mv $@+ $@
> >  
> > +bugreport-config-safelist.h: generate-bugreport-config-safelist.sh
> > +
> > +bugreport-config-safelist.h: Documentation/config/*.txt
> > +	$(QUIET_GEN)$(SHELL_PATH) ./generate-bugreport-config-safelist.sh \
> > +		>$@+ && mv $@+ $@
> 
> OK.
> 
> >  SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):$(GIT_VERSION):\
> >  	$(localedir_SQ):$(NO_CURL):$(USE_GETTEXT_SCHEME):$(SANE_TOOL_PATH_SQ):\
> >  	$(gitwebdir_SQ):$(PERL_PATH_SQ):$(SANE_TEXT_GREP):$(PAGER_ENV):\
> 
> But bugreport.o needs this *.h file generated before a compiler
> attempts to produce it out of bugreport.c; that dependency is
> missing from this update to the Makefile.

Hm. Somewhere I misformatted my commits, then. My goal was to add the new header here, but not add it as a dependency to anybody; then, in the next commit, add it as a dependency to git-bugreport and start to use it (commit 1: make the thing; commit 2: use the thing). But now when I look at the next commit, I don't see a Makefile change. So I missed something while I was shuffling around fixups. Thanks for noticing.

Show 21 quoted lines
> 
> > @@ -2791,7 +2798,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE
> >  .PHONY: sparse $(SP_OBJ)
> >  sparse: $(SP_OBJ)
> >  
> > -GEN_HDRS := config-list.h command-list.h unicode-width.h
> > +GEN_HDRS := config-list.h command-list.h unicode-width.h bugreport-config-safelist.h
> >  EXCEPT_HDRS := $(GEN_HDRS) compat/% xdiff/%
> >  ifndef GCRYPT_SHA256
> >  	EXCEPT_HDRS += sha256/gcrypt.h
> > @@ -3117,7 +3124,8 @@ clean: profile-clean coverage-clean cocciclean
> >  	$(RM) $(HCC)
> >  	$(RM) -r bin-wrappers $(dep_dirs)
> >  	$(RM) -r po/build/
> > -	$(RM) *.pyc *.pyo */*.pyc */*.pyo config-list.h command-list.h
> > +	$(RM) *.pyc *.pyo */*.pyc */*.pyo
> > +	$(RM) config-list.h command-list.h bugreport-config-safelist.h
> 
> It is kind of sad that GEN_HDRS defines the list of build artifact
> that we should be able to clean, and then we manually list them to
> be removed here independently.  Can we fix it up too?
Yeah, I can do that. I thought so too when I was updating this.

I *think* what happened is that when I split config-list away (in the earleir commit in this chain) I didn't notice that command-list.h was already in GEN_HDRS, and instead just grepped Makefile for "command-list.h" and added config-list.h next to it. So I'll try to fix it much earlier, in that commit, and then simply add to the appropriate variables in this commit.

Show 7 quoted lines
> 
> We probably clean up the build/update procedure around unicode-width.h
> before we can do so, probably.  It may be generatable using contrib/
> script, but as far as our normal build is concerned, it is a tracked
> source and not part of the build byproducts, so we probably would
> want to remove it from GEN_HDRS.  When that happens, we can $(RM)
> all of the $(GEN_HDRS) in the "clean" target.
Noted. Thanks.
 - Emily
Emily Shaffer· Dec 17, 2019, 00:41 UTC · re: Emily Shaffer · lore

Re: [PATCH v4 09/15] bugreport: generate config safelist based on docs

On Mon, Dec 16, 2019 at 03:01:24PM -0800, Emily Shaffer wrote:
Show 93 quoted lines
> On Fri, Dec 13, 2019 at 02:57:17PM -0800, Junio C Hamano wrote:
> > Emily Shaffer <emilyshaffer@google.com> writes:
> > 
> > > diff --git a/Makefile b/Makefile
> > > index c49f55a521..76dc51e2b1 100644
> > > --- a/Makefile
> > > +++ b/Makefile
> > > @@ -651,7 +651,7 @@ install-perl-script: $(SCRIPT_PERL_GEN)
> > >  install-python-script: $(SCRIPT_PYTHON_GEN)
> > >  	$(INSTALL) $^ '$(DESTDIR_SQ)$(gitexec_instdir_SQ)'
> > >  
> > > -.PHONY: clean-perl-script clean-sh-script clean-python-script
> > > +.PHONY: clean-perl-script clean-sh-script clean-python-script clean-script-dependencies
> > >  clean-sh-script:
> > >  	$(RM) $(SCRIPT_SH_GEN)
> > >  clean-perl-script:
> > > @@ -817,6 +817,7 @@ VCSSVN_LIB = vcs-svn/lib.a
> > >  
> > >  GENERATED_H += config-list.h
> > >  GENERATED_H += command-list.h
> > > +GENERATED_H += bugreport-config-safelist.h
> > 
> > OK.
> > 
> > >  LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
> > >  	$(FIND) . \
> > > @@ -2161,6 +2162,12 @@ command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Doc
> > >  		$(patsubst %,--exclude-program %,$(EXCLUDED_PROGRAMS)) \
> > >  		command-list.txt >$@+ && mv $@+ $@
> > >  
> > > +bugreport-config-safelist.h: generate-bugreport-config-safelist.sh
> > > +
> > > +bugreport-config-safelist.h: Documentation/config/*.txt
> > > +	$(QUIET_GEN)$(SHELL_PATH) ./generate-bugreport-config-safelist.sh \
> > > +		>$@+ && mv $@+ $@
> > 
> > OK.
> > 
> > >  SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):$(GIT_VERSION):\
> > >  	$(localedir_SQ):$(NO_CURL):$(USE_GETTEXT_SCHEME):$(SANE_TOOL_PATH_SQ):\
> > >  	$(gitwebdir_SQ):$(PERL_PATH_SQ):$(SANE_TEXT_GREP):$(PAGER_ENV):\
> > 
> > But bugreport.o needs this *.h file generated before a compiler
> > attempts to produce it out of bugreport.c; that dependency is
> > missing from this update to the Makefile.
> 
> Hm. Somewhere I misformatted my commits, then. My goal was to add the
> new header here, but not add it as a dependency to anybody; then, in the
> next commit, add it as a dependency to git-bugreport and start to use it
> (commit 1: make the thing; commit 2: use the thing). But now when I look
> at the next commit, I don't see a Makefile change. So I missed something
> while I was shuffling around fixups.  Thanks for noticing.
> 
> > 
> > > @@ -2791,7 +2798,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE
> > >  .PHONY: sparse $(SP_OBJ)
> > >  sparse: $(SP_OBJ)
> > >  
> > > -GEN_HDRS := config-list.h command-list.h unicode-width.h
> > > +GEN_HDRS := config-list.h command-list.h unicode-width.h bugreport-config-safelist.h
> > >  EXCEPT_HDRS := $(GEN_HDRS) compat/% xdiff/%
> > >  ifndef GCRYPT_SHA256
> > >  	EXCEPT_HDRS += sha256/gcrypt.h
> > > @@ -3117,7 +3124,8 @@ clean: profile-clean coverage-clean cocciclean
> > >  	$(RM) $(HCC)
> > >  	$(RM) -r bin-wrappers $(dep_dirs)
> > >  	$(RM) -r po/build/
> > > -	$(RM) *.pyc *.pyo */*.pyc */*.pyo config-list.h command-list.h
> > > +	$(RM) *.pyc *.pyo */*.pyc */*.pyo
> > > +	$(RM) config-list.h command-list.h bugreport-config-safelist.h
> > 
> > It is kind of sad that GEN_HDRS defines the list of build artifact
> > that we should be able to clean, and then we manually list them to
> > be removed here independently.  Can we fix it up too?
> 
> Yeah, I can do that. I thought so too when I was updating this.
> 
> I *think* what happened is that when I split config-list away (in the
> earleir commit in this chain) I didn't notice that command-list.h was
> already in GEN_HDRS, and instead just grepped Makefile for
> "command-list.h" and added config-list.h next to it. So I'll try to fix
> it much earlier, in that commit, and then simply add to the appropriate
> variables in this commit.
> 
> > 
> > We probably clean up the build/update procedure around unicode-width.h
> > before we can do so, probably.  It may be generatable using contrib/
> > script, but as far as our normal build is concerned, it is a tracked
> > source and not part of the build byproducts, so we probably would
> > want to remove it from GEN_HDRS.  When that happens, we can $(RM)
> > all of the $(GEN_HDRS) in the "clean" target.
> 
> Noted. Thanks.

Ah, after I sent this mail I saw your patch. For those playing along at home, https://lore.kernel.org/xmqq1rt7hkp6.fsf@gitster-ct.c.googlers.com

Will review.
 - Emily
Johannes Schindelin· Dec 15, 2019, 20:17 UTC · re: Emily Shaffer · lore

Re: [PATCH v4 09/15] bugreport: generate config safelist based on docs

Hi Emily,
On Thu, 12 Dec 2019, Emily Shaffer wrote:
Show 19 quoted lines
> diff --git a/generate-bugreport-config-safelist.sh b/generate-bugreport-config-safelist.sh
> new file mode 100755
> index 0000000000..06b8e0c3c4
> --- /dev/null
> +++ b/generate-bugreport-config-safelist.sh
> @@ -0,0 +1,22 @@
> +#!/bin/sh
> +
> +cat <<EOF
> +/* Automatically generated by bugreport-generate-config-safelist.sh */
> +
> +
> +static const char *bugreport_config_safelist[] = {
> +EOF
> +
> +# cat all regular files in Documentation/config
> +find Documentation/config -type f -exec cat {} \; |
> +# print the command name which matches the bugreport-include macro
> +sed -n 's/^\(.*\) \+bugreport:include.* ::$/\1/p' |

If you use `/ "\1",/` as replacement, you can totally avoid that ugly `while` loop (that is unfortunately quite slow in MSYS2/Cygwin). You can still pipe the result into `sort` just the same.

Ciao, Dscho

Show 13 quoted lines
> +sort |
> +while read line
> +do
> +	echo "	\"$line\","
> +done
> +
> +cat <<EOF
> +};
> +EOF
> --
> 2.24.1.735.g03f4e72817-goog
>
>
Emily Shaffer· Dec 16, 2019, 22:52 UTC · re: Johannes Schindelin · lore

Re: [PATCH v4 09/15] bugreport: generate config safelist based on docs

On Sun, Dec 15, 2019 at 09:17:57PM +0100, Johannes Schindelin wrote:
Show 27 quoted lines
> Hi Emily,
> 
> On Thu, 12 Dec 2019, Emily Shaffer wrote:
> 
> > diff --git a/generate-bugreport-config-safelist.sh b/generate-bugreport-config-safelist.sh
> > new file mode 100755
> > index 0000000000..06b8e0c3c4
> > --- /dev/null
> > +++ b/generate-bugreport-config-safelist.sh
> > @@ -0,0 +1,22 @@
> > +#!/bin/sh
> > +
> > +cat <<EOF
> > +/* Automatically generated by bugreport-generate-config-safelist.sh */
> > +
> > +
> > +static const char *bugreport_config_safelist[] = {
> > +EOF
> > +
> > +# cat all regular files in Documentation/config
> > +find Documentation/config -type f -exec cat {} \; |
> > +# print the command name which matches the bugreport-include macro
> > +sed -n 's/^\(.*\) \+bugreport:include.* ::$/\1/p' |
> 
> If you use `/  "\1",/` as replacement, you can totally avoid that ugly
> `while` loop (that is unfortunately quite slow in MSYS2/Cygwin). You can
> still pipe the result into `sort` just the same.
Oooof. How embarrassing - I will do that.
Johannes Schindelin· Dec 17, 2019, 18:38 UTC · re: Emily Shaffer · lore

Re: [PATCH v4 09/15] bugreport: generate config safelist based on docs

Hi Emily,
On Thu, 12 Dec 2019, Emily Shaffer wrote:
Show 12 quoted lines
> diff --git a/Documentation/asciidoctor-extensions.rb b/Documentation/asciidoctor-extensions.rb
> index d906a00803..750bdff9af 100644
> --- a/Documentation/asciidoctor-extensions.rb
> +++ b/Documentation/asciidoctor-extensions.rb
> @@ -37,6 +37,10 @@ module Git
>            output = output.sub(/<\/refmeta>/, new_tags + "</refmeta>")
>          end
>          output
> +
> +    class BugReportProcessor < Asciidoctor::Extensions::InlineMacroProcessor
> +      def process(parent, action, attrs)
> +        ""
The Azure Pipeline fails on this hunk, saying:

/usr/bin/asciidoctor: /home/vsts/work/1/s/Documentation/asciidoctor-extensions.rb:41: class definition in method body (SyntaxError) /home/vsts/work/1/s/Documentation/asciidoctor-extensions.rb:55: syntax error, unexpected end-of-input, expecting keyword_end

See https://dev.azure.com/gitgitgadget/git/_build/results?buildId=23830&view=logs&jobId=253e5128-1058-5bd4-fdf1-9b556d3207f8&j=253e5128-1058-5bd4-fdf1-9b556d3207f8&t=95c65aec-3627-54dc-fc17-47625f123bb3 for details.

Ciao, Dscho

Show 199 quoted lines
>        end
>      end
>    end
> @@ -45,4 +49,7 @@ end
>  Asciidoctor::Extensions.register do
>    inline_macro Git::Documentation::LinkGitProcessor, :linkgit
>    postprocessor Git::Documentation::DocumentPostProcessor
> +  # The bugreport macro does nothing as far as rendering is
> +  # concerned -- we just grep for it in the sources.
> +  inline_macro Git::Documentation::BugReportProcessor, :bugreport
>  end
> diff --git a/Documentation/config/sendemail.txt b/Documentation/config/sendemail.txt
> index 0006faf800..92f5082013 100644
> --- a/Documentation/config/sendemail.txt
> +++ b/Documentation/config/sendemail.txt
> @@ -1,63 +1,63 @@
> -sendemail.identity::
> +sendemail.identity bugreport:exclude[x] ::
>  	A configuration identity. When given, causes values in the
>  	'sendemail.<identity>' subsection to take precedence over
>  	values in the 'sendemail' section. The default identity is
>  	the value of `sendemail.identity`.
>
> -sendemail.smtpEncryption::
> +sendemail.smtpEncryption bugreport:include[x] ::
>  	See linkgit:git-send-email[1] for description.  Note that this
>  	setting is not subject to the 'identity' mechanism.
>
> -sendemail.smtpssl (deprecated)::
> +sendemail.smtpssl (deprecated) bugreport:exclude[x] ::
>  	Deprecated alias for 'sendemail.smtpEncryption = ssl'.
>
> -sendemail.smtpsslcertpath::
> +sendemail.smtpsslcertpath bugreport:exclude[x] ::
>  	Path to ca-certificates (either a directory or a single file).
>  	Set it to an empty string to disable certificate verification.
>
> -sendemail.<identity>.*::
> +sendemail.<identity>.* bugreport:exclude[x] ::
>  	Identity-specific versions of the 'sendemail.*' parameters
>  	found below, taking precedence over those when this
>  	identity is selected, through either the command-line or
>  	`sendemail.identity`.
>
> -sendemail.aliasesFile::
> -sendemail.aliasFileType::
> -sendemail.annotate::
> -sendemail.bcc::
> -sendemail.cc::
> -sendemail.ccCmd::
> -sendemail.chainReplyTo::
> -sendemail.confirm::
> -sendemail.envelopeSender::
> -sendemail.from::
> -sendemail.multiEdit::
> -sendemail.signedoffbycc::
> -sendemail.smtpPass::
> -sendemail.suppresscc::
> -sendemail.suppressFrom::
> -sendemail.to::
> -sendemail.tocmd::
> -sendemail.smtpDomain::
> -sendemail.smtpServer::
> -sendemail.smtpServerPort::
> -sendemail.smtpServerOption::
> -sendemail.smtpUser::
> -sendemail.thread::
> -sendemail.transferEncoding::
> -sendemail.validate::
> -sendemail.xmailer::
> +sendemail.aliasesFile bugreport:exclude[x] ::
> +sendemail.aliasFileType bugreport:exclude[x] ::
> +sendemail.annotate bugreport:include[x] ::
> +sendemail.bcc bugreport:include[x] ::
> +sendemail.cc bugreport:include[x] ::
> +sendemail.ccCmd bugreport:include[x] ::
> +sendemail.chainReplyTo bugreport:include[x] ::
> +sendemail.confirm bugreport:include[x] ::
> +sendemail.envelopeSender bugreport:include[x] ::
> +sendemail.from bugreport:include[x] ::
> +sendemail.multiEdit bugreport:include[x] ::
> +sendemail.signedoffbycc bugreport:include[x] ::
> +sendemail.smtpPass bugreport:exclude[x] ::
> +sendemail.suppresscc bugreport:include[x] ::
> +sendemail.suppressFrom bugreport:include[x] ::
> +sendemail.to bugreport:include[x] ::
> +sendemail.tocmd bugreport:include[x] ::
> +sendemail.smtpDomain bugreport:include[x] ::
> +sendemail.smtpServer bugreport:include[x] ::
> +sendemail.smtpServerPort bugreport:include[x] ::
> +sendemail.smtpServerOption bugreport:include[x] ::
> +sendemail.smtpUser bugreport:exclude[x] ::
> +sendemail.thread bugreport:include[x] ::
> +sendemail.transferEncoding bugreport:include[x] ::
> +sendemail.validate bugreport:include[x] ::
> +sendemail.xmailer bugreport:include[x] ::
>  	See linkgit:git-send-email[1] for description.
>
> -sendemail.signedoffcc (deprecated)::
> +sendemail.signedoffcc (deprecated) bugreport:exclude[x] ::
>  	Deprecated alias for `sendemail.signedoffbycc`.
>
> -sendemail.smtpBatchSize::
> +sendemail.smtpBatchSize bugreport:include[x] ::
>  	Number of messages to be sent per connection, after that a relogin
>  	will happen.  If the value is 0 or undefined, send all messages in
>  	one connection.
>  	See also the `--batch-size` option of linkgit:git-send-email[1].
>
> -sendemail.smtpReloginDelay::
> +sendemail.smtpReloginDelay bugreport:include[x] ::
>  	Seconds wait before reconnecting to smtp server.
>  	See also the `--relogin-delay` option of linkgit:git-send-email[1].
> diff --git a/Makefile b/Makefile
> index c49f55a521..76dc51e2b1 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -651,7 +651,7 @@ install-perl-script: $(SCRIPT_PERL_GEN)
>  install-python-script: $(SCRIPT_PYTHON_GEN)
>  	$(INSTALL) $^ '$(DESTDIR_SQ)$(gitexec_instdir_SQ)'
>
> -.PHONY: clean-perl-script clean-sh-script clean-python-script
> +.PHONY: clean-perl-script clean-sh-script clean-python-script clean-script-dependencies
>  clean-sh-script:
>  	$(RM) $(SCRIPT_SH_GEN)
>  clean-perl-script:
> @@ -817,6 +817,7 @@ VCSSVN_LIB = vcs-svn/lib.a
>
>  GENERATED_H += config-list.h
>  GENERATED_H += command-list.h
> +GENERATED_H += bugreport-config-safelist.h
>
>  LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
>  	$(FIND) . \
> @@ -2161,6 +2162,12 @@ command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Doc
>  		$(patsubst %,--exclude-program %,$(EXCLUDED_PROGRAMS)) \
>  		command-list.txt >$@+ && mv $@+ $@
>
> +bugreport-config-safelist.h: generate-bugreport-config-safelist.sh
> +
> +bugreport-config-safelist.h: Documentation/config/*.txt
> +	$(QUIET_GEN)$(SHELL_PATH) ./generate-bugreport-config-safelist.sh \
> +		>$@+ && mv $@+ $@
> +
>  SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):$(GIT_VERSION):\
>  	$(localedir_SQ):$(NO_CURL):$(USE_GETTEXT_SCHEME):$(SANE_TOOL_PATH_SQ):\
>  	$(gitwebdir_SQ):$(PERL_PATH_SQ):$(SANE_TEXT_GREP):$(PAGER_ENV):\
> @@ -2791,7 +2798,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE
>  .PHONY: sparse $(SP_OBJ)
>  sparse: $(SP_OBJ)
>
> -GEN_HDRS := config-list.h command-list.h unicode-width.h
> +GEN_HDRS := config-list.h command-list.h unicode-width.h bugreport-config-safelist.h
>  EXCEPT_HDRS := $(GEN_HDRS) compat/% xdiff/%
>  ifndef GCRYPT_SHA256
>  	EXCEPT_HDRS += sha256/gcrypt.h
> @@ -3117,7 +3124,8 @@ clean: profile-clean coverage-clean cocciclean
>  	$(RM) $(HCC)
>  	$(RM) -r bin-wrappers $(dep_dirs)
>  	$(RM) -r po/build/
> -	$(RM) *.pyc *.pyo */*.pyc */*.pyo config-list.h command-list.h
> +	$(RM) *.pyc *.pyo */*.pyc */*.pyo
> +	$(RM) config-list.h command-list.h bugreport-config-safelist.h
>  	$(RM) $(ETAGS_TARGET) tags cscope*
>  	$(RM) -r $(GIT_TARNAME) .doc-tmp-dir
>  	$(RM) $(GIT_TARNAME).tar.gz git-core_$(GIT_VERSION)-*.tar.gz
> diff --git a/generate-bugreport-config-safelist.sh b/generate-bugreport-config-safelist.sh
> new file mode 100755
> index 0000000000..06b8e0c3c4
> --- /dev/null
> +++ b/generate-bugreport-config-safelist.sh
> @@ -0,0 +1,22 @@
> +#!/bin/sh
> +
> +cat <<EOF
> +/* Automatically generated by bugreport-generate-config-safelist.sh */
> +
> +
> +static const char *bugreport_config_safelist[] = {
> +EOF
> +
> +# cat all regular files in Documentation/config
> +find Documentation/config -type f -exec cat {} \; |
> +# print the command name which matches the bugreport-include macro
> +sed -n 's/^\(.*\) \+bugreport:include.* ::$/\1/p' |
> +sort |
> +while read line
> +do
> +	echo "	\"$line\","
> +done
> +
> +cat <<EOF
> +};
> +EOF
> --
> 2.24.1.735.g03f4e72817-goog
>
>
>
Emily Shaffer· Dec 13, 2019, 00:43 UTC · re: Emily Shaffer · lore

[PATCH v4 10/15] bugreport: add config values from safelist

Teach bugreport to gather the values of config options which are present in 'bugreport-config-safelist.h'.

Many config options are sensitive, and many Git add-ons use config options which git-core does not know about; it is better only to gather config options which we know to be safe, rather than excluding options which we know to be unsafe.

Taking the build-time generated array and putting it into a set saves us time - since git_config_bugreport() is called for every option the user has configured, performing option lookup in constant time is a useful optimization.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 bugreport.c | 72 ++++++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 71 insertions(+), 1 deletion(-)
Show changes to bugreport.c +71 −1
diff --git a/bugreport.c b/bugreport.c
index 759cc0b0f8..1fca28f0b9 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -6,6 +6,9 @@
 #include "help.h"
 #include <gnu/libc-version.h>
 #include "run-command.h"
+#include "config.h"
+#include "bugreport-config-safelist.h"
+#include "khash.h"
 
 static void get_http_version_info(struct strbuf *http_info)
 {
@@ -18,6 +21,41 @@ static void get_http_version_info(struct strbuf *http_info)
 	    strbuf_addstr(http_info, "'git-http-fetch -V' not supported\n");
 }
 
+KHASH_INIT(cfg_set, const char*, int, 0, kh_str_hash_func, kh_str_hash_equal);
+
+struct cfgset {
+	kh_cfg_set_t set;
+};
+
+struct cfgset safelist;
+
+static void cfgset_init(struct cfgset *set, size_t initial_size)
+{
+	memset(&set->set, 0, sizeof(set->set));
+	if (initial_size)
+		kh_resize_cfg_set(&set->set, initial_size);
+}
+
+static int cfgset_insert(struct cfgset *set, const char *cfg_key)
+{
+	int added;
+	kh_put_cfg_set(&set->set, cfg_key, &added);
+	printf("ESS: added %s\n", cfg_key);
+	return !added;
+}
+
+static int cfgset_contains(struct cfgset *set, const char *cfg_key)
+{
+	khiter_t pos = kh_get_cfg_set(&set->set, cfg_key);
+	return pos != kh_end(&set->set);
+}
+
+static void cfgset_clear(struct cfgset *set)
+{
+	kh_release_cfg_set(&set->set);
+	cfgset_init(set, 0);
+}
+
 static void get_system_info(struct strbuf *sys_info)
 {
 	struct strbuf version_info = STRBUF_INIT;
@@ -53,6 +91,36 @@ static void get_system_info(struct strbuf *sys_info)
 	strbuf_complete_line(sys_info);
 }
 
+static void gather_safelist()
+{
+	int index;
+	int safelist_len = sizeof(bugreport_config_safelist) / sizeof(const char *);
+	cfgset_init(&safelist, safelist_len);
+	for (index = 0; index < safelist_len; index++)
+		cfgset_insert(&safelist, bugreport_config_safelist[index]);
+
+}
+
+static int git_config_bugreport(const char *var, const char *value, void *cb)
+{
+	struct strbuf *config_info = (struct strbuf *)cb;
+
+	if (cfgset_contains(&safelist, var))
+		strbuf_addf(config_info,
+			    "%s (%s) : %s\n",
+			    var, config_scope_to_string(current_config_scope()),
+			    value);
+
+	return 0;
+}
+
+static void get_safelisted_config(struct strbuf *config_info)
+{
+	gather_safelist();
+	git_config(git_config_bugreport, config_info);
+	cfgset_clear(&safelist);
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output <file>]"),
 	NULL
@@ -114,10 +182,12 @@ int cmd_main(int argc, const char **argv)
 
 	get_bug_template(&buffer);
 
-	// add other contents
 	get_header(&buffer, "System Info");
 	get_system_info(&buffer);
 
+	get_header(&buffer, "Safelisted Config Info");
+	get_safelisted_config(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 	strbuf_write(&buffer, report);
 	fclose(report);
-- 
2.24.1.735.g03f4e72817-goog
Junio C Hamano· Dec 13, 2019, 21:45 UTC · re: Emily Shaffer · lore

Re: [PATCH v4 10/15] bugreport: add config values from safelist

Emily Shaffer <emilyshaffer@google.com> writes:
Show 12 quoted lines
> Teach bugreport to gather the values of config options which are present
> in 'bugreport-config-safelist.h'.
>
> Many config options are sensitive, and many Git add-ons use config
> options which git-core does not know about; it is better only to gather
> config options which we know to be safe, rather than excluding options
> which we know to be unsafe.
>
> Taking the build-time generated array and putting it into a set saves us
> time - since git_config_bugreport() is called for every option the user
> has configured, performing option lookup in constant time is a useful
> optimization.

Interesting. I actually was expecting the look-up to go the other way around. We know the safe keys, so iterate over them and grab their values, if defined. No need for hashes or anything, but just a simple linear list of safe stuff.

But that is too simple-minded. If we wanted to safelist foo.*.bar, where '*' can be anything, walking on the list of safe variables would not work. We must have a hash table that knows "foo.*.bar" is allowed, and while walking all the configuration keys, when we see foo.a.bar, we consult "foo.*.bar" as well as "foo.a.bar" to see if it is whitelisted, or something like that.

But then I am not sure if this implementation does something like this for three-level names. If not, I do not see much point in use of the hash there either.

Puzzled.
Show 113 quoted lines
>
> Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
> ---
>  bugreport.c | 72 ++++++++++++++++++++++++++++++++++++++++++++++++++++-
>  1 file changed, 71 insertions(+), 1 deletion(-)
>
> diff --git a/bugreport.c b/bugreport.c
> index 759cc0b0f8..1fca28f0b9 100644
> --- a/bugreport.c
> +++ b/bugreport.c
> @@ -6,6 +6,9 @@
>  #include "help.h"
>  #include <gnu/libc-version.h>
>  #include "run-command.h"
> +#include "config.h"
> +#include "bugreport-config-safelist.h"
> +#include "khash.h"
>  
>  static void get_http_version_info(struct strbuf *http_info)
>  {
> @@ -18,6 +21,41 @@ static void get_http_version_info(struct strbuf *http_info)
>  	    strbuf_addstr(http_info, "'git-http-fetch -V' not supported\n");
>  }
>  
> +KHASH_INIT(cfg_set, const char*, int, 0, kh_str_hash_func, kh_str_hash_equal);
> +
> +struct cfgset {
> +	kh_cfg_set_t set;
> +};
> +
> +struct cfgset safelist;
> +
> +static void cfgset_init(struct cfgset *set, size_t initial_size)
> +{
> +	memset(&set->set, 0, sizeof(set->set));
> +	if (initial_size)
> +		kh_resize_cfg_set(&set->set, initial_size);
> +}
> +
> +static int cfgset_insert(struct cfgset *set, const char *cfg_key)
> +{
> +	int added;
> +	kh_put_cfg_set(&set->set, cfg_key, &added);
> +	printf("ESS: added %s\n", cfg_key);
> +	return !added;
> +}
> +
> +static int cfgset_contains(struct cfgset *set, const char *cfg_key)
> +{
> +	khiter_t pos = kh_get_cfg_set(&set->set, cfg_key);
> +	return pos != kh_end(&set->set);
> +}
> +
> +static void cfgset_clear(struct cfgset *set)
> +{
> +	kh_release_cfg_set(&set->set);
> +	cfgset_init(set, 0);
> +}
> +
>  static void get_system_info(struct strbuf *sys_info)
>  {
>  	struct strbuf version_info = STRBUF_INIT;
> @@ -53,6 +91,36 @@ static void get_system_info(struct strbuf *sys_info)
>  	strbuf_complete_line(sys_info);
>  }
>  
> +static void gather_safelist()
> +{
> +	int index;
> +	int safelist_len = sizeof(bugreport_config_safelist) / sizeof(const char *);
> +	cfgset_init(&safelist, safelist_len);
> +	for (index = 0; index < safelist_len; index++)
> +		cfgset_insert(&safelist, bugreport_config_safelist[index]);
> +
> +}
> +
> +static int git_config_bugreport(const char *var, const char *value, void *cb)
> +{
> +	struct strbuf *config_info = (struct strbuf *)cb;
> +
> +	if (cfgset_contains(&safelist, var))
> +		strbuf_addf(config_info,
> +			    "%s (%s) : %s\n",
> +			    var, config_scope_to_string(current_config_scope()),
> +			    value);
> +
> +	return 0;
> +}
> +
> +static void get_safelisted_config(struct strbuf *config_info)
> +{
> +	gather_safelist();
> +	git_config(git_config_bugreport, config_info);
> +	cfgset_clear(&safelist);
> +}
> +
>  static const char * const bugreport_usage[] = {
>  	N_("git bugreport [-o|--output <file>]"),
>  	NULL
> @@ -114,10 +182,12 @@ int cmd_main(int argc, const char **argv)
>  
>  	get_bug_template(&buffer);
>  
> -	// add other contents
>  	get_header(&buffer, "System Info");
>  	get_system_info(&buffer);
>  
> +	get_header(&buffer, "Safelisted Config Info");
> +	get_safelisted_config(&buffer);
> +
>  	report = fopen_for_writing(report_path.buf);
>  	strbuf_write(&buffer, report);
>  	fclose(report);
Emily Shaffer· Dec 16, 2019, 23:40 UTC · re: Junio C Hamano · lore

Re: [PATCH v4 10/15] bugreport: add config values from safelist

On Fri, Dec 13, 2019 at 01:45:47PM -0800, Junio C Hamano wrote:
Show 19 quoted lines
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> > Teach bugreport to gather the values of config options which are present
> > in 'bugreport-config-safelist.h'.
> >
> > Many config options are sensitive, and many Git add-ons use config
> > options which git-core does not know about; it is better only to gather
> > config options which we know to be safe, rather than excluding options
> > which we know to be unsafe.
> >
> > Taking the build-time generated array and putting it into a set saves us
> > time - since git_config_bugreport() is called for every option the user
> > has configured, performing option lookup in constant time is a useful
> > optimization.
> 
> Interesting.  I actually was expecting the look-up to go the other
> way around.  We know the safe keys, so iterate over them and grab
> their values, if defined.  No need for hashes or anything, but just
> a simple linear list of safe stuff.

Hm. Without looking at the code, I said to myself, "repo_config_get_value() will open all the available config files to find the resolved value, so I don't want to do n*4 file open/closes, I only want to do 4 total."

Now I look at the code and see that the configs are already being read into a hashset before now. So you're right that it doesn't make sense for me to do it this way....

Show 7 quoted lines
> 
> But that is too simple-minded.  If we wanted to safelist foo.*.bar,
> where '*' can be anything, walking on the list of safe variables
> would not work.  We must have a hash table that knows "foo.*.bar" is
> allowed, and while walking all the configuration keys, when we see
> foo.a.bar, we consult "foo.*.bar" as well as "foo.a.bar" to see if
> it is whitelisted, or something like that.
...unless we want to use wildcards like you suggest.

But I'm not sure it's a good idea. I envision someone writing another Git add-on, which offers someone to specify "user.password" and automagically do some Bitbucket interaction, or something. (An extreme and misguided example, but I hope the point remains.) Then if I say, "all the user.* configs I can see in 'git help config' are safe, so I will just safelist user.*," I'm not accounting for this other tool's known configs which are a superset of what Git knows and expects.

The point of this safelist-generation exercise is to avoid accidentally printing some sensitive user config, and the point of using a safelist instead of a blocklist is to avoid printing a third-party config we didn't know about (or messing up our pattern matching). So I suggest we avoid pattern matching entirely.

Show 6 quoted lines
> 
> But then I am not sure if this implementation does something like
> this for three-level names.  If not, I do not see much point in use
> of the hash there either.
> 
> Puzzled.

I'd prefer to solve it by iterating over the compile-time-generated safelist using get_config_value_multi(), which I clock at O(n) for n = safelist length: performing n calls to an O(1) hashset lookup. That saves us the O(n*log(n)) hashset population, and the implementation of Yet Another Set in the source.

Thanks.
 - Emily
Junio C Hamano· Dec 17, 2019, 17:43 UTC · re: Emily Shaffer · lore

Re: [PATCH v4 10/15] bugreport: add config values from safelist

Emily Shaffer <emilyshaffer@google.com> writes:
Show 11 quoted lines
>> But that is too simple-minded.  If we wanted to safelist foo.*.bar,
>> where '*' can be anything, walking on the list of safe variables
>> would not work.  We must have a hash table that knows "foo.*.bar" is
>> allowed, and while walking all the configuration keys, when we see
>> foo.a.bar, we consult "foo.*.bar" as well as "foo.a.bar" to see if
>> it is whitelisted, or something like that.
>
> ...unless we want to use wildcards like you suggest.
>
> But I'm not sure it's a good idea. I envision someone writing another
> Git add-on, which offers someone to specify "user.password" ...

Wildcarding the leaf level of two (or for that matter three) level names like "user.*" in your example is of course a nonsense way to use the safelist. But think about three-level names where the second level is designed to be used for user-supplied token to give things of similar nature a name the user can use to distinguish things. If remote.origin.url is worth reporting, wouldn't remote.upstream.url also be? Shouldn't remote.*.url be the way to say "the URL field for each and every remote is a safe thing to report" in the safelist?

Emily Shaffer· Jan 24, 2020, 03:29 UTC · re: Junio C Hamano · lore

Re: [PATCH v4 10/15] bugreport: add config values from safelist

On Tue, Dec 17, 2019 at 09:43:23AM -0800, Junio C Hamano wrote:
Show 23 quoted lines
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> >> But that is too simple-minded.  If we wanted to safelist foo.*.bar,
> >> where '*' can be anything, walking on the list of safe variables
> >> would not work.  We must have a hash table that knows "foo.*.bar" is
> >> allowed, and while walking all the configuration keys, when we see
> >> foo.a.bar, we consult "foo.*.bar" as well as "foo.a.bar" to see if
> >> it is whitelisted, or something like that.
> >
> > ...unless we want to use wildcards like you suggest.
> >
> > But I'm not sure it's a good idea. I envision someone writing another
> > Git add-on, which offers someone to specify "user.password" ...
> 
> Wildcarding the leaf level of two (or for that matter three) level
> names like "user.*" in your example is of course a nonsense way to
> use the safelist.  But think about three-level names where the
> second level is designed to be used for user-supplied token to give
> things of similar nature a name the user can use to distinguish
> things.  If remote.origin.url is worth reporting, wouldn't
> remote.upstream.url also be?  Shouldn't remote.*.url be the way to
> say "the URL field for each and every remote is a safe thing to
> report" in the safelist?
Bah, somehow this mail slipped through the cracks for me.

When you put it that way, I see how it can be useful. Although I admit to fatigue on this patchset and now I'm hesitant about feature creep. :)

I'm going to send the reroll for v5 as I have it, not having seen this mail, and think more about how I'd approach wildcarding in the middle like this. I want to take a look at the other configs which use a middle level as a user token and try to reason about whether we ought to be collecting them. I'll hope to report back sooner (or later).

 - Emily
Johannes Schindelin· Dec 29, 2019, 20:17 UTC · re: Emily Shaffer · lore

Re: [PATCH v4 10/15] bugreport: add config values from safelist

Hi Emily,
On Thu, 12 Dec 2019, Emily Shaffer wrote:
Show 11 quoted lines
> diff --git a/bugreport.c b/bugreport.c
> index 759cc0b0f8..1fca28f0b9 100644
> --- a/bugreport.c
> +++ b/bugreport.c
> @@ -6,6 +6,9 @@
>  #include "help.h"
>  #include <gnu/libc-version.h>
>  #include "run-command.h"
> +#include "config.h"
> +#include "bugreport-config-safelist.h"
> +#include "khash.h"

Seems that this patch makes things fail in the CI build (https://dev.azure.com/gitgitgadget/git/_build/results?buildId=23830&view=results):

-- snipsnap --
bugreport.c:10:10: fatal error: 'bugreport-config-safelist.h' file not found
#include "bugreport-config-safelist.h"
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
Makefile:2395: recipe for target 'bugreport.o' failed
make: *** [bugreport.o] Error 1
make: *** Waiting for unfinished jobs....
Emily Shaffer· Dec 13, 2019, 00:43 UTC · re: Emily Shaffer · lore

[PATCH v4 11/15] bugreport: collect list of populated hooks

Occasionally a failure a user is seeing may be related to a specific hook which is being run, perhaps without the user realizing. While the contents of hooks can be sensitive - containing user data or process information specific to the user's organization - simply knowing that a hook is being run at a certain stage can help us to understand whether something is going wrong.

Without a definitive list of hook names within the code, we compile our own list from the documentation. This is likely prone to bitrot. To reduce the amount of code humans need to read, we turn the list into a string_list and iterate over it (as we are calling the same find_hook operation on each string). However, since bugreport should primarily be called by the user, the performance loss from massaging the string seems acceptable.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 bugreport.c | 55 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 55 insertions(+)
Show changes to bugreport.c +55 −0
diff --git a/bugreport.c b/bugreport.c
index 1fca28f0b9..f89cb8d754 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -9,6 +9,7 @@
 #include "config.h"
 #include "bugreport-config-safelist.h"
 #include "khash.h"
+#include "run-command.h"
 
 static void get_http_version_info(struct strbuf *http_info)
 {
@@ -121,6 +122,57 @@ static void get_safelisted_config(struct strbuf *config_info)
 	cfgset_clear(&safelist);
 }
 
+static void get_populated_hooks(struct strbuf *hook_info)
+{
+	/*
+	 * Doesn't look like there is a list of all possible hooks; so below is
+	 * a transcription of `git help hook`.
+	 */
+	const char *hooks = "applypatch-msg,"
+			    "pre-applypatch,"
+			    "post-applypatch,"
+			    "pre-commit,"
+			    "pre-merge-commit,"
+			    "prepare-commit-msg,"
+			    "commit-msg,"
+			    "post-commit,"
+			    "pre-rebase,"
+			    "post-checkout,"
+			    "post-merge,"
+			    "pre-push,"
+			    "pre-receive,"
+			    "update,"
+			    "post-receive,"
+			    "post-update,"
+			    "push-to-checkout,"
+			    "pre-auto-gc,"
+			    "post-rewrite,"
+			    "sendemail-validate,"
+			    "fsmonitor-watchman,"
+			    "p4-pre-submit,"
+			    "post-index-changex";
+	struct string_list hooks_list = STRING_LIST_INIT_DUP;
+	struct string_list_item *iter = NULL;
+	int nongit_ok;
+
+	setup_git_directory_gently(&nongit_ok);
+
+	if (nongit_ok) {
+		strbuf_addstr(hook_info,
+			"not run from a git repository - no hooks to show\n");
+		return;
+	}
+
+	string_list_split(&hooks_list, hooks, ',', -1);
+
+	for_each_string_list_item(iter, &hooks_list) {
+		if (find_hook(iter->string)) {
+			strbuf_addstr(hook_info, iter->string);
+			strbuf_complete_line(hook_info);
+		}
+	}
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output <file>]"),
 	NULL
@@ -188,6 +240,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Safelisted Config Info");
 	get_safelisted_config(&buffer);
 
+	get_header(&buffer, "Configured Hooks");
+	get_populated_hooks(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 	strbuf_write(&buffer, report);
 	fclose(report);
-- 
2.24.1.735.g03f4e72817-goog
Junio C Hamano· Dec 13, 2019, 21:47 UTC · re: Emily Shaffer · lore

Re: [PATCH v4 11/15] bugreport: collect list of populated hooks

Emily Shaffer <emilyshaffer@google.com> writes:
> +	/*
> +	 * Doesn't look like there is a list of all possible hooks; so below is
> +	 * a transcription of `git help hook`.
> +	 */

We probably would want to employ technique similar to what you used for the list of safe configuration we saw in an earlier patch in the series. What's not even documented is not worth reporting ;-)

Show 58 quoted lines
> +	const char *hooks = "applypatch-msg,"
> +			    "pre-applypatch,"
> +			    "post-applypatch,"
> +			    "pre-commit,"
> +			    "pre-merge-commit,"
> +			    "prepare-commit-msg,"
> +			    "commit-msg,"
> +			    "post-commit,"
> +			    "pre-rebase,"
> +			    "post-checkout,"
> +			    "post-merge,"
> +			    "pre-push,"
> +			    "pre-receive,"
> +			    "update,"
> +			    "post-receive,"
> +			    "post-update,"
> +			    "push-to-checkout,"
> +			    "pre-auto-gc,"
> +			    "post-rewrite,"
> +			    "sendemail-validate,"
> +			    "fsmonitor-watchman,"
> +			    "p4-pre-submit,"
> +			    "post-index-changex";
> +	struct string_list hooks_list = STRING_LIST_INIT_DUP;
> +	struct string_list_item *iter = NULL;
> +	int nongit_ok;
> +
> +	setup_git_directory_gently(&nongit_ok);
> +
> +	if (nongit_ok) {
> +		strbuf_addstr(hook_info,
> +			"not run from a git repository - no hooks to show\n");
> +		return;
> +	}
> +
> +	string_list_split(&hooks_list, hooks, ',', -1);
> +
> +	for_each_string_list_item(iter, &hooks_list) {
> +		if (find_hook(iter->string)) {
> +			strbuf_addstr(hook_info, iter->string);
> +			strbuf_complete_line(hook_info);
> +		}
> +	}
> +}
> +
>  static const char * const bugreport_usage[] = {
>  	N_("git bugreport [-o|--output <file>]"),
>  	NULL
> @@ -188,6 +240,9 @@ int cmd_main(int argc, const char **argv)
>  	get_header(&buffer, "Safelisted Config Info");
>  	get_safelisted_config(&buffer);
>  
> +	get_header(&buffer, "Configured Hooks");
> +	get_populated_hooks(&buffer);
> +
>  	report = fopen_for_writing(report_path.buf);
>  	strbuf_write(&buffer, report);
>  	fclose(report);
Emily Shaffer· Dec 16, 2019, 23:51 UTC · re: Junio C Hamano · lore

Re: [PATCH v4 11/15] bugreport: collect list of populated hooks

On Fri, Dec 13, 2019 at 01:47:27PM -0800, Junio C Hamano wrote:
Show 10 quoted lines
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> > +	/*
> > +	 * Doesn't look like there is a list of all possible hooks; so below is
> > +	 * a transcription of `git help hook`.
> > +	 */
> 
> We probably would want to employ technique similar to what you used
> for the list of safe configuration we saw in an earlier patch in the
> series.  What's not even documented is not worth reporting ;-)

I somewhat slightly would rather not, since I hope to include a list of the available hooks in my work with git-hook: https://lore.kernel.org/20191211205114.GD107889@google.com

It's true that the way this patch is written now is prone to bitrot. It's also true that I could win the lottery tomorrow and never finish my hopes of git-hook implementation. And it's finally true that I don't want to write a fourth build-time-generated header which scrapes docs, especially if I also plan to delete it two months from now.

I wonder whether a better approach might be to drop hook support entirely here, and add it later on when this particular bit of tech debt (no canonical list of hook names) is solved, one way or another.

Another approach may be to just list all contents of $(core.hookdir)/ which doesn't end in ".sample", which is what the bash first attempt of bugreport did. This does mean that those who have something like:

  .git/hooks/
    pre-commit
    pre-commit.d/
  pre-commit:
    sh pre-commit.d/a
    sh pre-commit.d/b

will have their multihook config exposed, although the contents of those hooks won't be.

I'm interested to know everyone else's opinion, because mine is biased by my hope that I can solve all the world's problems with git-hook ;)

 - Emily
Emily Shaffer· Dec 13, 2019, 00:43 UTC · re: Emily Shaffer · lore

[PATCH v4 12/15] bugreport: count loose objects

The number of unpacked objects in a user's repository may help us understand the root of the problem they're seeing, especially if a command is running unusually slowly.

Rather than directly invoking 'git-count-objects', which may sometimes fail unexpectedly on Git for Windows, manually count the contents of .git/objects. Additionally, since we may wish to inspect other directories' contents for bugreport in the future, put the directory listing into a helper function.

Helped-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 bugreport.c | 64 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 64 insertions(+)
Show changes to bugreport.c +64 −0
diff --git a/bugreport.c b/bugreport.c
index f89cb8d754..3abb83d77f 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -173,6 +173,67 @@ static void get_populated_hooks(struct strbuf *hook_info)
 	}
 }
 
+static int is_hex(const char *string, size_t count)
+{
+	for (; count; string++, count--) {
+		if (!isxdigit(*string))
+			return 0;
+	}
+	return 1;
+}
+
+static void get_loose_object_summary(struct strbuf *obj_info) {
+	struct dirent *d = NULL;
+	DIR *dir, *subdir = NULL;
+	size_t dir_len;
+	struct strbuf dirpath = STRBUF_INIT;
+
+	strbuf_addstr(&dirpath, get_object_directory());
+	strbuf_complete(&dirpath, '/');
+
+	dir = opendir(dirpath.buf);
+	if (!dir) {
+		strbuf_addf(obj_info, "could not open object directory '%s'\n",
+			    dirpath.buf);
+		strbuf_release(&dirpath);
+		return;
+	}
+
+	dir_len = dirpath.len;
+
+	while ((d = readdir(dir))) {
+		int object_count = 0;
+		char subdir_name[3];
+
+		if (d->d_type != DT_DIR)
+			continue;
+
+		if ((strlen(d->d_name) != 2) || (!is_hex(d->d_name, 2)))
+			continue;
+
+		/* copy directory name + \0 */
+		memcpy(subdir_name, d->d_name, 3);
+
+		strbuf_setlen(&dirpath, dir_len);
+		strbuf_addstr(&dirpath, d->d_name);
+
+		subdir = opendir(dirpath.buf);
+		if (!subdir)
+			continue;
+		while ((d = readdir(subdir)))
+			if (d->d_type == DT_REG)
+				object_count++;
+
+		closedir(subdir);
+
+		strbuf_addf(obj_info, "%s: %d\n", subdir_name, object_count);
+	}
+
+
+	closedir(dir);
+	strbuf_release(&dirpath);
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output <file>]"),
 	NULL
@@ -243,6 +304,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Configured Hooks");
 	get_populated_hooks(&buffer);
 
+	get_header(&buffer, "Loose Object Counts");
+	get_loose_object_summary(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 	strbuf_write(&buffer, report);
 	fclose(report);
-- 
2.24.1.735.g03f4e72817-goog
Junio C Hamano· Dec 13, 2019, 21:51 UTC · re: Emily Shaffer · lore

Re: [PATCH v4 12/15] bugreport: count loose objects

Emily Shaffer <emilyshaffer@google.com> writes:
Show 7 quoted lines
> The number of unpacked objects in a user's repository may help us
> understand the root of the problem they're seeing, especially if a
> command is running unusually slowly.
>
> Rather than directly invoking 'git-count-objects', which may sometimes
> fail unexpectedly on Git for Windows, manually count the contents of
> .git/objects.

Is for_each_loose_object() or for_each_loose_file_in_objdir() not sufficient? We really do *not* want a redundant implementation of something we already use elsewhere in the system, especially for the purpose of this program, because you would end up reporting what you computed in a way that may be quite different from what the rest of the system that is actually used by the end users computes.

Emily Shaffer· Dec 16, 2019, 23:54 UTC · re: Junio C Hamano · lore

Re: [PATCH v4 12/15] bugreport: count loose objects

On Fri, Dec 13, 2019 at 01:51:27PM -0800, Junio C Hamano wrote:
Show 17 quoted lines
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> > The number of unpacked objects in a user's repository may help us
> > understand the root of the problem they're seeing, especially if a
> > command is running unusually slowly.
> >
> > Rather than directly invoking 'git-count-objects', which may sometimes
> > fail unexpectedly on Git for Windows, manually count the contents of
> > .git/objects.
> 
> Is for_each_loose_object() or for_each_loose_file_in_objdir() not
> sufficient?  We really do *not* want a redundant implementation of
> something we already use elsewhere in the system, especially for the
> purpose of this program, because you would end up reporting what you
> computed in a way that may be quite different from what the rest of
> the system that is actually used by the end users computes.
> 

Oh, thanks for the pointer. That looks sufficient - I will investigate the differences between the two and stop filesystem browsing by hand in this case.

 - Emily
Emily Shaffer· Dec 13, 2019, 00:43 UTC · re: Emily Shaffer · lore

[PATCH v4 13/15] bugreport: add packed object summary

Alongside the list of loose objects, it's useful to see the list of object packs as well. It can help us to examine what Git did and did not pack.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 bugreport.c | 31 +++++++++++++++++++++++++++++++
 1 file changed, 31 insertions(+)
Show changes to bugreport.c +31 −0
diff --git a/bugreport.c b/bugreport.c
index 3abb83d77f..992d8f9de7 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -234,6 +234,34 @@ static void get_loose_object_summary(struct strbuf *obj_info) {
 	strbuf_release(&dirpath);
 }
 
+static void get_packed_object_summary(struct strbuf *obj_info)
+{
+	struct strbuf dirpath = STRBUF_INIT;
+	struct dirent *d;
+	DIR *dir = NULL;
+
+	strbuf_addstr(&dirpath, get_object_directory());
+	strbuf_complete(&dirpath, '/');
+	strbuf_addstr(&dirpath, "pack/");
+
+	dir = opendir(dirpath.buf);
+	if (!dir) {
+		strbuf_addf(obj_info, "could not open packed object directory '%s'\n",
+			    dirpath.buf);
+		strbuf_release(&dirpath);
+		return;
+	}
+
+	while ((d = readdir(dir))) {
+		strbuf_addbuf(obj_info, &dirpath);
+		strbuf_addstr(obj_info, d->d_name);
+		strbuf_complete_line(obj_info);
+	}
+
+	closedir(dir);
+	strbuf_release(&dirpath);
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output <file>]"),
 	NULL
@@ -307,6 +335,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Loose Object Counts");
 	get_loose_object_summary(&buffer);
 
+	get_header(&buffer, "Packed Object Summary");
+	get_packed_object_summary(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 	strbuf_write(&buffer, report);
 	fclose(report);
-- 
2.24.1.735.g03f4e72817-goog
Junio C Hamano· Dec 13, 2019, 21:56 UTC · re: Emily Shaffer · lore

Re: [PATCH v4 13/15] bugreport: add packed object summary

Emily Shaffer <emilyshaffer@google.com> writes:
> Alongside the list of loose objects, it's useful to see the list of
> object packs as well. It can help us to examine what Git did and did not
> pack.

Not exactly the same comment, but is in the same spirit, as the previous step applies to this one.

Would it be too much work to libify the bulk of cmd_count_objects() that computes numbers, making the cmd_count_objects() into a thin wrapper that calls the libified "counter/collector" function and prints the resulting numbers received from it?

That way, the get_packed_object_summary() can be another consumer of the same "counter/collector" function, no?

Show 54 quoted lines
> Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
> ---
>  bugreport.c | 31 +++++++++++++++++++++++++++++++
>  1 file changed, 31 insertions(+)
>
> diff --git a/bugreport.c b/bugreport.c
> index 3abb83d77f..992d8f9de7 100644
> --- a/bugreport.c
> +++ b/bugreport.c
> @@ -234,6 +234,34 @@ static void get_loose_object_summary(struct strbuf *obj_info) {
>  	strbuf_release(&dirpath);
>  }
>  
> +static void get_packed_object_summary(struct strbuf *obj_info)
> +{
> +	struct strbuf dirpath = STRBUF_INIT;
> +	struct dirent *d;
> +	DIR *dir = NULL;
> +
> +	strbuf_addstr(&dirpath, get_object_directory());
> +	strbuf_complete(&dirpath, '/');
> +	strbuf_addstr(&dirpath, "pack/");
> +
> +	dir = opendir(dirpath.buf);
> +	if (!dir) {
> +		strbuf_addf(obj_info, "could not open packed object directory '%s'\n",
> +			    dirpath.buf);
> +		strbuf_release(&dirpath);
> +		return;
> +	}
> +
> +	while ((d = readdir(dir))) {
> +		strbuf_addbuf(obj_info, &dirpath);
> +		strbuf_addstr(obj_info, d->d_name);
> +		strbuf_complete_line(obj_info);
> +	}
> +
> +	closedir(dir);
> +	strbuf_release(&dirpath);
> +}
> +
>  static const char * const bugreport_usage[] = {
>  	N_("git bugreport [-o|--output <file>]"),
>  	NULL
> @@ -307,6 +335,9 @@ int cmd_main(int argc, const char **argv)
>  	get_header(&buffer, "Loose Object Counts");
>  	get_loose_object_summary(&buffer);
>  
> +	get_header(&buffer, "Packed Object Summary");
> +	get_packed_object_summary(&buffer);
> +
>  	report = fopen_for_writing(report_path.buf);
>  	strbuf_write(&buffer, report);
>  	fclose(report);
Emily Shaffer· Dec 16, 2019, 23:56 UTC · re: Junio C Hamano · lore

Re: [PATCH v4 13/15] bugreport: add packed object summary

On Fri, Dec 13, 2019 at 01:56:07PM -0800, Junio C Hamano wrote:
Show 16 quoted lines
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> > Alongside the list of loose objects, it's useful to see the list of
> > object packs as well. It can help us to examine what Git did and did not
> > pack.
> 
> Not exactly the same comment, but is in the same spirit, as the
> previous step applies to this one.
> 
> Would it be too much work to libify the bulk of cmd_count_objects()
> that computes numbers, making the cmd_count_objects() into a thin
> wrapper that calls the libified "counter/collector" function and
> prints the resulting numbers received from it?
> 
> That way, the get_packed_object_summary() can be another consumer       
> of the same "counter/collector" function, no?
That sounds like a good approach. Will do.
Show 55 quoted lines
> 
> > Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
> > ---
> >  bugreport.c | 31 +++++++++++++++++++++++++++++++
> >  1 file changed, 31 insertions(+)
> >
> > diff --git a/bugreport.c b/bugreport.c
> > index 3abb83d77f..992d8f9de7 100644
> > --- a/bugreport.c
> > +++ b/bugreport.c
> > @@ -234,6 +234,34 @@ static void get_loose_object_summary(struct strbuf *obj_info) {
> >  	strbuf_release(&dirpath);
> >  }
> >  
> > +static void get_packed_object_summary(struct strbuf *obj_info)
> > +{
> > +	struct strbuf dirpath = STRBUF_INIT;
> > +	struct dirent *d;
> > +	DIR *dir = NULL;
> > +
> > +	strbuf_addstr(&dirpath, get_object_directory());
> > +	strbuf_complete(&dirpath, '/');
> > +	strbuf_addstr(&dirpath, "pack/");
> > +
> > +	dir = opendir(dirpath.buf);
> > +	if (!dir) {
> > +		strbuf_addf(obj_info, "could not open packed object directory '%s'\n",
> > +			    dirpath.buf);
> > +		strbuf_release(&dirpath);
> > +		return;
> > +	}
> > +
> > +	while ((d = readdir(dir))) {
> > +		strbuf_addbuf(obj_info, &dirpath);
> > +		strbuf_addstr(obj_info, d->d_name);
> > +		strbuf_complete_line(obj_info);
> > +	}
> > +
> > +	closedir(dir);
> > +	strbuf_release(&dirpath);
> > +}
> > +
> >  static const char * const bugreport_usage[] = {
> >  	N_("git bugreport [-o|--output <file>]"),
> >  	NULL
> > @@ -307,6 +335,9 @@ int cmd_main(int argc, const char **argv)
> >  	get_header(&buffer, "Loose Object Counts");
> >  	get_loose_object_summary(&buffer);
> >  
> > +	get_header(&buffer, "Packed Object Summary");
> > +	get_packed_object_summary(&buffer);
> > +
> >  	report = fopen_for_writing(report_path.buf);
> >  	strbuf_write(&buffer, report);
> >  	fclose(report);
Emily Shaffer· Dec 13, 2019, 00:43 UTC · re: Emily Shaffer · lore

[PATCH v4 14/15] bugreport: list contents of $OBJDIR/info

Miscellaneous information used about the object store can end up in .git/objects/info; this can help us understand what may be going on with the object store when the user is reporting a bug. Otherwise, it could be difficult to track down what is going wrong with an object which isn't kept locally to .git/objects/ or .git/objects/pack. Having some understanding of where the user's objects may be kept can save us some hops during the bug reporting process.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 bugreport.c | 48 ++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 48 insertions(+)
Show changes to bugreport.c +48 −0
diff --git a/bugreport.c b/bugreport.c
index 992d8f9de7..5d300f1c5f 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -262,6 +262,51 @@ static void get_packed_object_summary(struct strbuf *obj_info)
 	strbuf_release(&dirpath);
 }
 
+static void list_contents_of_dir_recursively(struct strbuf *contents,
+				      	     struct strbuf *dirpath)
+{
+	struct dirent *d;
+	DIR *dir;
+	size_t path_len;
+
+	dir = opendir(dirpath->buf);
+	if (!dir)
+		return;
+
+	strbuf_complete(dirpath, '/');
+	path_len = dirpath->len;
+
+	while ((d = readdir(dir))) {
+		if (!strcmp(d->d_name, ".") || !strcmp(d->d_name, ".."))
+			continue;
+
+		strbuf_addbuf(contents, dirpath);
+		strbuf_addstr(contents, d->d_name);
+		strbuf_complete_line(contents);
+
+		if (d->d_type == DT_DIR) {
+			strbuf_addstr(dirpath, d->d_name);
+			list_contents_of_dir_recursively(contents, dirpath);
+		}
+		strbuf_setlen(dirpath, path_len);
+	}
+
+	closedir(dir);
+}
+
+static void get_object_info_summary(struct strbuf *obj_info)
+{
+	struct strbuf dirpath = STRBUF_INIT;
+
+	strbuf_addstr(&dirpath, get_object_directory());
+	strbuf_complete(&dirpath, '/');
+	strbuf_addstr(&dirpath, "info/");
+
+	list_contents_of_dir_recursively(obj_info, &dirpath);
+
+	strbuf_release(&dirpath);
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output <file>]"),
 	NULL
@@ -338,6 +383,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Packed Object Summary");
 	get_packed_object_summary(&buffer);
 
+	get_header(&buffer, "Object Info Summary");
+	get_object_info_summary(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 	strbuf_write(&buffer, report);
 	fclose(report);
-- 
2.24.1.735.g03f4e72817-goog
Emily Shaffer· Dec 13, 2019, 00:43 UTC · re: Emily Shaffer · lore

[PATCH v4 15/15] bugreport: summarize contents of alternates file

In some cases, it could be that the user is having a problem with an object which isn't present in their normal object directory. We can get a hint that that might be the case by examining the list of alternates where their object may be stored instead. Since paths to alternates may be sensitive, we'll instead count how many alternates have been specified and note how many of them exist or are broken.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 bugreport.c | 39 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 39 insertions(+)
Show changes to bugreport.c +39 −0
diff --git a/bugreport.c b/bugreport.c
index 5d300f1c5f..d2a3fb1f67 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -307,6 +307,42 @@ static void get_object_info_summary(struct strbuf *obj_info)
 	strbuf_release(&dirpath);
 }
 
+static void get_alternates_summary(struct strbuf *alternates_info)
+{
+	struct strbuf alternates_path = STRBUF_INIT;
+	struct strbuf alternate = STRBUF_INIT;
+	FILE *file;
+	size_t exists = 0, broken = 0;
+
+	strbuf_addstr(&alternates_path, get_object_directory());
+	strbuf_complete(&alternates_path, '/');
+	strbuf_addstr(&alternates_path, "info/alternates");
+
+	file = fopen(alternates_path.buf, "r");
+	if (!file) {
+		strbuf_addstr(alternates_info, "No alternates file found.\n");
+		strbuf_release(&alternates_path);
+		return;
+	}
+
+	while (strbuf_getline(&alternate, file) != EOF) {
+		if (!access(alternate.buf, F_OK))
+			exists++;
+		else
+			broken++;
+	}
+
+	strbuf_addf(alternates_info,
+		    "%zd alternates found (%zd working, %zd broken)\n",
+		    exists + broken,
+		    exists,
+		    broken);
+
+	fclose(file);
+	strbuf_release(&alternate);
+	strbuf_release(&alternates_path);
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output <file>]"),
 	NULL
@@ -386,6 +422,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Object Info Summary");
 	get_object_info_summary(&buffer);
 
+	get_header(&buffer, "Alternates");
+	get_alternates_summary(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 	strbuf_write(&buffer, report);
 	fclose(report);
-- 
2.24.1.735.g03f4e72817-goog
emilyshaffer@google.com· Jan 24, 2020, 03:34 UTC · re: Emily Shaffer · lore

[PATCH v5 00/15] add git-bugreport tool

From: Emily Shaffer <emilyshaffer@google.com>

This topic branch depends on the patch mailed in lore.kernel.org/git/20191211233820.185153-1-emilyshaffer@google.com in order to display scopes for configs gathered during "bugreport: add config values from safelist".

I'll summarize v4-v5. Since v4 has languished for some time, I don't think an interdiff is too helpful, so I won't include one. Bonus, the code is much simplified from some suggestions from Junio on how to inspect objects, so I hope it's easy to review anyways.

Throughout, did some style changes away from C++ bad habits. Otherwise, as listed:

Emily Shaffer (15):
  bugreport: add tool to generate debugging info
    (no change)
  help: move list_config_help to builtin/help
    Changed to agree with f3719846134
  bugreport: gather git version and build info
    Style only
  help: add shell-path to --build-options
    (no change)
  bugreport: add uname info
    Removed nodename for privacy reasons
  bugreport: add compiler info
    Moved glibc tattling into compat/.
    I appreciate a close look at this one - I think I understood the
    right way to go about a compat/ util but it's the first one I've
    done.
  bugreport: add curl version
    Moved curl tattling from git-http-fetch into git-remote-curl
  bugreport: include user interactive shell
    Stop depending on a compiler quirk to save us from segfault
  bugreport: generate config safelist based on docs
    Changed to agree with f3719846134
    Cleaned up script a little
  bugreport: add config values from safelist
    Made git-bugreport dependent on generated safelist header
  bugreport: collect list of populated hooks
    No change.
    Per https://lore.kernel.org/git/20191216235131.GL135450@google.com,
    should we even keep this patch?
  bugreport: count loose objects
    Use helpers from object-store.h instead of manually walking the
    filesystem.
  bugreport: add packed object summary
    Use helpers from object-store.h instead of manually walking the
    filesystem.
  bugreport: list contents of $OBJDIR/info
    (no change)
  bugreport: summarize contents of alternates file
    Rephrase commit message to explain why I can't use the helpers in
    object-store.h.
Thanks.
 - Emily
 .gitignore                              |   3 +
 Documentation/asciidoc.conf             |   8 +
 Documentation/asciidoctor-extensions.rb |   7 +
 Documentation/config/sendemail.txt      |  68 ++--
 Documentation/git-bugreport.txt         |  43 +++
 Makefile                                |  25 +-
 bugreport.c                             | 427 ++++++++++++++++++++++++
 builtin/help.c                          |  86 +++++
 compat/compiler.h                       |  24 ++
 generate-bugreport-config-safelist.sh   |  22 ++
 generate-cmdlist.sh                     |  19 --
 generate-configlist.sh                  |  24 ++
 help.c                                  | 133 ++------
 help.h                                  |   2 +-
 remote-curl.c                           |   8 +
 t/t0091-bugreport.sh                    |  41 +++
 16 files changed, 780 insertions(+), 160 deletions(-)
 create mode 100644 Documentation/git-bugreport.txt
 create mode 100644 bugreport.c
 create mode 100644 compat/compiler.h
 create mode 100755 generate-bugreport-config-safelist.sh
 create mode 100755 generate-configlist.sh
 create mode 100755 t/t0091-bugreport.sh
-- 
2.25.0.341.g760bfbb309-goog
emilyshaffer@google.com· Jan 24, 2020, 03:34 UTC · re: emilyshaffer@google.com · lore

[PATCH v5 01/15] bugreport: add tool to generate debugging info

From: Emily Shaffer <emilyshaffer@google.com>

Teach Git how to prompt the user for a good bug report: reproduction steps, expected behavior, and actual behavior. Later, Git can learn how to collect some diagnostic information from the repository.

If users can send us a well-written bug report which contains diagnostic information we would otherwise need to ask the user for, we can reduce the number of question-and-answer round trips between the reporter and the Git contributor.

Users may also wish to send a report like this to their local "Git expert" if they have put their repository into a state they are confused by.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 .gitignore                      |  1 +
 Documentation/git-bugreport.txt | 43 ++++++++++++++++++++
 Makefile                        |  5 +++
 bugreport.c                     | 69 +++++++++++++++++++++++++++++++++
 t/t0091-bugreport.sh            | 41 ++++++++++++++++++++
 5 files changed, 159 insertions(+)
 create mode 100644 Documentation/git-bugreport.txt
 create mode 100644 bugreport.c
 create mode 100755 t/t0091-bugreport.sh
Show changes to 5 files +159 −0

.gitignore, Documentation/git-bugreport.txt, Makefile, bugreport.c, t/t0091-bugreport.sh

diff --git a/.gitignore b/.gitignore
index aebe7c0908..ca301bc890 100644
--- a/.gitignore
+++ b/.gitignore
@@ -25,6 +25,7 @@
 /git-bisect--helper
 /git-blame
 /git-branch
+/git-bugreport
 /git-bundle
 /git-cat-file
 /git-check-attr
diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
new file mode 100644
index 0000000000..75f0c80acf
--- /dev/null
+++ b/Documentation/git-bugreport.txt
@@ -0,0 +1,43 @@
+git-bugreport(1)
+================
+
+NAME
+----
+git-bugreport - Collect information for user to file a bug report
+
+SYNOPSIS
+--------
+[verse]
+'git bugreport' [-o | --output <path>]
+
+DESCRIPTION
+-----------
+Captures information about the user's machine, Git client, and repository state,
+as well as a form requesting information about the behavior the user observed,
+into a single text file which the user can then share, for example to the Git
+mailing list, in order to report an observed bug.
+
+The following information is requested from the user:
+
+ - Reproduction steps
+ - Expected behavior
+ - Actual behavior
+
+The following information is captured automatically:
+
+ - Git version (`git version --build-options`)
+ - Machine information (`uname -a`)
+ - Versions of various dependencies
+ - Git config contents (`git config --show-origin --list`)
+ - The names of all configured git-hooks in `.git/hooks/`
+
+OPTIONS
+-------
+-o [<path>]::
+--output [<path>]::
+	Place the resulting bug report file in <path> instead of the root of the
+	Git repository.
+
+GIT
+---
+Part of the linkgit:git[1] suite
diff --git a/Makefile b/Makefile
index 09f98b777c..f271619371 100644
--- a/Makefile
+++ b/Makefile
@@ -681,6 +681,7 @@ EXTRA_PROGRAMS =
 # ... and all the rest that could be moved out of bindir to gitexecdir
 PROGRAMS += $(EXTRA_PROGRAMS)
 
+PROGRAM_OBJS += bugreport.o
 PROGRAM_OBJS += credential-store.o
 PROGRAM_OBJS += daemon.o
 PROGRAM_OBJS += fast-import.o
@@ -2450,6 +2451,10 @@ endif
 git-%$X: %.o GIT-LDFLAGS $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) $(LIBS)
 
+git-bugreport$X: bugreport.o GIT-LDFLAGS $(GITLIBS)
+	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
+		$(LIBS)
+
 git-imap-send$X: imap-send.o $(IMAP_SEND_BUILDDEPS) GIT-LDFLAGS $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
 		$(IMAP_SEND_LDFLAGS) $(LIBS)
diff --git a/bugreport.c b/bugreport.c
new file mode 100644
index 0000000000..5495b31674
--- /dev/null
+++ b/bugreport.c
@@ -0,0 +1,69 @@
+#include "builtin.h"
+#include "parse-options.h"
+#include "stdio.h"
+#include "strbuf.h"
+#include "time.h"
+
+static const char * const bugreport_usage[] = {
+	N_("git bugreport [-o|--output <file>]"),
+	NULL
+};
+
+static int get_bug_template(struct strbuf *template)
+{
+	const char template_text[] = N_(
+"Thank you for filling out a Git bug report!\n"
+"Please answer the following questions to help us understand your issue.\n"
+"\n"
+"What did you do before the bug happened? (Steps to reproduce your issue)\n"
+"\n"
+"What did you expect to happen? (Expected behavior)\n"
+"\n"
+"What happened instead? (Actual behavior)\n"
+"\n"
+"What's different between what you expected and what actually happened?\n"
+"\n"
+"Anything else you want to add:\n"
+"\n"
+"Please review the rest of the bug report below.\n"
+"You can delete any lines you don't wish to send.\n");
+
+	strbuf_addstr(template, template_text);
+	return 0;
+}
+
+int cmd_main(int argc, const char **argv)
+{
+	struct strbuf buffer = STRBUF_INIT;
+	struct strbuf report_path = STRBUF_INIT;
+	FILE *report;
+	time_t now = time(NULL);
+	char *option_output = NULL;
+
+	const struct option bugreport_options[] = {
+		OPT_STRING('o', "output", &option_output, N_("path"),
+			   N_("specify a destination for the bugreport file")),
+		OPT_END()
+	};
+	argc = parse_options(argc, argv, "", bugreport_options,
+			     bugreport_usage, 0);
+
+	if (option_output) {
+		strbuf_addstr(&report_path, option_output);
+		strbuf_complete(&report_path, '/');
+	}
+
+	strbuf_addstr(&report_path, "git-bugreport-");
+	strbuf_addftime(&report_path, "%F", gmtime(&now), 0, 0);
+	strbuf_addstr(&report_path, ".txt");
+
+
+	get_bug_template(&buffer);
+
+	report = fopen_for_writing(report_path.buf);
+	strbuf_write(&buffer, report);
+	fclose(report);
+
+	launch_editor(report_path.buf, NULL, NULL);
+	return 0;
+}
diff --git a/t/t0091-bugreport.sh b/t/t0091-bugreport.sh
new file mode 100755
index 0000000000..6eb2ee4f66
--- /dev/null
+++ b/t/t0091-bugreport.sh
@@ -0,0 +1,41 @@
+#!/bin/bash
+
+test_description='git bugreport'
+
+. ./test-lib.sh
+
+# Headers "[System Info]" will be followed by a non-empty line if we put some
+# information there; we can make sure all our headers were followed by some
+# information to check if the command was successful.
+HEADER_PATTERN="^\[.*\]$"
+check_all_headers_populated() {
+	while read -r line; do
+		if [$(grep $HEADER_PATTERN $line)]; then
+			read -r nextline
+			if [-z $nextline]; then
+				return 1;
+			fi
+		fi
+	done
+}
+
+test_expect_success 'creates a report with content in the right places' '
+	git bugreport &&
+	check_all_headers_populated <git-bugreport-* &&
+	rm git-bugreport-*
+'
+
+test_expect_success '--output puts the report in the provided dir' '
+	mkdir foo/ &&
+	git bugreport -o foo/ &&
+	test -f foo/git-bugreport-* &&
+	rm -fr foo/
+'
+
+test_expect_success 'incorrect arguments abort with usage' '
+	test_must_fail git bugreport --false 2>output &&
+	grep usage output &&
+	test ! -f git-bugreport-*
+'
+
+test_done
-- 
2.25.0.341.g760bfbb309-goog
Martin Ågren· Jan 30, 2020, 22:18 UTC · re: emilyshaffer@google.com · lore

Re: [PATCH v5 01/15] bugreport: add tool to generate debugging info

On Fri, 24 Jan 2020 at 05:56, <emilyshaffer@google.com> wrote:
Show 6 quoted lines
>
> From: Emily Shaffer <emilyshaffer@google.com>
>
> Teach Git how to prompt the user for a good bug report: reproduction
> steps, expected behavior, and actual behavior. Later, Git can learn how
> to collect some diagnostic information from the repository.
("Later" meaning "later in this series", or "any day now"? ;-) )
> +SYNOPSIS
> +--------
> +[verse]
> +'git bugreport' [-o | --output <path>]
Hmm. Should that be "[(-o | --output) <path>]"?
Show 6 quoted lines
> +DESCRIPTION
> +-----------
> +Captures information about the user's machine, Git client, and repository state,
> +as well as a form requesting information about the behavior the user observed,
> +into a single text file which the user can then share, for example to the Git
> +mailing list, in order to report an observed bug.
Nice description. Got it.
Show 13 quoted lines
> +The following information is requested from the user:
> +
> + - Reproduction steps
> + - Expected behavior
> + - Actual behavior
> +
> +The following information is captured automatically:
> +
> + - Git version (`git version --build-options`)
> + - Machine information (`uname -a`)
> + - Versions of various dependencies
> + - Git config contents (`git config --show-origin --list`)
> + - The names of all configured git-hooks in `.git/hooks/`

I would have expected these points to appear later, both to make it clear what this does commit does (and not), and to highlight what user-visible (documentation-worthy) changes later commits bring along.

> +OPTIONS
> +-------
> +-o [<path>]::
> +--output [<path>]::
Drop the "[" and "]"? If you give -o, you'd better give a path as well?
> +       Place the resulting bug report file in <path> instead of the root of the
`<path>`
> +"Please review the rest of the bug report below.\n"
> +"You can delete any lines you don't wish to send.\n");

"send" sounds like we're *just* about to send this report somewhere, but it's "only" going to be written to the disk. Maybe "share", instead?

> +       if (option_output) {
> +               strbuf_addstr(&report_path, option_output);
> +               strbuf_complete(&report_path, '/');
> +       }

I thought I'd use `-o` to indicate the filename, but it turns out it's the *directory* where the file (of some semi-random, generated name) will end up. Re-reading the docs further up, I can see how this is consistent. I sort of wonder if this should be `--output*-directory*` for symmetry with `git format-patch`.

Show 5 quoted lines
> +       strbuf_addstr(&report_path, "git-bugreport-");
> +       strbuf_addftime(&report_path, "%F", gmtime(&now), 0, 0);
> +       strbuf_addstr(&report_path, ".txt");
> +
> +
(Double blank line?)
> +       get_bug_template(&buffer);
> +
> +       report = fopen_for_writing(report_path.buf);
Report might be NULL here.

If there's already such a file, we overwrite. Should we generate the filename using not just today's date (two bug reports in a day wouldn't be unheard of?) but also something like hh:mm:ss?

> +       strbuf_write(&buffer, report);
> +       fclose(report);
Maybe clear the strbuf around here...
> +       launch_editor(report_path.buf, NULL, NULL);
> +       return 0;
... and/or UNLEAK it here, together with report_path.
Maybe "return -launch_editor(...)"?
> +#!/bin/bash
Use /bin/sh instead?
Show 7 quoted lines
> +# Headers "[System Info]" will be followed by a non-empty line if we put some
> +# information there; we can make sure all our headers were followed by some
> +# information to check if the command was successful.
> +HEADER_PATTERN="^\[.*\]$"
> +check_all_headers_populated() {
> +       while read -r line; do
> +               if [$(grep $HEADER_PATTERN $line)]; then
I think this is a bash-ism.
> +                       read -r nextline
> +                       if [-z $nextline]; then
Likewise.
Show 14 quoted lines
> +                               return 1;
> +                       fi
> +               fi
> +       done
> +}
> +
> +test_expect_success 'creates a report with content in the right places' '
> +       git bugreport &&
> +       check_all_headers_populated <git-bugreport-* &&
> +       rm git-bugreport-*
> +'
> +
> +test_expect_success '--output puts the report in the provided dir' '
> +       mkdir foo/ &&

If foo isn't there, do we not create it? Apparently not -- in my testing, we segfault. (We don't check for NULL after opening the file.)

> +       git bugreport -o foo/ &&
> +       test -f foo/git-bugreport-* &&
test_path_is_file
Show 7 quoted lines
> +       rm -fr foo/
> +'
> +
> +test_expect_success 'incorrect arguments abort with usage' '
> +       test_must_fail git bugreport --false 2>output &&
> +       grep usage output &&
> +       test ! -f git-bugreport-*
test_path_is_missing
Martin
Emily Shaffer· Feb 4, 2020, 22:00 UTC · re: Martin Ågren · lore

Re: [PATCH v5 01/15] bugreport: add tool to generate debugging info

On Thu, Jan 30, 2020 at 11:18:55PM +0100, Martin Ågren wrote:
Show 16 quoted lines
> On Fri, 24 Jan 2020 at 05:56, <emilyshaffer@google.com> wrote:
> >
> > From: Emily Shaffer <emilyshaffer@google.com>
> >
> > Teach Git how to prompt the user for a good bug report: reproduction
> > steps, expected behavior, and actual behavior. Later, Git can learn how
> > to collect some diagnostic information from the repository.
> 
> ("Later" meaning "later in this series", or "any day now"? ;-) )
> 
> > +SYNOPSIS
> > +--------
> > +[verse]
> > +'git bugreport' [-o | --output <path>]
> 
> Hmm. Should that be "[(-o | --output) <path>]"?
Done.
Show 26 quoted lines
> > +DESCRIPTION
> > +-----------
> > +Captures information about the user's machine, Git client, and repository state,
> > +as well as a form requesting information about the behavior the user observed,
> > +into a single text file which the user can then share, for example to the Git
> > +mailing list, in order to report an observed bug.
> 
> Nice description. Got it.
> 
> > +The following information is requested from the user:
> > +
> > + - Reproduction steps
> > + - Expected behavior
> > + - Actual behavior
> > +
> > +The following information is captured automatically:
> > +
> > + - Git version (`git version --build-options`)
> > + - Machine information (`uname -a`)
> > + - Versions of various dependencies
> > + - Git config contents (`git config --show-origin --list`)
> > + - The names of all configured git-hooks in `.git/hooks/`
> 
> I would have expected these points to appear later, both to make it
> clear what this does commit does (and not), and to highlight what
> user-visible (documentation-worthy) changes later commits bring along.
Sure, agreed.
Show 7 quoted lines
> 
> > +OPTIONS
> > +-------
> > +-o [<path>]::
> > +--output [<path>]::
> 
> Drop the "[" and "]"? If you give -o, you'd better give a path as well?
Done.
> > +       Place the resulting bug report file in <path> instead of the root of the
> 
> `<path>`
Done.
Show 5 quoted lines
> > +"Please review the rest of the bug report below.\n"
> > +"You can delete any lines you don't wish to send.\n");
> 
> "send" sounds like we're *just* about to send this report somewhere, but
> it's "only" going to be written to the disk. Maybe "share", instead?
Nice turn of phrase, done.
Show 10 quoted lines
> > +       if (option_output) {
> > +               strbuf_addstr(&report_path, option_output);
> > +               strbuf_complete(&report_path, '/');
> > +       }
> 
> I thought I'd use `-o` to indicate the filename, but it turns out it's
> the *directory* where the file (of some semi-random, generated name)
> will end up. Re-reading the docs further up, I can see how this is
> consistent. I sort of wonder if this should be `--output*-directory*`
> for symmetry with `git format-patch`.
Sure.
Show 7 quoted lines
> > +       strbuf_addstr(&report_path, "git-bugreport-");
> > +       strbuf_addftime(&report_path, "%F", gmtime(&now), 0, 0);
> > +       strbuf_addstr(&report_path, ".txt");
> > +
> > +
> 
> (Double blank line?)
Done.
 
Show 5 quoted lines
> > +       get_bug_template(&buffer);
> > +
> > +       report = fopen_for_writing(report_path.buf);
> 
> Report might be NULL here.
Nice.
> If there's already such a file, we overwrite. Should we generate the
> filename using not just today's date (two bug reports in a day wouldn't
> be unheard of?) but also something like hh:mm:ss?

Sure. For the sake of brevity I'll probably neglect seconds; I hope someone is spending more than 1 minute filling in the provided form.

I'm a little worried about including : in a filename, so I went for 'git-bugreport-YYYY-MM-DD-HHMM' (24-hour).

As I started to write a test to ensure duplicate filenames were handled well, Jonathan Tan pointed out that it would be easy to add an arg like --suffix to allow specifying a custom strftime string. That would allow users to easily create a file named `git-bugreport-fetch-failing.txt` or `git-bugreport-March-19.txt` or whatever they want; it also makes testing easy. So I'll add this for the next rollup.

Show 12 quoted lines
> 
> > +       strbuf_write(&buffer, report);
> > +       fclose(report);
> 
> Maybe clear the strbuf around here...
> 
> > +       launch_editor(report_path.buf, NULL, NULL);
> > +       return 0;
> 
> ... and/or UNLEAK it here, together with report_path.
> 
> Maybe "return -launch_editor(...)"?

Hm, sure. I see that builtin/tag.c does mark strbufs this way, so I don't see a problem using UNLEAK and tail-calling launch_editor().

As a final bonus, I also added a line to report to stderr the name of the file that was created. I noticed it's sort of unclear what the command actually did otherwise.

> > +#!/bin/bash
> 
> Use /bin/sh instead?

Yeah, doing so immediately pointed out the bashisms you mentioned, plus some more. :facepalm:

Show 33 quoted lines
> 
> > +# Headers "[System Info]" will be followed by a non-empty line if we put some
> > +# information there; we can make sure all our headers were followed by some
> > +# information to check if the command was successful.
> > +HEADER_PATTERN="^\[.*\]$"
> > +check_all_headers_populated() {
> > +       while read -r line; do
> > +               if [$(grep $HEADER_PATTERN $line)]; then
> 
> I think this is a bash-ism.
> 
> > +                       read -r nextline
> > +                       if [-z $nextline]; then
> 
> Likewise.
> 
> > +                               return 1;
> > +                       fi
> > +               fi
> > +       done
> > +}
> > +
> > +test_expect_success 'creates a report with content in the right places' '
> > +       git bugreport &&
> > +       check_all_headers_populated <git-bugreport-* &&
> > +       rm git-bugreport-*
> > +'
> > +
> > +test_expect_success '--output puts the report in the provided dir' '
> > +       mkdir foo/ &&
> 
> If foo isn't there, do we not create it? Apparently not -- in my
> testing, we segfault. (We don't check for NULL after opening the file.)

Yeah, at the moment I just added a die() if we can't open the provided path. I think other utilties can create the path (e.g. git-format-patch) but where it makes sense to do so, I'd prefer to keep bugreport very simple.

I'll die instead of overwriting, too; you're right that spending quite a while on a bug report and then accidentally writing over it with a blank one would be a very bad user experience.

Show 5 quoted lines
> 
> > +       git bugreport -o foo/ &&
> > +       test -f foo/git-bugreport-* &&
> 
> test_path_is_file
Sure.
Show 9 quoted lines
> > +       rm -fr foo/
> > +'
> > +
> > +test_expect_success 'incorrect arguments abort with usage' '
> > +       test_must_fail git bugreport --false 2>output &&
> > +       grep usage output &&
> > +       test ! -f git-bugreport-*
> 
> test_path_is_missing
OK.

Thanks very much, Martin, for the thorough review. This is incredibly helpful.

 - Emily
emilyshaffer@google.com· Jan 24, 2020, 03:34 UTC · re: emilyshaffer@google.com · lore

[PATCH v5 04/15] help: add shell-path to --build-options

From: Emily Shaffer <emilyshaffer@google.com>

It may be useful to know which shell Git was built to try to point to, in the event that shell-based Git commands are failing. $SHELL_PATH is set during the build and used to launch the manpage viewer, as well as by git-compat-util.h, and it's used during tests. 'git version --build-options' is encouraged for use in bug reports, so it makes sense to include this information there.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 help.c | 1 +
 1 file changed, 1 insertion(+)
Show changes to help.c +1 −0
diff --git a/help.c b/help.c
index 156a6cdea3..d3f941824f 100644
--- a/help.c
+++ b/help.c
@@ -642,6 +642,7 @@ void get_version_info(struct strbuf *buf, int show_build_options)
 			strbuf_addstr(buf, "no commit associated with this build\n");
 		strbuf_addf(buf, "sizeof-long: %d\n", (int)sizeof(long));
 		strbuf_addf(buf, "sizeof-size_t: %d\n", (int)sizeof(size_t));
+		strbuf_addf(buf, "shell-path: %s\n", SHELL_PATH);
 		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
 	}
 }
-- 
2.25.0.341.g760bfbb309-goog
Martin Ågren· Jan 30, 2020, 22:21 UTC · re: emilyshaffer@google.com · lore

Re: [PATCH v5 04/15] help: add shell-path to --build-options

On Fri, 24 Jan 2020 at 04:40, <emilyshaffer@google.com> wrote:
Show 6 quoted lines
> It may be useful to know which shell Git was built to try to point to,
> in the event that shell-based Git commands are failing. $SHELL_PATH is
> set during the build and used to launch the manpage viewer, as well as
> by git-compat-util.h, and it's used during tests. 'git version
> --build-options' is encouraged for use in bug reports, so it makes sense
> to include this information there.

Hmm, similar nit as for patch 02/15: This could go in as, e.g., 02/15 and your 01/15 would be 03/15: This patch would be useful even without `git bugreport`.

Martin
emilyshaffer@google.com· Jan 24, 2020, 03:34 UTC · re: emilyshaffer@google.com · lore

[PATCH v5 03/15] bugreport: gather git version and build info

From: Emily Shaffer <emilyshaffer@google.com>

Knowing which version of Git a user has and how it was built allows us to more precisely pin down the circumstances when a certain issue occurs, so teach bugreport how to tell us the same output as 'git version --build-options'.

It's not ideal to directly call 'git version --build-options' because that output goes to stdout. Instead, wrap the version string in a helper within help.[ch] library, and call that helper from within the bugreport library.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 bugreport.c | 22 +++++++++++++++++++++-
 help.c      | 47 +++++++++++++++++++++++++++++------------------
 help.h      |  1 +
 3 files changed, 51 insertions(+), 19 deletions(-)
Show changes to 3 files +51 −19

bugreport.c, help.c, help.h

diff --git a/bugreport.c b/bugreport.c
index 5495b31674..b1a5a279ac 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -1,8 +1,20 @@
-#include "builtin.h"
+#include "cache.h"
 #include "parse-options.h"
 #include "stdio.h"
 #include "strbuf.h"
 #include "time.h"
+#include "help.h"
+
+static void get_system_info(struct strbuf *sys_info)
+{
+	struct strbuf version_info = STRBUF_INIT;
+
+	/* get git version from native cmd */
+	strbuf_addstr(sys_info, "git version:\n");
+	get_version_info(&version_info, 1);
+	strbuf_addbuf(sys_info, &version_info);
+	strbuf_complete_line(sys_info);
+}
 
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output <file>]"),
@@ -32,6 +44,11 @@ static int get_bug_template(struct strbuf *template)
 	return 0;
 }
 
+static void get_header(struct strbuf *buf, const char *title)
+{
+	strbuf_addf(buf, "\n\n[%s]\n", title);
+}
+
 int cmd_main(int argc, const char **argv)
 {
 	struct strbuf buffer = STRBUF_INIT;
@@ -60,6 +77,9 @@ int cmd_main(int argc, const char **argv)
 
 	get_bug_template(&buffer);
 
+	get_header(&buffer, "System Info");
+	get_system_info(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 	strbuf_write(&buffer, report);
 	fclose(report);
diff --git a/help.c b/help.c
index a21487db77..156a6cdea3 100644
--- a/help.c
+++ b/help.c
@@ -622,8 +622,33 @@ const char *help_unknown_cmd(const char *cmd)
 	exit(1);
 }
 
+void get_version_info(struct strbuf *buf, int show_build_options)
+{
+	strbuf_reset(buf);
+	/*
+	 * The format of this string should be kept stable for compatibility
+	 * with external projects that rely on the output of "git version".
+	 *
+	 * Always show the version, even if other options are given.
+	 */
+	strbuf_addf(buf, "git version %s\n", git_version_string);
+
+	if (show_build_options) {
+		strbuf_addf(buf, "cpu: %s\n", GIT_HOST_CPU);
+		if (git_built_from_commit_string[0])
+			strbuf_addf(buf, "built from commit: %s\n",
+			       git_built_from_commit_string);
+		else
+			strbuf_addstr(buf, "no commit associated with this build\n");
+		strbuf_addf(buf, "sizeof-long: %d\n", (int)sizeof(long));
+		strbuf_addf(buf, "sizeof-size_t: %d\n", (int)sizeof(size_t));
+		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
+	}
+}
+
 int cmd_version(int argc, const char **argv, const char *prefix)
 {
+	struct strbuf buf = STRBUF_INIT;
 	int build_options = 0;
 	const char * const usage[] = {
 		N_("git version [<options>]"),
@@ -637,25 +662,11 @@ int cmd_version(int argc, const char **argv, const char *prefix)
 
 	argc = parse_options(argc, argv, prefix, options, usage, 0);
 
-	/*
-	 * The format of this string should be kept stable for compatibility
-	 * with external projects that rely on the output of "git version".
-	 *
-	 * Always show the version, even if other options are given.
-	 */
-	printf("git version %s\n", git_version_string);
+	get_version_info(&buf, build_options);
+	printf("%s", buf.buf);
+
+	strbuf_release(&buf);
 
-	if (build_options) {
-		printf("cpu: %s\n", GIT_HOST_CPU);
-		if (git_built_from_commit_string[0])
-			printf("built from commit: %s\n",
-			       git_built_from_commit_string);
-		else
-			printf("no commit associated with this build\n");
-		printf("sizeof-long: %d\n", (int)sizeof(long));
-		printf("sizeof-size_t: %d\n", (int)sizeof(size_t));
-		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
-	}
 	return 0;
 }
 
diff --git a/help.h b/help.h
index 9071894e8c..500521b908 100644
--- a/help.h
+++ b/help.h
@@ -37,6 +37,7 @@ void add_cmdname(struct cmdnames *cmds, const char *name, int len);
 void exclude_cmds(struct cmdnames *cmds, struct cmdnames *excludes);
 int is_in_cmdlist(struct cmdnames *cmds, const char *name);
 void list_commands(unsigned int colopts, struct cmdnames *main_cmds, struct cmdnames *other_cmds);
+void get_version_info(struct strbuf *buf, int show_build_options);
 
 /*
  * call this to die(), when it is suspected that the user mistyped a
-- 
2.25.0.341.g760bfbb309-goog
Martin Ågren· Jan 30, 2020, 22:19 UTC · re: emilyshaffer@google.com · lore

Re: [PATCH v5 03/15] bugreport: gather git version and build info

On Fri, 24 Jan 2020 at 04:40, <emilyshaffer@google.com> wrote:
Show 9 quoted lines
> +static void get_system_info(struct strbuf *sys_info)
> +{
> +       struct strbuf version_info = STRBUF_INIT;
> +
> +       /* get git version from native cmd */
> +       strbuf_addstr(sys_info, "git version:\n");
> +       get_version_info(&version_info, 1);
> +       strbuf_addbuf(sys_info, &version_info);
> +       strbuf_complete_line(sys_info);
This leaks version_info.
> +}
Martin
Emily Shaffer· Feb 4, 2020, 22:21 UTC · re: Martin Ågren · lore

Re: [PATCH v5 03/15] bugreport: gather git version and build info

On Thu, Jan 30, 2020 at 11:19:55PM +0100, Martin Ågren wrote:
Show 12 quoted lines
> On Fri, 24 Jan 2020 at 04:40, <emilyshaffer@google.com> wrote:
> > +static void get_system_info(struct strbuf *sys_info)
> > +{
> > +       struct strbuf version_info = STRBUF_INIT;
> > +
> > +       /* get git version from native cmd */
> > +       strbuf_addstr(sys_info, "git version:\n");
> > +       get_version_info(&version_info, 1);
> > +       strbuf_addbuf(sys_info, &version_info);
> > +       strbuf_complete_line(sys_info);
> 
> This leaks version_info.
So it does.

I think the easiest thing is to drop the 'strbuf_reset()' at the top of help.c:get_version_info(), and just pass in sys_info from bugreport.c.

 - Emily
emilyshaffer@google.com· Jan 24, 2020, 03:34 UTC · re: emilyshaffer@google.com · lore

[PATCH v5 02/15] help: move list_config_help to builtin/help

From: Emily Shaffer <emilyshaffer@google.com>

Starting in 3ac68a93fd2, help.o began to depend on builtin/branch.o, builtin/clean.o, and builtin/config.o. This meant that help.o was unusable outside of the context of the main Git executable.

To make help.o usable by other commands again, move list_config_help() into builtin/help.c (where it makes sense to assume other builtin libraries are present).

When command-list.h is included but a member is not used, we start to hear a compiler warning. Since the config list is generated in a fairly different way than the command list, and since commands and config options are semantically different, move the config list into its own header and move the generator into its own script and build rule.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 .gitignore             |  1 +
 Makefile               | 13 +++++--
 builtin/help.c         | 86 ++++++++++++++++++++++++++++++++++++++++++
 generate-cmdlist.sh    | 19 ----------
 generate-configlist.sh | 24 ++++++++++++
 help.c                 | 85 -----------------------------------------
 help.h                 |  1 -
 7 files changed, 121 insertions(+), 108 deletions(-)
 create mode 100755 generate-configlist.sh
Show changes to 7 files +121 −108

.gitignore, Makefile, builtin/help.c, generate-cmdlist.sh, generate-configlist.sh, help.c, help.h

diff --git a/.gitignore b/.gitignore
index ca301bc890..d89bf9e11e 100644
--- a/.gitignore
+++ b/.gitignore
@@ -190,6 +190,7 @@
 /gitweb/gitweb.cgi
 /gitweb/static/gitweb.js
 /gitweb/static/gitweb.min.*
+/config-list.h
 /command-list.h
 *.tar.gz
 *.dsc
diff --git a/Makefile b/Makefile
index f271619371..a01a050aa3 100644
--- a/Makefile
+++ b/Makefile
@@ -815,6 +815,7 @@ LIB_FILE = libgit.a
 XDIFF_LIB = xdiff/lib.a
 VCSSVN_LIB = vcs-svn/lib.a
 
+GENERATED_H += config-list.h
 GENERATED_H += command-list.h
 
 LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
@@ -2129,7 +2130,7 @@ git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS)
 
 help.sp help.s help.o: command-list.h
 
-builtin/help.sp builtin/help.s builtin/help.o: command-list.h GIT-PREFIX
+builtin/help.sp builtin/help.s builtin/help.o: config-list.h GIT-PREFIX
 builtin/help.sp builtin/help.s builtin/help.o: EXTRA_CPPFLAGS = \
 	'-DGIT_HTML_PATH="$(htmldir_relative_SQ)"' \
 	'-DGIT_MAN_PATH="$(mandir_relative_SQ)"' \
@@ -2149,6 +2150,12 @@ $(BUILT_INS): git$X
 	ln -s $< $@ 2>/dev/null || \
 	cp $< $@
 
+config-list.h: generate-configlist.sh
+
+config-list.h:
+	$(QUIET_GEN)$(SHELL_PATH) ./generate-configlist.sh \
+		>$@+ && mv $@+ $@
+
 command-list.h: generate-cmdlist.sh command-list.txt
 
 command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Documentation/config/*.txt
@@ -2786,7 +2793,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE
 .PHONY: sparse $(SP_OBJ)
 sparse: $(SP_OBJ)
 
-EXCEPT_HDRS := command-list.h unicode-width.h compat/% xdiff/%
+EXCEPT_HDRS := command-list.h config-list.h unicode-width.h compat/% xdiff/%
 ifndef GCRYPT_SHA256
 	EXCEPT_HDRS += sha256/gcrypt.h
 endif
@@ -2808,7 +2815,7 @@ hdr-check: $(HCO)
 style:
 	git clang-format --style file --diff --extensions c,h
 
-check: command-list.h
+check: config-list.h command-list.h
 	@if sparse; \
 	then \
 		echo >&2 "Use 'make sparse' instead"; \
diff --git a/builtin/help.c b/builtin/help.c
index e5590d7787..1c5f2b9255 100644
--- a/builtin/help.c
+++ b/builtin/help.c
@@ -8,6 +8,7 @@
 #include "parse-options.h"
 #include "run-command.h"
 #include "column.h"
+#include "config-list.h"
 #include "help.h"
 #include "alias.h"
 
@@ -62,6 +63,91 @@ static const char * const builtin_help_usage[] = {
 	NULL
 };
 
+struct slot_expansion {
+	const char *prefix;
+	const char *placeholder;
+	void (*fn)(struct string_list *list, const char *prefix);
+	int found;
+};
+
+static void list_config_help(int for_human)
+{
+	struct slot_expansion slot_expansions[] = {
+		{ "advice", "*", list_config_advices },
+		{ "color.branch", "<slot>", list_config_color_branch_slots },
+		{ "color.decorate", "<slot>", list_config_color_decorate_slots },
+		{ "color.diff", "<slot>", list_config_color_diff_slots },
+		{ "color.grep", "<slot>", list_config_color_grep_slots },
+		{ "color.interactive", "<slot>", list_config_color_interactive_slots },
+		{ "color.remote", "<slot>", list_config_color_sideband_slots },
+		{ "color.status", "<slot>", list_config_color_status_slots },
+		{ "fsck", "<msg-id>", list_config_fsck_msg_ids },
+		{ "receive.fsck", "<msg-id>", list_config_fsck_msg_ids },
+		{ NULL, NULL, NULL }
+	};
+	const char **p;
+	struct slot_expansion *e;
+	struct string_list keys = STRING_LIST_INIT_DUP;
+	int i;
+
+	for (p = config_name_list; *p; p++) {
+		const char *var = *p;
+		struct strbuf sb = STRBUF_INIT;
+
+		for (e = slot_expansions; e->prefix; e++) {
+
+			strbuf_reset(&sb);
+			strbuf_addf(&sb, "%s.%s", e->prefix, e->placeholder);
+			if (!strcasecmp(var, sb.buf)) {
+				e->fn(&keys, e->prefix);
+				e->found++;
+				break;
+			}
+		}
+		strbuf_release(&sb);
+		if (!e->prefix)
+			string_list_append(&keys, var);
+	}
+
+	for (e = slot_expansions; e->prefix; e++)
+		if (!e->found)
+			BUG("slot_expansion %s.%s is not used",
+			    e->prefix, e->placeholder);
+
+	string_list_sort(&keys);
+	for (i = 0; i < keys.nr; i++) {
+		const char *var = keys.items[i].string;
+		const char *wildcard, *tag, *cut;
+
+		if (for_human) {
+			puts(var);
+			continue;
+		}
+
+		wildcard = strchr(var, '*');
+		tag = strchr(var, '<');
+
+		if (!wildcard && !tag) {
+			puts(var);
+			continue;
+		}
+
+		if (wildcard && !tag)
+			cut = wildcard;
+		else if (!wildcard && tag)
+			cut = tag;
+		else
+			cut = wildcard < tag ? wildcard : tag;
+
+		/*
+		 * We may produce duplicates, but that's up to
+		 * git-completion.bash to handle
+		 */
+		printf("%.*s\n", (int)(cut - var), var);
+	}
+	string_list_clear(&keys, 0);
+}
+
 static enum help_format parse_help_format(const char *format)
 {
 	if (!strcmp(format, "man"))
diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
index 71158f7d8b..45fecf8bdf 100755
--- a/generate-cmdlist.sh
+++ b/generate-cmdlist.sh
@@ -76,23 +76,6 @@ print_command_list () {
 	echo "};"
 }
 
-print_config_list () {
-	cat <<EOF
-static const char *config_name_list[] = {
-EOF
-	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
-	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
-	sort |
-	while read line
-	do
-		echo "	\"$line\","
-	done
-	cat <<EOF
-	NULL,
-};
-EOF
-}
-
 exclude_programs=
 while test "--exclude-program" = "$1"
 do
@@ -113,5 +96,3 @@ echo
 define_category_names "$1"
 echo
 print_command_list "$1"
-echo
-print_config_list
diff --git a/generate-configlist.sh b/generate-configlist.sh
new file mode 100755
index 0000000000..eca6a00c30
--- /dev/null
+++ b/generate-configlist.sh
@@ -0,0 +1,24 @@
+#!/bin/sh
+
+echo "/* Automatically generated by generate-configlist.sh */"
+echo
+
+print_config_list () {
+	cat <<EOF
+static const char *config_name_list[] = {
+EOF
+	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
+	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
+	sort |
+	while read line
+	do
+		echo "	\"$line\","
+	done
+	cat <<EOF
+	NULL,
+};
+EOF
+}
+
+echo
+print_config_list
diff --git a/help.c b/help.c
index cf67624a94..a21487db77 100644
--- a/help.c
+++ b/help.c
@@ -407,91 +407,6 @@ void list_common_guides_help(void)
 	putchar('\n');
 }
 
-struct slot_expansion {
-	const char *prefix;
-	const char *placeholder;
-	void (*fn)(struct string_list *list, const char *prefix);
-	int found;
-};
-
-void list_config_help(int for_human)
-{
-	struct slot_expansion slot_expansions[] = {
-		{ "advice", "*", list_config_advices },
-		{ "color.branch", "<slot>", list_config_color_branch_slots },
-		{ "color.decorate", "<slot>", list_config_color_decorate_slots },
-		{ "color.diff", "<slot>", list_config_color_diff_slots },
-		{ "color.grep", "<slot>", list_config_color_grep_slots },
-		{ "color.interactive", "<slot>", list_config_color_interactive_slots },
-		{ "color.remote", "<slot>", list_config_color_sideband_slots },
-		{ "color.status", "<slot>", list_config_color_status_slots },
-		{ "fsck", "<msg-id>", list_config_fsck_msg_ids },
-		{ "receive.fsck", "<msg-id>", list_config_fsck_msg_ids },
-		{ NULL, NULL, NULL }
-	};
-	const char **p;
-	struct slot_expansion *e;
-	struct string_list keys = STRING_LIST_INIT_DUP;
-	int i;
-
-	for (p = config_name_list; *p; p++) {
-		const char *var = *p;
-		struct strbuf sb = STRBUF_INIT;
-
-		for (e = slot_expansions; e->prefix; e++) {
-
-			strbuf_reset(&sb);
-			strbuf_addf(&sb, "%s.%s", e->prefix, e->placeholder);
-			if (!strcasecmp(var, sb.buf)) {
-				e->fn(&keys, e->prefix);
-				e->found++;
-				break;
-			}
-		}
-		strbuf_release(&sb);
-		if (!e->prefix)
-			string_list_append(&keys, var);
-	}
-
-	for (e = slot_expansions; e->prefix; e++)
-		if (!e->found)
-			BUG("slot_expansion %s.%s is not used",
-			    e->prefix, e->placeholder);
-
-	string_list_sort(&keys);
-	for (i = 0; i < keys.nr; i++) {
-		const char *var = keys.items[i].string;
-		const char *wildcard, *tag, *cut;
-
-		if (for_human) {
-			puts(var);
-			continue;
-		}
-
-		wildcard = strchr(var, '*');
-		tag = strchr(var, '<');
-
-		if (!wildcard && !tag) {
-			puts(var);
-			continue;
-		}
-
-		if (wildcard && !tag)
-			cut = wildcard;
-		else if (!wildcard && tag)
-			cut = tag;
-		else
-			cut = wildcard < tag ? wildcard : tag;
-
-		/*
-		 * We may produce duplicates, but that's up to
-		 * git-completion.bash to handle
-		 */
-		printf("%.*s\n", (int)(cut - var), var);
-	}
-	string_list_clear(&keys, 0);
-}
-
 static int get_alias(const char *var, const char *value, void *data)
 {
 	struct string_list *list = data;
diff --git a/help.h b/help.h
index 7a455beeb7..9071894e8c 100644
--- a/help.h
+++ b/help.h
@@ -22,7 +22,6 @@ static inline void mput_char(char c, unsigned int num)
 void list_common_cmds_help(void);
 void list_all_cmds_help(void);
 void list_common_guides_help(void);
-void list_config_help(int for_human);
 
 void list_all_main_cmds(struct string_list *list);
 void list_all_other_cmds(struct string_list *list);
-- 
2.25.0.341.g760bfbb309-goog
Martin Ågren· Jan 30, 2020, 22:19 UTC · re: emilyshaffer@google.com · lore

Re: [PATCH v5 02/15] help: move list_config_help to builtin/help

On Fri, 24 Jan 2020 at 04:40, <emilyshaffer@google.com> wrote:
> To make help.o usable by other commands again, move list_config_help()
> into builtin/help.c (where it makes sense to assume other builtin libraries
> are present).
Nit: I think this could be patch 01/15. It shuffles things around --
*then*, starting with patch 02/15, we can add exciting stuff.
Martin
Emily Shaffer· Feb 4, 2020, 00:53 UTC · re: Martin Ågren · lore

Re: [PATCH v5 02/15] help: move list_config_help to builtin/help

On Thu, Jan 30, 2020 at 11:19:26PM +0100, Martin Ågren wrote:
Show 7 quoted lines
> On Fri, 24 Jan 2020 at 04:40, <emilyshaffer@google.com> wrote:
> > To make help.o usable by other commands again, move list_config_help()
> > into builtin/help.c (where it makes sense to assume other builtin libraries
> > are present).
> 
> Nit: I think this could be patch 01/15. It shuffles things around --
> *then*, starting with patch 02/15, we can add exciting stuff.
Sounds reasonable, here and 04/15. Will do.
emilyshaffer@google.com· Jan 24, 2020, 03:34 UTC · re: emilyshaffer@google.com · lore

[PATCH v5 05/15] bugreport: add uname info

From: Emily Shaffer <emilyshaffer@google.com>

The contents of uname() can give us some insight into what sort of system the user is running on, and help us replicate their setup if need be. The domainname field is not guaranteed to be available, so don't collect it.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 bugreport.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)
Show changes to bugreport.c +12 −0
diff --git a/bugreport.c b/bugreport.c
index b1a5a279ac..720c91e1bd 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -8,12 +8,24 @@
 static void get_system_info(struct strbuf *sys_info)
 {
 	struct strbuf version_info = STRBUF_INIT;
+	struct utsname uname_info;
 
 	/* get git version from native cmd */
 	strbuf_addstr(sys_info, "git version:\n");
 	get_version_info(&version_info, 1);
 	strbuf_addbuf(sys_info, &version_info);
 	strbuf_complete_line(sys_info);
+
+	/* system call for other version info */
+	strbuf_addstr(sys_info, "uname -a: ");
+	if (uname(&uname_info))
+		strbuf_addf(sys_info, "uname() failed with code %d\n", errno);
+	else
+		strbuf_addf(sys_info, "%s %s %s %s\n",
+			    uname_info.sysname,
+			    uname_info.release,
+			    uname_info.version,
+			    uname_info.machine);
 }
 
 static const char * const bugreport_usage[] = {
-- 
2.25.0.341.g760bfbb309-goog
emilyshaffer@google.com· Jan 24, 2020, 03:34 UTC · re: emilyshaffer@google.com · lore

[PATCH v5 06/15] bugreport: add compiler info

From: Emily Shaffer <emilyshaffer@google.com>

To help pinpoint the source of a regression, it is useful to know some info about the compiler which the user's Git client was built with. By adding a generic get_compiler_info() in 'compat/' we can choose which relevant information to share per compiler; to get started, let's demonstrate the version of glibc if the user built with 'gcc'.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 bugreport.c       |  5 +++++
 compat/compiler.h | 24 ++++++++++++++++++++++++
 2 files changed, 29 insertions(+)
 create mode 100644 compat/compiler.h
Show changes to 2 files +29 −0

bugreport.c, compat/compiler.h

diff --git a/bugreport.c b/bugreport.c
index 720c91e1bd..818ccb385c 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -4,6 +4,7 @@
 #include "strbuf.h"
 #include "time.h"
 #include "help.h"
+#include "compat/compiler.h"
 
 static void get_system_info(struct strbuf *sys_info)
 {
@@ -26,6 +27,10 @@ static void get_system_info(struct strbuf *sys_info)
 			    uname_info.release,
 			    uname_info.version,
 			    uname_info.machine);
+
+	strbuf_addstr(sys_info, "compiler info: ");
+	get_compiler_info(sys_info);
+	strbuf_complete_line(sys_info);
 }
 
 static const char * const bugreport_usage[] = {
diff --git a/compat/compiler.h b/compat/compiler.h
new file mode 100644
index 0000000000..bda5098e1b
--- /dev/null
+++ b/compat/compiler.h
@@ -0,0 +1,24 @@
+#ifndef COMPILER_H
+#define COMPILER_H
+
+#include "git-compat-util.h"
+#include "strbuf.h"
+
+#ifdef __GLIBC__
+#include <gnu/libc-version.h>
+
+static inline void get_compiler_info(struct strbuf *info)
+{
+	strbuf_addf(info, "glibc: %s", gnu_get_libc_version());
+}
+
+#else
+
+static inline void get_compiler_info(struct strbuf *info)
+{
+	strbuf_addstr(info, "get_compiler_info() not implemented");
+}
+
+#endif
+
+#endif /* COMPILER_H */
-- 
2.25.0.341.g760bfbb309-goog
Martin Ågren· Jan 30, 2020, 22:21 UTC · re: emilyshaffer@google.com · lore

Re: [PATCH v5 06/15] bugreport: add compiler info

On Fri, 24 Jan 2020 at 04:40, <emilyshaffer@google.com> wrote:
Show 5 quoted lines
> +#else
> +
> +static inline void get_compiler_info(struct strbuf *info)
> +{
> +       strbuf_addstr(info, "get_compiler_info() not implemented");

Maybe "no compiler info available" (or s/ available//, or s/available/reported/), or something else more human-readable?

Show 5 quoted lines
> +}
> +
> +#endif
> +
> +#endif /* COMPILER_H */
Martin
Emily Shaffer· Feb 4, 2020, 22:51 UTC · re: Martin Ågren · lore

Re: [PATCH v5 06/15] bugreport: add compiler info

On Thu, Jan 30, 2020 at 11:21:39PM +0100, Martin Ågren wrote:
Show 9 quoted lines
> On Fri, 24 Jan 2020 at 04:40, <emilyshaffer@google.com> wrote:
> > +#else
> > +
> > +static inline void get_compiler_info(struct strbuf *info)
> > +{
> > +       strbuf_addstr(info, "get_compiler_info() not implemented");
> 
> Maybe "no compiler info available" (or s/ available//, or
> s/available/reported/), or something else more human-readable?

Hm. I envisioned the target audience for this as "Git developers" (and that's why I didn't mark the unimplemented string for translation); Git developers know exactly what "get_compiler_info() not implemented" means. But I suppose it's just as easy to grep for one string as the other.

I am hesitant to say "no info available" or "no info reported" when I'm certain there is some environment variable set at build time which contains the info we'd want to show here; it really is just a question of some Git developer having put in the time to implement this function.

So, I think I will leave it. Thanks for your suggestions.
 - Emily
Martin Ågren· Feb 5, 2020, 19:47 UTC · re: Emily Shaffer · lore

Re: [PATCH v5 06/15] bugreport: add compiler info

On Tue, 4 Feb 2020 at 23:51, Emily Shaffer <emilyshaffer@google.com> wrote:
Show 24 quoted lines
>
> On Thu, Jan 30, 2020 at 11:21:39PM +0100, Martin Ågren wrote:
> > On Fri, 24 Jan 2020 at 04:40, <emilyshaffer@google.com> wrote:
> > > +#else
> > > +
> > > +static inline void get_compiler_info(struct strbuf *info)
> > > +{
> > > +       strbuf_addstr(info, "get_compiler_info() not implemented");
> >
> > Maybe "no compiler info available" (or s/ available//, or
> > s/available/reported/), or something else more human-readable?
>
> Hm. I envisioned the target audience for this as "Git developers" (and
> that's why I didn't mark the unimplemented string for translation); Git
> developers know exactly what "get_compiler_info() not implemented"
> means. But I suppose it's just as easy to grep for one string as the
> other.
>
> I am hesitant to say "no info available" or "no info reported" when I'm
> certain there is some environment variable set at build time which
> contains the info we'd want to show here; it really is just a question
> of some Git developer having put in the time to implement this function.
>
> So, I think I will leave it. Thanks for your suggestions.
Ok, that makes sense.
Martin
emilyshaffer@google.com· Jan 24, 2020, 03:34 UTC · re: emilyshaffer@google.com · lore

[PATCH v5 07/15] bugreport: add curl version

From: Emily Shaffer <emilyshaffer@google.com>

It's possible for git-remote-curl to be built separately from git; in that case we want to know what version of cURL is used by git-remote-curl, not necessarily which version was present at git-bugreport's build time. So instead, ask git-remote-curl for the version information it knows about.

For longevity purposes, invoke the alias "git-remote-https" instead of "git-remote-http".

Since it could have been built at a different time, also report the version and built-from commit of git-remote-curl alongside the cURL info.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 bugreport.c   | 16 ++++++++++++++++
 remote-curl.c |  8 ++++++++
 2 files changed, 24 insertions(+)
Show changes to 2 files +24 −0

bugreport.c, remote-curl.c

diff --git a/bugreport.c b/bugreport.c
index 818ccb385c..73f6d39517 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -5,6 +5,18 @@
 #include "time.h"
 #include "help.h"
 #include "compat/compiler.h"
+#include "run-command.h"
+
+static void get_curl_version_info(struct strbuf *curl_info)
+{
+	struct child_process cp = CHILD_PROCESS_INIT;
+
+	argv_array_push(&cp.args, "git");
+	argv_array_push(&cp.args, "remote-https");
+	argv_array_push(&cp.args, "--build-info");
+	if (capture_command(&cp, curl_info, 0))
+	    strbuf_addstr(curl_info, "'git-remote-https --build-info' not supported\n");
+}
 
 static void get_system_info(struct strbuf *sys_info)
 {
@@ -31,6 +43,10 @@ static void get_system_info(struct strbuf *sys_info)
 	strbuf_addstr(sys_info, "compiler info: ");
 	get_compiler_info(sys_info);
 	strbuf_complete_line(sys_info);
+
+	strbuf_addstr(sys_info, "git-remote-https --build-info:\n");
+	get_curl_version_info(sys_info);
+	strbuf_complete_line(sys_info);
 }
 
 static const char * const bugreport_usage[] = {
diff --git a/remote-curl.c b/remote-curl.c
index 350d92a074..c590fbfae3 100644
--- a/remote-curl.c
+++ b/remote-curl.c
@@ -17,6 +17,7 @@
 #include "protocol.h"
 #include "quote.h"
 #include "transport.h"
+#include "version.h"
 
 static struct remote *remote;
 /* always ends with a trailing slash */
@@ -1374,6 +1375,13 @@ int cmd_main(int argc, const char **argv)
 	string_list_init(&options.deepen_not, 1);
 	string_list_init(&options.push_options, 1);
 
+	if (!strcmp("--build-info", argv[1])) {
+		printf("git-http-fetch version: %s\n", git_version_string);
+		printf("built from commit: %s\n", git_built_from_commit_string);
+		printf("curl version: %s\n", curl_version());
+		return 0;
+	}
+
 	/*
 	 * Just report "remote-curl" here (folding all the various aliases
 	 * ("git-remote-http", "git-remote-https", and etc.) here since they
-- 
2.25.0.341.g760bfbb309-goog
Martin Ågren· Jan 30, 2020, 22:27 UTC · re: emilyshaffer@google.com · lore

Re: [PATCH v5 07/15] bugreport: add curl version

On Fri, 24 Jan 2020 at 04:41, <emilyshaffer@google.com> wrote:
Show 20 quoted lines
> +static void get_curl_version_info(struct strbuf *curl_info)
> +{
> +       struct child_process cp = CHILD_PROCESS_INIT;
> +
> +       argv_array_push(&cp.args, "git");
> +       argv_array_push(&cp.args, "remote-https");
> +       argv_array_push(&cp.args, "--build-info");
> +       if (capture_command(&cp, curl_info, 0))
> +           strbuf_addstr(curl_info, "'git-remote-https --build-info' not supported\n");
> +}
>
>  static void get_system_info(struct strbuf *sys_info)
>  {
> @@ -31,6 +43,10 @@ static void get_system_info(struct strbuf *sys_info)
>         strbuf_addstr(sys_info, "compiler info: ");
>         get_compiler_info(sys_info);
>         strbuf_complete_line(sys_info);
> +
> +       strbuf_addstr(sys_info, "git-remote-https --build-info:\n");
> +       get_curl_version_info(sys_info);

The header here looks a lot like an implementation detail of `get_curl_version_info()`. Or put differently, these risk getting out of sync. Maybe frame the header a bit more human readable: "curl version". But is this "curl version", or more like "git-remote-https version"? There's some discrepancy here.

> +       strbuf_complete_line(sys_info);
>  }
Martin
Emily Shaffer· Feb 4, 2020, 22:54 UTC · re: Martin Ågren · lore

Re: [PATCH v5 07/15] bugreport: add curl version

On Thu, Jan 30, 2020 at 11:27:45PM +0100, Martin Ågren wrote:
Show 27 quoted lines
> On Fri, 24 Jan 2020 at 04:41, <emilyshaffer@google.com> wrote:
> > +static void get_curl_version_info(struct strbuf *curl_info)
> > +{
> > +       struct child_process cp = CHILD_PROCESS_INIT;
> > +
> > +       argv_array_push(&cp.args, "git");
> > +       argv_array_push(&cp.args, "remote-https");
> > +       argv_array_push(&cp.args, "--build-info");
> > +       if (capture_command(&cp, curl_info, 0))
> > +           strbuf_addstr(curl_info, "'git-remote-https --build-info' not supported\n");
> > +}
> >
> >  static void get_system_info(struct strbuf *sys_info)
> >  {
> > @@ -31,6 +43,10 @@ static void get_system_info(struct strbuf *sys_info)
> >         strbuf_addstr(sys_info, "compiler info: ");
> >         get_compiler_info(sys_info);
> >         strbuf_complete_line(sys_info);
> > +
> > +       strbuf_addstr(sys_info, "git-remote-https --build-info:\n");
> > +       get_curl_version_info(sys_info);
> 
> The header here looks a lot like an implementation detail of
> `get_curl_version_info()`. Or put differently, these risk getting out of
> sync. Maybe frame the header a bit more human readable: "curl version".
> But is this "curl version", or more like "git-remote-https version"?
> There's some discrepancy here.

Hm, I think you're saying "If we switch to future-https-lib instead of cURL for git-remote-https, then this command will be incorrectly named." Sure, I agree. It's true that with this change git-remote-https also tells us some info about itself.

Thanks.
 - Emily
emilyshaffer@google.com· Jan 24, 2020, 03:34 UTC · re: emilyshaffer@google.com · lore

[PATCH v5 08/15] bugreport: include user interactive shell

From: Emily Shaffer <emilyshaffer@google.com>

It's possible a user may complain about the way that Git interacts with their interactive shell, e.g. autocompletion or shell prompt. In that case, it's useful for us to know which shell they're using interactively.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 bugreport.c | 5 +++++
 1 file changed, 5 insertions(+)
Show changes to bugreport.c +5 −0
diff --git a/bugreport.c b/bugreport.c
index 73f6d39517..07b84b9c94 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -22,6 +22,7 @@ static void get_system_info(struct strbuf *sys_info)
 {
 	struct strbuf version_info = STRBUF_INIT;
 	struct utsname uname_info;
+	char *shell = NULL;
 
 	/* get git version from native cmd */
 	strbuf_addstr(sys_info, "git version:\n");
@@ -44,6 +45,10 @@ static void get_system_info(struct strbuf *sys_info)
 	get_compiler_info(sys_info);
 	strbuf_complete_line(sys_info);
 
+	shell = getenv("SHELL");
+	strbuf_addf(sys_info, "$SHELL (typically, interactive shell): %s\n",
+		    shell ? shell : "(NULL)");
+
 	strbuf_addstr(sys_info, "git-remote-https --build-info:\n");
 	get_curl_version_info(sys_info);
 	strbuf_complete_line(sys_info);
-- 
2.25.0.341.g760bfbb309-goog
Martin Ågren· Jan 30, 2020, 22:28 UTC · re: emilyshaffer@google.com · lore

Re: [PATCH v5 08/15] bugreport: include user interactive shell

On Fri, 24 Jan 2020 at 04:41, <emilyshaffer@google.com> wrote:
> +       char *shell = NULL;
(Unnecessary initialization.)
> +       shell = getenv("SHELL");
> +       strbuf_addf(sys_info, "$SHELL (typically, interactive shell): %s\n",
> +                   shell ? shell : "(NULL)");
Thanks for avoiding a classic pitfall. :-)

"<unused>" instead of "(NULL)"? "NULL" is mostly an implementation detail.

Martin
Emily Shaffer· Feb 4, 2020, 23:16 UTC · re: Martin Ågren · lore

Re: [PATCH v5 08/15] bugreport: include user interactive shell

On Thu, Jan 30, 2020 at 11:28:40PM +0100, Martin Ågren wrote:
Show 10 quoted lines
> On Fri, 24 Jan 2020 at 04:41, <emilyshaffer@google.com> wrote:
> > +       char *shell = NULL;
> 
> (Unnecessary initialization.)
> 
> > +       shell = getenv("SHELL");
> > +       strbuf_addf(sys_info, "$SHELL (typically, interactive shell): %s\n",
> > +                   shell ? shell : "(NULL)");
> 
> Thanks for avoiding a classic pitfall. :-)
Thank Junio. I fell right into it because it Just Works in gcc. ;)
> "<unused>" instead of "(NULL)"? "NULL" is mostly an implementation
> detail.
Sure, why not.
Junio C Hamano· Feb 5, 2020, 20:06 UTC · re: Martin Ågren · lore

Re: [PATCH v5 08/15] bugreport: include user interactive shell

Martin Ågren <martin.agren@gmail.com> writes:
Show 13 quoted lines
> On Fri, 24 Jan 2020 at 04:41, <emilyshaffer@google.com> wrote:
>> +       char *shell = NULL;
>
> (Unnecessary initialization.)
>
>> +       shell = getenv("SHELL");
>> +       strbuf_addf(sys_info, "$SHELL (typically, interactive shell): %s\n",
>> +                   shell ? shell : "(NULL)");
>
> Thanks for avoiding a classic pitfall. :-)
>
> "<unused>" instead of "(NULL)"? "NULL" is mostly an implementation
> detail.
Isn't that <unset>?
Martin Ågren· Feb 5, 2020, 20:14 UTC · re: Junio C Hamano · lore

Re: [PATCH v5 08/15] bugreport: include user interactive shell

On Wed, 5 Feb 2020 at 21:06, Junio C Hamano <gitster@pobox.com> wrote:
Show 18 quoted lines
>
> Martin Ågren <martin.agren@gmail.com> writes:
>
> > On Fri, 24 Jan 2020 at 04:41, <emilyshaffer@google.com> wrote:
> >> +       char *shell = NULL;
> >
> > (Unnecessary initialization.)
> >
> >> +       shell = getenv("SHELL");
> >> +       strbuf_addf(sys_info, "$SHELL (typically, interactive shell): %s\n",
> >> +                   shell ? shell : "(NULL)");
> >
> > Thanks for avoiding a classic pitfall. :-)
> >
> > "<unused>" instead of "(NULL)"? "NULL" is mostly an implementation
> > detail.
>
> Isn't that <unset>?
Heh, yes, "SHELL: unused" sounds wrong. :-)
Martin
emilyshaffer@google.com· Jan 24, 2020, 03:34 UTC · re: emilyshaffer@google.com · lore

[PATCH v5 09/15] bugreport: generate config safelist based on docs

From: Emily Shaffer <emilyshaffer@google.com>

Add a new step to the build to generate a safelist of git-config variables which are appropriate to include in the output of git-bugreport. New variables can be added to the safelist by annotating their documentation in Documentation/config with the "bugreport" macro, which is recognized by AsciiDoc and AsciiDoctor.

Some configs are private in nature, and can contain remote URLs, passwords, or other sensitive information. In the event that a user doesn't notice their information while reviewing a bugreport, that user may leak their credentials to other individuals, mailing lists, or bug tracking tools inadvertently. Heuristic blocklisting of configuration keys is imperfect and prone to false negatives; given the nature of the information which can be leaked, a safelist is more reliable.

In order to prevent staleness of the safelist, add a mechanism to generate the safelist from annotations in the config documentation, where contributors are already used to documenting their new config keys.

Implement a new no-op "bugreport" macro for use as "bugreport:include[x]" to annotate the config keys that should be included in the automatically generated safelist. Use "exclude" for the others.

With Asciidoctor, it's ok to say "bugreport:include[]", but AsciiDoc seems to want something between the brackets. A bit unfortunate, but not a huge problem -- we'll just provide an "x".

"doc-diff" reports that this macro doesn't render at all. That is, these are both empty after this commit:

  cd Documentation
  ./doc-diff --asciidoctor :/"bugreport: add tool" HEAD
  ./doc-diff --asciidoc    :/"bugreport: add tool" HEAD

Diffing the rendered HTML shows that there is some small amount of whitespace and comments added. That shouldn't be a problem.

We could perhaps let the implementation verify that the "action" is one of "include" and "exclude". For the Asciidoctor implementation that should be straightforward, but for AsciiDoc I don't immediately know how to do it. Anyway, if someone stumbles on the keyboard and writes "bugreport:icndule", they'll "only" miss out on the config key being included in the safelist. If this were a blocklist, the consequences of a misspelled target could be a lot more severe.

Additionally, add annotations to the sendemail config documentation in order to demonstrate a proof of concept.

Helped-by: Martin Ågren <martin.agren@gmail.com>
Helped-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 .gitignore                              |  1 +
 Documentation/asciidoc.conf             |  8 +++
 Documentation/asciidoctor-extensions.rb |  7 +++
 Documentation/config/sendemail.txt      | 68 ++++++++++++-------------
 Makefile                                |  7 +++
 generate-bugreport-config-safelist.sh   | 17 +++++++
 6 files changed, 74 insertions(+), 34 deletions(-)
 create mode 100755 generate-bugreport-config-safelist.sh
Show changes to 6 files +74 −34

.gitignore, Documentation/asciidoc.conf, Documentation/asciidoctor-extensions.rb, Documentation/config/sendemail.txt, Makefile, generate-bugreport-config-safelist.sh

diff --git a/.gitignore b/.gitignore
index d89bf9e11e..bd2f49b996 100644
--- a/.gitignore
+++ b/.gitignore
@@ -192,6 +192,7 @@
 /gitweb/static/gitweb.min.*
 /config-list.h
 /command-list.h
+/bugreport-config-safelist.h
 *.tar.gz
 *.dsc
 *.deb
diff --git a/Documentation/asciidoc.conf b/Documentation/asciidoc.conf
index 8fc4b67081..5d5359fcf9 100644
--- a/Documentation/asciidoc.conf
+++ b/Documentation/asciidoc.conf
@@ -6,9 +6,13 @@
 #
 # Show Git link as: <command>(<section>); if section is defined, else just show
 # the command.
+#
+# The bugreport macro does nothing as far as rendering is
+# concerned -- we just grep for it in the sources.
 
 [macros]
 (?su)[\\]?(?P<name>linkgit):(?P<target>\S*?)\[(?P<attrlist>.*?)\]=
+(?su)[\\]?(?P<name>bugreport):(?P<action>\S*?)\[(?P<attrlist>.*?)\]=
 
 [attributes]
 asterisk=&#42;
@@ -28,6 +32,8 @@ ifdef::backend-docbook[]
 {0#<citerefentry>}
 {0#<refentrytitle>{target}</refentrytitle><manvolnum>{0}</manvolnum>}
 {0#</citerefentry>}
+[bugreport-inlinemacro]
+{0#}
 endif::backend-docbook[]
 
 ifdef::backend-docbook[]
@@ -94,4 +100,6 @@ ifdef::backend-xhtml11[]
 git-relative-html-prefix=
 [linkgit-inlinemacro]
 <a href="{git-relative-html-prefix}{target}.html">{target}{0?({0})}</a>
+[bugreport-inlinemacro]
+<!-- -->
 endif::backend-xhtml11[]
diff --git a/Documentation/asciidoctor-extensions.rb b/Documentation/asciidoctor-extensions.rb
index d906a00803..750bdff9af 100644
--- a/Documentation/asciidoctor-extensions.rb
+++ b/Documentation/asciidoctor-extensions.rb
@@ -37,6 +37,10 @@ module Git
           output = output.sub(/<\/refmeta>/, new_tags + "</refmeta>")
         end
         output
+
+    class BugReportProcessor < Asciidoctor::Extensions::InlineMacroProcessor
+      def process(parent, action, attrs)
+        ""
       end
     end
   end
@@ -45,4 +49,7 @@ end
 Asciidoctor::Extensions.register do
   inline_macro Git::Documentation::LinkGitProcessor, :linkgit
   postprocessor Git::Documentation::DocumentPostProcessor
+  # The bugreport macro does nothing as far as rendering is
+  # concerned -- we just grep for it in the sources.
+  inline_macro Git::Documentation::BugReportProcessor, :bugreport
 end
diff --git a/Documentation/config/sendemail.txt b/Documentation/config/sendemail.txt
index 0006faf800..92f5082013 100644
--- a/Documentation/config/sendemail.txt
+++ b/Documentation/config/sendemail.txt
@@ -1,63 +1,63 @@
-sendemail.identity::
+sendemail.identity bugreport:exclude[x] ::
 	A configuration identity. When given, causes values in the
 	'sendemail.<identity>' subsection to take precedence over
 	values in the 'sendemail' section. The default identity is
 	the value of `sendemail.identity`.
 
-sendemail.smtpEncryption::
+sendemail.smtpEncryption bugreport:include[x] ::
 	See linkgit:git-send-email[1] for description.  Note that this
 	setting is not subject to the 'identity' mechanism.
 
-sendemail.smtpssl (deprecated)::
+sendemail.smtpssl (deprecated) bugreport:exclude[x] ::
 	Deprecated alias for 'sendemail.smtpEncryption = ssl'.
 
-sendemail.smtpsslcertpath::
+sendemail.smtpsslcertpath bugreport:exclude[x] ::
 	Path to ca-certificates (either a directory or a single file).
 	Set it to an empty string to disable certificate verification.
 
-sendemail.<identity>.*::
+sendemail.<identity>.* bugreport:exclude[x] ::
 	Identity-specific versions of the 'sendemail.*' parameters
 	found below, taking precedence over those when this
 	identity is selected, through either the command-line or
 	`sendemail.identity`.
 
-sendemail.aliasesFile::
-sendemail.aliasFileType::
-sendemail.annotate::
-sendemail.bcc::
-sendemail.cc::
-sendemail.ccCmd::
-sendemail.chainReplyTo::
-sendemail.confirm::
-sendemail.envelopeSender::
-sendemail.from::
-sendemail.multiEdit::
-sendemail.signedoffbycc::
-sendemail.smtpPass::
-sendemail.suppresscc::
-sendemail.suppressFrom::
-sendemail.to::
-sendemail.tocmd::
-sendemail.smtpDomain::
-sendemail.smtpServer::
-sendemail.smtpServerPort::
-sendemail.smtpServerOption::
-sendemail.smtpUser::
-sendemail.thread::
-sendemail.transferEncoding::
-sendemail.validate::
-sendemail.xmailer::
+sendemail.aliasesFile bugreport:exclude[x] ::
+sendemail.aliasFileType bugreport:exclude[x] ::
+sendemail.annotate bugreport:include[x] ::
+sendemail.bcc bugreport:include[x] ::
+sendemail.cc bugreport:include[x] ::
+sendemail.ccCmd bugreport:include[x] ::
+sendemail.chainReplyTo bugreport:include[x] ::
+sendemail.confirm bugreport:include[x] ::
+sendemail.envelopeSender bugreport:include[x] ::
+sendemail.from bugreport:include[x] ::
+sendemail.multiEdit bugreport:include[x] ::
+sendemail.signedoffbycc bugreport:include[x] ::
+sendemail.smtpPass bugreport:exclude[x] ::
+sendemail.suppresscc bugreport:include[x] ::
+sendemail.suppressFrom bugreport:include[x] ::
+sendemail.to bugreport:include[x] ::
+sendemail.tocmd bugreport:include[x] ::
+sendemail.smtpDomain bugreport:include[x] ::
+sendemail.smtpServer bugreport:include[x] ::
+sendemail.smtpServerPort bugreport:include[x] ::
+sendemail.smtpServerOption bugreport:include[x] ::
+sendemail.smtpUser bugreport:exclude[x] ::
+sendemail.thread bugreport:include[x] ::
+sendemail.transferEncoding bugreport:include[x] ::
+sendemail.validate bugreport:include[x] ::
+sendemail.xmailer bugreport:include[x] ::
 	See linkgit:git-send-email[1] for description.
 
-sendemail.signedoffcc (deprecated)::
+sendemail.signedoffcc (deprecated) bugreport:exclude[x] ::
 	Deprecated alias for `sendemail.signedoffbycc`.
 
-sendemail.smtpBatchSize::
+sendemail.smtpBatchSize bugreport:include[x] ::
 	Number of messages to be sent per connection, after that a relogin
 	will happen.  If the value is 0 or undefined, send all messages in
 	one connection.
 	See also the `--batch-size` option of linkgit:git-send-email[1].
 
-sendemail.smtpReloginDelay::
+sendemail.smtpReloginDelay bugreport:include[x] ::
 	Seconds wait before reconnecting to smtp server.
 	See also the `--relogin-delay` option of linkgit:git-send-email[1].
diff --git a/Makefile b/Makefile
index a01a050aa3..2bc9f112ea 100644
--- a/Makefile
+++ b/Makefile
@@ -817,6 +817,7 @@ VCSSVN_LIB = vcs-svn/lib.a
 
 GENERATED_H += config-list.h
 GENERATED_H += command-list.h
+GENERATED_H += bugreport-config-safelist.h
 
 LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
 	$(FIND) . \
@@ -2163,6 +2164,12 @@ command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Doc
 		$(patsubst %,--exclude-program %,$(EXCLUDED_PROGRAMS)) \
 		command-list.txt >$@+ && mv $@+ $@
 
+bugreport-config-safelist.h: generate-bugreport-config-safelist.sh
+
+bugreport-config-safelist.h: Documentation/config/*.txt
+	$(QUIET_GEN)$(SHELL_PATH) ./generate-bugreport-config-safelist.sh \
+		>$@+ && mv $@+ $@
+
 SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):$(GIT_VERSION):\
 	$(localedir_SQ):$(NO_CURL):$(USE_GETTEXT_SCHEME):$(SANE_TOOL_PATH_SQ):\
 	$(gitwebdir_SQ):$(PERL_PATH_SQ):$(SANE_TEXT_GREP):$(PAGER_ENV):\
diff --git a/generate-bugreport-config-safelist.sh b/generate-bugreport-config-safelist.sh
new file mode 100755
index 0000000000..44612d5538
--- /dev/null
+++ b/generate-bugreport-config-safelist.sh
@@ -0,0 +1,17 @@
+#!/bin/sh
+
+cat <<EOF
+/* Automatically generated by bugreport-generate-config-safelist.sh */
+
+
+static const char *bugreport_config_safelist[] = {
+EOF
+
+# cat all regular files in Documentation/config
+find Documentation/config -type f -exec cat {} \; |
+# print the command name which matches the bugreport-include macro
+sed -n 's/^\(.*\) \+bugreport:include.* ::$/  "\1",/p' | sort
+
+cat <<EOF
+};
+EOF
-- 
2.25.0.341.g760bfbb309-goog
Martin Ågren· Jan 30, 2020, 22:34 UTC · re: emilyshaffer@google.com · lore

Re: [PATCH v5 09/15] bugreport: generate config safelist based on docs

On Fri, 24 Jan 2020 at 04:35, <emilyshaffer@google.com> wrote:
Show 14 quoted lines
> Add a new step to the build to generate a safelist of git-config
> variables which are appropriate to include in the output of
> git-bugreport. New variables can be added to the safelist by annotating
> their documentation in Documentation/config with the
> "bugreport" macro, which is recognized by AsciiDoc and
> AsciiDoctor.
>
> Some configs are private in nature, and can contain remote URLs,
> passwords, or other sensitive information. In the event that a user
> doesn't notice their information while reviewing a bugreport, that user
> may leak their credentials to other individuals, mailing lists, or bug
> tracking tools inadvertently. Heuristic blocklisting of configuration
> keys is imperfect and prone to false negatives; given the nature of the
> information which can be leaked, a safelist is more reliable.

I sort of wonder whether safelist/blocklist is to prefer over whitelist/blacklist, or if it's the other way round. The former are new to me, whereas the latter are the terms I would have used. But that's just me, of course. I was a little surprised, that's all.

Show 8 quoted lines
> Implement a new no-op "bugreport" macro for use as
> "bugreport:include[x]" to annotate the config keys that should be
> included in the automatically generated safelist. Use "exclude" for the
> others.
>
> With Asciidoctor, it's ok to say "bugreport:include[]", but AsciiDoc
> seems to want something between the brackets. A bit unfortunate, but
> not a huge problem -- we'll just provide an "x".

I recognize this reasoning :-) and I'm not terribly opposed to it, but after some nights' sleeping on this, I have to wonder if "annotate:bugreport[include]" wouldn't be better than "bugreport[x]" with that ugly "x". Maybe this isn't the biggest problem, but if we expect this macro to eventually sit right next to ~90% of all our config variables...

Show 6 quoted lines
> "doc-diff" reports that this macro doesn't render at all. That is,
> these are both empty after this commit:
>
>   cd Documentation
>   ./doc-diff --asciidoctor :/"bugreport: add tool" HEAD
>   ./doc-diff --asciidoc    :/"bugreport: add tool" HEAD

That was true in [1], but alas, no more. In that patch, it's sort of obvious from the diff how it adds a "class" which "end"s.

[1] https://lore.kernel.org/git/20190817203846.31609-1-martin.agren@gmail.com/
Show 13 quoted lines
> --- a/Documentation/asciidoctor-extensions.rb
> +++ b/Documentation/asciidoctor-extensions.rb
> @@ -37,6 +37,10 @@ module Git
>            output = output.sub(/<\/refmeta>/, new_tags + "</refmeta>")
>          end
>          output
> +
> +    class BugReportProcessor < Asciidoctor::Extensions::InlineMacroProcessor
> +      def process(parent, action, attrs)
> +        ""
>        end
>      end
>    end
But this one doesn't add an "end", and Asciidoctor trips up badly.
> +  # The bugreport macro does nothing as far as rendering is
> +  # concerned -- we just grep for it in the sources.
> +  inline_macro Git::Documentation::BugReportProcessor, :bugreport

(I never much liked this copy-paste comment then, and I still don't, to be honest.)

Martin
Emily Shaffer· Feb 5, 2020, 00:44 UTC · re: Martin Ågren · lore

Re: [PATCH v5 09/15] bugreport: generate config safelist based on docs

On Thu, Jan 30, 2020 at 11:34:24PM +0100, Martin Ågren wrote:
Show 20 quoted lines
> On Fri, 24 Jan 2020 at 04:35, <emilyshaffer@google.com> wrote:
> > Add a new step to the build to generate a safelist of git-config
> > variables which are appropriate to include in the output of
> > git-bugreport. New variables can be added to the safelist by annotating
> > their documentation in Documentation/config with the
> > "bugreport" macro, which is recognized by AsciiDoc and
> > AsciiDoctor.
> >
> > Some configs are private in nature, and can contain remote URLs,
> > passwords, or other sensitive information. In the event that a user
> > doesn't notice their information while reviewing a bugreport, that user
> > may leak their credentials to other individuals, mailing lists, or bug
> > tracking tools inadvertently. Heuristic blocklisting of configuration
> > keys is imperfect and prone to false negatives; given the nature of the
> > information which can be leaked, a safelist is more reliable.
> 
> I sort of wonder whether safelist/blocklist is to prefer over
> whitelist/blacklist, or if it's the other way round. The former are new
> to me, whereas the latter are the terms I would have used. But that's
> just me, of course. I was a little surprised, that's all.
Eh. I think the following things are true:
 - Whitelist/blacklist has a "smell" of discrimination, whether that's
   the true etymology or not.
 - Those with experience in the field can easily understand what
   whitelist or blacklist means.
 - Safelist/blocklist do not "smell" the same way.
 - It is easy to tell what "safelist" means: "a list of stuff which is
   safe." No experience needed.

So, while it's new, I think it's not harmful. I see only a no-op or positive impact from using this term instead of whitelist/blacklist. Computer science seems to have quite a few terms which fall into this long-standing but potentially negative area, so I don't mind looking for alternatives where it's harmless to do so.

Show 16 quoted lines
> 
> > Implement a new no-op "bugreport" macro for use as
> > "bugreport:include[x]" to annotate the config keys that should be
> > included in the automatically generated safelist. Use "exclude" for the
> > others.
> >
> > With Asciidoctor, it's ok to say "bugreport:include[]", but AsciiDoc
> > seems to want something between the brackets. A bit unfortunate, but
> > not a huge problem -- we'll just provide an "x".
> 
> I recognize this reasoning :-) and I'm not terribly opposed to it, but
> after some nights' sleeping on this, I have to wonder if
> "annotate:bugreport[include]" wouldn't be better than "bugreport[x]"
> with that ugly "x". Maybe this isn't the biggest problem, but if we
> expect this macro to eventually sit right next to ~90% of all our config
> variables...

Hm. I wanted to say, "Ok, but I don't know how to do that, so can you help?" But I think that's all the more reason that I should do it ;)

Ok. I will try and change it to annotate:bugreport[include] like you suggested, and hopefully learn more about asciidoc macros :)

Show 28 quoted lines
> 
> > "doc-diff" reports that this macro doesn't render at all. That is,
> > these are both empty after this commit:
> >
> >   cd Documentation
> >   ./doc-diff --asciidoctor :/"bugreport: add tool" HEAD
> >   ./doc-diff --asciidoc    :/"bugreport: add tool" HEAD
> 
> That was true in [1], but alas, no more. In that patch, it's sort of
> obvious from the diff how it adds a "class" which "end"s.
> 
> [1] https://lore.kernel.org/git/20190817203846.31609-1-martin.agren@gmail.com/
> 
> > --- a/Documentation/asciidoctor-extensions.rb
> > +++ b/Documentation/asciidoctor-extensions.rb
> > @@ -37,6 +37,10 @@ module Git
> >            output = output.sub(/<\/refmeta>/, new_tags + "</refmeta>")
> >          end
> >          output
> > +
> > +    class BugReportProcessor < Asciidoctor::Extensions::InlineMacroProcessor
> > +      def process(parent, action, attrs)
> > +        ""
> >        end
> >      end
> >    end
> 
> But this one doesn't add an "end", and Asciidoctor trips up badly.
Ok, I'll have a look. I'm sure I copied something badly.
Show 7 quoted lines
> 
> > +  # The bugreport macro does nothing as far as rendering is
> > +  # concerned -- we just grep for it in the sources.
> > +  inline_macro Git::Documentation::BugReportProcessor, :bugreport
> 
> (I never much liked this copy-paste comment then, and I still don't, to
> be honest.)
I'll see if I can find a reasonable alternative (or remove it).
 - Emily
Martin Ågren· Feb 5, 2020, 19:53 UTC · re: Emily Shaffer · lore

Re: [PATCH v5 09/15] bugreport: generate config safelist based on docs

On Wed, 5 Feb 2020 at 01:45, Emily Shaffer <emilyshaffer@google.com> wrote:
Show 21 quoted lines
>
> > I sort of wonder whether safelist/blocklist is to prefer over
> > whitelist/blacklist, or if it's the other way round. The former are new
> > to me, whereas the latter are the terms I would have used. But that's
> > just me, of course. I was a little surprised, that's all.
>
> Eh. I think the following things are true:
>
>  - Whitelist/blacklist has a "smell" of discrimination, whether that's
>    the true etymology or not.
>  - Those with experience in the field can easily understand what
>    whitelist or blacklist means.
>  - Safelist/blocklist do not "smell" the same way.
>  - It is easy to tell what "safelist" means: "a list of stuff which is
>    safe." No experience needed.
>
> So, while it's new, I think it's not harmful. I see only a no-op or
> positive impact from using this term instead of whitelist/blacklist.
> Computer science seems to have quite a few terms which fall into this
> long-standing but potentially negative area, so I don't mind looking for
> alternatives where it's harmless to do so.
Ok, that all makes sense.

Thanks Martin

Martin Ågren· Jan 31, 2020, 21:20 UTC · re: emilyshaffer@google.com · lore

Re: [PATCH v5 09/15] bugreport: generate config safelist based on docs

On Fri, 24 Jan 2020 at 04:35, <emilyshaffer@google.com> wrote:
Show 6 quoted lines
> Add a new step to the build to generate a safelist of git-config
> variables which are appropriate to include in the output of
> git-bugreport. New variables can be added to the safelist by annotating
> their documentation in Documentation/config with the
> "bugreport" macro, which is recognized by AsciiDoc and
> AsciiDoctor.

"which is recognized by" sounds like it's built-in. Maybe "with a new no-op 'bugreport' macro" or something like that.

Show 10 quoted lines
> -sendemail.signedoffcc (deprecated)::
> +sendemail.signedoffcc (deprecated) bugreport:exclude[x] ::
>         Deprecated alias for `sendemail.signedoffbycc`.
>
> -sendemail.smtpBatchSize::
> +sendemail.smtpBatchSize bugreport:include[x] ::
>         Number of messages to be sent per connection, after that a relogin
>         will happen.  If the value is 0 or undefined, send all messages in
>         one connection.
>         See also the `--batch-size` option of linkgit:git-send-email[1].

Do we really need to list includes *and* excludes? I could see the point of adding an exclude here and there to signal that "this might look innocent enough, but trust me, we really need to exclude this" in order to avoid future commits to more or less accidentally over-include. Should we add some internal documentation and/or a remark in the commit message about this? As an example, is "sendemail.signedofcc" sensitive enough that we need to explicitly exclude it? If someone wants to come along and include it, they don't just need to argue for an inclusion, but also for lifting the exclusion. Hmm?

Martin
Emily Shaffer· Feb 5, 2020, 00:30 UTC · re: Martin Ågren · lore

Re: [PATCH v5 09/15] bugreport: generate config safelist based on docs

On Fri, Jan 31, 2020 at 10:20:05PM +0100, Martin Ågren wrote:
Show 10 quoted lines
> On Fri, 24 Jan 2020 at 04:35, <emilyshaffer@google.com> wrote:
> > Add a new step to the build to generate a safelist of git-config
> > variables which are appropriate to include in the output of
> > git-bugreport. New variables can be added to the safelist by annotating
> > their documentation in Documentation/config with the
> > "bugreport" macro, which is recognized by AsciiDoc and
> > AsciiDoctor.
> 
> "which is recognized by" sounds like it's built-in. Maybe "with a new
> no-op 'bugreport' macro" or something like that.
Sure, I'll wordsmith this a little.
Show 13 quoted lines
> 
> > -sendemail.signedoffcc (deprecated)::
> > +sendemail.signedoffcc (deprecated) bugreport:exclude[x] ::
> >         Deprecated alias for `sendemail.signedoffbycc`.
> >
> > -sendemail.smtpBatchSize::
> > +sendemail.smtpBatchSize bugreport:include[x] ::
> >         Number of messages to be sent per connection, after that a relogin
> >         will happen.  If the value is 0 or undefined, send all messages in
> >         one connection.
> >         See also the `--batch-size` option of linkgit:git-send-email[1].
> 
> Do we really need to list includes *and* excludes?

Need? Nah. The exclude markers are not even acknowledged by the current implementation.

> I could see the point of adding an exclude here and there to signal
> that "this might look innocent enough, but trust me, we really need to
> exclude this" in order to avoid future commits to more or less
> accidentally over-include.

This was more of my line of thinking, along with the thought that some folks may use bug reporting internally, where they have A) more control over which extensions/tools are in use, and B) less concern about sharing things like remote URLs, branch names, etc. and may want to go for a blocklist as opposed to a safelist, to make it easier for their support folks to diagnose.

Show 5 quoted lines
> Should we add some internal documentation and/or a remark in the commit
> message about this? As an example, is "sendemail.signedofcc" sensitive
> enough that we need to explicitly exclude it? If someone wants to come
> along and include it, they don't just need to argue for an inclusion,
> but also for lifting the exclusion. Hmm?

Yeah, I think this is a good point. I'll try to figure out how to reword the commit message, and take another look at the sample include/exclude change I made to see where I can omit entirely.

 - Emily
Emily Shaffer· Feb 5, 2020, 00:52 UTC · re: Emily Shaffer · lore

Re: [PATCH v5 09/15] bugreport: generate config safelist based on docs

On Tue, Feb 04, 2020 at 04:30:43PM -0800, Emily Shaffer wrote:
> Yeah, I think this is a good point. I'll try to figure out how to reword
> the commit message, and take another look at the sample include/exclude
> change I made to see where I can omit entirely.

Eh. I'm going to try and reword the whole commit message. I combined your scissors patch, plus something from Dscho's scissors patch, plus my original commit message, and now it's huge and hard to follow.

 - Emily
emilyshaffer@google.com· Jan 24, 2020, 03:34 UTC · re: emilyshaffer@google.com · lore

[PATCH v5 10/15] bugreport: add config values from safelist

From: Emily Shaffer <emilyshaffer@google.com>

Teach bugreport to gather the values of config options which are present in 'bugreport-config-safelist.h'.

Many config options are sensitive, and many Git add-ons use config options which git-core does not know about; it is better only to gather config options which we know to be safe, rather than excluding options which we know to be unsafe.

Taking the build-time generated array and putting it into a set saves us time - since git_config_bugreport() is called for every option the user has configured, performing option lookup in constant time is a useful optimization.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Makefile    |  2 +-
 bugreport.c | 75 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 76 insertions(+), 1 deletion(-)
Show changes to 2 files +76 −1

Makefile, bugreport.c

diff --git a/Makefile b/Makefile
index 2bc9f112ea..eb17ece120 100644
--- a/Makefile
+++ b/Makefile
@@ -2465,7 +2465,7 @@ endif
 git-%$X: %.o GIT-LDFLAGS $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) $(LIBS)
 
-git-bugreport$X: bugreport.o GIT-LDFLAGS $(GITLIBS)
+git-bugreport$X: bugreport-config-safelist.h bugreport.o GIT-LDFLAGS $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
 		$(LIBS)
 
diff --git a/bugreport.c b/bugreport.c
index 07b84b9c94..7a9fd36b60 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -6,6 +6,9 @@
 #include "help.h"
 #include "compat/compiler.h"
 #include "run-command.h"
+#include "config.h"
+#include "bugreport-config-safelist.h"
+#include "khash.h"
 
 static void get_curl_version_info(struct strbuf *curl_info)
 {
@@ -18,6 +21,40 @@ static void get_curl_version_info(struct strbuf *curl_info)
 	    strbuf_addstr(curl_info, "'git-remote-https --build-info' not supported\n");
 }
 
+KHASH_INIT(cfg_set, const char*, int, 0, kh_str_hash_func, kh_str_hash_equal);
+
+struct cfgset {
+	kh_cfg_set_t set;
+};
+
+struct cfgset safelist;
+
+static void cfgset_init(struct cfgset *set, size_t initial_size)
+{
+	memset(&set->set, 0, sizeof(set->set));
+	if (initial_size)
+		kh_resize_cfg_set(&set->set, initial_size);
+}
+
+static int cfgset_insert(struct cfgset *set, const char *cfg_key)
+{
+	int added;
+	kh_put_cfg_set(&set->set, cfg_key, &added);
+	return !added;
+}
+
+static int cfgset_contains(struct cfgset *set, const char *cfg_key)
+{
+	khiter_t pos = kh_get_cfg_set(&set->set, cfg_key);
+	return pos != kh_end(&set->set);
+}
+
+static void cfgset_clear(struct cfgset *set)
+{
+	kh_release_cfg_set(&set->set);
+	cfgset_init(set, 0);
+}
+
 static void get_system_info(struct strbuf *sys_info)
 {
 	struct strbuf version_info = STRBUF_INIT;
@@ -54,6 +91,36 @@ static void get_system_info(struct strbuf *sys_info)
 	strbuf_complete_line(sys_info);
 }
 
+static void gather_safelist(void)
+{
+	int index;
+	int safelist_len = sizeof(bugreport_config_safelist) / sizeof(const char *);
+	cfgset_init(&safelist, safelist_len);
+	for (index = 0; index < safelist_len; index++)
+		cfgset_insert(&safelist, bugreport_config_safelist[index]);
+
+}
+
+static int git_config_bugreport(const char *var, const char *value, void *cb)
+{
+	struct strbuf *config_info = (struct strbuf *)cb;
+
+	if (cfgset_contains(&safelist, var))
+		strbuf_addf(config_info,
+			    "%s (%s) : %s\n",
+			    var, config_scope_to_string(current_config_scope()),
+			    value);
+
+	return 0;
+}
+
+static void get_safelisted_config(struct strbuf *config_info)
+{
+	gather_safelist();
+	git_config(git_config_bugreport, config_info);
+	cfgset_clear(&safelist);
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output <file>]"),
 	NULL
@@ -94,12 +161,17 @@ int cmd_main(int argc, const char **argv)
 	FILE *report;
 	time_t now = time(NULL);
 	char *option_output = NULL;
+	int nongit_ok = 0;
 
 	const struct option bugreport_options[] = {
 		OPT_STRING('o', "output", &option_output, N_("path"),
 			   N_("specify a destination for the bugreport file")),
 		OPT_END()
 	};
+
+	/* Prerequisite for hooks and config checks */
+	setup_git_directory_gently(&nongit_ok);
+
 	argc = parse_options(argc, argv, "", bugreport_options,
 			     bugreport_usage, 0);
 
@@ -118,6 +190,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "System Info");
 	get_system_info(&buffer);
 
+	get_header(&buffer, "Safelisted Config Info");
+	get_safelisted_config(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 	strbuf_write(&buffer, report);
 	fclose(report);
-- 
2.25.0.341.g760bfbb309-goog
Martin Ågren· Jan 30, 2020, 22:36 UTC · re: emilyshaffer@google.com · lore

Re: [PATCH v5 10/15] bugreport: add config values from safelist

On Fri, 24 Jan 2020 at 04:40, <emilyshaffer@google.com> wrote:
Show 10 quoted lines
> --- a/Makefile
> +++ b/Makefile
> @@ -2465,7 +2465,7 @@ endif
>  git-%$X: %.o GIT-LDFLAGS $(GITLIBS)
>         $(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) $(LIBS)
>
> -git-bugreport$X: bugreport.o GIT-LDFLAGS $(GITLIBS)
> +git-bugreport$X: bugreport-config-safelist.h bugreport.o GIT-LDFLAGS $(GITLIBS)
>         $(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
>                 $(LIBS)

Haven't looked at this patch at all, except I've noticed that something is up with the dependencies. I think bugreport.o needs to depend on bugreport-config-safelist.h. As it is, the latter may or may not be available as bugreport.o is built. (Reproduces fairly well for me with something like `make clean && make -j16`.)

I'll try to continue tomorrow. :-)
Martin
Emily Shaffer· Feb 5, 2020, 01:34 UTC · re: Martin Ågren · lore

Re: [PATCH v5 10/15] bugreport: add config values from safelist

On Thu, Jan 30, 2020 at 11:36:43PM +0100, Martin Ågren wrote:
Show 17 quoted lines
> On Fri, 24 Jan 2020 at 04:40, <emilyshaffer@google.com> wrote:
> > --- a/Makefile
> > +++ b/Makefile
> > @@ -2465,7 +2465,7 @@ endif
> >  git-%$X: %.o GIT-LDFLAGS $(GITLIBS)
> >         $(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) $(LIBS)
> >
> > -git-bugreport$X: bugreport.o GIT-LDFLAGS $(GITLIBS)
> > +git-bugreport$X: bugreport-config-safelist.h bugreport.o GIT-LDFLAGS $(GITLIBS)
> >         $(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
> >                 $(LIBS)
> 
> Haven't looked at this patch at all, except I've noticed that something
> is up with the dependencies. I think bugreport.o needs to depend on
> bugreport-config-safelist.h. As it is, the latter may or may not be
> available as bugreport.o is built. (Reproduces fairly well for me with
> something like `make clean && make -j16`.)

Hum, I thought that the ordering of the dependency list here was ensured. But I see where help.o relies directly on command-list.h, so I'll add a dependency for bugreport.o (and drop it from the direct dependency for git-bugreport).

 - Emily
Martin Ågren· Jan 31, 2020, 21:25 UTC · re: emilyshaffer@google.com · lore

Re: [PATCH v5 10/15] bugreport: add config values from safelist

On Fri, 24 Jan 2020 at 04:40, <emilyshaffer@google.com> wrote:
> Taking the build-time generated array and putting it into a set saves us
> time - since git_config_bugreport() is called for every option the user
> has configured, performing option lookup in constant time is a useful
> optimization.

I'm sympathetic to your sending out what you have to obtain comments, knowing that it's not perfect. It would have saved me some time and effort if I'd known that this was the case though. An "[RFC]" tag, perhaps. Or at least tweaking the above part of this commit message to say that this might be over-engineered, with a reference to [1].

[1] https://lore.kernel.org/git/20200124032905.GA37541@google.com/
> +       int safelist_len = sizeof(bugreport_config_safelist) / sizeof(const char *);

I was going to suggest ARRAY_SIZE, but then I realized there are some outstanding questions around whether you need this stuff in the first place. I'd be inclined to guess that the first version of this would be "for each safelisted item, obtain it and include", ignoring any "a.*.b" business. In which case you wouldn't really need this hashset stuff.

Martin
Emily Shaffer· Feb 5, 2020, 02:31 UTC · re: Martin Ågren · lore

Re: [PATCH v5 10/15] bugreport: add config values from safelist

On Fri, Jan 31, 2020 at 10:25:06PM +0100, Martin Ågren wrote:
Show 13 quoted lines
> On Fri, 24 Jan 2020 at 04:40, <emilyshaffer@google.com> wrote:
> > Taking the build-time generated array and putting it into a set saves us
> > time - since git_config_bugreport() is called for every option the user
> > has configured, performing option lookup in constant time is a useful
> > optimization.
> 
> I'm sympathetic to your sending out what you have to obtain comments,
> knowing that it's not perfect. It would have saved me some time and
> effort if I'd known that this was the case though. An "[RFC]" tag,
> perhaps. Or at least tweaking the above part of this commit message to
> say that this might be over-engineered, with a reference to [1].
> 
> [1] https://lore.kernel.org/git/20200124032905.GA37541@google.com/

Yeah, you are right, and thanks for the feedback. I think I had wrapped up the series before I realized there were those significant outstanding comments, but I definitely should have said so in this patch.

Show 8 quoted lines
> 
> > +       int safelist_len = sizeof(bugreport_config_safelist) / sizeof(const char *);
> 
> I was going to suggest ARRAY_SIZE, but then I realized there are some
> outstanding questions around whether you need this stuff in the first
> place. I'd be inclined to guess that the first version of this would be
> "for each safelisted item, obtain it and include", ignoring any "a.*.b"
> business. In which case you wouldn't really need this hashset stuff.
Regardless, I think it's worth doing nicely for now.

It seems to me that supporting wildcarding in the config safelist is a superset of supporting static strings in that safelist - that is, if I write it simply to support static strings, a later change to support wildcards would be welcome and non-breaking.

So I'm going to clean this up without making it more featureful, for now.

Sorry about the confusion!
 - Emily
Martin Ågren· Feb 5, 2020, 20:12 UTC · re: Emily Shaffer · lore

Re: [PATCH v5 10/15] bugreport: add config values from safelist

On Wed, 5 Feb 2020 at 03:31, Emily Shaffer <emilyshaffer@google.com> wrote:
Show 35 quoted lines
>
> On Fri, Jan 31, 2020 at 10:25:06PM +0100, Martin Ågren wrote:
> > On Fri, 24 Jan 2020 at 04:40, <emilyshaffer@google.com> wrote:
> > > Taking the build-time generated array and putting it into a set saves us
> > > time - since git_config_bugreport() is called for every option the user
> > > has configured, performing option lookup in constant time is a useful
> > > optimization.
> >
> > I'm sympathetic to your sending out what you have to obtain comments,
> > knowing that it's not perfect. It would have saved me some time and
> > effort if I'd known that this was the case though. An "[RFC]" tag,
> > perhaps. Or at least tweaking the above part of this commit message to
> > say that this might be over-engineered, with a reference to [1].
> >
> > [1] https://lore.kernel.org/git/20200124032905.GA37541@google.com/
>
> Yeah, you are right, and thanks for the feedback. I think I had wrapped
> up the series before I realized there were those significant outstanding
> comments, but I definitely should have said so in this patch.
>
> >
> > > +       int safelist_len = sizeof(bugreport_config_safelist) / sizeof(const char *);
> >
> > I was going to suggest ARRAY_SIZE, but then I realized there are some
> > outstanding questions around whether you need this stuff in the first
> > place. I'd be inclined to guess that the first version of this would be
> > "for each safelisted item, obtain it and include", ignoring any "a.*.b"
> > business. In which case you wouldn't really need this hashset stuff.
>
> Regardless, I think it's worth doing nicely for now.
>
> It seems to me that supporting wildcarding in the config safelist is a
> superset of supporting static strings in that safelist - that is, if I
> write it simply to support static strings, a later change to support
> wildcards would be welcome and non-breaking.

Yeah, agreed. Also, whatever magic the code for gathering the config items knows, the annotations in Documentation/, i.e., the generated list in bugreport-config-safelist.h will be "synced". That is, until we know how to handle "a.*.b" in the safelist, we probably won't be marking "a.*.b" as safe. And even if we have some crazy mixed build, it's a *safe*list, so we should be fine. (Famous last words?)

On that topic though, and just so we don't mess up from the beginning, in the documentation, we typically write something like "branch.<name>.merge", i.e., not "branch.*.merge". So I guess the "a.*.b" feature would be more like "look for '.<', something, something, '>.'." Would that be sufficient? Or would we actually want more fine-grained control, i.e., something like regexes?

If so, we'd need to provide those regexes somehow, e.g., as part of the asciidoc macro notation. :-/ Do we need to prepare somehow for such a future? I don't think we do.

Show 5 quoted lines
> So I'm going to clean this up without making it more featureful, for
> now.
>
> Sorry about the confusion!
>  - Emily
No worries. :)
Martin
emilyshaffer@google.com· Jan 24, 2020, 03:34 UTC · re: emilyshaffer@google.com · lore

[PATCH v5 11/15] bugreport: collect list of populated hooks

From: Emily Shaffer <emilyshaffer@google.com>

Occasionally a failure a user is seeing may be related to a specific hook which is being run, perhaps without the user realizing. While the contents of hooks can be sensitive - containing user data or process information specific to the user's organization - simply knowing that a hook is being run at a certain stage can help us to understand whether something is going wrong.

Without a definitive list of hook names within the code, we compile our own list from the documentation. This is likely prone to bitrot. To reduce the amount of code humans need to read, we turn the list into a string_list and iterate over it (as we are calling the same find_hook operation on each string). However, since bugreport should primarily be called by the user, the performance loss from massaging the string seems acceptable.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 bugreport.c | 53 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 53 insertions(+)
Show changes to bugreport.c +53 −0
diff --git a/bugreport.c b/bugreport.c
index 7a9fd36b60..4c77009f1b 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -9,6 +9,7 @@
 #include "config.h"
 #include "bugreport-config-safelist.h"
 #include "khash.h"
+#include "run-command.h"
 
 static void get_curl_version_info(struct strbuf *curl_info)
 {
@@ -121,6 +122,55 @@ static void get_safelisted_config(struct strbuf *config_info)
 	cfgset_clear(&safelist);
 }
 
+static void get_populated_hooks(struct strbuf *hook_info, int nongit)
+{
+	/*
+	 * Doesn't look like there is a list of all possible hooks; so below is
+	 * a transcription of `git help hook`.
+	 */
+	const char *hooks = "applypatch-msg,"
+			    "pre-applypatch,"
+			    "post-applypatch,"
+			    "pre-commit,"
+			    "pre-merge-commit,"
+			    "prepare-commit-msg,"
+			    "commit-msg,"
+			    "post-commit,"
+			    "pre-rebase,"
+			    "post-checkout,"
+			    "post-merge,"
+			    "pre-push,"
+			    "pre-receive,"
+			    "update,"
+			    "post-receive,"
+			    "post-update,"
+			    "push-to-checkout,"
+			    "pre-auto-gc,"
+			    "post-rewrite,"
+			    "sendemail-validate,"
+			    "fsmonitor-watchman,"
+			    "p4-pre-submit,"
+			    "post-index-changex";
+	struct string_list hooks_list = STRING_LIST_INIT_DUP;
+	struct string_list_item *iter = NULL;
+
+
+	if (nongit) {
+		strbuf_addstr(hook_info,
+			"not run from a git repository - no hooks to show\n");
+		return;
+	}
+
+	string_list_split(&hooks_list, hooks, ',', -1);
+
+	for_each_string_list_item(iter, &hooks_list) {
+		if (find_hook(iter->string)) {
+			strbuf_addstr(hook_info, iter->string);
+			strbuf_complete_line(hook_info);
+		}
+	}
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output <file>]"),
 	NULL
@@ -193,6 +243,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Safelisted Config Info");
 	get_safelisted_config(&buffer);
 
+	get_header(&buffer, "Configured Hooks");
+	get_populated_hooks(&buffer, nongit_ok);
+
 	report = fopen_for_writing(report_path.buf);
 	strbuf_write(&buffer, report);
 	fclose(report);
-- 
2.25.0.341.g760bfbb309-goog
Junio C Hamano· Feb 4, 2020, 18:44 UTC · re: emilyshaffer@google.com · lore

Re: [PATCH v5 11/15] bugreport: collect list of populated hooks

emilyshaffer@google.com writes:
> +	/*
> +	 * Doesn't look like there is a list of all possible hooks; so below is
> +	 * a transcription of `git help hook`.

That's "git help hooks", if I tried my reproduction correctly. A straight-forward (in the sense of "what we want in the outcome is quite clear" and not in the sense of "anybody can design and implement it with a single 30-line patch") follow-up we can make after this series lands is to rethink how Documentation/githooks.txt is maintained and the list we have here is synchronized with it.

The design could me just the matter of running "grep" of some sort, with appropriate markups that are no-op to AsciiDoctor/AsciiDoc added to the documentation source, to produce this list.

Show 52 quoted lines
> +	 */
> +	const char *hooks = "applypatch-msg,"
> +			    "pre-applypatch,"
> +			    "post-applypatch,"
> +			    "pre-commit,"
> +			    "pre-merge-commit,"
> +			    "prepare-commit-msg,"
> +			    "commit-msg,"
> +			    "post-commit,"
> +			    "pre-rebase,"
> +			    "post-checkout,"
> +			    "post-merge,"
> +			    "pre-push,"
> +			    "pre-receive,"
> +			    "update,"
> +			    "post-receive,"
> +			    "post-update,"
> +			    "push-to-checkout,"
> +			    "pre-auto-gc,"
> +			    "post-rewrite,"
> +			    "sendemail-validate,"
> +			    "fsmonitor-watchman,"
> +			    "p4-pre-submit,"
> +			    "post-index-changex";
> +	struct string_list hooks_list = STRING_LIST_INIT_DUP;
> +	struct string_list_item *iter = NULL;
> +
> +
> +	if (nongit) {
> +		strbuf_addstr(hook_info,
> +			"not run from a git repository - no hooks to show\n");
> +		return;
> +	}
> +
> +	string_list_split(&hooks_list, hooks, ',', -1);
> +
> +	for_each_string_list_item(iter, &hooks_list) {
> +		if (find_hook(iter->string)) {
> +			strbuf_addstr(hook_info, iter->string);
> +			strbuf_complete_line(hook_info);
> +		}
> +	}
> +}
> +
>  static const char * const bugreport_usage[] = {
>  	N_("git bugreport [-o|--output <file>]"),
>  	NULL
> @@ -193,6 +243,9 @@ int cmd_main(int argc, const char **argv)
>  	get_header(&buffer, "Safelisted Config Info");
>  	get_safelisted_config(&buffer);
>  
> +	get_header(&buffer, "Configured Hooks");

Phrase nit. There may be many people who just enabled hooks without configuring, so "Enabled Hooks" may be more appropriate. We do not have to inspect what is in the hook to determine if it is enabled, but we do need to if we want to tell if a hook is "configured".

Show 5 quoted lines
> +	get_populated_hooks(&buffer, nongit_ok);
> +
>  	report = fopen_for_writing(report_path.buf);
>  	strbuf_write(&buffer, report);
>  	fclose(report);
Emily Shaffer· Feb 5, 2020, 02:48 UTC · re: Junio C Hamano · lore

Re: [PATCH v5 11/15] bugreport: collect list of populated hooks

On Tue, Feb 04, 2020 at 10:44:39AM -0800, Junio C Hamano wrote:
Show 7 quoted lines
> emilyshaffer@google.com writes:
> 
> > +	/*
> > +	 * Doesn't look like there is a list of all possible hooks; so below is
> > +	 * a transcription of `git help hook`.
> 
> That's "git help hooks", if I tried my reproduction correctly.
Yep. I'll fix it.
Show 9 quoted lines
> A straight-forward (in the sense of "what we want in the outcome is
> quite clear" and not in the sense of "anybody can design and implement
> it with a single 30-line patch") follow-up we can make after this
> series lands is to rethink how Documentation/githooks.txt is
> maintained and the list we have here is synchronized with it.
> 
> The design could me just the matter of running "grep" of some sort,
> with appropriate markups that are no-op to AsciiDoctor/AsciiDoc
> added to the documentation source, to produce this list.

Well, with Martin's suggestion[1] to use annotate:bugreport[include] instead of bugreport:include[x], this kind of change becomes a matter of course to just use "annotate:hook[x]" and write a script nearly identical to generate-bugreport-config-safelist.sh, and add it to the Makefile.

Like I said up-thread, I don't like the idea just because it's awfully temporary (pending work that's right below this on my list for the coming weeks).

It sounds like you're saying you don't mind any work of this nature being done as a follow-up - and that's fine with me, too. I'd just rather do the follow-up with enums and a library than with grep and sh. :)

Show 13 quoted lines
> >  static const char * const bugreport_usage[] = {
> >  	N_("git bugreport [-o|--output <file>]"),
> >  	NULL
> > @@ -193,6 +243,9 @@ int cmd_main(int argc, const char **argv)
> >  	get_header(&buffer, "Safelisted Config Info");
> >  	get_safelisted_config(&buffer);
> >  
> > +	get_header(&buffer, "Configured Hooks");
> 
> Phrase nit.  There may be many people who just enabled hooks without
> configuring, so "Enabled Hooks" may be more appropriate.  We do not
> have to inspect what is in the hook to determine if it is enabled,
> but we do need to if we want to tell if a hook is "configured".
Ah, that's a good point. OK, I'll change the wording as you suggested.
 - Emily
Emily Shaffer· Feb 5, 2020, 03:00 UTC · re: Emily Shaffer · lore

Re: [PATCH v5 11/15] bugreport: collect list of populated hooks

On Tue, Feb 04, 2020 at 06:48:59PM -0800, Emily Shaffer wrote:
Show 5 quoted lines
> Well, with Martin's suggestion[1] to use annotate:bugreport[include]
> instead of bugreport:include[x], this kind of change becomes a matter of
> course to just use "annotate:hook[x]" and write a script nearly
> identical to generate-bugreport-config-safelist.sh, and add it to the
> Makefile.
Bah.
[1]: https://lore.kernel.org/git/CAN0heSq_i4EitqYH-qrZyXBU+=PUNcSXOOJDHLSnJ1ufV0WtaQ@mail.gmail.com
emilyshaffer@google.com· Jan 24, 2020, 03:34 UTC · re: emilyshaffer@google.com · lore

[PATCH v5 13/15] bugreport: add packed object summary

From: Emily Shaffer <emilyshaffer@google.com>

Alongside the loose object counts, it can be useful to show the number of packs and packed objects. This way we can check whether the repo has an appropriate ratio of packed to loose objects to help determine whether it's behaving correctly.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 bugreport.c | 38 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 38 insertions(+)
Show changes to bugreport.c +38 −0
diff --git a/bugreport.c b/bugreport.c
index bf10857183..45cc1764e0 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -214,6 +214,41 @@ static void get_loose_object_summary(struct strbuf *obj_info) {
 		    total_count_questionable ? " (problem during count)" : "");
 }
 
+struct packed_object_cb_data {
+	struct packed_git *last_pack;
+	int pack_count;
+	int object_count;
+};
+
+static int packed_object_cb(const struct object_id *oid,
+			    struct packed_git *pack,
+			    uint32_t pos,
+			    void *data) {
+	struct packed_object_cb_data *cb_data = data;
+
+	if (!cb_data)
+		return 1;
+
+	if (pack && pack != cb_data->last_pack) {
+		cb_data->last_pack = pack;
+		cb_data->pack_count++;
+	}
+
+	cb_data->object_count++;
+
+	return 0;
+}
+
+static void get_packed_object_summary(struct strbuf *obj_info)
+{
+	struct packed_object_cb_data cb_data = {NULL, 0, 0};
+
+	for_each_packed_object(packed_object_cb, &cb_data, 0);
+
+	strbuf_addf(obj_info, "%d total packs (%d objects)\n",
+		    cb_data.pack_count, cb_data.object_count);
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output <file>]"),
 	NULL
@@ -292,6 +327,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Loose Object Counts");
 	get_loose_object_summary(&buffer);
 
+	get_header(&buffer, "Packed Object Summary");
+	get_packed_object_summary(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 	strbuf_write(&buffer, report);
 	fclose(report);
-- 
2.25.0.341.g760bfbb309-goog
Junio C Hamano· Feb 4, 2020, 19:00 UTC · re: emilyshaffer@google.com · lore

Re: [PATCH v5 13/15] bugreport: add packed object summary

emilyshaffer@google.com writes:
Show 6 quoted lines
> From: Emily Shaffer <emilyshaffer@google.com>
>
> Alongside the loose object counts, it can be useful to show the number
> of packs and packed objects. This way we can check whether the repo has
> an appropriate ratio of packed to loose objects to help determine
> whether it's behaving correctly.

This lets the enumeration machinery to enumerate at the individual object level, relies on the enumeration machinery to show your callback all objects from the same pack before an object from a different pack, to count both objects and packs.

But given that packfiles record how many objects there are in the pack, and that the packfile.c layer must know what are the pack files we have available, I wonder if we have an API already to allow us to enumerate each packfile, and while counting the number of times our callback was called, add the objects contained in each of them. If there isn't, it does not sound like a bad API function to add (i.e. given a repository r, iterate over r->objects->packed_git and let the API caller see each of them; the API caller can grab num_objects and add them up).

Emily Shaffer· Feb 5, 2020, 03:15 UTC · re: Junio C Hamano · lore

Re: [PATCH v5 13/15] bugreport: add packed object summary

On Tue, Feb 04, 2020 at 11:00:41AM -0800, Junio C Hamano wrote:
Show 23 quoted lines
> emilyshaffer@google.com writes:
> 
> > From: Emily Shaffer <emilyshaffer@google.com>
> >
> > Alongside the loose object counts, it can be useful to show the number
> > of packs and packed objects. This way we can check whether the repo has
> > an appropriate ratio of packed to loose objects to help determine
> > whether it's behaving correctly.
> 
> This lets the enumeration machinery to enumerate at the individual
> object level, relies on the enumeration machinery to show your
> callback all objects from the same pack before an object from a
> different pack, to count both objects and packs.
> 
> But given that packfiles record how many objects there are in the
> pack, and that the packfile.c layer must know what are the pack
> files we have available, I wonder if we have an API already to allow
> us to enumerate each packfile, and while counting the number of
> times our callback was called, add the objects contained in each of
> them.  If there isn't, it does not sound like a bad API function to
> add (i.e. given a repository r, iterate over r->objects->packed_git
> and let the API caller see each of them; the API caller can grab
> num_objects and add them up).

Yeah, I don't see one. I'll go ahead and add it - that sounds straightforward enough.

 - Emily
Junio C Hamano· Feb 4, 2020, 19:03 UTC · re: emilyshaffer@google.com · lore

Re: [PATCH v5 13/15] bugreport: add packed object summary

emilyshaffer@google.com writes:
Show 6 quoted lines
> From: Emily Shaffer <emilyshaffer@google.com>
>
> Alongside the loose object counts, it can be useful to show the number
> of packs and packed objects. This way we can check whether the repo has
> an appropriate ratio of packed to loose objects to help determine
> whether it's behaving correctly.

This step makes me wonder if we want to see the midx as well. Didn't we have a bug that manifests only when midx is (or is not) in use?

Similarly, I think use (or non-use) of the commit graph may also be a useful information for diagnosing.

Emily Shaffer· Feb 5, 2020, 03:09 UTC · re: Junio C Hamano · lore

Re: [PATCH v5 13/15] bugreport: add packed object summary

On Tue, Feb 04, 2020 at 11:03:57AM -0800, Junio C Hamano wrote:
Show 16 quoted lines
> emilyshaffer@google.com writes:
> 
> > From: Emily Shaffer <emilyshaffer@google.com>
> >
> > Alongside the loose object counts, it can be useful to show the number
> > of packs and packed objects. This way we can check whether the repo has
> > an appropriate ratio of packed to loose objects to help determine
> > whether it's behaving correctly.
> 
> This step makes me wonder if we want to see the midx as well.
> Didn't we have a bug that manifests only when midx is (or is not) in
> use?
> 
> Similarly, I think use (or non-use) of the commit graph may also be
> a useful information for diagnosing.
> 

Sure. I'll look into both of these. Would you rather see them as add-ons later, or do you want me to hold off on a v6 of this set until they're ready?

 - Emily
emilyshaffer@google.com· Jan 24, 2020, 03:34 UTC · re: emilyshaffer@google.com · lore

[PATCH v5 12/15] bugreport: count loose objects

From: Emily Shaffer <emilyshaffer@google.com>

The number of unpacked objects in a user's repository may help us understand the root of the problem they're seeing, especially if a command is running unusually slowly.

Helped-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
The refactor removed much of the code Dscho suggested; and yet it
remains true that he helped me while developing this commit (although
his suggestions didn't survive). Shall I leave the Helped-by line or
remove it?
 - Emily
 bugreport.c | 46 ++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 46 insertions(+)
Show changes to bugreport.c +46 −0
diff --git a/bugreport.c b/bugreport.c
index 4c77009f1b..bf10857183 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -10,6 +10,7 @@
 #include "bugreport-config-safelist.h"
 #include "khash.h"
 #include "run-command.h"
+#include "object-store.h"
 
 static void get_curl_version_info(struct strbuf *curl_info)
 {
@@ -171,6 +172,48 @@ static void get_populated_hooks(struct strbuf *hook_info, int nongit)
 	}
 }
 
+static int loose_object_cb(const struct object_id *oid, const char *path,
+			   void *data) {
+	int *loose_object_count = data;
+
+	if (loose_object_count) {
+		(*loose_object_count)++;
+		return 0;
+	}
+
+	return 1;
+}
+
+static void get_loose_object_summary(struct strbuf *obj_info) {
+
+	int local_loose_object_count = 0, total_loose_object_count = 0;
+	int local_count_questionable = 0, total_count_questionable = 0;
+
+	local_count_questionable = for_each_loose_object(
+					loose_object_cb,
+					&local_loose_object_count,
+					FOR_EACH_OBJECT_LOCAL_ONLY);
+
+	total_count_questionable = for_each_loose_object(
+					loose_object_cb,
+					&total_loose_object_count,
+					0);
+
+	strbuf_addf(obj_info, "%d local loose objects%s\n",
+		    local_loose_object_count,
+		    local_count_questionable ? " (problem during count)" : "");
+
+	strbuf_addf(obj_info, "%d alternate loose objects%s\n",
+		    total_loose_object_count - local_loose_object_count,
+		    (local_count_questionable || total_count_questionable)
+			? " (problem during count)"
+			: "");
+
+	strbuf_addf(obj_info, "%d total loose objects%s\n",
+		    total_loose_object_count,
+		    total_count_questionable ? " (problem during count)" : "");
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output <file>]"),
 	NULL
@@ -246,6 +289,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Configured Hooks");
 	get_populated_hooks(&buffer, nongit_ok);
 
+	get_header(&buffer, "Loose Object Counts");
+	get_loose_object_summary(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 	strbuf_write(&buffer, report);
 	fclose(report);
-- 
2.25.0.341.g760bfbb309-goog
Junio C Hamano· Feb 4, 2020, 18:48 UTC · re: emilyshaffer@google.com · lore

Re: [PATCH v5 12/15] bugreport: count loose objects

emilyshaffer@google.com writes:
Show 13 quoted lines
> From: Emily Shaffer <emilyshaffer@google.com>
>
> The number of unpacked objects in a user's repository may help us
> understand the root of the problem they're seeing, especially if a
> command is running unusually slowly.
>
> Helped-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
> Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
> ---
> The refactor removed much of the code Dscho suggested; and yet it
> remains true that he helped me while developing this commit (although
> his suggestions didn't survive). Shall I leave the Helped-by line or
> remove it?

You two collectively thought about viable alternatives and decided to reject what was not wanted in the final result, and not having that rejected code was good for the project, right? If so, I would say it still is the help that deserves recognition. After all, making the result better by removing things is harder than by adding things ;-)

>  bugreport.c | 46 ++++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 46 insertions(+)
> ...
The patch text looked sensible.
Emily Shaffer· Feb 5, 2020, 02:50 UTC · re: Junio C Hamano · lore

Re: [PATCH v5 12/15] bugreport: count loose objects

On Tue, Feb 04, 2020 at 10:48:30AM -0800, Junio C Hamano wrote:
Show 22 quoted lines
> emilyshaffer@google.com writes:
> 
> > From: Emily Shaffer <emilyshaffer@google.com>
> >
> > The number of unpacked objects in a user's repository may help us
> > understand the root of the problem they're seeing, especially if a
> > command is running unusually slowly.
> >
> > Helped-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
> > Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
> > ---
> > The refactor removed much of the code Dscho suggested; and yet it
> > remains true that he helped me while developing this commit (although
> > his suggestions didn't survive). Shall I leave the Helped-by line or
> > remove it?
> 
> You two collectively thought about viable alternatives and decided
> to reject what was not wanted in the final result, and not having
> that rejected code was good for the project, right?  If so, I would
> say it still is the help that deserves recognition.  After all,
> making the result better by removing things is harder than by adding
> things ;-)

Sounds great to me. I'd rather over-assign credit than under-assign (but I don't think this is a case of over-assigning here).

Show 5 quoted lines
> >  bugreport.c | 46 ++++++++++++++++++++++++++++++++++++++++++++++
> >  1 file changed, 46 insertions(+)
> > ...
> 
> The patch text looked sensible.
Thanks.
 - Emily
emilyshaffer@google.com· Jan 24, 2020, 03:34 UTC · re: emilyshaffer@google.com · lore

[PATCH v5 14/15] bugreport: list contents of $OBJDIR/info

From: Emily Shaffer <emilyshaffer@google.com>

Miscellaneous information used about the object store can end up in .git/objects/info; this can help us understand what may be going on with the object store when the user is reporting a bug. Otherwise, it could be difficult to track down what is going wrong with an object which isn't kept locally to .git/objects/ or .git/objects/pack. Having some understanding of where the user's objects may be kept can save us some hops during the bug reporting process.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 bugreport.c | 48 ++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 48 insertions(+)
Show changes to bugreport.c +48 −0
diff --git a/bugreport.c b/bugreport.c
index 45cc1764e0..12463ffdc7 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -249,6 +249,51 @@ static void get_packed_object_summary(struct strbuf *obj_info)
 		    cb_data.pack_count, cb_data.object_count);
 }
 
+static void list_contents_of_dir_recursively(struct strbuf *contents,
+				      	     struct strbuf *dirpath)
+{
+	struct dirent *d;
+	DIR *dir;
+	size_t path_len;
+
+	dir = opendir(dirpath->buf);
+	if (!dir)
+		return;
+
+	strbuf_complete(dirpath, '/');
+	path_len = dirpath->len;
+
+	while ((d = readdir(dir))) {
+		if (!strcmp(d->d_name, ".") || !strcmp(d->d_name, ".."))
+			continue;
+
+		strbuf_addbuf(contents, dirpath);
+		strbuf_addstr(contents, d->d_name);
+		strbuf_complete_line(contents);
+
+		if (d->d_type == DT_DIR) {
+			strbuf_addstr(dirpath, d->d_name);
+			list_contents_of_dir_recursively(contents, dirpath);
+		}
+		strbuf_setlen(dirpath, path_len);
+	}
+
+	closedir(dir);
+}
+
+static void get_object_info_summary(struct strbuf *obj_info)
+{
+	struct strbuf dirpath = STRBUF_INIT;
+
+	strbuf_addstr(&dirpath, get_object_directory());
+	strbuf_complete(&dirpath, '/');
+	strbuf_addstr(&dirpath, "info/");
+
+	list_contents_of_dir_recursively(obj_info, &dirpath);
+
+	strbuf_release(&dirpath);
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output <file>]"),
 	NULL
@@ -330,6 +375,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Packed Object Summary");
 	get_packed_object_summary(&buffer);
 
+	get_header(&buffer, "Object Info Summary");
+	get_object_info_summary(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 	strbuf_write(&buffer, report);
 	fclose(report);
-- 
2.25.0.341.g760bfbb309-goog
emilyshaffer@google.com· Jan 24, 2020, 03:34 UTC · re: emilyshaffer@google.com · lore

[PATCH v5 15/15] bugreport: summarize contents of alternates file

From: Emily Shaffer <emilyshaffer@google.com>

In some cases, it could be that the user is having a problem with an object which isn't present in their normal object directory. We can get a hint that that might be the case by examining the list of alternates where their object may be stored instead. Since paths to alternates may be sensitive, we'll instead count how many alternates have been specified and note how many of them exist or are broken.

While object-cache.h describes a function "foreach_alt_odb()", this function does not provide information on broken alternates, which are skipped over in "link_alt_odb_entry()". Since the goal is to identify missing alternates, we can gather the contents of .git/objects/info/alternates manually.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 bugreport.c | 39 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 39 insertions(+)
Show changes to bugreport.c +39 −0
diff --git a/bugreport.c b/bugreport.c
index 12463ffdc7..2c540182dd 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -294,6 +294,42 @@ static void get_object_info_summary(struct strbuf *obj_info)
 	strbuf_release(&dirpath);
 }
 
+static void get_alternates_summary(struct strbuf *alternates_info)
+{
+	struct strbuf alternates_path = STRBUF_INIT;
+	struct strbuf alternate = STRBUF_INIT;
+	FILE *file;
+	size_t exists = 0, broken = 0;
+
+	strbuf_addstr(&alternates_path, get_object_directory());
+	strbuf_complete(&alternates_path, '/');
+	strbuf_addstr(&alternates_path, "info/alternates");
+
+	file = fopen(alternates_path.buf, "r");
+	if (!file) {
+		strbuf_addstr(alternates_info, "No alternates file found.\n");
+		strbuf_release(&alternates_path);
+		return;
+	}
+
+	while (strbuf_getline(&alternate, file) != EOF) {
+		if (!access(alternate.buf, F_OK))
+			exists++;
+		else
+			broken++;
+	}
+
+	strbuf_addf(alternates_info,
+		    "%zd alternates found (%zd working, %zd broken)\n",
+		    exists + broken,
+		    exists,
+		    broken);
+
+	fclose(file);
+	strbuf_release(&alternate);
+	strbuf_release(&alternates_path);
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output <file>]"),
 	NULL
@@ -378,6 +414,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Object Info Summary");
 	get_object_info_summary(&buffer);
 
+	get_header(&buffer, "Alternates");
+	get_alternates_summary(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 	strbuf_write(&buffer, report);
 	fclose(report);
-- 
2.25.0.341.g760bfbb309-goog
Emily Shaffer· Jan 24, 2020, 03:38 UTC · re: emilyshaffer@google.com · lore

Re: [PATCH v5 00/15] add git-bugreport tool

A sample bugreport generated from this patchset follows:

Thank you for filling out a Git bug report! Please answer the following questions to help us understand your issue.

What did you do before the bug happened? (Steps to reproduce your issue)
What did you expect to happen? (Expected behavior)
What happened instead? (Actual behavior)
What's different between what you expected and what actually happened?
Anything else you want to add:

Please review the rest of the bug report below. You can delete any lines you don't wish to send.

[System Info] git version: git version 2.25.0.18.g682ab0d3eb cpu: x86_64 built from commit: 682ab0d3eb8b84f8af4db1a161d24ca53d2f39fc sizeof-long: 8 sizeof-size_t: 8 shell-path: /bin/sh uname -a: Linux 5.2.17-1rodete3-amd64 #1 SMP Debian 5.2.17-1rodete3 (2019-10-21 > 2018) x86_64 compiler info: glibc: 2.28 $SHELL (typically, interactive shell): /bin/bash git-remote-https --build-info: git-http-fetch version: 2.25.0.18.g682ab0d3eb built from commit: 682ab0d3eb8b84f8af4db1a161d24ca53d2f39fc curl version: libcurl/7.66.0 GnuTLS/3.6.9 zlib/1.2.11 brotli/1.0.7 libidn2/2.2.0 libpsl/0.20.2 (+libidn2/2.2.0) libssh2/1.8.0 nghttp2/1.39.2 librtmp/2.3

[Safelisted Config Info] sendemail.from (global) : emilyshaffer@google.com sendemail.from (repo) : emilyshaffer@google.com

[Configured Hooks] pre-commit prepare-commit-msg

[Loose Object Counts] 2549 local loose objects 2641 alternate loose objects 5190 total loose objects

[Packed Object Summary] 60 total packs (591308 objects)

[Object Info Summary] .git/objects/info/foo .git/objects/info/bar .git/objects/info/bar/baz .git/objects/info/alternates

[Alternates] 2 alternates found (1 working, 1 broken)

Jonathan Tan· Jan 28, 2020, 23:04 UTC · re: emilyshaffer@google.com · lore

Re: [PATCH v5 00/15] add git-bugreport tool

Show 11 quoted lines
> From: Emily Shaffer <emilyshaffer@google.com>
> 
> This topic branch depends on the patch mailed in
> lore.kernel.org/git/20191211233820.185153-1-emilyshaffer@google.com in order to
> display scopes for configs gathered during "bugreport: add config values from
> safelist".
> 
> I'll summarize v4-v5. Since v4 has languished for some time, I don't
> think an interdiff is too helpful, so I won't include one. Bonus, the
> code is much simplified from some suggestions from Junio on how to
> inspect objects, so I hope it's easy to review anyways.

To everyone in the developer community interested in this set: what is the status of this?

If this needs further review, then maybe it would be best if only patches 1-4 were put up for submission first, with a note in the bugreport documentation that more information may be added in future Git versions. For me, patches 1-4 look good and I don't have enough experience with uname (especially across libcs and OSes) to determine what should or should not be included - if this is typical of reviewers in the Git project, it might be better to submit patches 1-4 first, and then send each additional diagnostic separately, so that people who know what's going on in one area but not another can just comment on the area they know about.

Having said that, I see that a few people have already looked at the entire patchset and made comments, so if they are OK with it, we don't need to split it up.

Emily Shaffer· Jan 28, 2020, 23:26 UTC · re: Jonathan Tan · lore

Re: [PATCH v5 00/15] add git-bugreport tool

On Tue, Jan 28, 2020 at 03:04:21PM -0800, Jonathan Tan wrote:
Show 29 quoted lines
> > From: Emily Shaffer <emilyshaffer@google.com>
> > 
> > This topic branch depends on the patch mailed in
> > lore.kernel.org/git/20191211233820.185153-1-emilyshaffer@google.com in order to
> > display scopes for configs gathered during "bugreport: add config values from
> > safelist".
> > 
> > I'll summarize v4-v5. Since v4 has languished for some time, I don't
> > think an interdiff is too helpful, so I won't include one. Bonus, the
> > code is much simplified from some suggestions from Junio on how to
> > inspect objects, so I hope it's easy to review anyways.
> 
> To everyone in the developer community interested in this set: what is
> the status of this?
> 
> If this needs further review, then maybe it would be best if only
> patches 1-4 were put up for submission first, with a note in the
> bugreport documentation that more information may be added in future Git
> versions. For me, patches 1-4 look good and I don't have enough
> experience with uname (especially across libcs and OSes) to determine
> what should or should not be included - if this is typical of reviewers
> in the Git project, it might be better to submit patches 1-4 first, and
> then send each additional diagnostic separately, so that people who know
> what's going on in one area but not another can just comment on the area
> they know about.
> 
> Having said that, I see that a few people have already looked at the
> entire patchset and made comments, so if they are OK with it, we don't
> need to split it up.

I'd be fine with either; from now, my gut says the only ones I'm not comfortable merging today are maybe 10, as Junio had some concerns about whether to allow glob expansion, and 11, as it's prone to rot and I'm doing other work in that area which would cause 11 to be dropped/refactored anyways once it goes in. Otherwise I think they're all pretty OK to go in and let the other work continue later in a different thread.

Of course, as an author my opinion that they're good to go doesn't mean much ;)

 - Emily
Martin Ågren· Jan 30, 2020, 22:15 UTC · re: emilyshaffer@google.com · lore

Re: [PATCH v5 00/15] add git-bugreport tool

On Fri, 24 Jan 2020 at 04:35, <emilyshaffer@google.com> wrote:
> This topic branch depends on the patch mailed in
> lore.kernel.org/git/20191211233820.185153-1-emilyshaffer@google.com in order to
> display scopes for configs gathered during "bugreport: add config values from
> safelist".

Should this use `config_scope_name()` which looks like it's about to graduate [1]? Disclaimer: I haven't followed that patch set too closely.

[1] https://lore.kernel.org/git/xmqqzhe66dav.fsf@gitster-ct.c.googlers.com/
Martin
Emily Shaffer· Feb 4, 2020, 00:07 UTC · re: Martin Ågren · lore

Re: [PATCH v5 00/15] add git-bugreport tool

On Thu, Jan 30, 2020 at 11:15:25PM +0100, Martin Ågren wrote:
Show 8 quoted lines
> On Fri, 24 Jan 2020 at 04:35, <emilyshaffer@google.com> wrote:
> > This topic branch depends on the patch mailed in
> > lore.kernel.org/git/20191211233820.185153-1-emilyshaffer@google.com in order to
> > display scopes for configs gathered during "bugreport: add config values from
> > safelist".
> 
> Should this use `config_scope_name()` which looks like it's about
> to graduate [1]? Disclaimer: I haven't followed that patch set too closely.

Yeah, you're right that I can use it. config.h:config_scope_name() is a new feature on that patch since last time I looked. Thanks for the tip!

 - Emily
> [1] https://lore.kernel.org/git/xmqqzhe66dav.fsf@gitster-ct.c.googlers.com/
Emily Shaffer· Feb 6, 2020, 00:40 UTC · re: emilyshaffer@google.com · lore

[PATCH v6 00/15] add git-bugreport tool

This patchset is based on mr/show-config-scope.
Since v5, I addressed comments from Martin and Junio.
 - Reorganized the series so the ecosystem cleanup comes before the
   interesting new features.
 - Spread out manpage updates to indicate each new feature as it's added.
 - Fixed a number of patches which assumed a repo, so bugreport can now
   run anywhere.
 - Wordsmithing in the report template.
 - die() instead of overwriting an existing report.
 - Added --suffix to allow user to change the name of the report to
   git-bugreport-<strftime format>.txt.
 - General NULL-ptr, leak, and bashism cleanup.
 - Changed safelist macro from bugreport:include[x] to
   annotate:bugreport[include].
 - Use git_config_get_value_multi() instead of a config callback to
   gather the safelist configs.
 - Added a convenience for_each_pack() macro to object-store.h and began
   using it in packfile.c and bugreport.c for pack summary
Emily Shaffer (15):
  help: move list_config_help to builtin/help
  help: add shell-path to --build-options
  bugreport: add tool to generate debugging info
  bugreport: gather git version and build info
  bugreport: add uname info
  bugreport: add compiler info
  bugreport: add git-remote-https version
  bugreport: include user interactive shell
  bugreport: generate config safelist based on docs
  bugreport: add config values from safelist
  bugreport: collect list of populated hooks
  bugreport: count loose objects
  bugreport: add packed object summary
  bugreport: list contents of $OBJDIR/info
  bugreport: summarize contents of alternates file
 .gitignore                              |   3 +
 Documentation/asciidoc.conf             |   9 +
 Documentation/asciidoctor-extensions.rb |   5 +
 Documentation/config/sendemail.txt      |  56 ++--
 Documentation/git-bugreport.txt         |  55 ++++
 Makefile                                |  27 +-
 bugreport.c                             | 409 ++++++++++++++++++++++++
 builtin/help.c                          |  86 +++++
 compat/compiler.h                       |  24 ++
 generate-bugreport-config-safelist.sh   |  17 +
 generate-cmdlist.sh                     |  19 --
 generate-configlist.sh                  |  24 ++
 help.c                                  | 132 ++------
 help.h                                  |   2 +-
 object-store.h                          |   6 +
 packfile.c                              |   3 +-
 remote-curl.c                           |   8 +
 t/t0091-bugreport.sh                    |  49 +++
 18 files changed, 778 insertions(+), 156 deletions(-)
 create mode 100644 Documentation/git-bugreport.txt
 create mode 100644 bugreport.c
 create mode 100644 compat/compiler.h
 create mode 100755 generate-bugreport-config-safelist.sh
 create mode 100755 generate-configlist.sh
 create mode 100755 t/t0091-bugreport.sh
-- 
2.25.0.341.g760bfbb309-goog
Emily Shaffer· Feb 6, 2020, 00:40 UTC · re: Emily Shaffer · lore

[PATCH v6 01/15] help: move list_config_help to builtin/help

Starting in 3ac68a93fd2, help.o began to depend on builtin/branch.o, builtin/clean.o, and builtin/config.o. This meant that help.o was unusable outside of the context of the main Git executable.

To make help.o usable by other commands again, move list_config_help() into builtin/help.c (where it makes sense to assume other builtin libraries are present).

When command-list.h is included but a member is not used, we start to hear a compiler warning. Since the config list is generated in a fairly different way than the command list, and since commands and config options are semantically different, move the config list into its own header and move the generator into its own script and build rule.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 .gitignore             |  1 +
 Makefile               | 13 +++++--
 builtin/help.c         | 86 ++++++++++++++++++++++++++++++++++++++++++
 generate-cmdlist.sh    | 19 ----------
 generate-configlist.sh | 24 ++++++++++++
 help.c                 | 85 -----------------------------------------
 help.h                 |  1 -
 7 files changed, 121 insertions(+), 108 deletions(-)
 create mode 100755 generate-configlist.sh
Show changes to 7 files +121 −108

.gitignore, Makefile, builtin/help.c, generate-cmdlist.sh, generate-configlist.sh, help.c, help.h

diff --git a/.gitignore b/.gitignore
index aebe7c0908..ea97de83f3 100644
--- a/.gitignore
+++ b/.gitignore
@@ -189,6 +189,7 @@
 /gitweb/gitweb.cgi
 /gitweb/static/gitweb.js
 /gitweb/static/gitweb.min.*
+/config-list.h
 /command-list.h
 *.tar.gz
 *.dsc
diff --git a/Makefile b/Makefile
index 09f98b777c..5a022367d4 100644
--- a/Makefile
+++ b/Makefile
@@ -814,6 +814,7 @@ LIB_FILE = libgit.a
 XDIFF_LIB = xdiff/lib.a
 VCSSVN_LIB = vcs-svn/lib.a
 
+GENERATED_H += config-list.h
 GENERATED_H += command-list.h
 
 LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
@@ -2128,7 +2129,7 @@ git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS)
 
 help.sp help.s help.o: command-list.h
 
-builtin/help.sp builtin/help.s builtin/help.o: command-list.h GIT-PREFIX
+builtin/help.sp builtin/help.s builtin/help.o: config-list.h GIT-PREFIX
 builtin/help.sp builtin/help.s builtin/help.o: EXTRA_CPPFLAGS = \
 	'-DGIT_HTML_PATH="$(htmldir_relative_SQ)"' \
 	'-DGIT_MAN_PATH="$(mandir_relative_SQ)"' \
@@ -2148,6 +2149,12 @@ $(BUILT_INS): git$X
 	ln -s $< $@ 2>/dev/null || \
 	cp $< $@
 
+config-list.h: generate-configlist.sh
+
+config-list.h:
+	$(QUIET_GEN)$(SHELL_PATH) ./generate-configlist.sh \
+		>$@+ && mv $@+ $@
+
 command-list.h: generate-cmdlist.sh command-list.txt
 
 command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Documentation/config/*.txt
@@ -2781,7 +2788,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE
 .PHONY: sparse $(SP_OBJ)
 sparse: $(SP_OBJ)
 
-EXCEPT_HDRS := command-list.h unicode-width.h compat/% xdiff/%
+EXCEPT_HDRS := command-list.h config-list.h unicode-width.h compat/% xdiff/%
 ifndef GCRYPT_SHA256
 	EXCEPT_HDRS += sha256/gcrypt.h
 endif
@@ -2803,7 +2810,7 @@ hdr-check: $(HCO)
 style:
 	git clang-format --style file --diff --extensions c,h
 
-check: command-list.h
+check: config-list.h command-list.h
 	@if sparse; \
 	then \
 		echo >&2 "Use 'make sparse' instead"; \
diff --git a/builtin/help.c b/builtin/help.c
index e5590d7787..1c5f2b9255 100644
--- a/builtin/help.c
+++ b/builtin/help.c
@@ -8,6 +8,7 @@
 #include "parse-options.h"
 #include "run-command.h"
 #include "column.h"
+#include "config-list.h"
 #include "help.h"
 #include "alias.h"
 
@@ -62,6 +63,91 @@ static const char * const builtin_help_usage[] = {
 	NULL
 };
 
+struct slot_expansion {
+	const char *prefix;
+	const char *placeholder;
+	void (*fn)(struct string_list *list, const char *prefix);
+	int found;
+};
+
+static void list_config_help(int for_human)
+{
+	struct slot_expansion slot_expansions[] = {
+		{ "advice", "*", list_config_advices },
+		{ "color.branch", "<slot>", list_config_color_branch_slots },
+		{ "color.decorate", "<slot>", list_config_color_decorate_slots },
+		{ "color.diff", "<slot>", list_config_color_diff_slots },
+		{ "color.grep", "<slot>", list_config_color_grep_slots },
+		{ "color.interactive", "<slot>", list_config_color_interactive_slots },
+		{ "color.remote", "<slot>", list_config_color_sideband_slots },
+		{ "color.status", "<slot>", list_config_color_status_slots },
+		{ "fsck", "<msg-id>", list_config_fsck_msg_ids },
+		{ "receive.fsck", "<msg-id>", list_config_fsck_msg_ids },
+		{ NULL, NULL, NULL }
+	};
+	const char **p;
+	struct slot_expansion *e;
+	struct string_list keys = STRING_LIST_INIT_DUP;
+	int i;
+
+	for (p = config_name_list; *p; p++) {
+		const char *var = *p;
+		struct strbuf sb = STRBUF_INIT;
+
+		for (e = slot_expansions; e->prefix; e++) {
+
+			strbuf_reset(&sb);
+			strbuf_addf(&sb, "%s.%s", e->prefix, e->placeholder);
+			if (!strcasecmp(var, sb.buf)) {
+				e->fn(&keys, e->prefix);
+				e->found++;
+				break;
+			}
+		}
+		strbuf_release(&sb);
+		if (!e->prefix)
+			string_list_append(&keys, var);
+	}
+
+	for (e = slot_expansions; e->prefix; e++)
+		if (!e->found)
+			BUG("slot_expansion %s.%s is not used",
+			    e->prefix, e->placeholder);
+
+	string_list_sort(&keys);
+	for (i = 0; i < keys.nr; i++) {
+		const char *var = keys.items[i].string;
+		const char *wildcard, *tag, *cut;
+
+		if (for_human) {
+			puts(var);
+			continue;
+		}
+
+		wildcard = strchr(var, '*');
+		tag = strchr(var, '<');
+
+		if (!wildcard && !tag) {
+			puts(var);
+			continue;
+		}
+
+		if (wildcard && !tag)
+			cut = wildcard;
+		else if (!wildcard && tag)
+			cut = tag;
+		else
+			cut = wildcard < tag ? wildcard : tag;
+
+		/*
+		 * We may produce duplicates, but that's up to
+		 * git-completion.bash to handle
+		 */
+		printf("%.*s\n", (int)(cut - var), var);
+	}
+	string_list_clear(&keys, 0);
+}
+
 static enum help_format parse_help_format(const char *format)
 {
 	if (!strcmp(format, "man"))
diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
index 71158f7d8b..45fecf8bdf 100755
--- a/generate-cmdlist.sh
+++ b/generate-cmdlist.sh
@@ -76,23 +76,6 @@ print_command_list () {
 	echo "};"
 }
 
-print_config_list () {
-	cat <<EOF
-static const char *config_name_list[] = {
-EOF
-	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
-	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
-	sort |
-	while read line
-	do
-		echo "	\"$line\","
-	done
-	cat <<EOF
-	NULL,
-};
-EOF
-}
-
 exclude_programs=
 while test "--exclude-program" = "$1"
 do
@@ -113,5 +96,3 @@ echo
 define_category_names "$1"
 echo
 print_command_list "$1"
-echo
-print_config_list
diff --git a/generate-configlist.sh b/generate-configlist.sh
new file mode 100755
index 0000000000..eca6a00c30
--- /dev/null
+++ b/generate-configlist.sh
@@ -0,0 +1,24 @@
+#!/bin/sh
+
+echo "/* Automatically generated by generate-configlist.sh */"
+echo
+
+print_config_list () {
+	cat <<EOF
+static const char *config_name_list[] = {
+EOF
+	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
+	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
+	sort |
+	while read line
+	do
+		echo "	\"$line\","
+	done
+	cat <<EOF
+	NULL,
+};
+EOF
+}
+
+echo
+print_config_list
diff --git a/help.c b/help.c
index cf67624a94..a21487db77 100644
--- a/help.c
+++ b/help.c
@@ -407,91 +407,6 @@ void list_common_guides_help(void)
 	putchar('\n');
 }
 
-struct slot_expansion {
-	const char *prefix;
-	const char *placeholder;
-	void (*fn)(struct string_list *list, const char *prefix);
-	int found;
-};
-
-void list_config_help(int for_human)
-{
-	struct slot_expansion slot_expansions[] = {
-		{ "advice", "*", list_config_advices },
-		{ "color.branch", "<slot>", list_config_color_branch_slots },
-		{ "color.decorate", "<slot>", list_config_color_decorate_slots },
-		{ "color.diff", "<slot>", list_config_color_diff_slots },
-		{ "color.grep", "<slot>", list_config_color_grep_slots },
-		{ "color.interactive", "<slot>", list_config_color_interactive_slots },
-		{ "color.remote", "<slot>", list_config_color_sideband_slots },
-		{ "color.status", "<slot>", list_config_color_status_slots },
-		{ "fsck", "<msg-id>", list_config_fsck_msg_ids },
-		{ "receive.fsck", "<msg-id>", list_config_fsck_msg_ids },
-		{ NULL, NULL, NULL }
-	};
-	const char **p;
-	struct slot_expansion *e;
-	struct string_list keys = STRING_LIST_INIT_DUP;
-	int i;
-
-	for (p = config_name_list; *p; p++) {
-		const char *var = *p;
-		struct strbuf sb = STRBUF_INIT;
-
-		for (e = slot_expansions; e->prefix; e++) {
-
-			strbuf_reset(&sb);
-			strbuf_addf(&sb, "%s.%s", e->prefix, e->placeholder);
-			if (!strcasecmp(var, sb.buf)) {
-				e->fn(&keys, e->prefix);
-				e->found++;
-				break;
-			}
-		}
-		strbuf_release(&sb);
-		if (!e->prefix)
-			string_list_append(&keys, var);
-	}
-
-	for (e = slot_expansions; e->prefix; e++)
-		if (!e->found)
-			BUG("slot_expansion %s.%s is not used",
-			    e->prefix, e->placeholder);
-
-	string_list_sort(&keys);
-	for (i = 0; i < keys.nr; i++) {
-		const char *var = keys.items[i].string;
-		const char *wildcard, *tag, *cut;
-
-		if (for_human) {
-			puts(var);
-			continue;
-		}
-
-		wildcard = strchr(var, '*');
-		tag = strchr(var, '<');
-
-		if (!wildcard && !tag) {
-			puts(var);
-			continue;
-		}
-
-		if (wildcard && !tag)
-			cut = wildcard;
-		else if (!wildcard && tag)
-			cut = tag;
-		else
-			cut = wildcard < tag ? wildcard : tag;
-
-		/*
-		 * We may produce duplicates, but that's up to
-		 * git-completion.bash to handle
-		 */
-		printf("%.*s\n", (int)(cut - var), var);
-	}
-	string_list_clear(&keys, 0);
-}
-
 static int get_alias(const char *var, const char *value, void *data)
 {
 	struct string_list *list = data;
diff --git a/help.h b/help.h
index 7a455beeb7..9071894e8c 100644
--- a/help.h
+++ b/help.h
@@ -22,7 +22,6 @@ static inline void mput_char(char c, unsigned int num)
 void list_common_cmds_help(void);
 void list_all_cmds_help(void);
 void list_common_guides_help(void);
-void list_config_help(int for_human);
 
 void list_all_main_cmds(struct string_list *list);
 void list_all_other_cmds(struct string_list *list);
-- 
2.25.0.341.g760bfbb309-goog
Danh Doan· Feb 6, 2020, 01:35 UTC · re: Emily Shaffer · lore

Re: [PATCH v6 01/15] help: move list_config_help to builtin/help

On 2020-02-05 16:40:54-0800, Emily Shaffer <emilyshaffer@google.com> wrote:
Show 56 quoted lines
> diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
> index 71158f7d8b..45fecf8bdf 100755
> --- a/generate-cmdlist.sh
> +++ b/generate-cmdlist.sh
> @@ -76,23 +76,6 @@ print_command_list () {
>  	echo "};"
>  }
>  
> -print_config_list () {
> -	cat <<EOF
> -static const char *config_name_list[] = {
> -EOF
> -	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
> -	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
> -	sort |
> -	while read line
> -	do
> -		echo "	\"$line\","
> -	done
> -	cat <<EOF
> -	NULL,
> -};
> -EOF
> -}
> -
>  exclude_programs=
>  while test "--exclude-program" = "$1"
>  do
> @@ -113,5 +96,3 @@ echo
>  define_category_names "$1"
>  echo
>  print_command_list "$1"
> -echo
> -print_config_list
> diff --git a/generate-configlist.sh b/generate-configlist.sh
> new file mode 100755
> index 0000000000..eca6a00c30
> --- /dev/null
> +++ b/generate-configlist.sh
> @@ -0,0 +1,24 @@
> +#!/bin/sh
> +
> +echo "/* Automatically generated by generate-configlist.sh */"
> +echo
> +
> +print_config_list () {
> +	cat <<EOF
> +static const char *config_name_list[] = {
> +EOF
> +	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
> +	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
> +	sort |
> +	while read line
> +	do
> +		echo "	\"$line\","
> +	done

This while-read-echo was moved from generate-cmdlist.sh, which has some logic to work with read-variable.

We're moving it out, I think apply this diff on top of it will make the code easier to read.
Show changes to generate-configlist.sh +1 −4
diff --git a/generate-configlist.sh b/generate-configlist.sh
index eca6a00c30..163dbf30bb 100755
--- a/generate-configlist.sh
+++ b/generate-configlist.sh
@@ -10,10 +10,7 @@ EOF
 	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
 	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
 	sort |
-	while read line
-	do
-		echo "	\"$line\","
-	done
+	sed 's/^/	"/; s/$/",/'
 	cat <<EOF
 	NULL,
 };
-- 
Danh
Emily Shaffer· Feb 13, 2020, 22:58 UTC · re: Danh Doan · lore

Re: [PATCH v6 01/15] help: move list_config_help to builtin/help

On Thu, Feb 06, 2020 at 08:35:33AM +0700, Danh Doan wrote:
Show 79 quoted lines
> On 2020-02-05 16:40:54-0800, Emily Shaffer <emilyshaffer@google.com> wrote:
> > diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
> > index 71158f7d8b..45fecf8bdf 100755
> > --- a/generate-cmdlist.sh
> > +++ b/generate-cmdlist.sh
> > @@ -76,23 +76,6 @@ print_command_list () {
> >  	echo "};"
> >  }
> >  
> > -print_config_list () {
> > -	cat <<EOF
> > -static const char *config_name_list[] = {
> > -EOF
> > -	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
> > -	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
> > -	sort |
> > -	while read line
> > -	do
> > -		echo "	\"$line\","
> > -	done
> > -	cat <<EOF
> > -	NULL,
> > -};
> > -EOF
> > -}
> > -
> >  exclude_programs=
> >  while test "--exclude-program" = "$1"
> >  do
> > @@ -113,5 +96,3 @@ echo
> >  define_category_names "$1"
> >  echo
> >  print_command_list "$1"
> > -echo
> > -print_config_list
> > diff --git a/generate-configlist.sh b/generate-configlist.sh
> > new file mode 100755
> > index 0000000000..eca6a00c30
> > --- /dev/null
> > +++ b/generate-configlist.sh
> > @@ -0,0 +1,24 @@
> > +#!/bin/sh
> > +
> > +echo "/* Automatically generated by generate-configlist.sh */"
> > +echo
> > +
> > +print_config_list () {
> > +	cat <<EOF
> > +static const char *config_name_list[] = {
> > +EOF
> > +	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
> > +	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
> > +	sort |
> > +	while read line
> > +	do
> > +		echo "	\"$line\","
> > +	done
> 
> This while-read-echo was moved from generate-cmdlist.sh,
> which has some logic to work with read-variable.
> 
> We're moving it out, I think apply this diff on top of it will make the code easier to read.
> 
> diff --git a/generate-configlist.sh b/generate-configlist.sh
> index eca6a00c30..163dbf30bb 100755
> --- a/generate-configlist.sh
> +++ b/generate-configlist.sh
> @@ -10,10 +10,7 @@ EOF
>  	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
>  	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
>  	sort |
> -	while read line
> -	do
> -		echo "	\"$line\","
> -	done
> +	sed 's/^/	"/; s/$/",/'
>  	cat <<EOF
>  	NULL,
>  };

Thanks for the suggestion. You're right that I didn't look into the contents of this script much, because I was just moving it; I modified your sed expression slightly so it performs just one operation:

  sed 's/^.*$/    "\0",/'
 - Emily
Eric Sunshine· Feb 13, 2020, 23:07 UTC · re: Emily Shaffer · lore

Re: [PATCH v6 01/15] help: move list_config_help to builtin/help

On Thu, Feb 13, 2020 at 5:58 PM Emily Shaffer <emilyshaffer@google.com> wrote:
Show 12 quoted lines
> On Thu, Feb 06, 2020 at 08:35:33AM +0700, Danh Doan wrote:
> > -     while read line
> > -     do
> > -             echo "  \"$line\","
> > -     done
> > +     sed 's/^/       "/; s/$/",/'
>
> Thanks for the suggestion. You're right that I didn't look into the
> contents of this script much, because I was just moving it; I modified
> your sed expression slightly so it performs just one operation:
>
>   sed 's/^.*$/    "\0",/'

This isn't portable and won't work with 'sed' from BSD lineage, including Mac OS. More portable would be:

    sed 's/^\(.*\)$/    "\1",/'
Junio C Hamano· Feb 13, 2020, 23:24 UTC · re: Eric Sunshine · lore

Re: [PATCH v6 01/15] help: move list_config_help to builtin/help

Eric Sunshine <sunshine@sunshineco.com> writes:
Show 18 quoted lines
> On Thu, Feb 13, 2020 at 5:58 PM Emily Shaffer <emilyshaffer@google.com> wrote:
>> On Thu, Feb 06, 2020 at 08:35:33AM +0700, Danh Doan wrote:
>> > -     while read line
>> > -     do
>> > -             echo "  \"$line\","
>> > -     done
>> > +     sed 's/^/       "/; s/$/",/'
>>
>> Thanks for the suggestion. You're right that I didn't look into the
>> contents of this script much, because I was just moving it; I modified
>> your sed expression slightly so it performs just one operation:
>>
>>   sed 's/^.*$/    "\0",/'
>
> This isn't portable and won't work with 'sed' from BSD lineage,
> including Mac OS. More portable would be:
>
>     sed 's/^\(.*\)$/    "\1",/'
Or just replace Emily's \0 with &; that should be in POSIX.
Eric Sunshine· Feb 13, 2020, 23:29 UTC · re: Junio C Hamano · lore

Re: [PATCH v6 01/15] help: move list_config_help to builtin/help

On Thu, Feb 13, 2020 at 6:24 PM Junio C Hamano <gitster@pobox.com> wrote:
Show 14 quoted lines
> Eric Sunshine <sunshine@sunshineco.com> writes:
> > On Thu, Feb 13, 2020 at 5:58 PM Emily Shaffer <emilyshaffer@google.com> wrote:
> >> Thanks for the suggestion. You're right that I didn't look into the
> >> contents of this script much, because I was just moving it; I modified
> >> your sed expression slightly so it performs just one operation:
> >>
> >>   sed 's/^.*$/    "\0",/'
> >
> > This isn't portable and won't work with 'sed' from BSD lineage,
> > including Mac OS. More portable would be:
> >
> >     sed 's/^\(.*\)$/    "\1",/'
>
> Or just replace Emily's \0 with &; that should be in POSIX.
Yes, that's nicer.
Emily Shaffer· Feb 14, 2020, 01:20 UTC · re: Eric Sunshine · lore

Re: [PATCH v6 01/15] help: move list_config_help to builtin/help

On Thu, Feb 13, 2020 at 06:29:39PM -0500, Eric Sunshine wrote:
Show 17 quoted lines
> On Thu, Feb 13, 2020 at 6:24 PM Junio C Hamano <gitster@pobox.com> wrote:
> > Eric Sunshine <sunshine@sunshineco.com> writes:
> > > On Thu, Feb 13, 2020 at 5:58 PM Emily Shaffer <emilyshaffer@google.com> wrote:
> > >> Thanks for the suggestion. You're right that I didn't look into the
> > >> contents of this script much, because I was just moving it; I modified
> > >> your sed expression slightly so it performs just one operation:
> > >>
> > >>   sed 's/^.*$/    "\0",/'
> > >
> > > This isn't portable and won't work with 'sed' from BSD lineage,
> > > including Mac OS. More portable would be:
> > >
> > >     sed 's/^\(.*\)$/    "\1",/'
> >
> > Or just replace Emily's \0 with &; that should be in POSIX.
> 
> Yes, that's nicer.
Sure, thanks.
Emily Shaffer· Feb 6, 2020, 00:40 UTC · re: Emily Shaffer · lore

[PATCH v6 02/15] help: add shell-path to --build-options

It may be useful to know which shell Git was built to try to point to, in the event that shell-based Git commands are failing. $SHELL_PATH is set during the build and used to launch the manpage viewer, as well as by git-compat-util.h, and it's used during tests. 'git version --build-options' is encouraged for use in bug reports, so it makes sense to include this information there.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 help.c | 1 +
 1 file changed, 1 insertion(+)
Show changes to help.c +1 −0
diff --git a/help.c b/help.c
index a21487db77..190722fb0a 100644
--- a/help.c
+++ b/help.c
@@ -654,6 +654,7 @@ int cmd_version(int argc, const char **argv, const char *prefix)
 			printf("no commit associated with this build\n");
 		printf("sizeof-long: %d\n", (int)sizeof(long));
 		printf("sizeof-size_t: %d\n", (int)sizeof(size_t));
+		printf("shell-path: %s\n", SHELL_PATH);
 		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
 	}
 	return 0;
-- 
2.25.0.341.g760bfbb309-goog
Emily Shaffer· Feb 6, 2020, 00:40 UTC · re: Emily Shaffer · lore

[PATCH v6 03/15] bugreport: add tool to generate debugging info

Teach Git how to prompt the user for a good bug report: reproduction steps, expected behavior, and actual behavior. Later, Git can learn how to collect some diagnostic information from the repository.

If users can send us a well-written bug report which contains diagnostic information we would otherwise need to ask the user for, we can reduce the number of question-and-answer round trips between the reporter and the Git contributor.

Users may also wish to send a report like this to their local "Git expert" if they have put their repository into a state they are confused by.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 .gitignore                      |  1 +
 Documentation/git-bugreport.txt | 41 ++++++++++++++++
 Makefile                        |  5 ++
 bugreport.c                     | 85 +++++++++++++++++++++++++++++++++
 t/t0091-bugreport.sh            | 49 +++++++++++++++++++
 5 files changed, 181 insertions(+)
 create mode 100644 Documentation/git-bugreport.txt
 create mode 100644 bugreport.c
 create mode 100755 t/t0091-bugreport.sh
Show changes to 5 files +181 −0

.gitignore, Documentation/git-bugreport.txt, Makefile, bugreport.c, t/t0091-bugreport.sh

diff --git a/.gitignore b/.gitignore
index ea97de83f3..d89bf9e11e 100644
--- a/.gitignore
+++ b/.gitignore
@@ -25,6 +25,7 @@
 /git-bisect--helper
 /git-blame
 /git-branch
+/git-bugreport
 /git-bundle
 /git-cat-file
 /git-check-attr
diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
new file mode 100644
index 0000000000..52d49ed7aa
--- /dev/null
+++ b/Documentation/git-bugreport.txt
@@ -0,0 +1,41 @@
+git-bugreport(1)
+================
+
+NAME
+----
+git-bugreport - Collect information for user to file a bug report
+
+SYNOPSIS
+--------
+[verse]
+'git bugreport' [(-o | --output-directory) <path>] [(-s | --suffix) <format>]
+
+DESCRIPTION
+-----------
+Captures information about the user's machine, Git client, and repository state,
+as well as a form requesting information about the behavior the user observed,
+into a single text file which the user can then share, for example to the Git
+mailing list, in order to report an observed bug.
+
+The following information is requested from the user:
+
+ - Reproduction steps
+ - Expected behavior
+ - Actual behavior
+
+OPTIONS
+-------
+-o <path>::
+--output-directory <path>::
+	Place the resulting bug report file in `<path>` instead of the root of
+	the Git repository.
+
+-s <format>::
+--suffix <format>::
+	Specify an alternate suffix for the bugreport name, to create a file
+	named 'git-bugreport-<formatted suffix>'. This should take the form of a
+	link:strftime[3] format string; the current local time will be used.
+
+GIT
+---
+Part of the linkgit:git[1] suite
diff --git a/Makefile b/Makefile
index 5a022367d4..a01a050aa3 100644
--- a/Makefile
+++ b/Makefile
@@ -681,6 +681,7 @@ EXTRA_PROGRAMS =
 # ... and all the rest that could be moved out of bindir to gitexecdir
 PROGRAMS += $(EXTRA_PROGRAMS)
 
+PROGRAM_OBJS += bugreport.o
 PROGRAM_OBJS += credential-store.o
 PROGRAM_OBJS += daemon.o
 PROGRAM_OBJS += fast-import.o
@@ -2457,6 +2458,10 @@ endif
 git-%$X: %.o GIT-LDFLAGS $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) $(LIBS)
 
+git-bugreport$X: bugreport.o GIT-LDFLAGS $(GITLIBS)
+	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
+		$(LIBS)
+
 git-imap-send$X: imap-send.o $(IMAP_SEND_BUILDDEPS) GIT-LDFLAGS $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
 		$(IMAP_SEND_LDFLAGS) $(LIBS)
diff --git a/bugreport.c b/bugreport.c
new file mode 100644
index 0000000000..db46fb88be
--- /dev/null
+++ b/bugreport.c
@@ -0,0 +1,85 @@
+#include "builtin.h"
+#include "parse-options.h"
+#include "stdio.h"
+#include "strbuf.h"
+#include "time.h"
+
+static const char * const bugreport_usage[] = {
+	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
+	NULL
+};
+
+static int get_bug_template(struct strbuf *template)
+{
+	const char template_text[] = N_(
+"Thank you for filling out a Git bug report!\n"
+"Please answer the following questions to help us understand your issue.\n"
+"\n"
+"What did you do before the bug happened? (Steps to reproduce your issue)\n"
+"\n"
+"What did you expect to happen? (Expected behavior)\n"
+"\n"
+"What happened instead? (Actual behavior)\n"
+"\n"
+"What's different between what you expected and what actually happened?\n"
+"\n"
+"Anything else you want to add:\n"
+"\n"
+"Please review the rest of the bug report below.\n"
+"You can delete any lines you don't wish to share.\n");
+
+	strbuf_addstr(template, template_text);
+	return 0;
+}
+
+int cmd_main(int argc, const char **argv)
+{
+	struct strbuf buffer = STRBUF_INIT;
+	struct strbuf report_path = STRBUF_INIT;
+	FILE *report;
+	time_t now = time(NULL);
+	char *option_output = NULL;
+	char *option_suffix = "%F-%H%M";
+	struct stat statbuf;
+
+	const struct option bugreport_options[] = {
+		OPT_STRING('o', "output-directory", &option_output, N_("path"),
+			   N_("specify a destination for the bugreport file")),
+		OPT_STRING('s', "suffix", &option_suffix, N_("format"),
+			   N_("specify a strftime format suffix for the filename")),
+		OPT_END()
+	};
+	argc = parse_options(argc, argv, "", bugreport_options,
+			     bugreport_usage, 0);
+
+	if (option_output) {
+		strbuf_addstr(&report_path, option_output);
+		strbuf_complete(&report_path, '/');
+	}
+
+
+	strbuf_addstr(&report_path, "git-bugreport-");
+	strbuf_addftime(&report_path, option_suffix, localtime(&now), 0, 0);
+	strbuf_addstr(&report_path, ".txt");
+
+	if (!stat(report_path.buf, &statbuf))
+		die("'%s' already exists", report_path.buf);
+
+	get_bug_template(&buffer);
+
+	report = fopen_for_writing(report_path.buf);
+
+	if (report == NULL) {
+		strbuf_release(&report_path);
+		die("couldn't open '%s' for writing", report_path.buf);
+	}
+
+	strbuf_write(&buffer, report);
+	fclose(report);
+
+	fprintf(stderr, _("Created new report at '%s'.\n"), report_path.buf);
+
+	UNLEAK(buffer);
+	UNLEAK(report_path);
+	return -launch_editor(report_path.buf, NULL, NULL);
+}
diff --git a/t/t0091-bugreport.sh b/t/t0091-bugreport.sh
new file mode 100755
index 0000000000..451badff0c
--- /dev/null
+++ b/t/t0091-bugreport.sh
@@ -0,0 +1,49 @@
+#!/bin/sh
+
+test_description='git bugreport'
+
+. ./test-lib.sh
+
+# Headers "[System Info]" will be followed by a non-empty line if we put some
+# information there; we can make sure all our headers were followed by some
+# information to check if the command was successful.
+HEADER_PATTERN="^\[.*\]$"
+check_all_headers_populated() {
+	while read -r line; do
+		if test "$(grep "$HEADER_PATTERN" "$line")"; then
+			echo "$line"
+			read -r nextline
+			if test -z "$nextline"; then
+				return 1;
+			fi
+		fi
+	done
+}
+
+test_expect_success 'creates a report with content in the right places' '
+	git bugreport &&
+	REPORT="$(ls git-bugreport-*)" &&
+	check_all_headers_populated <$REPORT &&
+	rm $REPORT
+'
+
+test_expect_success 'dies if file with same name as report already exists' '
+	touch git-bugreport-duplicate.txt &&
+	test_must_fail git bugreport --suffix duplicate &&
+	rm git-bugreport-duplicate.txt
+'
+
+test_expect_success '--output-directory puts the report in the provided dir' '
+	mkdir foo/ &&
+	git bugreport -o foo/ &&
+	test_path_is_file foo/git-bugreport-* &&
+	rm -fr foo/
+'
+
+test_expect_success 'incorrect arguments abort with usage' '
+	test_must_fail git bugreport --false 2>output &&
+	grep usage output &&
+	test_path_is_missing git-bugreport-*
+'
+
+test_done
-- 
2.25.0.341.g760bfbb309-goog
SZEDER Gábor· Feb 7, 2020, 14:18 UTC · re: Emily Shaffer · lore

Re: [PATCH v6 03/15] bugreport: add tool to generate debugging info

On Wed, Feb 05, 2020 at 04:40:56PM -0800, Emily Shaffer wrote:
Show 31 quoted lines
> diff --git a/t/t0091-bugreport.sh b/t/t0091-bugreport.sh
> new file mode 100755
> index 0000000000..451badff0c
> --- /dev/null
> +++ b/t/t0091-bugreport.sh
> @@ -0,0 +1,49 @@
> +#!/bin/sh
> +
> +test_description='git bugreport'
> +
> +. ./test-lib.sh
> +
> +# Headers "[System Info]" will be followed by a non-empty line if we put some
> +# information there; we can make sure all our headers were followed by some
> +# information to check if the command was successful.
> +HEADER_PATTERN="^\[.*\]$"
> +check_all_headers_populated() {
> +	while read -r line; do
> +		if test "$(grep "$HEADER_PATTERN" "$line")"; then
> +			echo "$line"
> +			read -r nextline
> +			if test -z "$nextline"; then
> +				return 1;
> +			fi
> +		fi
> +	done
> +}
> +
> +test_expect_success 'creates a report with content in the right places' '
> +	git bugreport &&
> +	REPORT="$(ls git-bugreport-*)" &&
What if the globbing were to match more than one file?
> +	check_all_headers_populated <$REPORT &&
> +	rm $REPORT
Please register a cleanup commands like this with 'test_when_finished'.
Show 10 quoted lines
> +'
> +
> +test_expect_success 'dies if file with same name as report already exists' '
> +	touch git-bugreport-duplicate.txt &&
> +	test_must_fail git bugreport --suffix duplicate &&
> +	rm git-bugreport-duplicate.txt
> +'
> +
> +test_expect_success '--output-directory puts the report in the provided dir' '
> +	mkdir foo/ &&

Is it really necessary to create the directory in advance? Or to put it in another way: shouldn't 'git bugreport' create any missing leading directories of the path given for its '-o' option? FWIW, 'git format-patch -o dir ...' does create all necessary directories.

> +	git bugreport -o foo/ &&
> +	test_path_is_file foo/git-bugreport-* &&
What if the globbing were to match more than one file? :)
Show 6 quoted lines
> +	rm -fr foo/
> +'
> +
> +test_expect_success 'incorrect arguments abort with usage' '
> +	test_must_fail git bugreport --false 2>output &&
> +	grep usage output &&

This breaks the GETTEXT_POISON CI job, because "usage" is translated, so the test should not look for it with plain 'grep'. Please use 'test_i18ngrep' instead.

Show 7 quoted lines
> +	test_path_is_missing git-bugreport-*
> +'
> +
> +test_done
> -- 
> 2.25.0.341.g760bfbb309-goog
> 
Junio C Hamano· Feb 7, 2020, 18:51 UTC · re: SZEDER Gábor · lore

Re: [PATCH v6 03/15] bugreport: add tool to generate debugging info

SZEDER Gábor <szeder.dev@gmail.com> writes:
Show 34 quoted lines
> On Wed, Feb 05, 2020 at 04:40:56PM -0800, Emily Shaffer wrote:
>> diff --git a/t/t0091-bugreport.sh b/t/t0091-bugreport.sh
>> new file mode 100755
>> index 0000000000..451badff0c
>> --- /dev/null
>> +++ b/t/t0091-bugreport.sh
>> @@ -0,0 +1,49 @@
>> +#!/bin/sh
>> +
>> +test_description='git bugreport'
>> +
>> +. ./test-lib.sh
>> +
>> +# Headers "[System Info]" will be followed by a non-empty line if we put some
>> +# information there; we can make sure all our headers were followed by some
>> +# information to check if the command was successful.
>> +HEADER_PATTERN="^\[.*\]$"
>> +check_all_headers_populated() {
>> +	while read -r line; do
>> +		if test "$(grep "$HEADER_PATTERN" "$line")"; then
>> +			echo "$line"
>> +			read -r nextline
>> +			if test -z "$nextline"; then
>> +				return 1;
>> +			fi
>> +		fi
>> +	done
>> +}
>> +
>> +test_expect_success 'creates a report with content in the right places' '
>> +	git bugreport &&
>> +	REPORT="$(ls git-bugreport-*)" &&
>
> What if the globbing were to match more than one file?

An often-useful pattern is to make the command report the output filename, i.e.

	REPORT=$(git butreport) &&

if the design insists that "git bugreport" should allocate a filename in order to make it easy to guarantee uniqueness.

Of course, we can make the invoker supply filename, e.g.
	REPORT=$(generate-output-filename) &&
	git bugreport -o "$REPORT" ;# or git bugreport >"$REPORT"
Emily Shaffer· Feb 11, 2020, 22:40 UTC · re: Junio C Hamano · lore

Re: [PATCH v6 03/15] bugreport: add tool to generate debugging info

On Fri, Feb 07, 2020 at 10:51:29AM -0800, Junio C Hamano wrote:
Show 49 quoted lines
> SZEDER Gábor <szeder.dev@gmail.com> writes:
> 
> > On Wed, Feb 05, 2020 at 04:40:56PM -0800, Emily Shaffer wrote:
> >> diff --git a/t/t0091-bugreport.sh b/t/t0091-bugreport.sh
> >> new file mode 100755
> >> index 0000000000..451badff0c
> >> --- /dev/null
> >> +++ b/t/t0091-bugreport.sh
> >> @@ -0,0 +1,49 @@
> >> +#!/bin/sh
> >> +
> >> +test_description='git bugreport'
> >> +
> >> +. ./test-lib.sh
> >> +
> >> +# Headers "[System Info]" will be followed by a non-empty line if we put some
> >> +# information there; we can make sure all our headers were followed by some
> >> +# information to check if the command was successful.
> >> +HEADER_PATTERN="^\[.*\]$"
> >> +check_all_headers_populated() {
> >> +	while read -r line; do
> >> +		if test "$(grep "$HEADER_PATTERN" "$line")"; then
> >> +			echo "$line"
> >> +			read -r nextline
> >> +			if test -z "$nextline"; then
> >> +				return 1;
> >> +			fi
> >> +		fi
> >> +	done
> >> +}
> >> +
> >> +test_expect_success 'creates a report with content in the right places' '
> >> +	git bugreport &&
> >> +	REPORT="$(ls git-bugreport-*)" &&
> >
> > What if the globbing were to match more than one file?
> 
> An often-useful pattern is to make the command report the output
> filename, i.e.
> 
> 	REPORT=$(git butreport) &&
> 
> if the design insists that "git bugreport" should allocate a
> filename in order to make it easy to guarantee uniqueness.
> 
> Of course, we can make the invoker supply filename, e.g.
> 
> 	REPORT=$(generate-output-filename) &&
> 	git bugreport -o "$REPORT" ;# or git bugreport >"$REPORT"

Yeah, this is an option and I've changed the offending test - which captures a variable based on globbing - to use --suffix and force a filename.

SZEDER Gábor· Feb 7, 2020, 14:54 UTC · re: Emily Shaffer · lore

Re: [PATCH v6 03/15] bugreport: add tool to generate debugging info

On Wed, Feb 05, 2020 at 04:40:56PM -0800, Emily Shaffer wrote:
Show 24 quoted lines
> Teach Git how to prompt the user for a good bug report: reproduction
> steps, expected behavior, and actual behavior. Later, Git can learn how
> to collect some diagnostic information from the repository.
> 
> If users can send us a well-written bug report which contains diagnostic
> information we would otherwise need to ask the user for, we can reduce
> the number of question-and-answer round trips between the reporter and
> the Git contributor.
> 
> Users may also wish to send a report like this to their local "Git
> expert" if they have put their repository into a state they are confused
> by.
> 
> Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
> ---
>  .gitignore                      |  1 +
>  Documentation/git-bugreport.txt | 41 ++++++++++++++++
>  Makefile                        |  5 ++
>  bugreport.c                     | 85 +++++++++++++++++++++++++++++++++
>  t/t0091-bugreport.sh            | 49 +++++++++++++++++++
>  5 files changed, 181 insertions(+)
>  create mode 100644 Documentation/git-bugreport.txt
>  create mode 100644 bugreport.c
>  create mode 100755 t/t0091-bugreport.sh

Please add a corresponding entry to 'command-list.txt' as well. According to the comment at the beginning of that file, all, even external commands should be included there.

Junio C Hamano· Feb 12, 2020, 18:06 UTC · re: Emily Shaffer · lore

Re: [PATCH v6 03/15] bugreport: add tool to generate debugging info

As this topic seems to break under GIT_TEST_GETTEXT_POISON=yes, I'll apply the following band-aid on top before merging it to 'pu'.

Most of them are style fixes, but quoting $REPORT, not just makes the redirection honor the coding guidelines, will ensure that an error is caught if git-bugreport-* glob matches no paths or more than on paths.

 t/t0091-bugreport.sh | 17 ++++++++++-------
 1 file changed, 10 insertions(+), 7 deletions(-)
Show changes to t/t0091-bugreport.sh +10 −7
diff --git a/t/t0091-bugreport.sh b/t/t0091-bugreport.sh
index 451badff0c..f2186941ce 100755
--- a/t/t0091-bugreport.sh
+++ b/t/t0091-bugreport.sh
@@ -8,9 +8,12 @@ test_description='git bugreport'
 # information there; we can make sure all our headers were followed by some
 # information to check if the command was successful.
 HEADER_PATTERN="^\[.*\]$"
-check_all_headers_populated() {
-	while read -r line; do
-		if test "$(grep "$HEADER_PATTERN" "$line")"; then
+
+check_all_headers_populated () {
+	while read -r line
+	do
+		if test "$(grep "$HEADER_PATTERN" "$line")"
+		then
 			echo "$line"
 			read -r nextline
 			if test -z "$nextline"; then
@@ -23,12 +26,12 @@ check_all_headers_populated() {
 test_expect_success 'creates a report with content in the right places' '
 	git bugreport &&
 	REPORT="$(ls git-bugreport-*)" &&
-	check_all_headers_populated <$REPORT &&
-	rm $REPORT
+	check_all_headers_populated <"$REPORT" &&
+	rm "$REPORT"
 '
 
 test_expect_success 'dies if file with same name as report already exists' '
-	touch git-bugreport-duplicate.txt &&
+	>>git-bugreport-duplicate.txt &&
 	test_must_fail git bugreport --suffix duplicate &&
 	rm git-bugreport-duplicate.txt
 '
@@ -42,7 +45,7 @@ test_expect_success '--output-directory puts the report in the provided dir' '
 
 test_expect_success 'incorrect arguments abort with usage' '
 	test_must_fail git bugreport --false 2>output &&
-	grep usage output &&
+	test_i18ngrep usage output &&
 	test_path_is_missing git-bugreport-*
 '
 
-- 
2.25.0-455-gcf3c3a5ab4
Emily Shaffer· Feb 12, 2020, 22:36 UTC · re: Junio C Hamano · lore

Re: [PATCH v6 03/15] bugreport: add tool to generate debugging info

On Wed, Feb 12, 2020 at 10:06:44AM -0800, Junio C Hamano wrote:
Show 7 quoted lines
> As this topic seems to break under GIT_TEST_GETTEXT_POISON=yes, I'll
> apply the following band-aid on top before merging it to 'pu'.  
> 
> Most of them are style fixes, but quoting $REPORT, not just makes
> the redirection honor the coding guidelines, will ensure that an
> error is caught if git-bugreport-* glob matches no paths or more
> than on paths.

Thanks. Most of these I have locally, but not all. I'll apply this as a fixup on patch 3/n (the one that introduces the tests) and add a Helped-by line.

I'm hoping to get a reroll out today; it was the first thing on my list this week, and then a high-priority internal issue shows up (so it goes). Sorry for the inconvenience with the CI failures.

 - Emily
Show 51 quoted lines
>  t/t0091-bugreport.sh | 17 ++++++++++-------
>  1 file changed, 10 insertions(+), 7 deletions(-)
> 
> diff --git a/t/t0091-bugreport.sh b/t/t0091-bugreport.sh
> index 451badff0c..f2186941ce 100755
> --- a/t/t0091-bugreport.sh
> +++ b/t/t0091-bugreport.sh
> @@ -8,9 +8,12 @@ test_description='git bugreport'
>  # information there; we can make sure all our headers were followed by some
>  # information to check if the command was successful.
>  HEADER_PATTERN="^\[.*\]$"
> -check_all_headers_populated() {
> -	while read -r line; do
> -		if test "$(grep "$HEADER_PATTERN" "$line")"; then
> +
> +check_all_headers_populated () {
> +	while read -r line
> +	do
> +		if test "$(grep "$HEADER_PATTERN" "$line")"
> +		then
>  			echo "$line"
>  			read -r nextline
>  			if test -z "$nextline"; then
> @@ -23,12 +26,12 @@ check_all_headers_populated() {
>  test_expect_success 'creates a report with content in the right places' '
>  	git bugreport &&
>  	REPORT="$(ls git-bugreport-*)" &&
> -	check_all_headers_populated <$REPORT &&
> -	rm $REPORT
> +	check_all_headers_populated <"$REPORT" &&
> +	rm "$REPORT"
>  '
>  
>  test_expect_success 'dies if file with same name as report already exists' '
> -	touch git-bugreport-duplicate.txt &&
> +	>>git-bugreport-duplicate.txt &&
>  	test_must_fail git bugreport --suffix duplicate &&
>  	rm git-bugreport-duplicate.txt
>  '
> @@ -42,7 +45,7 @@ test_expect_success '--output-directory puts the report in the provided dir' '
>  
>  test_expect_success 'incorrect arguments abort with usage' '
>  	test_must_fail git bugreport --false 2>output &&
> -	grep usage output &&
> +	test_i18ngrep usage output &&
>  	test_path_is_missing git-bugreport-*
>  '
>  
> -- 
> 2.25.0-455-gcf3c3a5ab4
> 
Emily Shaffer· Feb 6, 2020, 00:40 UTC · re: Emily Shaffer · lore

[PATCH v6 04/15] bugreport: gather git version and build info

Knowing which version of Git a user has and how it was built allows us to more precisely pin down the circumstances when a certain issue occurs, so teach bugreport how to tell us the same output as 'git version --build-options'.

It's not ideal to directly call 'git version --build-options' because that output goes to stdout. Instead, wrap the version string in a helper within help.[ch] library, and call that helper from within the bugreport library.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  4 +++
 bugreport.c                     | 19 ++++++++++++-
 help.c                          | 48 ++++++++++++++++++++-------------
 help.h                          |  1 +
 4 files changed, 52 insertions(+), 20 deletions(-)
Show changes to 4 files +52 −20

Documentation/git-bugreport.txt, bugreport.c, help.c, help.h

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index 52d49ed7aa..6ce2ae2eff 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -23,6 +23,10 @@ The following information is requested from the user:
  - Expected behavior
  - Actual behavior
 
+The following information is captured automatically:
+
+ - 'git version --build-options'
+
 OPTIONS
 -------
 -o <path>::
diff --git a/bugreport.c b/bugreport.c
index db46fb88be..dd21be9d68 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -1,8 +1,17 @@
-#include "builtin.h"
+#include "cache.h"
 #include "parse-options.h"
 #include "stdio.h"
 #include "strbuf.h"
 #include "time.h"
+#include "help.h"
+
+static void get_system_info(struct strbuf *sys_info)
+{
+	/* get git version from native cmd */
+	strbuf_addstr(sys_info, "git version:\n");
+	get_version_info(sys_info, 1);
+	strbuf_complete_line(sys_info);
+}
 
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
@@ -32,6 +41,11 @@ static int get_bug_template(struct strbuf *template)
 	return 0;
 }
 
+static void get_header(struct strbuf *buf, const char *title)
+{
+	strbuf_addf(buf, "\n\n[%s]\n", title);
+}
+
 int cmd_main(int argc, const char **argv)
 {
 	struct strbuf buffer = STRBUF_INIT;
@@ -67,6 +81,9 @@ int cmd_main(int argc, const char **argv)
 
 	get_bug_template(&buffer);
 
+	get_header(&buffer, "System Info");
+	get_system_info(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 
 	if (report == NULL) {
diff --git a/help.c b/help.c
index 190722fb0a..44cee69c11 100644
--- a/help.c
+++ b/help.c
@@ -622,8 +622,33 @@ const char *help_unknown_cmd(const char *cmd)
 	exit(1);
 }
 
+void get_version_info(struct strbuf *buf, int show_build_options)
+{
+	/*
+	 * The format of this string should be kept stable for compatibility
+	 * with external projects that rely on the output of "git version".
+	 *
+	 * Always show the version, even if other options are given.
+	 */
+	strbuf_addf(buf, "git version %s\n", git_version_string);
+
+	if (show_build_options) {
+		strbuf_addf(buf, "cpu: %s\n", GIT_HOST_CPU);
+		if (git_built_from_commit_string[0])
+			strbuf_addf(buf, "built from commit: %s\n",
+			       git_built_from_commit_string);
+		else
+			strbuf_addstr(buf, "no commit associated with this build\n");
+		strbuf_addf(buf, "sizeof-long: %d\n", (int)sizeof(long));
+		strbuf_addf(buf, "sizeof-size_t: %d\n", (int)sizeof(size_t));
+		strbuf_addf(buf, "shell-path: %s\n", SHELL_PATH);
+		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
+	}
+}
+
 int cmd_version(int argc, const char **argv, const char *prefix)
 {
+	struct strbuf buf = STRBUF_INIT;
 	int build_options = 0;
 	const char * const usage[] = {
 		N_("git version [<options>]"),
@@ -637,26 +662,11 @@ int cmd_version(int argc, const char **argv, const char *prefix)
 
 	argc = parse_options(argc, argv, prefix, options, usage, 0);
 
-	/*
-	 * The format of this string should be kept stable for compatibility
-	 * with external projects that rely on the output of "git version".
-	 *
-	 * Always show the version, even if other options are given.
-	 */
-	printf("git version %s\n", git_version_string);
+	get_version_info(&buf, build_options);
+	printf("%s", buf.buf);
+
+	strbuf_release(&buf);
 
-	if (build_options) {
-		printf("cpu: %s\n", GIT_HOST_CPU);
-		if (git_built_from_commit_string[0])
-			printf("built from commit: %s\n",
-			       git_built_from_commit_string);
-		else
-			printf("no commit associated with this build\n");
-		printf("sizeof-long: %d\n", (int)sizeof(long));
-		printf("sizeof-size_t: %d\n", (int)sizeof(size_t));
-		printf("shell-path: %s\n", SHELL_PATH);
-		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
-	}
 	return 0;
 }
 
diff --git a/help.h b/help.h
index 9071894e8c..500521b908 100644
--- a/help.h
+++ b/help.h
@@ -37,6 +37,7 @@ void add_cmdname(struct cmdnames *cmds, const char *name, int len);
 void exclude_cmds(struct cmdnames *cmds, struct cmdnames *excludes);
 int is_in_cmdlist(struct cmdnames *cmds, const char *name);
 void list_commands(unsigned int colopts, struct cmdnames *main_cmds, struct cmdnames *other_cmds);
+void get_version_info(struct strbuf *buf, int show_build_options);
 
 /*
  * call this to die(), when it is suspected that the user mistyped a
-- 
2.25.0.341.g760bfbb309-goog
Emily Shaffer· Feb 6, 2020, 00:40 UTC · re: Emily Shaffer · lore

[PATCH v6 05/15] bugreport: add uname info

The contents of uname() can give us some insight into what sort of system the user is running on, and help us replicate their setup if need be. The domainname field is not guaranteed to be available, so don't collect it.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 13 +++++++++++++
 2 files changed, 14 insertions(+)
Show changes to 2 files +14 −0

Documentation/git-bugreport.txt, bugreport.c

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index 6ce2ae2eff..4dd72c60f5 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -26,6 +26,7 @@ The following information is requested from the user:
 The following information is captured automatically:
 
  - 'git version --build-options'
+ - uname sysname, release, version, and machine strings
 
 OPTIONS
 -------
diff --git a/bugreport.c b/bugreport.c
index dd21be9d68..4c4b877ea1 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -7,10 +7,23 @@
 
 static void get_system_info(struct strbuf *sys_info)
 {
+	struct utsname uname_info;
+
 	/* get git version from native cmd */
 	strbuf_addstr(sys_info, "git version:\n");
 	get_version_info(sys_info, 1);
 	strbuf_complete_line(sys_info);
+
+	/* system call for other version info */
+	strbuf_addstr(sys_info, "uname -a: ");
+	if (uname(&uname_info))
+		strbuf_addf(sys_info, "uname() failed with code %d\n", errno);
+	else
+		strbuf_addf(sys_info, "%s %s %s %s\n",
+			    uname_info.sysname,
+			    uname_info.release,
+			    uname_info.version,
+			    uname_info.machine);
 }
 
 static const char * const bugreport_usage[] = {
-- 
2.25.0.341.g760bfbb309-goog
Emily Shaffer· Feb 6, 2020, 00:40 UTC · re: Emily Shaffer · lore

[PATCH v6 06/15] bugreport: add compiler info

To help pinpoint the source of a regression, it is useful to know some info about the compiler which the user's Git client was built with. By adding a generic get_compiler_info() in 'compat/' we can choose which relevant information to share per compiler; to get started, let's demonstrate the version of glibc if the user built with 'gcc'.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     |  5 +++++
 compat/compiler.h               | 24 ++++++++++++++++++++++++
 3 files changed, 30 insertions(+)
 create mode 100644 compat/compiler.h
Show changes to 3 files +30 −0

Documentation/git-bugreport.txt, bugreport.c, compat/compiler.h

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index 4dd72c60f5..8bbc4c960c 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -27,6 +27,7 @@ The following information is captured automatically:
 
  - 'git version --build-options'
  - uname sysname, release, version, and machine strings
+ - Compiler-specific info string
 
 OPTIONS
 -------
diff --git a/bugreport.c b/bugreport.c
index 4c4b877ea1..aaeb94c10c 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -4,6 +4,7 @@
 #include "strbuf.h"
 #include "time.h"
 #include "help.h"
+#include "compat/compiler.h"
 
 static void get_system_info(struct strbuf *sys_info)
 {
@@ -24,6 +25,10 @@ static void get_system_info(struct strbuf *sys_info)
 			    uname_info.release,
 			    uname_info.version,
 			    uname_info.machine);
+
+	strbuf_addstr(sys_info, "compiler info: ");
+	get_compiler_info(sys_info);
+	strbuf_complete_line(sys_info);
 }
 
 static const char * const bugreport_usage[] = {
diff --git a/compat/compiler.h b/compat/compiler.h
new file mode 100644
index 0000000000..bda5098e1b
--- /dev/null
+++ b/compat/compiler.h
@@ -0,0 +1,24 @@
+#ifndef COMPILER_H
+#define COMPILER_H
+
+#include "git-compat-util.h"
+#include "strbuf.h"
+
+#ifdef __GLIBC__
+#include <gnu/libc-version.h>
+
+static inline void get_compiler_info(struct strbuf *info)
+{
+	strbuf_addf(info, "glibc: %s", gnu_get_libc_version());
+}
+
+#else
+
+static inline void get_compiler_info(struct strbuf *info)
+{
+	strbuf_addstr(info, "get_compiler_info() not implemented");
+}
+
+#endif
+
+#endif /* COMPILER_H */
-- 
2.25.0.341.g760bfbb309-goog
Emily Shaffer· Feb 6, 2020, 00:41 UTC · re: Emily Shaffer · lore

[PATCH v6 07/15] bugreport: add git-remote-https version

It's possible for git-remote-curl to be built separately from git; in that case we want to know what version of cURL is used by git-remote-curl, not necessarily which version was present at git-bugreport's build time. So instead, ask git-remote-curl for the version information it knows about.

Today, "git-remote-http" and "git-remote-https" are aliased to "git-remote-curl"; but in case we rely on a different library than cURL in the future, let's not explicitly reference cURL from bugreport.

For longevity purposes, invoke the alias "git-remote-https" instead of "git-remote-http".

Since it could have been built at a different time, also report the version and built-from commit of git-remote-curl alongside the cURL info.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 16 ++++++++++++++++
 remote-curl.c                   |  8 ++++++++
 3 files changed, 25 insertions(+)
Show changes to 3 files +25 −0

Documentation/git-bugreport.txt, bugreport.c, remote-curl.c

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index 8bbc4c960c..33df4dec7f 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -28,6 +28,7 @@ The following information is captured automatically:
  - 'git version --build-options'
  - uname sysname, release, version, and machine strings
  - Compiler-specific info string
+ - 'git remote-https --build-info'
 
 OPTIONS
 -------
diff --git a/bugreport.c b/bugreport.c
index aaeb94c10c..da5859c131 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -5,6 +5,18 @@
 #include "time.h"
 #include "help.h"
 #include "compat/compiler.h"
+#include "run-command.h"
+
+static void get_git_remote_https_version_info(struct strbuf *version_info)
+{
+	struct child_process cp = CHILD_PROCESS_INIT;
+
+	argv_array_push(&cp.args, "git");
+	argv_array_push(&cp.args, "remote-https");
+	argv_array_push(&cp.args, "--build-info");
+	if (capture_command(&cp, version_info, 0))
+	    strbuf_addstr(version_info, "'git-remote-https --build-info' not supported\n");
+}
 
 static void get_system_info(struct strbuf *sys_info)
 {
@@ -29,6 +41,10 @@ static void get_system_info(struct strbuf *sys_info)
 	strbuf_addstr(sys_info, "compiler info: ");
 	get_compiler_info(sys_info);
 	strbuf_complete_line(sys_info);
+
+	strbuf_addstr(sys_info, "git-remote-https --build-info:\n");
+	get_git_remote_https_version_info(sys_info);
+	strbuf_complete_line(sys_info);
 }
 
 static const char * const bugreport_usage[] = {
diff --git a/remote-curl.c b/remote-curl.c
index 350d92a074..c590fbfae3 100644
--- a/remote-curl.c
+++ b/remote-curl.c
@@ -17,6 +17,7 @@
 #include "protocol.h"
 #include "quote.h"
 #include "transport.h"
+#include "version.h"
 
 static struct remote *remote;
 /* always ends with a trailing slash */
@@ -1374,6 +1375,13 @@ int cmd_main(int argc, const char **argv)
 	string_list_init(&options.deepen_not, 1);
 	string_list_init(&options.push_options, 1);
 
+	if (!strcmp("--build-info", argv[1])) {
+		printf("git-http-fetch version: %s\n", git_version_string);
+		printf("built from commit: %s\n", git_built_from_commit_string);
+		printf("curl version: %s\n", curl_version());
+		return 0;
+	}
+
 	/*
 	 * Just report "remote-curl" here (folding all the various aliases
 	 * ("git-remote-http", "git-remote-https", and etc.) here since they
-- 
2.25.0.341.g760bfbb309-goog
Emily Shaffer· Feb 6, 2020, 00:41 UTC · re: Emily Shaffer · lore

[PATCH v6 08/15] bugreport: include user interactive shell

It's possible a user may complain about the way that Git interacts with their interactive shell, e.g. autocompletion or shell prompt. In that case, it's useful for us to know which shell they're using interactively.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt | 1 +
 bugreport.c                     | 5 +++++
 2 files changed, 6 insertions(+)
Show changes to 2 files +6 −0

Documentation/git-bugreport.txt, bugreport.c

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index 33df4dec7f..23265b0d74 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -29,6 +29,7 @@ The following information is captured automatically:
  - uname sysname, release, version, and machine strings
  - Compiler-specific info string
  - 'git remote-https --build-info'
+ - $SHELL
 
 OPTIONS
 -------
diff --git a/bugreport.c b/bugreport.c
index da5859c131..9b51250155 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -21,6 +21,7 @@ static void get_git_remote_https_version_info(struct strbuf *version_info)
 static void get_system_info(struct strbuf *sys_info)
 {
 	struct utsname uname_info;
+	char *shell = NULL;
 
 	/* get git version from native cmd */
 	strbuf_addstr(sys_info, "git version:\n");
@@ -42,6 +43,10 @@ static void get_system_info(struct strbuf *sys_info)
 	get_compiler_info(sys_info);
 	strbuf_complete_line(sys_info);
 
+	shell = getenv("SHELL");
+	strbuf_addf(sys_info, "$SHELL (typically, interactive shell): %s\n",
+		    shell ? shell : "<unset>");
+
 	strbuf_addstr(sys_info, "git-remote-https --build-info:\n");
 	get_git_remote_https_version_info(sys_info);
 	strbuf_complete_line(sys_info);
-- 
2.25.0.341.g760bfbb309-goog
Emily Shaffer· Feb 6, 2020, 00:41 UTC · re: Emily Shaffer · lore

[PATCH v6 09/15] bugreport: generate config safelist based on docs

Add a new step to the build to generate a safelist of git-config variables which are appropriate to include in the output of git-bugreport. New variables can be added to the safelist by annotating their documentation in Documentation/config with the "annotate" macro, which is a no-op in AsciiDoc and AsciiDoctor.

Some configs are private in nature, and can contain remote URLs, passwords, or other sensitive information. In the event that a user doesn't notice their information while reviewing a bugreport, that user may leak their credentials to other individuals, mailing lists, or bug tracking tools inadvertently. Heuristic blocklisting of configuration keys is imperfect and prone to false negatives; given the nature of the information which can be leaked, a safelist is more reliable.

However, it's possible that in some situations, an organization may be less concerned with privacy of things like remote URLs and branch names, and more concerned with ease of diagnosis for their support staff. In those cases, it may make more sense for that organization to modify the code to use a blocklist. To that end, we should try to mark configs which are definitely safe, and configs which are definitely unsafe, and leave blank configs which are somewhere in between. To mark a config as safe, add "annotate:bugreport[include]" to the corresponding line in the config documentation; to mark it as unsafe, add "annotate:bugreport[exclude]" instead.

Generating bugreport-config-safelist.h at build time by grepping the documentation for this new macro helps us prevent staleness. The macro itself is a no-op and should not alter the appearance of the documentation in either AsciiDoc or AsciiDoctor, confirmable by running:

  cd Documentation
  ./doc-diff --asciidoctor HEAD^ HEAD
  ./doc-diff --asciidoc HEAD^ HEAD

Diffing the rendered HTML shows that only inline comments were added, which shouldn't be a problem.

Additionally, add annotations to the sendemail config documentation in order to demonstrate a proof of concept.

Helped-by: Martin Ågren <martin.agren@gmail.com>
Helped-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 .gitignore                              |  1 +
 Documentation/asciidoc.conf             |  9 ++++
 Documentation/asciidoctor-extensions.rb |  5 +++
 Documentation/config/sendemail.txt      | 56 ++++++++++++-------------
 Makefile                                |  7 ++++
 generate-bugreport-config-safelist.sh   | 17 ++++++++
 6 files changed, 67 insertions(+), 28 deletions(-)
 create mode 100755 generate-bugreport-config-safelist.sh
Show changes to 6 files +67 −28

.gitignore, Documentation/asciidoc.conf, Documentation/asciidoctor-extensions.rb, Documentation/config/sendemail.txt, Makefile, generate-bugreport-config-safelist.sh

diff --git a/.gitignore b/.gitignore
index d89bf9e11e..bd2f49b996 100644
--- a/.gitignore
+++ b/.gitignore
@@ -192,6 +192,7 @@
 /gitweb/static/gitweb.min.*
 /config-list.h
 /command-list.h
+/bugreport-config-safelist.h
 *.tar.gz
 *.dsc
 *.deb
diff --git a/Documentation/asciidoc.conf b/Documentation/asciidoc.conf
index 8fc4b67081..663e06481f 100644
--- a/Documentation/asciidoc.conf
+++ b/Documentation/asciidoc.conf
@@ -6,9 +6,14 @@
 #
 # Show Git link as: <command>(<section>); if section is defined, else just show
 # the command.
+#
+# The annotate macro does nothing as far as rendering is
+# concerned -- we just grep for it in the sources to populate
+# things like the bugreport safelist.
 
 [macros]
 (?su)[\\]?(?P<name>linkgit):(?P<target>\S*?)\[(?P<attrlist>.*?)\]=
+(?su)[\\]?(?P<name>annotate):(?P<target>\S*?)\[(?P<attrlist>.*?)\]=
 
 [attributes]
 asterisk=&#42;
@@ -28,6 +33,8 @@ ifdef::backend-docbook[]
 {0#<citerefentry>}
 {0#<refentrytitle>{target}</refentrytitle><manvolnum>{0}</manvolnum>}
 {0#</citerefentry>}
+[annotate-inlinemacro]
+{0#}
 endif::backend-docbook[]
 
 ifdef::backend-docbook[]
@@ -94,4 +101,6 @@ ifdef::backend-xhtml11[]
 git-relative-html-prefix=
 [linkgit-inlinemacro]
 <a href="{git-relative-html-prefix}{target}.html">{target}{0?({0})}</a>
+[annotate-inlinemacro]
+<!-- -->
 endif::backend-xhtml11[]
diff --git a/Documentation/asciidoctor-extensions.rb b/Documentation/asciidoctor-extensions.rb
index d906a00803..382bd8f6f4 100644
--- a/Documentation/asciidoctor-extensions.rb
+++ b/Documentation/asciidoctor-extensions.rb
@@ -37,6 +37,10 @@ module Git
           output = output.sub(/<\/refmeta>/, new_tags + "</refmeta>")
         end
         output
+
+    class AnnotateProcessor < Asciidoctor::Extensions::InlineMacroProcessor
+      def process(parent, target, attrs)
+        ""
       end
     end
   end
@@ -45,4 +49,5 @@ end
 Asciidoctor::Extensions.register do
   inline_macro Git::Documentation::LinkGitProcessor, :linkgit
   postprocessor Git::Documentation::DocumentPostProcessor
+  inline_macro Git::Documentation::AnnotateProcessor, :annotate
 end
diff --git a/Documentation/config/sendemail.txt b/Documentation/config/sendemail.txt
index 0006faf800..fe27473e44 100644
--- a/Documentation/config/sendemail.txt
+++ b/Documentation/config/sendemail.txt
@@ -4,7 +4,7 @@ sendemail.identity::
 	values in the 'sendemail' section. The default identity is
 	the value of `sendemail.identity`.
 
-sendemail.smtpEncryption::
+sendemail.smtpEncryption annotate:bugreport[include] ::
 	See linkgit:git-send-email[1] for description.  Note that this
 	setting is not subject to the 'identity' mechanism.
 
@@ -15,7 +15,7 @@ sendemail.smtpsslcertpath::
 	Path to ca-certificates (either a directory or a single file).
 	Set it to an empty string to disable certificate verification.
 
-sendemail.<identity>.*::
+sendemail.<identity>.* annotate:bugreport[exclude] ::
 	Identity-specific versions of the 'sendemail.*' parameters
 	found below, taking precedence over those when this
 	identity is selected, through either the command-line or
@@ -23,41 +23,41 @@ sendemail.<identity>.*::
 
 sendemail.aliasesFile::
 sendemail.aliasFileType::
-sendemail.annotate::
-sendemail.bcc::
-sendemail.cc::
-sendemail.ccCmd::
-sendemail.chainReplyTo::
-sendemail.confirm::
-sendemail.envelopeSender::
-sendemail.from::
-sendemail.multiEdit::
-sendemail.signedoffbycc::
-sendemail.smtpPass::
-sendemail.suppresscc::
-sendemail.suppressFrom::
-sendemail.to::
-sendemail.tocmd::
-sendemail.smtpDomain::
-sendemail.smtpServer::
-sendemail.smtpServerPort::
-sendemail.smtpServerOption::
-sendemail.smtpUser::
-sendemail.thread::
-sendemail.transferEncoding::
-sendemail.validate::
-sendemail.xmailer::
+sendemail.annotate annotate:bugreport[include] ::
+sendemail.bcc annotate:bugreport[include] ::
+sendemail.cc annotate:bugreport[include] ::
+sendemail.ccCmd annotate:bugreport[include] ::
+sendemail.chainReplyTo annotate:bugreport[include] ::
+sendemail.confirm annotate:bugreport[include] ::
+sendemail.envelopeSender annotate:bugreport[include] ::
+sendemail.from annotate:bugreport[include] ::
+sendemail.multiEdit annotate:bugreport[include] ::
+sendemail.signedoffbycc annotate:bugreport[include] ::
+sendemail.smtpPass annotate:bugreport[exclude] ::
+sendemail.suppresscc annotate:bugreport[include] ::
+sendemail.suppressFrom annotate:bugreport[include] ::
+sendemail.to annotate:bugreport[include] ::
+sendemail.tocmd annotate:bugreport[include] ::
+sendemail.smtpDomain annotate:bugreport[include] ::
+sendemail.smtpServer annotate:bugreport[include] ::
+sendemail.smtpServerPort annotate:bugreport[include] ::
+sendemail.smtpServerOption annotate:bugreport[include] ::
+sendemail.smtpUser annotate:bugreport[exclude] ::
+sendemail.thread annotate:bugreport[include] ::
+sendemail.transferEncoding annotate:bugreport[include] ::
+sendemail.validate annotate:bugreport[include] ::
+sendemail.xmailer annotate:bugreport[include] ::
 	See linkgit:git-send-email[1] for description.
 
 sendemail.signedoffcc (deprecated)::
 	Deprecated alias for `sendemail.signedoffbycc`.
 
-sendemail.smtpBatchSize::
+sendemail.smtpBatchSize annotate:bugreport[include] ::
 	Number of messages to be sent per connection, after that a relogin
 	will happen.  If the value is 0 or undefined, send all messages in
 	one connection.
 	See also the `--batch-size` option of linkgit:git-send-email[1].
 
-sendemail.smtpReloginDelay::
+sendemail.smtpReloginDelay annotate:bugreport[include] ::
 	Seconds wait before reconnecting to smtp server.
 	See also the `--relogin-delay` option of linkgit:git-send-email[1].
diff --git a/Makefile b/Makefile
index a01a050aa3..2bc9f112ea 100644
--- a/Makefile
+++ b/Makefile
@@ -817,6 +817,7 @@ VCSSVN_LIB = vcs-svn/lib.a
 
 GENERATED_H += config-list.h
 GENERATED_H += command-list.h
+GENERATED_H += bugreport-config-safelist.h
 
 LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
 	$(FIND) . \
@@ -2163,6 +2164,12 @@ command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Doc
 		$(patsubst %,--exclude-program %,$(EXCLUDED_PROGRAMS)) \
 		command-list.txt >$@+ && mv $@+ $@
 
+bugreport-config-safelist.h: generate-bugreport-config-safelist.sh
+
+bugreport-config-safelist.h: Documentation/config/*.txt
+	$(QUIET_GEN)$(SHELL_PATH) ./generate-bugreport-config-safelist.sh \
+		>$@+ && mv $@+ $@
+
 SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):$(GIT_VERSION):\
 	$(localedir_SQ):$(NO_CURL):$(USE_GETTEXT_SCHEME):$(SANE_TOOL_PATH_SQ):\
 	$(gitwebdir_SQ):$(PERL_PATH_SQ):$(SANE_TEXT_GREP):$(PAGER_ENV):\
diff --git a/generate-bugreport-config-safelist.sh b/generate-bugreport-config-safelist.sh
new file mode 100755
index 0000000000..17d92a91c5
--- /dev/null
+++ b/generate-bugreport-config-safelist.sh
@@ -0,0 +1,17 @@
+#!/bin/sh
+
+cat <<EOF
+/* Automatically generated by bugreport-generate-config-safelist.sh */
+
+
+static const char *bugreport_config_safelist[] = {
+EOF
+
+# cat all regular files in Documentation/config
+find Documentation/config -type f -exec cat {} \; |
+# print the command name which matches the annotate-bugreport macro
+sed -n 's/^\(.*\) \+annotate:bugreport\[include\].* ::$/  "\1",/p' | sort
+
+cat <<EOF
+};
+EOF
-- 
2.25.0.341.g760bfbb309-goog
SZEDER Gábor· Feb 7, 2020, 15:30 UTC · re: Emily Shaffer · lore

Re: [PATCH v6 09/15] bugreport: generate config safelist based on docs

On Wed, Feb 05, 2020 at 04:41:02PM -0800, Emily Shaffer wrote:
Show 112 quoted lines
> Add a new step to the build to generate a safelist of git-config
> variables which are appropriate to include in the output of
> git-bugreport. New variables can be added to the safelist by annotating
> their documentation in Documentation/config with the "annotate" macro,
> which is a no-op in AsciiDoc and AsciiDoctor.
> 
> Some configs are private in nature, and can contain remote URLs,
> passwords, or other sensitive information. In the event that a user
> doesn't notice their information while reviewing a bugreport, that user
> may leak their credentials to other individuals, mailing lists, or bug
> tracking tools inadvertently. Heuristic blocklisting of configuration
> keys is imperfect and prone to false negatives; given the nature of the
> information which can be leaked, a safelist is more reliable.
> 
> However, it's possible that in some situations, an organization may be
> less concerned with privacy of things like remote URLs and branch names,
> and more concerned with ease of diagnosis for their support staff. In
> those cases, it may make more sense for that organization to modify the
> code to use a blocklist. To that end, we should try to mark configs which
> are definitely safe, and configs which are definitely unsafe, and leave
> blank configs which are somewhere in between. To mark a config as safe,
> add "annotate:bugreport[include]" to the corresponding line in the
> config documentation; to mark it as unsafe, add
> "annotate:bugreport[exclude]" instead.
> 
> Generating bugreport-config-safelist.h at build time by grepping the
> documentation for this new macro helps us prevent staleness. The macro
> itself is a no-op and should not alter the appearance of the
> documentation in either AsciiDoc or AsciiDoctor, confirmable by running:
> 
>   cd Documentation
>   ./doc-diff --asciidoctor HEAD^ HEAD
>   ./doc-diff --asciidoc HEAD^ HEAD
> 
> Diffing the rendered HTML shows that only inline comments were added,
> which shouldn't be a problem.
> 
> Additionally, add annotations to the sendemail config documentation in
> order to demonstrate a proof of concept.
> 
> Helped-by: Martin Ågren <martin.agren@gmail.com>
> Helped-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
> Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
> ---
>  .gitignore                              |  1 +
>  Documentation/asciidoc.conf             |  9 ++++
>  Documentation/asciidoctor-extensions.rb |  5 +++
>  Documentation/config/sendemail.txt      | 56 ++++++++++++-------------
>  Makefile                                |  7 ++++
>  generate-bugreport-config-safelist.sh   | 17 ++++++++
>  6 files changed, 67 insertions(+), 28 deletions(-)
>  create mode 100755 generate-bugreport-config-safelist.sh
> 
> diff --git a/.gitignore b/.gitignore
> index d89bf9e11e..bd2f49b996 100644
> --- a/.gitignore
> +++ b/.gitignore
> @@ -192,6 +192,7 @@
>  /gitweb/static/gitweb.min.*
>  /config-list.h
>  /command-list.h
> +/bugreport-config-safelist.h
>  *.tar.gz
>  *.dsc
>  *.deb
> diff --git a/Documentation/asciidoc.conf b/Documentation/asciidoc.conf
> index 8fc4b67081..663e06481f 100644
> --- a/Documentation/asciidoc.conf
> +++ b/Documentation/asciidoc.conf
> @@ -6,9 +6,14 @@
>  #
>  # Show Git link as: <command>(<section>); if section is defined, else just show
>  # the command.
> +#
> +# The annotate macro does nothing as far as rendering is
> +# concerned -- we just grep for it in the sources to populate
> +# things like the bugreport safelist.
>  
>  [macros]
>  (?su)[\\]?(?P<name>linkgit):(?P<target>\S*?)\[(?P<attrlist>.*?)\]=
> +(?su)[\\]?(?P<name>annotate):(?P<target>\S*?)\[(?P<attrlist>.*?)\]=
>  
>  [attributes]
>  asterisk=&#42;
> @@ -28,6 +33,8 @@ ifdef::backend-docbook[]
>  {0#<citerefentry>}
>  {0#<refentrytitle>{target}</refentrytitle><manvolnum>{0}</manvolnum>}
>  {0#</citerefentry>}
> +[annotate-inlinemacro]
> +{0#}
>  endif::backend-docbook[]
>  
>  ifdef::backend-docbook[]
> @@ -94,4 +101,6 @@ ifdef::backend-xhtml11[]
>  git-relative-html-prefix=
>  [linkgit-inlinemacro]
>  <a href="{git-relative-html-prefix}{target}.html">{target}{0?({0})}</a>
> +[annotate-inlinemacro]
> +<!-- -->
>  endif::backend-xhtml11[]
> diff --git a/Documentation/asciidoctor-extensions.rb b/Documentation/asciidoctor-extensions.rb
> index d906a00803..382bd8f6f4 100644
> --- a/Documentation/asciidoctor-extensions.rb
> +++ b/Documentation/asciidoctor-extensions.rb
> @@ -37,6 +37,10 @@ module Git
>            output = output.sub(/<\/refmeta>/, new_tags + "</refmeta>")
>          end
>          output
> +
> +    class AnnotateProcessor < Asciidoctor::Extensions::InlineMacroProcessor
> +      def process(parent, target, attrs)
> +        ""
This change breaks building the documentation with Asciidoctor v1.5.8:
  /home/travis/.rvm/rubies/ruby-2.5.3/lib/ruby/site_ruby/2.5.0/rubygems/core_ext/kernel_require.rb:59:in `require': /home/travis/build/git/git/Documentation/asciidoctor-extensions.rb:41: class definition in method body (SyntaxError)
  
  ...xtensions::InlineMacroProcessor
  
  ...                               ^
  
  /home/travis/build/git/git/Documentation/asciidoctor-extensions.rb:53: syntax error, unexpected end-of-input, expecting keyword_end
  https://travis-ci.org/git/git/jobs/647093871#L1127
Show 9 quoted lines
>        end
>      end
>    end
> @@ -45,4 +49,5 @@ end
>  Asciidoctor::Extensions.register do
>    inline_macro Git::Documentation::LinkGitProcessor, :linkgit
>    postprocessor Git::Documentation::DocumentPostProcessor
> +  inline_macro Git::Documentation::AnnotateProcessor, :annotate
>  end
Emily Shaffer· Feb 13, 2020, 23:14 UTC · re: SZEDER Gábor · lore

Re: [PATCH v6 09/15] bugreport: generate config safelist based on docs

On Fri, Feb 07, 2020 at 04:30:42PM +0100, SZEDER Gábor wrote:
Show 26 quoted lines
> On Wed, Feb 05, 2020 at 04:41:02PM -0800, Emily Shaffer wrote:
> > diff --git a/Documentation/asciidoctor-extensions.rb b/Documentation/asciidoctor-extensions.rb
> > index d906a00803..382bd8f6f4 100644
> > --- a/Documentation/asciidoctor-extensions.rb
> > +++ b/Documentation/asciidoctor-extensions.rb
> > @@ -37,6 +37,10 @@ module Git
> >            output = output.sub(/<\/refmeta>/, new_tags + "</refmeta>")
> >          end
> >          output
> > +
> > +    class AnnotateProcessor < Asciidoctor::Extensions::InlineMacroProcessor
> > +      def process(parent, target, attrs)
> > +        ""
> 
> This change breaks building the documentation with Asciidoctor v1.5.8:
> 
>   /home/travis/.rvm/rubies/ruby-2.5.3/lib/ruby/site_ruby/2.5.0/rubygems/core_ext/kernel_require.rb:59:in `require': /home/travis/build/git/git/Documentation/asciidoctor-extensions.rb:41: class definition in method body (SyntaxError)
>   
>   ...xtensions::InlineMacroProcessor
>   
>   ...                               ^
>   
>   /home/travis/build/git/git/Documentation/asciidoctor-extensions.rb:53: syntax error, unexpected end-of-input, expecting keyword_end
> 
> 
>   https://travis-ci.org/git/git/jobs/647093871#L1127
Done. It passes locally for me now if I run:
  USE_ASCIIDOCTOR=1 make doc
Thanks for the heads up.
 - Emily
Emily Shaffer· Feb 6, 2020, 00:41 UTC · re: Emily Shaffer · lore

[PATCH v6 10/15] bugreport: add config values from safelist

Teach bugreport to gather the values of config options which are present in 'bugreport-config-safelist.h', and show their origin scope.

Many config options are sensitive, and many Git add-ons use config options which git-core does not know about; it is better only to gather config options which we know to be safe, rather than excluding options which we know to be unsafe.

Rather than including the path to someone's config, which can reveal filesystem layout and project names, just name the scope (e.g. system, global, local) of the config source.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  1 +
 Makefile                        |  2 ++
 bugreport.c                     | 34 +++++++++++++++++++++++++++++++++
 3 files changed, 37 insertions(+)
Show changes to 3 files +37 −0

Documentation/git-bugreport.txt, Makefile, bugreport.c

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index 23265b0d74..4e9171d1bd 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -30,6 +30,7 @@ The following information is captured automatically:
  - Compiler-specific info string
  - 'git remote-https --build-info'
  - $SHELL
+ - Selected config values
 
 OPTIONS
 -------
diff --git a/Makefile b/Makefile
index 2bc9f112ea..7b139230ae 100644
--- a/Makefile
+++ b/Makefile
@@ -2131,6 +2131,8 @@ git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS)
 
 help.sp help.s help.o: command-list.h
 
+bugreport.sp bugreport.s bugreport.o: bugreport-config-safelist.h
+
 builtin/help.sp builtin/help.s builtin/help.o: config-list.h GIT-PREFIX
 builtin/help.sp builtin/help.s builtin/help.o: EXTRA_CPPFLAGS = \
 	'-DGIT_HTML_PATH="$(htmldir_relative_SQ)"' \
diff --git a/bugreport.c b/bugreport.c
index 9b51250155..16216bff8e 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -6,6 +6,9 @@
 #include "help.h"
 #include "compat/compiler.h"
 #include "run-command.h"
+#include "config.h"
+#include "bugreport-config-safelist.h"
+#include "khash.h"
 
 static void get_git_remote_https_version_info(struct strbuf *version_info)
 {
@@ -52,6 +55,29 @@ static void get_system_info(struct strbuf *sys_info)
 	strbuf_complete_line(sys_info);
 }
 
+static void get_safelisted_config(struct strbuf *config_info)
+{
+	size_t idx;
+	struct string_list_item *it = NULL;
+	struct key_value_info *kv_info = NULL;
+
+	for (idx = 0; idx < ARRAY_SIZE(bugreport_config_safelist); idx++) {
+		const struct string_list *list =
+			git_config_get_value_multi(bugreport_config_safelist[idx]);
+
+		if (!list)
+			continue;
+
+		strbuf_addf(config_info, "%s:\n", bugreport_config_safelist[idx]);
+		for_each_string_list_item(it, list) {
+			kv_info = it->util;
+			strbuf_addf(config_info, "  %s (%s)\n", it->string,
+				    kv_info ? config_scope_name(kv_info->scope)
+					    : "source unknown");
+		}
+	}
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
 	NULL
@@ -94,6 +120,7 @@ int cmd_main(int argc, const char **argv)
 	char *option_output = NULL;
 	char *option_suffix = "%F-%H%M";
 	struct stat statbuf;
+	int nongit_ok = 0;
 
 	const struct option bugreport_options[] = {
 		OPT_STRING('o', "output-directory", &option_output, N_("path"),
@@ -102,6 +129,10 @@ int cmd_main(int argc, const char **argv)
 			   N_("specify a strftime format suffix for the filename")),
 		OPT_END()
 	};
+
+	/* Prerequisite for hooks and config checks */
+	setup_git_directory_gently(&nongit_ok);
+
 	argc = parse_options(argc, argv, "", bugreport_options,
 			     bugreport_usage, 0);
 
@@ -123,6 +154,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "System Info");
 	get_system_info(&buffer);
 
+	get_header(&buffer, "Safelisted Config Info");
+	get_safelisted_config(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 
 	if (report == NULL) {
-- 
2.25.0.341.g760bfbb309-goog
SZEDER Gábor· Feb 7, 2020, 14:47 UTC · re: Emily Shaffer · lore

Re: [PATCH v6 10/15] bugreport: add config values from safelist

On Wed, Feb 05, 2020 at 04:41:03PM -0800, Emily Shaffer wrote:
Show 7 quoted lines
> +static void get_safelisted_config(struct strbuf *config_info)
> +{
> +	size_t idx;
> +	struct string_list_item *it = NULL;
> +	struct key_value_info *kv_info = NULL;
> +
> +	for (idx = 0; idx < ARRAY_SIZE(bugreport_config_safelist); idx++) {
GCC 9 complains about this loop condition:
      CC bugreport.o
  
  bugreport.c: In function 'get_safelisted_config':
  
  bugreport.c:66:20: error: comparison of unsigned expression < 0 is always false [-Werror=type-limits]
  
     66 |  for (idx = 0; idx < ARRAY_SIZE(bugreport_config_safelist); idx++) {
  
        |                    ^

I don't understand this error, that autogenerated 'bugreport_config_safelist' array clearly has a a non-zero size. What am I missing?

Show 16 quoted lines
> +		const struct string_list *list =
> +			git_config_get_value_multi(bugreport_config_safelist[idx]);
> +
> +		if (!list)
> +			continue;
> +
> +		strbuf_addf(config_info, "%s:\n", bugreport_config_safelist[idx]);
> +		for_each_string_list_item(it, list) {
> +			kv_info = it->util;
> +			strbuf_addf(config_info, "  %s (%s)\n", it->string,
> +				    kv_info ? config_scope_name(kv_info->scope)
> +					    : "source unknown");
> +		}
> +	}
> +}
> +
SZEDER Gábor· Feb 7, 2020, 15:08 UTC · re: SZEDER Gábor · lore

Re: [PATCH v6 10/15] bugreport: add config values from safelist

On Fri, Feb 07, 2020 at 03:47:25PM +0100, SZEDER Gábor wrote:
Show 24 quoted lines
> On Wed, Feb 05, 2020 at 04:41:03PM -0800, Emily Shaffer wrote:
> > +static void get_safelisted_config(struct strbuf *config_info)
> > +{
> > +	size_t idx;
> > +	struct string_list_item *it = NULL;
> > +	struct key_value_info *kv_info = NULL;
> > +
> > +	for (idx = 0; idx < ARRAY_SIZE(bugreport_config_safelist); idx++) {
> 
> GCC 9 complains about this loop condition:
> 
>       CC bugreport.o
>   
>   bugreport.c: In function 'get_safelisted_config':
>   
>   bugreport.c:66:20: error: comparison of unsigned expression < 0 is always false [-Werror=type-limits]
>   
>      66 |  for (idx = 0; idx < ARRAY_SIZE(bugreport_config_safelist); idx++) {
>   
>         |                    ^
> 
> I don't understand this error, that autogenerated
> 'bugreport_config_safelist' array clearly has a a non-zero size.
> What am I missing?
macOS 'sed', that's what I was missing :)

So that array is in fact empty on macOS, because the entries of that array are generated with:

  # cat all regular files in Documentation/config
  find Documentation/config -type f -exec cat {} \; |
  # print the command name which matches the annotate-bugreport macro
  sed -n 's/^\(.*\) \+annotate:bugreport\[include\].* ::$/  "\1",/p' | sort

and the 'sed' included in macOS apparently interprets that '\+' differently than GNU 'sed', and as a result won't match anything.

FWIW, that '\+' doesn't seem to be necessary, though, and after removing it the resulting generated array looked good to me (and to the compiler) both on Linux and macOS.

Eric Sunshine· Feb 7, 2020, 16:24 UTC · re: SZEDER Gábor · lore

Re: [PATCH v6 10/15] bugreport: add config values from safelist

On Fri, Feb 7, 2020 at 10:09 AM SZEDER Gábor <szeder.dev@gmail.com> wrote:
Show 6 quoted lines
> macOS 'sed', that's what I was missing :)
>
>   sed -n 's/^\(.*\) \+annotate:bugreport\[include\].* ::$/  "\1",/p' | sort
>
> and the 'sed' included in macOS apparently interprets that '\+'
> differently than GNU 'sed', and as a result won't match anything.
More generally, this would be a problem with any 'sed' of BSD lineage.
> FWIW, that '\+' doesn't seem to be necessary, though, and after
> removing it the resulting generated array looked good to me [...]

A reasonable replacement for "<SP>\+" would be "<SP><SP>*" (where <SP> represents 'space').

Andreas Schwab· Feb 7, 2020, 16:51 UTC · re: Eric Sunshine · lore

Re: [PATCH v6 10/15] bugreport: add config values from safelist

On Feb 07 2020, Eric Sunshine wrote:
Show 15 quoted lines
> On Fri, Feb 7, 2020 at 10:09 AM SZEDER Gábor <szeder.dev@gmail.com> wrote:
>> macOS 'sed', that's what I was missing :)
>>
>>   sed -n 's/^\(.*\) \+annotate:bugreport\[include\].* ::$/  "\1",/p' | sort
>>
>> and the 'sed' included in macOS apparently interprets that '\+'
>> differently than GNU 'sed', and as a result won't match anything.
>
> More generally, this would be a problem with any 'sed' of BSD lineage.
>
>> FWIW, that '\+' doesn't seem to be necessary, though, and after
>> removing it the resulting generated array looked good to me [...]
>
> A reasonable replacement for "<SP>\+" would be "<SP><SP>*" (where <SP>
> represents 'space').

Another problem with that regexp is that it contains two adjacent repetitions matching the same character. When there are two or more spaces before "annotate:" all but the last of them can be matched by either '\(.*\)' or ' \+'. To fix that '\(.*\)' needs to be modified to not match a trailing space.

Andreas.
-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."
Emily Shaffer· Feb 13, 2020, 22:02 UTC · re: Andreas Schwab · lore

Re: [PATCH v6 10/15] bugreport: add config values from safelist

On Fri, Feb 07, 2020 at 05:51:05PM +0100, Andreas Schwab wrote:
Show 23 quoted lines
> On Feb 07 2020, Eric Sunshine wrote:
> 
> > On Fri, Feb 7, 2020 at 10:09 AM SZEDER Gábor <szeder.dev@gmail.com> wrote:
> >> macOS 'sed', that's what I was missing :)
> >>
> >>   sed -n 's/^\(.*\) \+annotate:bugreport\[include\].* ::$/  "\1",/p' | sort
> >>
> >> and the 'sed' included in macOS apparently interprets that '\+'
> >> differently than GNU 'sed', and as a result won't match anything.
> >
> > More generally, this would be a problem with any 'sed' of BSD lineage.
> >
> >> FWIW, that '\+' doesn't seem to be necessary, though, and after
> >> removing it the resulting generated array looked good to me [...]
> >
> > A reasonable replacement for "<SP>\+" would be "<SP><SP>*" (where <SP>
> > represents 'space').
> 
> Another problem with that regexp is that it contains two adjacent
> repetitions matching the same character.  When there are two or more
> spaces before "annotate:" all but the last of them can be matched by
> either '\(.*\)' or ' \+'.  To fix that '\(.*\)' needs to be modified to
> not match a trailing space.

Hum. I had assumed since the capture group was not greedy, it would not capture any trailing spaces that could be captured by ' \+'. I don't see a problem in making it non-explicit, though.

I'll hack on this some more. I do find myself pretty annoyed at the matcher difference between 'sed' and 'also sed', though, and don't see in the manpage a way to guarantee which matcher 'sed' should use (a la 'grep -[EFGP]'). :)

 - Emily
Emily Shaffer· Feb 6, 2020, 00:41 UTC · re: Emily Shaffer · lore

[PATCH v6 11/15] bugreport: collect list of populated hooks

Occasionally a failure a user is seeing may be related to a specific hook which is being run, perhaps without the user realizing. While the contents of hooks can be sensitive - containing user data or process information specific to the user's organization - simply knowing that a hook is being run at a certain stage can help us to understand whether something is going wrong.

Without a definitive list of hook names within the code, we compile our own list from the documentation. This is likely prone to bitrot. To reduce the amount of code humans need to read, we turn the list into a string_list and iterate over it (as we are calling the same find_hook operation on each string). However, since bugreport should primarily be called by the user, the performance loss from massaging the string seems acceptable.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 53 +++++++++++++++++++++++++++++++++
 2 files changed, 54 insertions(+)
Show changes to 2 files +54 −0

Documentation/git-bugreport.txt, bugreport.c

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index 4e9171d1bd..779c9c7fd9 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -31,6 +31,7 @@ The following information is captured automatically:
  - 'git remote-https --build-info'
  - $SHELL
  - Selected config values
+ - A list of enabled hooks
 
 OPTIONS
 -------
diff --git a/bugreport.c b/bugreport.c
index 16216bff8e..3b0a2b12cc 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -9,6 +9,7 @@
 #include "config.h"
 #include "bugreport-config-safelist.h"
 #include "khash.h"
+#include "run-command.h"
 
 static void get_git_remote_https_version_info(struct strbuf *version_info)
 {
@@ -78,6 +79,55 @@ static void get_safelisted_config(struct strbuf *config_info)
 	}
 }
 
+static void get_populated_hooks(struct strbuf *hook_info, int nongit)
+{
+	/*
+	 * Doesn't look like there is a list of all possible hooks; so below is
+	 * a transcription of `git help hooks`.
+	 */
+	const char *hooks = "applypatch-msg,"
+			    "pre-applypatch,"
+			    "post-applypatch,"
+			    "pre-commit,"
+			    "pre-merge-commit,"
+			    "prepare-commit-msg,"
+			    "commit-msg,"
+			    "post-commit,"
+			    "pre-rebase,"
+			    "post-checkout,"
+			    "post-merge,"
+			    "pre-push,"
+			    "pre-receive,"
+			    "update,"
+			    "post-receive,"
+			    "post-update,"
+			    "push-to-checkout,"
+			    "pre-auto-gc,"
+			    "post-rewrite,"
+			    "sendemail-validate,"
+			    "fsmonitor-watchman,"
+			    "p4-pre-submit,"
+			    "post-index-changex";
+	struct string_list hooks_list = STRING_LIST_INIT_DUP;
+	struct string_list_item *iter = NULL;
+
+
+	if (nongit) {
+		strbuf_addstr(hook_info,
+			"not run from a git repository - no hooks to show\n");
+		return;
+	}
+
+	string_list_split(&hooks_list, hooks, ',', -1);
+
+	for_each_string_list_item(iter, &hooks_list) {
+		if (find_hook(iter->string)) {
+			strbuf_addstr(hook_info, iter->string);
+			strbuf_complete_line(hook_info);
+		}
+	}
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
 	NULL
@@ -157,6 +207,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Safelisted Config Info");
 	get_safelisted_config(&buffer);
 
+	get_header(&buffer, "Enabled Hooks");
+	get_populated_hooks(&buffer, nongit_ok);
+
 	report = fopen_for_writing(report_path.buf);
 
 	if (report == NULL) {
-- 
2.25.0.341.g760bfbb309-goog
Emily Shaffer· Feb 6, 2020, 00:41 UTC · re: Emily Shaffer · lore

[PATCH v6 12/15] bugreport: count loose objects

The number of unpacked objects in a user's repository may help us understand the root of the problem they're seeing, especially if a command is running unusually slowly.

Helped-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 52 +++++++++++++++++++++++++++++++++
 2 files changed, 53 insertions(+)
Show changes to 2 files +53 −0

Documentation/git-bugreport.txt, bugreport.c

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index 779c9c7fd9..7e1ed44cb4 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -32,6 +32,7 @@ The following information is captured automatically:
  - $SHELL
  - Selected config values
  - A list of enabled hooks
+ - The number of loose objects in the repository
 
 OPTIONS
 -------
diff --git a/bugreport.c b/bugreport.c
index 3b0a2b12cc..9552e2922e 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -10,6 +10,7 @@
 #include "bugreport-config-safelist.h"
 #include "khash.h"
 #include "run-command.h"
+#include "object-store.h"
 
 static void get_git_remote_https_version_info(struct strbuf *version_info)
 {
@@ -128,6 +129,54 @@ static void get_populated_hooks(struct strbuf *hook_info, int nongit)
 	}
 }
 
+static int loose_object_cb(const struct object_id *oid, const char *path,
+			   void *data) {
+	int *loose_object_count = data;
+
+	if (loose_object_count) {
+		(*loose_object_count)++;
+		return 0;
+	}
+
+	return 1;
+}
+
+static void get_loose_object_summary(struct strbuf *obj_info, int nongit) {
+
+	int local_loose_object_count = 0, total_loose_object_count = 0;
+	int local_count_questionable = 0, total_count_questionable = 0;
+
+	if (nongit) {
+		strbuf_addstr(obj_info,
+			"not run from a git repository - no objects to show\n");
+		return;
+	}
+
+	local_count_questionable = for_each_loose_object(
+					loose_object_cb,
+					&local_loose_object_count,
+					FOR_EACH_OBJECT_LOCAL_ONLY);
+
+	total_count_questionable = for_each_loose_object(
+					loose_object_cb,
+					&total_loose_object_count,
+					0);
+
+	strbuf_addf(obj_info, "%d local loose objects%s\n",
+		    local_loose_object_count,
+		    local_count_questionable ? " (problem during count)" : "");
+
+	strbuf_addf(obj_info, "%d alternate loose objects%s\n",
+		    total_loose_object_count - local_loose_object_count,
+		    (local_count_questionable || total_count_questionable)
+			? " (problem during count)"
+			: "");
+
+	strbuf_addf(obj_info, "%d total loose objects%s\n",
+		    total_loose_object_count,
+		    total_count_questionable ? " (problem during count)" : "");
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
 	NULL
@@ -210,6 +259,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Enabled Hooks");
 	get_populated_hooks(&buffer, nongit_ok);
 
+	get_header(&buffer, "Loose Object Counts");
+	get_loose_object_summary(&buffer, nongit_ok);
+
 	report = fopen_for_writing(report_path.buf);
 
 	if (report == NULL) {
-- 
2.25.0.341.g760bfbb309-goog
Emily Shaffer· Feb 6, 2020, 00:41 UTC · re: Emily Shaffer · lore

[PATCH v6 13/15] bugreport: add packed object summary

Alongside the loose object counts, it can be useful to show the number of packs and packed objects. This way we can check whether the repo has an appropriate ratio of packed to loose objects to help determine whether it's behaving correctly.

Add a utility to easily traverse all packfiles in a given repository. Use it in packfile.c and remove a redundant call to prepare_packed_git(), which is already called in get_all_packs().

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
Perhaps the macro should be in another patch? I didn't spend a lot of
time looking for anybody who could conceivably be iterating packs; if I
were to do so, I'd break this change out separately.

I considered following the pattern in the rest of the file, where the foreach takes a callback and data pointer, but found it would be cumbersome to use with for_each_packed_object() - I'd have to add a new callback just to handle the pack, and bundle that function's callback data along with the callback pointer. It seemed like overkill.

 - Emily
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 30 ++++++++++++++++++++++++++++++
 object-store.h                  |  6 ++++++
 packfile.c                      |  3 +--
 4 files changed, 38 insertions(+), 2 deletions(-)
Show changes to 4 files +38 −2

Documentation/git-bugreport.txt, bugreport.c, object-store.h, packfile.c

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index 7e1ed44cb4..309b34fc42 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -33,6 +33,7 @@ The following information is captured automatically:
  - Selected config values
  - A list of enabled hooks
  - The number of loose objects in the repository
+ - The number of packs and packed objects in the repository
 
 OPTIONS
 -------
diff --git a/bugreport.c b/bugreport.c
index 9552e2922e..d554488f3e 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -177,6 +177,33 @@ static void get_loose_object_summary(struct strbuf *obj_info, int nongit) {
 		    total_count_questionable ? " (problem during count)" : "");
 }
 
+static void get_packed_object_summary(struct strbuf *obj_info, int nongit)
+{
+	struct packed_git *pack = NULL;
+	int pack_count = 0;
+	int object_count = 0;
+
+	if (nongit) {
+		strbuf_addstr(obj_info,
+			"not run from a git repository - no objects to show\n");
+		return;
+	}
+
+	for_each_pack(the_repository, pack) {
+		pack_count++;
+		/*
+		 * To accurately count how many objects are packed, look inside
+		 * the packfile's index.
+		 */
+		open_pack_index(pack);
+		object_count += pack->num_objects;
+	}
+
+	strbuf_addf(obj_info, "%d total packs (%d objects)\n", pack_count,
+		    object_count);
+
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
 	NULL
@@ -262,6 +289,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Loose Object Counts");
 	get_loose_object_summary(&buffer, nongit_ok);
 
+	get_header(&buffer, "Packed Object Summary");
+	get_packed_object_summary(&buffer, nongit_ok);
+
 	report = fopen_for_writing(report_path.buf);
 
 	if (report == NULL) {
diff --git a/object-store.h b/object-store.h
index 55ee639350..518c73469d 100644
--- a/object-store.h
+++ b/object-store.h
@@ -6,6 +6,7 @@
 #include "list.h"
 #include "sha1-array.h"
 #include "strbuf.h"
+#include "packfile.h"
 
 struct object_directory {
 	struct object_directory *next;
@@ -403,4 +404,9 @@ int for_each_object_in_pack(struct packed_git *p,
 int for_each_packed_object(each_packed_object_fn, void *,
 			   enum for_each_object_flags flags);
 
+#define for_each_pack(repo, pack) 		\
+		for (pack = get_all_packs(repo);\
+		     pack;			\
+		     pack = pack->next)
+
 #endif /* OBJECT_STORE_H */
diff --git a/packfile.c b/packfile.c
index 7e7c04e4d8..6873827fb5 100644
--- a/packfile.c
+++ b/packfile.c
@@ -2061,8 +2061,7 @@ int for_each_packed_object(each_packed_object_fn cb, void *data,
 	int r = 0;
 	int pack_errors = 0;
 
-	prepare_packed_git(the_repository);
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for_each_pack(the_repository, p) {
 		if ((flags & FOR_EACH_OBJECT_LOCAL_ONLY) && !p->pack_local)
 			continue;
 		if ((flags & FOR_EACH_OBJECT_PROMISOR_ONLY) &&
-- 
2.25.0.341.g760bfbb309-goog
Emily Shaffer· Feb 6, 2020, 00:41 UTC · re: Emily Shaffer · lore

[PATCH v6 14/15] bugreport: list contents of $OBJDIR/info

Miscellaneous information used about the object store can end up in .git/objects/info; this can help us understand what may be going on with the object store when the user is reporting a bug. Otherwise, it could be difficult to track down what is going wrong with an object which isn't kept locally to .git/objects/ or .git/objects/pack. Having some understanding of where the user's objects may be kept can save us some hops during the bug reporting process.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 54 +++++++++++++++++++++++++++++++++
 2 files changed, 55 insertions(+)
Show changes to 2 files +55 −0

Documentation/git-bugreport.txt, bugreport.c

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index 309b34fc42..a21d081616 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -34,6 +34,7 @@ The following information is captured automatically:
  - A list of enabled hooks
  - The number of loose objects in the repository
  - The number of packs and packed objects in the repository
+ - A list of the contents of .git/objects/info (or equivalent)
 
 OPTIONS
 -------
diff --git a/bugreport.c b/bugreport.c
index d554488f3e..3a3d33f680 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -204,6 +204,57 @@ static void get_packed_object_summary(struct strbuf *obj_info, int nongit)
 
 }
 
+static void list_contents_of_dir_recursively(struct strbuf *contents,
+				      	     struct strbuf *dirpath)
+{
+	struct dirent *d;
+	DIR *dir;
+	size_t path_len;
+
+	dir = opendir(dirpath->buf);
+	if (!dir)
+		return;
+
+	strbuf_complete(dirpath, '/');
+	path_len = dirpath->len;
+
+	while ((d = readdir(dir))) {
+		if (!strcmp(d->d_name, ".") || !strcmp(d->d_name, ".."))
+			continue;
+
+		strbuf_addbuf(contents, dirpath);
+		strbuf_addstr(contents, d->d_name);
+		strbuf_complete_line(contents);
+
+		if (d->d_type == DT_DIR) {
+			strbuf_addstr(dirpath, d->d_name);
+			list_contents_of_dir_recursively(contents, dirpath);
+		}
+		strbuf_setlen(dirpath, path_len);
+	}
+
+	closedir(dir);
+}
+
+static void get_object_info_summary(struct strbuf *obj_info, int nongit)
+{
+	struct strbuf dirpath = STRBUF_INIT;
+
+	if (nongit) {
+		strbuf_addstr(obj_info,
+			"not run from a git repository - object info unavailable\n");
+		return;
+	}
+
+	strbuf_addstr(&dirpath, get_object_directory());
+	strbuf_complete(&dirpath, '/');
+	strbuf_addstr(&dirpath, "info/");
+
+	list_contents_of_dir_recursively(obj_info, &dirpath);
+
+	strbuf_release(&dirpath);
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
 	NULL
@@ -292,6 +343,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Packed Object Summary");
 	get_packed_object_summary(&buffer, nongit_ok);
 
+	get_header(&buffer, "Object Info Summary");
+	get_object_info_summary(&buffer, nongit_ok);
+
 	report = fopen_for_writing(report_path.buf);
 
 	if (report == NULL) {
-- 
2.25.0.341.g760bfbb309-goog
Emily Shaffer· Feb 6, 2020, 00:41 UTC · re: Emily Shaffer · lore

[PATCH v6 15/15] bugreport: summarize contents of alternates file

In some cases, it could be that the user is having a problem with an object which isn't present in their normal object directory. We can get a hint that that might be the case by examining the list of alternates where their object may be stored instead. Since paths to alternates may be sensitive, we'll instead count how many alternates have been specified and note how many of them exist or are broken.

While object-cache.h describes a function "foreach_alt_odb()", this function does not provide information on broken alternates, which are skipped over in "link_alt_odb_entry()". Since the goal is to identify missing alternates, we can gather the contents of .git/objects/info/alternates manually.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 45 +++++++++++++++++++++++++++++++++
 2 files changed, 46 insertions(+)
Show changes to 2 files +46 −0

Documentation/git-bugreport.txt, bugreport.c

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index a21d081616..e870900c80 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -35,6 +35,7 @@ The following information is captured automatically:
  - The number of loose objects in the repository
  - The number of packs and packed objects in the repository
  - A list of the contents of .git/objects/info (or equivalent)
+ - The number of valid and invalid alternates
 
 OPTIONS
 -------
diff --git a/bugreport.c b/bugreport.c
index 3a3d33f680..9d4d5c8e6f 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -255,6 +255,48 @@ static void get_object_info_summary(struct strbuf *obj_info, int nongit)
 	strbuf_release(&dirpath);
 }
 
+static void get_alternates_summary(struct strbuf *alternates_info, int nongit)
+{
+	struct strbuf alternates_path = STRBUF_INIT;
+	struct strbuf alternate = STRBUF_INIT;
+	FILE *file;
+	size_t exists = 0, broken = 0;
+
+	if (nongit) {
+		strbuf_addstr(alternates_info,
+			"not run from a git repository - alternates unavailable\n");
+		return;
+	}
+
+	strbuf_addstr(&alternates_path, get_object_directory());
+	strbuf_complete(&alternates_path, '/');
+	strbuf_addstr(&alternates_path, "info/alternates");
+
+	file = fopen(alternates_path.buf, "r");
+	if (!file) {
+		strbuf_addstr(alternates_info, "No alternates file found.\n");
+		strbuf_release(&alternates_path);
+		return;
+	}
+
+	while (strbuf_getline(&alternate, file) != EOF) {
+		if (!access(alternate.buf, F_OK))
+			exists++;
+		else
+			broken++;
+	}
+
+	strbuf_addf(alternates_info,
+		    "%zd alternates found (%zd working, %zd broken)\n",
+		    exists + broken,
+		    exists,
+		    broken);
+
+	fclose(file);
+	strbuf_release(&alternate);
+	strbuf_release(&alternates_path);
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
 	NULL
@@ -346,6 +388,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Object Info Summary");
 	get_object_info_summary(&buffer, nongit_ok);
 
+	get_header(&buffer, "Alternates");
+	get_alternates_summary(&buffer, nongit_ok);
+
 	report = fopen_for_writing(report_path.buf);
 
 	if (report == NULL) {
-- 
2.25.0.341.g760bfbb309-goog
Emily Shaffer· Feb 14, 2020, 01:53 UTC · re: Emily Shaffer · lore

[PATCH v7 00/15] add git-bugreport tool

This patchset is based on mr/show-config-scope, which looks close to ready for 'next'.

Since v6:
 - Applied Danh's suggestion[1] (with modifications recommended by Eric and
   Junio downthread)
 - Fixed a missed end-of-macro in the Asciidoctor config which was
   pointed out most recently by SZEDER[2]
 - Fixed generate-bugreport-safelist-config.sh to be more friendly to
   BSD-derived 'sed' and to capture trailing ' ' when extra whitespace is
   present(patch 9/15). This now uses POSIX character classes and {}
   notation instead of + and has been tested on OSX; I'd love to hear if
   it's still not working correctly.
 - Applied Junio's patch[3] to make GIT_TEST_GETTEXT_POISON happy and to
   make the tests more idiomatic.
 - Stop globbing into variables, start making the output directory if it
   doesn't exist before[4]
 - Add an entry in 'command-list.txt'[5]. I sure wish someone had
   explained how to do this somewhere in a tutorial about writing new
   commands ;)
 - Added a test to ensure that git-bugreport runs gracefully in a
   non-repo directory.
 - Added nongit_ok flags to relevant pieces of info to ensure the above
   test passes.

[1] https://lore.kernel.org/git/20200206013533.GA3993@danh.dev [2] https://lore.kernel.org/git/20200207153042.GI2868@szeder.dev [3] https://lore.kernel.org/git/xmqq36bfvfwr.fsf@gitster-ct.c.googlers.com [4] https://lore.kernel.org/git/20200207141802.GE2868@szeder.dev [5] https://lore.kernel.org/git/20200207145409.GG2868@szeder.dev

Emily Shaffer (15):
  help: move list_config_help to builtin/help
  help: add shell-path to --build-options
  bugreport: add tool to generate debugging info
  bugreport: gather git version and build info
  bugreport: add uname info
  bugreport: add compiler info
  bugreport: add git-remote-https version
  bugreport: include user interactive shell
  bugreport: generate config safelist based on docs
  bugreport: add config values from safelist
  bugreport: collect list of populated hooks
  bugreport: count loose objects
  bugreport: add packed object summary
  bugreport: list contents of $OBJDIR/info
  bugreport: summarize contents of alternates file
 .gitignore                              |   3 +
 Documentation/asciidoc.conf             |   9 +
 Documentation/asciidoctor-extensions.rb |   7 +
 Documentation/config/sendemail.txt      |  56 ++--
 Documentation/git-bugreport.txt         |  55 ++++
 Makefile                                |  27 +-
 bugreport.c                             | 418 ++++++++++++++++++++++++
 builtin/help.c                          |  86 +++++
 command-list.txt                        |   1 +
 compat/compiler.h                       |  24 ++
 generate-bugreport-config-safelist.sh   |  18 +
 generate-cmdlist.sh                     |  19 --
 generate-configlist.sh                  |  21 ++
 help.c                                  | 132 ++------
 help.h                                  |   2 +-
 object-store.h                          |   6 +
 packfile.c                              |   3 +-
 remote-curl.c                           |   8 +
 t/t0091-bugreport.sh                    |  55 ++++
 19 files changed, 794 insertions(+), 156 deletions(-)
 create mode 100644 Documentation/git-bugreport.txt
 create mode 100644 bugreport.c
 create mode 100644 compat/compiler.h
 create mode 100755 generate-bugreport-config-safelist.sh
 create mode 100755 generate-configlist.sh
 create mode 100755 t/t0091-bugreport.sh
v6 to v7 range-diff:
Show changes to 6 files +33 −17

Documentation/asciidoctor-extensions.rb, bugreport.c, command-list.txt, generate-bugreport-config-safelist.sh, generate-configlist.sh, t/t0091-bugreport.sh

diff --git a/Documentation/asciidoctor-extensions.rb b/Documentation/asciidoctor-extensions.rb
index 382bd8f6f4..03c80af0e5 100644
--- a/Documentation/asciidoctor-extensions.rb
+++ b/Documentation/asciidoctor-extensions.rb
@@ -37,6 +37,8 @@ module Git
           output = output.sub(/<\/refmeta>/, new_tags + "</refmeta>")
         end
         output
+      end
+    end
 
     class AnnotateProcessor < Asciidoctor::Extensions::InlineMacroProcessor
       def process(parent, target, attrs)
diff --git a/bugreport.c b/bugreport.c
index 9d4d5c8e6f..3bc8cb3579 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -368,6 +368,15 @@ int cmd_main(int argc, const char **argv)
 	if (!stat(report_path.buf, &statbuf))
 		die("'%s' already exists", report_path.buf);
 
+	switch (safe_create_leading_directories(report_path.buf)) {
+	case SCLD_OK:
+	case SCLD_EXISTS:
+		break;
+	default:
+		die(_("could not create leading directories for '%s'"),
+		    report_path.buf);
+	}
+
 	get_bug_template(&buffer);
 
 	get_header(&buffer, "System Info");
diff --git a/command-list.txt b/command-list.txt
index 2087894655..185e5e3f05 100644
--- a/command-list.txt
+++ b/command-list.txt
@@ -54,6 +54,7 @@ git-archive                             mainporcelain
 git-bisect                              mainporcelain           info
 git-blame                               ancillaryinterrogators          complete
 git-branch                              mainporcelain           history
+git-bugreport                           ancillaryinterrogators
 git-bundle                              mainporcelain
 git-cat-file                            plumbinginterrogators
 git-check-attr                          purehelpers
diff --git a/generate-bugreport-config-safelist.sh b/generate-bugreport-config-safelist.sh
index 17d92a91c5..2794a8ae77 100755
--- a/generate-bugreport-config-safelist.sh
+++ b/generate-bugreport-config-safelist.sh
@@ -10,7 +10,8 @@ EOF
 # cat all regular files in Documentation/config
 find Documentation/config -type f -exec cat {} \; |
 # print the command name which matches the annotate-bugreport macro
-sed -n 's/^\(.*\) \+annotate:bugreport\[include\].* ::$/  "\1",/p' | sort
+sed -n 's/^\([^[:blank:]]*\)[[:blank:]]\{1,\}annotate:bugreport\[include\].* ::$/  "\1",/p' \
+	| sort
 
 cat <<EOF
 };
diff --git a/generate-configlist.sh b/generate-configlist.sh
index eca6a00c30..8692fe5cf4 100755
--- a/generate-configlist.sh
+++ b/generate-configlist.sh
@@ -10,10 +10,7 @@ EOF
 	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
 	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
 	sort |
-	while read line
-	do
-		echo "	\"$line\","
-	done
+	sed 's/^.*$/	"&",/'
 	cat <<EOF
 	NULL,
 };
diff --git a/t/t0091-bugreport.sh b/t/t0091-bugreport.sh
index 451badff0c..6585a2d144 100755
--- a/t/t0091-bugreport.sh
+++ b/t/t0091-bugreport.sh
@@ -8,9 +8,12 @@ test_description='git bugreport'
 # information there; we can make sure all our headers were followed by some
 # information to check if the command was successful.
 HEADER_PATTERN="^\[.*\]$"
-check_all_headers_populated() {
-	while read -r line; do
-		if test "$(grep "$HEADER_PATTERN" "$line")"; then
+
+check_all_headers_populated () {
+	while read -r line
+	do
+		if test "$(grep "$HEADER_PATTERN" "$line")"
+		then
 			echo "$line"
 			read -r nextline
 			if test -z "$nextline"; then
@@ -21,29 +24,32 @@ check_all_headers_populated() {
 }
 
 test_expect_success 'creates a report with content in the right places' '
-	git bugreport &&
-	REPORT="$(ls git-bugreport-*)" &&
-	check_all_headers_populated <$REPORT &&
-	rm $REPORT
+	git bugreport -s check-headers &&
+	check_all_headers_populated <git-bugreport-check-headers.txt &&
+	test_when_finished rm git-bugreport-check-headers.txt
 '
 
 test_expect_success 'dies if file with same name as report already exists' '
-	touch git-bugreport-duplicate.txt &&
+	>>git-bugreport-duplicate.txt &&
 	test_must_fail git bugreport --suffix duplicate &&
-	rm git-bugreport-duplicate.txt
+	test_when_finished rm git-bugreport-duplicate.txt
 '
 
 test_expect_success '--output-directory puts the report in the provided dir' '
-	mkdir foo/ &&
 	git bugreport -o foo/ &&
 	test_path_is_file foo/git-bugreport-* &&
-	rm -fr foo/
+	test_when_finished rm -fr foo/
 '
 
 test_expect_success 'incorrect arguments abort with usage' '
 	test_must_fail git bugreport --false 2>output &&
-	grep usage output &&
+	test_i18ngrep usage output &&
 	test_path_is_missing git-bugreport-*
 '
 
+test_expect_success 'runs outside of a git dir' '
+	nongit git bugreport &&
+	test_when_finished rm non-repo/git-bugreport-*
+'
+
 test_done
-- 
2.25.0.265.gbab2e86ba0-goog
Emily Shaffer· Feb 14, 2020, 01:53 UTC · re: Emily Shaffer · lore

[PATCH v7 02/15] help: add shell-path to --build-options

It may be useful to know which shell Git was built to try to point to, in the event that shell-based Git commands are failing. $SHELL_PATH is set during the build and used to launch the manpage viewer, as well as by git-compat-util.h, and it's used during tests. 'git version --build-options' is encouraged for use in bug reports, so it makes sense to include this information there.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 help.c | 1 +
 1 file changed, 1 insertion(+)
Show changes to help.c +1 −0
diff --git a/help.c b/help.c
index a21487db77..190722fb0a 100644
--- a/help.c
+++ b/help.c
@@ -654,6 +654,7 @@ int cmd_version(int argc, const char **argv, const char *prefix)
 			printf("no commit associated with this build\n");
 		printf("sizeof-long: %d\n", (int)sizeof(long));
 		printf("sizeof-size_t: %d\n", (int)sizeof(size_t));
+		printf("shell-path: %s\n", SHELL_PATH);
 		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
 	}
 	return 0;
-- 
2.25.0.265.gbab2e86ba0-goog
Emily Shaffer· Feb 14, 2020, 01:53 UTC · re: Emily Shaffer · lore

[PATCH v7 01/15] help: move list_config_help to builtin/help

Starting in 3ac68a93fd2, help.o began to depend on builtin/branch.o, builtin/clean.o, and builtin/config.o. This meant that help.o was unusable outside of the context of the main Git executable.

To make help.o usable by other commands again, move list_config_help() into builtin/help.c (where it makes sense to assume other builtin libraries are present).

When command-list.h is included but a member is not used, we start to hear a compiler warning. Since the config list is generated in a fairly different way than the command list, and since commands and config options are semantically different, move the config list into its own header and move the generator into its own script and build rule.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 .gitignore             |  1 +
 Makefile               | 13 +++++--
 builtin/help.c         | 86 ++++++++++++++++++++++++++++++++++++++++++
 generate-cmdlist.sh    | 19 ----------
 generate-configlist.sh | 21 +++++++++++
 help.c                 | 85 -----------------------------------------
 help.h                 |  1 -
 7 files changed, 118 insertions(+), 108 deletions(-)
 create mode 100755 generate-configlist.sh
Show changes to 7 files +118 −108

.gitignore, Makefile, builtin/help.c, generate-cmdlist.sh, generate-configlist.sh, help.c, help.h

diff --git a/.gitignore b/.gitignore
index aebe7c0908..ea97de83f3 100644
--- a/.gitignore
+++ b/.gitignore
@@ -189,6 +189,7 @@
 /gitweb/gitweb.cgi
 /gitweb/static/gitweb.js
 /gitweb/static/gitweb.min.*
+/config-list.h
 /command-list.h
 *.tar.gz
 *.dsc
diff --git a/Makefile b/Makefile
index 09f98b777c..5a022367d4 100644
--- a/Makefile
+++ b/Makefile
@@ -814,6 +814,7 @@ LIB_FILE = libgit.a
 XDIFF_LIB = xdiff/lib.a
 VCSSVN_LIB = vcs-svn/lib.a
 
+GENERATED_H += config-list.h
 GENERATED_H += command-list.h
 
 LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
@@ -2128,7 +2129,7 @@ git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS)
 
 help.sp help.s help.o: command-list.h
 
-builtin/help.sp builtin/help.s builtin/help.o: command-list.h GIT-PREFIX
+builtin/help.sp builtin/help.s builtin/help.o: config-list.h GIT-PREFIX
 builtin/help.sp builtin/help.s builtin/help.o: EXTRA_CPPFLAGS = \
 	'-DGIT_HTML_PATH="$(htmldir_relative_SQ)"' \
 	'-DGIT_MAN_PATH="$(mandir_relative_SQ)"' \
@@ -2148,6 +2149,12 @@ $(BUILT_INS): git$X
 	ln -s $< $@ 2>/dev/null || \
 	cp $< $@
 
+config-list.h: generate-configlist.sh
+
+config-list.h:
+	$(QUIET_GEN)$(SHELL_PATH) ./generate-configlist.sh \
+		>$@+ && mv $@+ $@
+
 command-list.h: generate-cmdlist.sh command-list.txt
 
 command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Documentation/config/*.txt
@@ -2781,7 +2788,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE
 .PHONY: sparse $(SP_OBJ)
 sparse: $(SP_OBJ)
 
-EXCEPT_HDRS := command-list.h unicode-width.h compat/% xdiff/%
+EXCEPT_HDRS := command-list.h config-list.h unicode-width.h compat/% xdiff/%
 ifndef GCRYPT_SHA256
 	EXCEPT_HDRS += sha256/gcrypt.h
 endif
@@ -2803,7 +2810,7 @@ hdr-check: $(HCO)
 style:
 	git clang-format --style file --diff --extensions c,h
 
-check: command-list.h
+check: config-list.h command-list.h
 	@if sparse; \
 	then \
 		echo >&2 "Use 'make sparse' instead"; \
diff --git a/builtin/help.c b/builtin/help.c
index e5590d7787..1c5f2b9255 100644
--- a/builtin/help.c
+++ b/builtin/help.c
@@ -8,6 +8,7 @@
 #include "parse-options.h"
 #include "run-command.h"
 #include "column.h"
+#include "config-list.h"
 #include "help.h"
 #include "alias.h"
 
@@ -62,6 +63,91 @@ static const char * const builtin_help_usage[] = {
 	NULL
 };
 
+struct slot_expansion {
+	const char *prefix;
+	const char *placeholder;
+	void (*fn)(struct string_list *list, const char *prefix);
+	int found;
+};
+
+static void list_config_help(int for_human)
+{
+	struct slot_expansion slot_expansions[] = {
+		{ "advice", "*", list_config_advices },
+		{ "color.branch", "<slot>", list_config_color_branch_slots },
+		{ "color.decorate", "<slot>", list_config_color_decorate_slots },
+		{ "color.diff", "<slot>", list_config_color_diff_slots },
+		{ "color.grep", "<slot>", list_config_color_grep_slots },
+		{ "color.interactive", "<slot>", list_config_color_interactive_slots },
+		{ "color.remote", "<slot>", list_config_color_sideband_slots },
+		{ "color.status", "<slot>", list_config_color_status_slots },
+		{ "fsck", "<msg-id>", list_config_fsck_msg_ids },
+		{ "receive.fsck", "<msg-id>", list_config_fsck_msg_ids },
+		{ NULL, NULL, NULL }
+	};
+	const char **p;
+	struct slot_expansion *e;
+	struct string_list keys = STRING_LIST_INIT_DUP;
+	int i;
+
+	for (p = config_name_list; *p; p++) {
+		const char *var = *p;
+		struct strbuf sb = STRBUF_INIT;
+
+		for (e = slot_expansions; e->prefix; e++) {
+
+			strbuf_reset(&sb);
+			strbuf_addf(&sb, "%s.%s", e->prefix, e->placeholder);
+			if (!strcasecmp(var, sb.buf)) {
+				e->fn(&keys, e->prefix);
+				e->found++;
+				break;
+			}
+		}
+		strbuf_release(&sb);
+		if (!e->prefix)
+			string_list_append(&keys, var);
+	}
+
+	for (e = slot_expansions; e->prefix; e++)
+		if (!e->found)
+			BUG("slot_expansion %s.%s is not used",
+			    e->prefix, e->placeholder);
+
+	string_list_sort(&keys);
+	for (i = 0; i < keys.nr; i++) {
+		const char *var = keys.items[i].string;
+		const char *wildcard, *tag, *cut;
+
+		if (for_human) {
+			puts(var);
+			continue;
+		}
+
+		wildcard = strchr(var, '*');
+		tag = strchr(var, '<');
+
+		if (!wildcard && !tag) {
+			puts(var);
+			continue;
+		}
+
+		if (wildcard && !tag)
+			cut = wildcard;
+		else if (!wildcard && tag)
+			cut = tag;
+		else
+			cut = wildcard < tag ? wildcard : tag;
+
+		/*
+		 * We may produce duplicates, but that's up to
+		 * git-completion.bash to handle
+		 */
+		printf("%.*s\n", (int)(cut - var), var);
+	}
+	string_list_clear(&keys, 0);
+}
+
 static enum help_format parse_help_format(const char *format)
 {
 	if (!strcmp(format, "man"))
diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
index 71158f7d8b..45fecf8bdf 100755
--- a/generate-cmdlist.sh
+++ b/generate-cmdlist.sh
@@ -76,23 +76,6 @@ print_command_list () {
 	echo "};"
 }
 
-print_config_list () {
-	cat <<EOF
-static const char *config_name_list[] = {
-EOF
-	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
-	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
-	sort |
-	while read line
-	do
-		echo "	\"$line\","
-	done
-	cat <<EOF
-	NULL,
-};
-EOF
-}
-
 exclude_programs=
 while test "--exclude-program" = "$1"
 do
@@ -113,5 +96,3 @@ echo
 define_category_names "$1"
 echo
 print_command_list "$1"
-echo
-print_config_list
diff --git a/generate-configlist.sh b/generate-configlist.sh
new file mode 100755
index 0000000000..8692fe5cf4
--- /dev/null
+++ b/generate-configlist.sh
@@ -0,0 +1,21 @@
+#!/bin/sh
+
+echo "/* Automatically generated by generate-configlist.sh */"
+echo
+
+print_config_list () {
+	cat <<EOF
+static const char *config_name_list[] = {
+EOF
+	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
+	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
+	sort |
+	sed 's/^.*$/	"&",/'
+	cat <<EOF
+	NULL,
+};
+EOF
+}
+
+echo
+print_config_list
diff --git a/help.c b/help.c
index cf67624a94..a21487db77 100644
--- a/help.c
+++ b/help.c
@@ -407,91 +407,6 @@ void list_common_guides_help(void)
 	putchar('\n');
 }
 
-struct slot_expansion {
-	const char *prefix;
-	const char *placeholder;
-	void (*fn)(struct string_list *list, const char *prefix);
-	int found;
-};
-
-void list_config_help(int for_human)
-{
-	struct slot_expansion slot_expansions[] = {
-		{ "advice", "*", list_config_advices },
-		{ "color.branch", "<slot>", list_config_color_branch_slots },
-		{ "color.decorate", "<slot>", list_config_color_decorate_slots },
-		{ "color.diff", "<slot>", list_config_color_diff_slots },
-		{ "color.grep", "<slot>", list_config_color_grep_slots },
-		{ "color.interactive", "<slot>", list_config_color_interactive_slots },
-		{ "color.remote", "<slot>", list_config_color_sideband_slots },
-		{ "color.status", "<slot>", list_config_color_status_slots },
-		{ "fsck", "<msg-id>", list_config_fsck_msg_ids },
-		{ "receive.fsck", "<msg-id>", list_config_fsck_msg_ids },
-		{ NULL, NULL, NULL }
-	};
-	const char **p;
-	struct slot_expansion *e;
-	struct string_list keys = STRING_LIST_INIT_DUP;
-	int i;
-
-	for (p = config_name_list; *p; p++) {
-		const char *var = *p;
-		struct strbuf sb = STRBUF_INIT;
-
-		for (e = slot_expansions; e->prefix; e++) {
-
-			strbuf_reset(&sb);
-			strbuf_addf(&sb, "%s.%s", e->prefix, e->placeholder);
-			if (!strcasecmp(var, sb.buf)) {
-				e->fn(&keys, e->prefix);
-				e->found++;
-				break;
-			}
-		}
-		strbuf_release(&sb);
-		if (!e->prefix)
-			string_list_append(&keys, var);
-	}
-
-	for (e = slot_expansions; e->prefix; e++)
-		if (!e->found)
-			BUG("slot_expansion %s.%s is not used",
-			    e->prefix, e->placeholder);
-
-	string_list_sort(&keys);
-	for (i = 0; i < keys.nr; i++) {
-		const char *var = keys.items[i].string;
-		const char *wildcard, *tag, *cut;
-
-		if (for_human) {
-			puts(var);
-			continue;
-		}
-
-		wildcard = strchr(var, '*');
-		tag = strchr(var, '<');
-
-		if (!wildcard && !tag) {
-			puts(var);
-			continue;
-		}
-
-		if (wildcard && !tag)
-			cut = wildcard;
-		else if (!wildcard && tag)
-			cut = tag;
-		else
-			cut = wildcard < tag ? wildcard : tag;
-
-		/*
-		 * We may produce duplicates, but that's up to
-		 * git-completion.bash to handle
-		 */
-		printf("%.*s\n", (int)(cut - var), var);
-	}
-	string_list_clear(&keys, 0);
-}
-
 static int get_alias(const char *var, const char *value, void *data)
 {
 	struct string_list *list = data;
diff --git a/help.h b/help.h
index 7a455beeb7..9071894e8c 100644
--- a/help.h
+++ b/help.h
@@ -22,7 +22,6 @@ static inline void mput_char(char c, unsigned int num)
 void list_common_cmds_help(void);
 void list_all_cmds_help(void);
 void list_common_guides_help(void);
-void list_config_help(int for_human);
 
 void list_all_main_cmds(struct string_list *list);
 void list_all_other_cmds(struct string_list *list);
-- 
2.25.0.265.gbab2e86ba0-goog
Emily Shaffer· Feb 14, 2020, 01:53 UTC · re: Emily Shaffer · lore

[PATCH v7 04/15] bugreport: gather git version and build info

Knowing which version of Git a user has and how it was built allows us to more precisely pin down the circumstances when a certain issue occurs, so teach bugreport how to tell us the same output as 'git version --build-options'.

It's not ideal to directly call 'git version --build-options' because that output goes to stdout. Instead, wrap the version string in a helper within help.[ch] library, and call that helper from within the bugreport library.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  4 +++
 bugreport.c                     | 19 ++++++++++++-
 help.c                          | 48 ++++++++++++++++++++-------------
 help.h                          |  1 +
 4 files changed, 52 insertions(+), 20 deletions(-)
Show changes to 4 files +52 −20

Documentation/git-bugreport.txt, bugreport.c, help.c, help.h

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index 52d49ed7aa..6ce2ae2eff 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -23,6 +23,10 @@ The following information is requested from the user:
  - Expected behavior
  - Actual behavior
 
+The following information is captured automatically:
+
+ - 'git version --build-options'
+
 OPTIONS
 -------
 -o <path>::
diff --git a/bugreport.c b/bugreport.c
index a9398e6a2a..b7504fe322 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -1,8 +1,17 @@
-#include "builtin.h"
+#include "cache.h"
 #include "parse-options.h"
 #include "stdio.h"
 #include "strbuf.h"
 #include "time.h"
+#include "help.h"
+
+static void get_system_info(struct strbuf *sys_info)
+{
+	/* get git version from native cmd */
+	strbuf_addstr(sys_info, "git version:\n");
+	get_version_info(sys_info, 1);
+	strbuf_complete_line(sys_info);
+}
 
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
@@ -32,6 +41,11 @@ static int get_bug_template(struct strbuf *template)
 	return 0;
 }
 
+static void get_header(struct strbuf *buf, const char *title)
+{
+	strbuf_addf(buf, "\n\n[%s]\n", title);
+}
+
 int cmd_main(int argc, const char **argv)
 {
 	struct strbuf buffer = STRBUF_INIT;
@@ -76,6 +90,9 @@ int cmd_main(int argc, const char **argv)
 
 	get_bug_template(&buffer);
 
+	get_header(&buffer, "System Info");
+	get_system_info(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 
 	if (report == NULL) {
diff --git a/help.c b/help.c
index 190722fb0a..44cee69c11 100644
--- a/help.c
+++ b/help.c
@@ -622,8 +622,33 @@ const char *help_unknown_cmd(const char *cmd)
 	exit(1);
 }
 
+void get_version_info(struct strbuf *buf, int show_build_options)
+{
+	/*
+	 * The format of this string should be kept stable for compatibility
+	 * with external projects that rely on the output of "git version".
+	 *
+	 * Always show the version, even if other options are given.
+	 */
+	strbuf_addf(buf, "git version %s\n", git_version_string);
+
+	if (show_build_options) {
+		strbuf_addf(buf, "cpu: %s\n", GIT_HOST_CPU);
+		if (git_built_from_commit_string[0])
+			strbuf_addf(buf, "built from commit: %s\n",
+			       git_built_from_commit_string);
+		else
+			strbuf_addstr(buf, "no commit associated with this build\n");
+		strbuf_addf(buf, "sizeof-long: %d\n", (int)sizeof(long));
+		strbuf_addf(buf, "sizeof-size_t: %d\n", (int)sizeof(size_t));
+		strbuf_addf(buf, "shell-path: %s\n", SHELL_PATH);
+		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
+	}
+}
+
 int cmd_version(int argc, const char **argv, const char *prefix)
 {
+	struct strbuf buf = STRBUF_INIT;
 	int build_options = 0;
 	const char * const usage[] = {
 		N_("git version [<options>]"),
@@ -637,26 +662,11 @@ int cmd_version(int argc, const char **argv, const char *prefix)
 
 	argc = parse_options(argc, argv, prefix, options, usage, 0);
 
-	/*
-	 * The format of this string should be kept stable for compatibility
-	 * with external projects that rely on the output of "git version".
-	 *
-	 * Always show the version, even if other options are given.
-	 */
-	printf("git version %s\n", git_version_string);
+	get_version_info(&buf, build_options);
+	printf("%s", buf.buf);
+
+	strbuf_release(&buf);
 
-	if (build_options) {
-		printf("cpu: %s\n", GIT_HOST_CPU);
-		if (git_built_from_commit_string[0])
-			printf("built from commit: %s\n",
-			       git_built_from_commit_string);
-		else
-			printf("no commit associated with this build\n");
-		printf("sizeof-long: %d\n", (int)sizeof(long));
-		printf("sizeof-size_t: %d\n", (int)sizeof(size_t));
-		printf("shell-path: %s\n", SHELL_PATH);
-		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
-	}
 	return 0;
 }
 
diff --git a/help.h b/help.h
index 9071894e8c..500521b908 100644
--- a/help.h
+++ b/help.h
@@ -37,6 +37,7 @@ void add_cmdname(struct cmdnames *cmds, const char *name, int len);
 void exclude_cmds(struct cmdnames *cmds, struct cmdnames *excludes);
 int is_in_cmdlist(struct cmdnames *cmds, const char *name);
 void list_commands(unsigned int colopts, struct cmdnames *main_cmds, struct cmdnames *other_cmds);
+void get_version_info(struct strbuf *buf, int show_build_options);
 
 /*
  * call this to die(), when it is suspected that the user mistyped a
-- 
2.25.0.265.gbab2e86ba0-goog
Emily Shaffer· Feb 14, 2020, 01:53 UTC · re: Emily Shaffer · lore

[PATCH v7 03/15] bugreport: add tool to generate debugging info

Teach Git how to prompt the user for a good bug report: reproduction steps, expected behavior, and actual behavior. Later, Git can learn how to collect some diagnostic information from the repository.

If users can send us a well-written bug report which contains diagnostic information we would otherwise need to ask the user for, we can reduce the number of question-and-answer round trips between the reporter and the Git contributor.

Users may also wish to send a report like this to their local "Git expert" if they have put their repository into a state they are confused by.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 .gitignore                      |  1 +
 Documentation/git-bugreport.txt | 41 ++++++++++++++
 Makefile                        |  5 ++
 bugreport.c                     | 94 +++++++++++++++++++++++++++++++++
 command-list.txt                |  1 +
 t/t0091-bugreport.sh            | 55 +++++++++++++++++++
 6 files changed, 197 insertions(+)
 create mode 100644 Documentation/git-bugreport.txt
 create mode 100644 bugreport.c
 create mode 100755 t/t0091-bugreport.sh
Show changes to 6 files +197 −0

.gitignore, Documentation/git-bugreport.txt, Makefile, bugreport.c, command-list.txt, t/t0091-bugreport.sh

diff --git a/.gitignore b/.gitignore
index ea97de83f3..d89bf9e11e 100644
--- a/.gitignore
+++ b/.gitignore
@@ -25,6 +25,7 @@
 /git-bisect--helper
 /git-blame
 /git-branch
+/git-bugreport
 /git-bundle
 /git-cat-file
 /git-check-attr
diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
new file mode 100644
index 0000000000..52d49ed7aa
--- /dev/null
+++ b/Documentation/git-bugreport.txt
@@ -0,0 +1,41 @@
+git-bugreport(1)
+================
+
+NAME
+----
+git-bugreport - Collect information for user to file a bug report
+
+SYNOPSIS
+--------
+[verse]
+'git bugreport' [(-o | --output-directory) <path>] [(-s | --suffix) <format>]
+
+DESCRIPTION
+-----------
+Captures information about the user's machine, Git client, and repository state,
+as well as a form requesting information about the behavior the user observed,
+into a single text file which the user can then share, for example to the Git
+mailing list, in order to report an observed bug.
+
+The following information is requested from the user:
+
+ - Reproduction steps
+ - Expected behavior
+ - Actual behavior
+
+OPTIONS
+-------
+-o <path>::
+--output-directory <path>::
+	Place the resulting bug report file in `<path>` instead of the root of
+	the Git repository.
+
+-s <format>::
+--suffix <format>::
+	Specify an alternate suffix for the bugreport name, to create a file
+	named 'git-bugreport-<formatted suffix>'. This should take the form of a
+	link:strftime[3] format string; the current local time will be used.
+
+GIT
+---
+Part of the linkgit:git[1] suite
diff --git a/Makefile b/Makefile
index 5a022367d4..a01a050aa3 100644
--- a/Makefile
+++ b/Makefile
@@ -681,6 +681,7 @@ EXTRA_PROGRAMS =
 # ... and all the rest that could be moved out of bindir to gitexecdir
 PROGRAMS += $(EXTRA_PROGRAMS)
 
+PROGRAM_OBJS += bugreport.o
 PROGRAM_OBJS += credential-store.o
 PROGRAM_OBJS += daemon.o
 PROGRAM_OBJS += fast-import.o
@@ -2457,6 +2458,10 @@ endif
 git-%$X: %.o GIT-LDFLAGS $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) $(LIBS)
 
+git-bugreport$X: bugreport.o GIT-LDFLAGS $(GITLIBS)
+	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
+		$(LIBS)
+
 git-imap-send$X: imap-send.o $(IMAP_SEND_BUILDDEPS) GIT-LDFLAGS $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
 		$(IMAP_SEND_LDFLAGS) $(LIBS)
diff --git a/bugreport.c b/bugreport.c
new file mode 100644
index 0000000000..a9398e6a2a
--- /dev/null
+++ b/bugreport.c
@@ -0,0 +1,94 @@
+#include "builtin.h"
+#include "parse-options.h"
+#include "stdio.h"
+#include "strbuf.h"
+#include "time.h"
+
+static const char * const bugreport_usage[] = {
+	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
+	NULL
+};
+
+static int get_bug_template(struct strbuf *template)
+{
+	const char template_text[] = N_(
+"Thank you for filling out a Git bug report!\n"
+"Please answer the following questions to help us understand your issue.\n"
+"\n"
+"What did you do before the bug happened? (Steps to reproduce your issue)\n"
+"\n"
+"What did you expect to happen? (Expected behavior)\n"
+"\n"
+"What happened instead? (Actual behavior)\n"
+"\n"
+"What's different between what you expected and what actually happened?\n"
+"\n"
+"Anything else you want to add:\n"
+"\n"
+"Please review the rest of the bug report below.\n"
+"You can delete any lines you don't wish to share.\n");
+
+	strbuf_addstr(template, template_text);
+	return 0;
+}
+
+int cmd_main(int argc, const char **argv)
+{
+	struct strbuf buffer = STRBUF_INIT;
+	struct strbuf report_path = STRBUF_INIT;
+	FILE *report;
+	time_t now = time(NULL);
+	char *option_output = NULL;
+	char *option_suffix = "%F-%H%M";
+	struct stat statbuf;
+
+	const struct option bugreport_options[] = {
+		OPT_STRING('o', "output-directory", &option_output, N_("path"),
+			   N_("specify a destination for the bugreport file")),
+		OPT_STRING('s', "suffix", &option_suffix, N_("format"),
+			   N_("specify a strftime format suffix for the filename")),
+		OPT_END()
+	};
+	argc = parse_options(argc, argv, "", bugreport_options,
+			     bugreport_usage, 0);
+
+	if (option_output) {
+		strbuf_addstr(&report_path, option_output);
+		strbuf_complete(&report_path, '/');
+	}
+
+
+	strbuf_addstr(&report_path, "git-bugreport-");
+	strbuf_addftime(&report_path, option_suffix, localtime(&now), 0, 0);
+	strbuf_addstr(&report_path, ".txt");
+
+	if (!stat(report_path.buf, &statbuf))
+		die("'%s' already exists", report_path.buf);
+
+	switch (safe_create_leading_directories(report_path.buf)) {
+	case SCLD_OK:
+	case SCLD_EXISTS:
+		break;
+	default:
+		die(_("could not create leading directories for '%s'"),
+		    report_path.buf);
+	}
+
+	get_bug_template(&buffer);
+
+	report = fopen_for_writing(report_path.buf);
+
+	if (report == NULL) {
+		strbuf_release(&report_path);
+		die("couldn't open '%s' for writing", report_path.buf);
+	}
+
+	strbuf_write(&buffer, report);
+	fclose(report);
+
+	fprintf(stderr, _("Created new report at '%s'.\n"), report_path.buf);
+
+	UNLEAK(buffer);
+	UNLEAK(report_path);
+	return -launch_editor(report_path.buf, NULL, NULL);
+}
diff --git a/command-list.txt b/command-list.txt
index 2087894655..185e5e3f05 100644
--- a/command-list.txt
+++ b/command-list.txt
@@ -54,6 +54,7 @@ git-archive                             mainporcelain
 git-bisect                              mainporcelain           info
 git-blame                               ancillaryinterrogators          complete
 git-branch                              mainporcelain           history
+git-bugreport                           ancillaryinterrogators
 git-bundle                              mainporcelain
 git-cat-file                            plumbinginterrogators
 git-check-attr                          purehelpers
diff --git a/t/t0091-bugreport.sh b/t/t0091-bugreport.sh
new file mode 100755
index 0000000000..6585a2d144
--- /dev/null
+++ b/t/t0091-bugreport.sh
@@ -0,0 +1,55 @@
+#!/bin/sh
+
+test_description='git bugreport'
+
+. ./test-lib.sh
+
+# Headers "[System Info]" will be followed by a non-empty line if we put some
+# information there; we can make sure all our headers were followed by some
+# information to check if the command was successful.
+HEADER_PATTERN="^\[.*\]$"
+
+check_all_headers_populated () {
+	while read -r line
+	do
+		if test "$(grep "$HEADER_PATTERN" "$line")"
+		then
+			echo "$line"
+			read -r nextline
+			if test -z "$nextline"; then
+				return 1;
+			fi
+		fi
+	done
+}
+
+test_expect_success 'creates a report with content in the right places' '
+	git bugreport -s check-headers &&
+	check_all_headers_populated <git-bugreport-check-headers.txt &&
+	test_when_finished rm git-bugreport-check-headers.txt
+'
+
+test_expect_success 'dies if file with same name as report already exists' '
+	>>git-bugreport-duplicate.txt &&
+	test_must_fail git bugreport --suffix duplicate &&
+	test_when_finished rm git-bugreport-duplicate.txt
+'
+
+test_expect_success '--output-directory puts the report in the provided dir' '
+	git bugreport -o foo/ &&
+	test_path_is_file foo/git-bugreport-* &&
+	test_when_finished rm -fr foo/
+'
+
+test_expect_success 'incorrect arguments abort with usage' '
+	test_must_fail git bugreport --false 2>output &&
+	test_i18ngrep usage output &&
+	test_path_is_missing git-bugreport-*
+'
+
+test_expect_success 'runs outside of a git dir' '
+	nongit git bugreport &&
+	test_when_finished rm non-repo/git-bugreport-*
+'
+
+test_done
-- 
2.25.0.265.gbab2e86ba0-goog
Junio C Hamano· Feb 14, 2020, 17:25 UTC · re: Emily Shaffer · lore

Re: [PATCH v7 03/15] bugreport: add tool to generate debugging info

Emily Shaffer <emilyshaffer@google.com> writes:
> +	switch (safe_create_leading_directories(report_path.buf)) {

This helper is about creating paths in the working tree and Git repository, hence it has a call to adjust_shared_perm() which in turn calls get_shared_repo(), i.e. requiring a repository.

I thought I read somewhere that this tool is meant to be usable outside a repository? If that is not the case, then the use of this helper is OK. If not, we may want to make sure that it will stay to be safe to use the helper (I think it happens to be OK right now, but if the reason why the user is trying to run the tool is because the user broke Git by writing garbage into .git/config, we may die("your configuration file is broken") before this helper returns).

Thanks.
Emily Shaffer· Feb 15, 2020, 01:57 UTC · re: Junio C Hamano · lore

Re: [PATCH v7 03/15] bugreport: add tool to generate debugging info

On Fri, Feb 14, 2020 at 09:25:12AM -0800, Junio C Hamano wrote:
Show 6 quoted lines
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> > +	switch (safe_create_leading_directories(report_path.buf)) {
> 
> This helper is about creating paths in the working tree and Git
> repository,

It's also used by cmd_format_patch() with --output-directory specified, which is how I found it. My usual workflow is to run format-patch with -o ~/mailed-patches/topic/ specified so I don't clutter my repo, so I'm surprised to hear the intent of safe_create_leading_directores() is for paths in the working tree or repo.

> hence it has a call to adjust_shared_perm() which in
> turn calls get_shared_repo(), i.e. requiring a repository.
Hmmm. I was able to run it pretty happily from a nongit dir:
  emilyshaffer@podkayne:~$ tg bugreport -o other/very/deep/path/
  Created new report at 'other/very/deep/path/git-bugreport-2020-02-14-1721.txt'.
  emilyshaffer@podkayne:~$ cat other/very/deep/path/git-bugreport-2020-02-14-1721.txt
  Thank you for filling out a Git bug report!
Show 7 quoted lines
> I thought I read somewhere that this tool is meant to be usable
> outside a repository?  If that is not the case, then the use of this
> helper is OK.  If not, we may want to make sure that it will stay to
> be safe to use the helper (I think it happens to be OK right now,
> but if the reason why the user is trying to run the tool is because
> the user broke Git by writing garbage into .git/config, we may
> die("your configuration file is broken") before this helper returns).

Can you explain a little more about what you mean? A broken local .git/config seems to be preventing git.c from dispatching the 'bugreport' subcommand (lots of other Git commands are broken) at all - breakpoints in cmd_main() right after the variable declarations are not being hit.

With junk in the .git/config, I can't run bugreport from within that repo. With junk in ~/.gitconfig, I can't run bugreport anywhere.

(To make the configs invalid, I leaned on my keyboard and added a line 'garbaghe~*~$%)%)(@' to the bottom of the config file. Most commands terminate early with "fatal: bad config line 37 in file .git/config".)

Do you mean there's some specific config that could be misconfigured and prevent that utility from working?

 - Emily
Junio C Hamano· Feb 15, 2020, 18:24 UTC · re: Emily Shaffer · lore

Re: [PATCH v7 03/15] bugreport: add tool to generate debugging info

Emily Shaffer <emilyshaffer@google.com> writes:
Show 10 quoted lines
> On Fri, Feb 14, 2020 at 09:25:12AM -0800, Junio C Hamano wrote:
>> Emily Shaffer <emilyshaffer@google.com> writes:
>> 
>> > +	switch (safe_create_leading_directories(report_path.buf)) {
>> 
>> This helper is about creating paths in the working tree and Git
>> repository,
>
> It's also used by cmd_format_patch() with --output-directory specified,
> which is how I found it.

And that is an example of a good use of this helper. What will be written out there should be visible by the same people as those who have access to the repository; get_shared_repo() and adjust_perm() based on what the repository you are taking patches from is perfectly sensible. And as it is format-patch, we know we have get_shared_repo() working, and we know which repository we are working on.

Output directory for bugreport is on the same boat when we know the users are producing a report on their use of Git within a repository. It is not clear if the tool is started without any repository---it happens to do a random safe thing (i.e. I think get_shared_repo() gives PERM_UMASK, which tells adjust_perm() not to do anything especial) right now, but there is no guarantee that we will keep it working like that. Somebody may come and demaind get_shared_perm() to die() when outside a repository, for example, and that would break the nice property that lets bugreport working outside a repository.

I just wanted to make sure that somebody will be keeping an eye to remind those who propose such a change in the future. A comment near where get_shared_repo() happens may be something that can be done with a minimum effort when code that depends on that property is introduced at the same time.

>> I thought I read somewhere that this tool is meant to be usable
>> outside a repository?  If that is not the case, then the use of this
>> helper is OK.  If not, we may want to make sure that it will stay to
>> be safe to use the helper (I think it happens to be OK right now,

I am actually OK if we limit the use of this tool to "use with a repository that is not corrupt", as coping with these kinds of breakages that in the main Git executable we deem "needs manual intervention" inside a single process is too painful (it would have been easier to cope with these too if we stuck with a script that invokes many discrete commands and acts on their errors, but that is optimizing for rare case and not recommended). But we should tell users about the limitation and encourage them to ask for help in non automatable means.

Thanks.
Emily Shaffer· Feb 18, 2020, 23:46 UTC · re: Junio C Hamano · lore

Re: [PATCH v7 03/15] bugreport: add tool to generate debugging info

On Sat, Feb 15, 2020 at 10:24:40AM -0800, Junio C Hamano wrote:
Show 31 quoted lines
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> > On Fri, Feb 14, 2020 at 09:25:12AM -0800, Junio C Hamano wrote:
> >> Emily Shaffer <emilyshaffer@google.com> writes:
> >> 
> >> > +	switch (safe_create_leading_directories(report_path.buf)) {
> >> 
> >> This helper is about creating paths in the working tree and Git
> >> repository,
> >
> > It's also used by cmd_format_patch() with --output-directory specified,
> > which is how I found it.
> 
> And that is an example of a good use of this helper.  What will be
> written out there should be visible by the same people as those who
> have access to the repository; get_shared_repo() and adjust_perm()
> based on what the repository you are taking patches from is
> perfectly sensible.  And as it is format-patch, we know we have
> get_shared_repo() working, and we know which repository we are
> working on.
> 
> Output directory for bugreport is on the same boat when we know the
> users are producing a report on their use of Git within a
> repository.  It is not clear if the tool is started without any
> repository---it happens to do a random safe thing (i.e. I think
> get_shared_repo() gives PERM_UMASK, which tells adjust_perm() not to
> do anything especial) right now, but there is no guarantee that we
> will keep it working like that.  Somebody may come and demaind
> get_shared_perm() to die() when outside a repository, for example,
> and that would break the nice property that lets bugreport working
> outside a repository.

Hm, this would break the convention of the name of safe_create_leading_directories() too though right? As I understood it, "safe_foo" in this codebase means "this function will not die()"?

Show 6 quoted lines
> 
> I just wanted to make sure that somebody will be keeping an eye to
> remind those who propose such a change in the future.  A comment
> near where get_shared_repo() happens may be something that can be
> done with a minimum effort when code that depends on that property
> is introduced at the same time.

Ok. I want to make sure I understand you right. I think you're saying, "This is OK, except if someone changes get_shared_repo() it could break, so we need a way to warn someone that it could break." It sounds like you suggested leaving a comment in the safe_create_leading_directories() helper. My own preference is to write a test so that it's explicit that we depend on that behavior, instead - it's easy to gloss over a comment or read a different part of the codebase, but it's hard to ignore a breaking test. It'd be trivial to add one, so I will in v8 - unless I misunderstood you.

Show 15 quoted lines
> 
> >> I thought I read somewhere that this tool is meant to be usable
> >> outside a repository?  If that is not the case, then the use of this
> >> helper is OK.  If not, we may want to make sure that it will stay to
> >> be safe to use the helper (I think it happens to be OK right now,
> 
> I am actually OK if we limit the use of this tool to "use with a
> repository that is not corrupt", as coping with these kinds of
> breakages that in the main Git executable we deem "needs manual
> intervention" inside a single process is too painful (it would have
> been easier to cope with these too if we stuck with a script that
> invokes many discrete commands and acts on their errors, but that is
> optimizing for rare case and not recommended).  But we should tell
> users about the limitation and encourage them to ask for help in non
> automatable means.

I think you're saying, "Mention this drawback in the manpage for git-bugreport." Sounds like a good idea to me, so I'll add it for v8.

 - Emily
Emily Shaffer· Feb 18, 2020, 23:56 UTC · re: Emily Shaffer · lore

Re: [PATCH v7 03/15] bugreport: add tool to generate debugging info

On Tue, Feb 18, 2020 at 03:46:28PM -0800, Emily Shaffer wrote:
Show 13 quoted lines
> On Sat, Feb 15, 2020 at 10:24:40AM -0800, Junio C Hamano wrote:
> > I am actually OK if we limit the use of this tool to "use with a
> > repository that is not corrupt", as coping with these kinds of
> > breakages that in the main Git executable we deem "needs manual
> > intervention" inside a single process is too painful (it would have
> > been easier to cope with these too if we stuck with a script that
> > invokes many discrete commands and acts on their errors, but that is
> > optimizing for rare case and not recommended).  But we should tell
> > users about the limitation and encourage them to ask for help in non
> > automatable means.
> 
> I think you're saying, "Mention this drawback in the manpage for
> git-bugreport." Sounds like a good idea to me, so I'll add it for v8.

I'm pretty unsure about how you wanted this to sound; rather than sending a v8 only to revise it a lot, I'll send the paragraph for wordsmithing beforehand. Is this what you meant?

  This tool is invoked via the typical Git setup process, which means that in some
  cases, it might not be able to launch - for example, if a relevant config file
  is unreadable. In this kind of scenario, it may be helpful to manually gather
  the kind of information listed above when manually asking for help.
 - Emily
Emily Shaffer· Feb 19, 2020, 23:15 UTC · re: Emily Shaffer · lore

Re: [PATCH v7 03/15] bugreport: add tool to generate debugging info

On Tue, Feb 18, 2020 at 03:56:22PM -0800, Emily Shaffer wrote:
Show 23 quoted lines
> On Tue, Feb 18, 2020 at 03:46:28PM -0800, Emily Shaffer wrote:
> > On Sat, Feb 15, 2020 at 10:24:40AM -0800, Junio C Hamano wrote:
> > > I am actually OK if we limit the use of this tool to "use with a
> > > repository that is not corrupt", as coping with these kinds of
> > > breakages that in the main Git executable we deem "needs manual
> > > intervention" inside a single process is too painful (it would have
> > > been easier to cope with these too if we stuck with a script that
> > > invokes many discrete commands and acts on their errors, but that is
> > > optimizing for rare case and not recommended).  But we should tell
> > > users about the limitation and encourage them to ask for help in non
> > > automatable means.
> > 
> > I think you're saying, "Mention this drawback in the manpage for
> > git-bugreport." Sounds like a good idea to me, so I'll add it for v8.
> 
> I'm pretty unsure about how you wanted this to sound; rather than
> sending a v8 only to revise it a lot, I'll send the paragraph for
> wordsmithing beforehand. Is this what you meant?
> 
>   This tool is invoked via the typical Git setup process, which means that in some
>   cases, it might not be able to launch - for example, if a relevant config file
>   is unreadable. In this kind of scenario, it may be helpful to manually gather
>   the kind of information listed above when manually asking for help.

Since I saw lots of replies from Junio elsewhere in this thread, I'll take the silence here as assent. v8 on its way momentarily.

 - Emily
Junio C Hamano· Feb 19, 2020, 23:24 UTC · re: Emily Shaffer · lore

Re: [PATCH v7 03/15] bugreport: add tool to generate debugging info

Emily Shaffer <emilyshaffer@google.com> writes:
Show 27 quoted lines
> On Tue, Feb 18, 2020 at 03:56:22PM -0800, Emily Shaffer wrote:
>> On Tue, Feb 18, 2020 at 03:46:28PM -0800, Emily Shaffer wrote:
>> > On Sat, Feb 15, 2020 at 10:24:40AM -0800, Junio C Hamano wrote:
>> > > I am actually OK if we limit the use of this tool to "use with a
>> > > repository that is not corrupt", as coping with these kinds of
>> > > breakages that in the main Git executable we deem "needs manual
>> > > intervention" inside a single process is too painful (it would have
>> > > been easier to cope with these too if we stuck with a script that
>> > > invokes many discrete commands and acts on their errors, but that is
>> > > optimizing for rare case and not recommended).  But we should tell
>> > > users about the limitation and encourage them to ask for help in non
>> > > automatable means.
>> > 
>> > I think you're saying, "Mention this drawback in the manpage for
>> > git-bugreport." Sounds like a good idea to me, so I'll add it for v8.
>> 
>> I'm pretty unsure about how you wanted this to sound; rather than
>> sending a v8 only to revise it a lot, I'll send the paragraph for
>> wordsmithing beforehand. Is this what you meant?
>> 
>>   This tool is invoked via the typical Git setup process, which means that in some
>>   cases, it might not be able to launch - for example, if a relevant config file
>>   is unreadable. In this kind of scenario, it may be helpful to manually gather
>>   the kind of information listed above when manually asking for help.
>
> Since I saw lots of replies from Junio elsewhere in this thread, I'll
> take the silence here as assent. v8 on its way momentarily.

Heh, it's just I have too little time to allocate on this single topic X-<. Don't take silence as anything else.

Johannes Schindelin· Feb 19, 2020, 14:18 UTC · re: Emily Shaffer · lore

Re: [PATCH v7 03/15] bugreport: add tool to generate debugging info

Hi Emily,
On Thu, 13 Feb 2020, Emily Shaffer wrote:
Show 99 quoted lines
> diff --git a/bugreport.c b/bugreport.c
> new file mode 100644
> index 0000000000..a9398e6a2a
> --- /dev/null
> +++ b/bugreport.c
> @@ -0,0 +1,94 @@
> +#include "builtin.h"
> +#include "parse-options.h"
> +#include "stdio.h"
> +#include "strbuf.h"
> +#include "time.h"
> +
> +static const char * const bugreport_usage[] = {
> +	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
> +	NULL
> +};
> +
> +static int get_bug_template(struct strbuf *template)
> +{
> +	const char template_text[] = N_(
> +"Thank you for filling out a Git bug report!\n"
> +"Please answer the following questions to help us understand your issue.\n"
> +"\n"
> +"What did you do before the bug happened? (Steps to reproduce your issue)\n"
> +"\n"
> +"What did you expect to happen? (Expected behavior)\n"
> +"\n"
> +"What happened instead? (Actual behavior)\n"
> +"\n"
> +"What's different between what you expected and what actually happened?\n"
> +"\n"
> +"Anything else you want to add:\n"
> +"\n"
> +"Please review the rest of the bug report below.\n"
> +"You can delete any lines you don't wish to share.\n");
> +
> +	strbuf_addstr(template, template_text);
> +	return 0;
> +}
> +
> +int cmd_main(int argc, const char **argv)
> +{
> +	struct strbuf buffer = STRBUF_INIT;
> +	struct strbuf report_path = STRBUF_INIT;
> +	FILE *report;
> +	time_t now = time(NULL);
> +	char *option_output = NULL;
> +	char *option_suffix = "%F-%H%M";
> +	struct stat statbuf;
> +
> +	const struct option bugreport_options[] = {
> +		OPT_STRING('o', "output-directory", &option_output, N_("path"),
> +			   N_("specify a destination for the bugreport file")),
> +		OPT_STRING('s', "suffix", &option_suffix, N_("format"),
> +			   N_("specify a strftime format suffix for the filename")),
> +		OPT_END()
> +	};
> +	argc = parse_options(argc, argv, "", bugreport_options,
> +			     bugreport_usage, 0);
> +
> +	if (option_output) {
> +		strbuf_addstr(&report_path, option_output);
> +		strbuf_complete(&report_path, '/');
> +	}
> +
> +
> +	strbuf_addstr(&report_path, "git-bugreport-");
> +	strbuf_addftime(&report_path, option_suffix, localtime(&now), 0, 0);
> +	strbuf_addstr(&report_path, ".txt");
> +
> +	if (!stat(report_path.buf, &statbuf))
> +		die("'%s' already exists", report_path.buf);
> +
> +	switch (safe_create_leading_directories(report_path.buf)) {
> +	case SCLD_OK:
> +	case SCLD_EXISTS:
> +		break;
> +	default:
> +		die(_("could not create leading directories for '%s'"),
> +		    report_path.buf);
> +	}
> +
> +	get_bug_template(&buffer);
> +
> +	report = fopen_for_writing(report_path.buf);
> +
> +	if (report == NULL) {
> +		strbuf_release(&report_path);
> +		die("couldn't open '%s' for writing", report_path.buf);
> +	}
> +
> +	strbuf_write(&buffer, report);
> +	fclose(report);
> +
> +	fprintf(stderr, _("Created new report at '%s'.\n"), report_path.buf);
> +
> +	UNLEAK(buffer);
> +	UNLEAK(report_path);
> +	return -launch_editor(report_path.buf, NULL, NULL);

This would be the first time (at least that _I_ know of) that we use `-` in this way. We seem to use `!!` a lot more often. And now I wonder whether there is a reason for that `-` that I missed?

Ciao, Dscho

Junio C Hamano· Feb 19, 2020, 16:55 UTC · re: Johannes Schindelin · lore

Re: [PATCH v7 03/15] bugreport: add tool to generate debugging info

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
Show 20 quoted lines
>> +int cmd_main(int argc, const char **argv)
>> +{
>> +...
>> +	if (report == NULL) {
>> +		strbuf_release(&report_path);
>> +		die("couldn't open '%s' for writing", report_path.buf);
>> +	}
>> +
>> +	strbuf_write(&buffer, report);
>> +	fclose(report);
>> +
>> +	fprintf(stderr, _("Created new report at '%s'.\n"), report_path.buf);
>> +
>> +	UNLEAK(buffer);
>> +	UNLEAK(report_path);
>> +	return -launch_editor(report_path.buf, NULL, NULL);
>
> This would be the first time (at least that _I_ know of) that we use `-`
> in this way. We seem to use `!!` a lot more often. And now I wonder
> whether there is a reason for that `-` that I missed?

In general, our preferred way to report an error from API functions is to return a negative number and a success is reported by returning zero.

The argument to exit(3), which the return value from the main() function essentially is, on the other hand, is expected to be a small non-negative integer. As long as we are in tight control of the range of the returned value from launch_editor() (i.e. it must return a small non-positive integer whose negation is suitable to be fed to exit(3)), the above is fine.

The idiom "return !!fn();" is to canonicalize the value to 0 or 1 for the caller who asked "so, did fn() give us 0 or non-zero?", which can also be used as the value given to exit(3), and could be more appropriate under some conditions:

 - we MUST know launch_editor() returns zero if and only if it is
   successful.
 - we do not have to be confident to be in tight control of the
   range of the returned value from launch_editor() (e.g. it could
   return a positive upon an error).
 - we MUST NOT care to differenciate different error codes returned
   from launch_editor().  IOW, we must be fine to give the invoker
   of the program only 0 (success) or 1 (unspecified failure).

Use of "return !!fn();" that is not to give to exit(3) is probably more common in our codebase. See apply.c::try_create_file() and the comment before the function about its possible return values for example.

Emily Shaffer· Feb 19, 2020, 21:52 UTC · re: Junio C Hamano · lore

Re: [PATCH v7 03/15] bugreport: add tool to generate debugging info

On Wed, Feb 19, 2020 at 08:55:04AM -0800, Junio C Hamano wrote:
Show 38 quoted lines
> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
> >> +int cmd_main(int argc, const char **argv)
> >> +{
> >> +...
> >> +	if (report == NULL) {
> >> +		strbuf_release(&report_path);
> >> +		die("couldn't open '%s' for writing", report_path.buf);
> >> +	}
> >> +
> >> +	strbuf_write(&buffer, report);
> >> +	fclose(report);
> >> +
> >> +	fprintf(stderr, _("Created new report at '%s'.\n"), report_path.buf);
> >> +
> >> +	UNLEAK(buffer);
> >> +	UNLEAK(report_path);
> >> +	return -launch_editor(report_path.buf, NULL, NULL);
> >
> > This would be the first time (at least that _I_ know of) that we use `-`
> > in this way. We seem to use `!!` a lot more often. And now I wonder
> > whether there is a reason for that `-` that I missed?
> 
> In general, our preferred way to report an error from API functions
> is to return a negative number and a success is reported by
> returning zero.
> 
> The argument to exit(3), which the return value from the main()
> function essentially is, on the other hand, is expected to be a
> small non-negative integer.  As long as we are in tight control of
> the range of the returned value from launch_editor() (i.e. it must
> return a small non-positive integer whose negation is suitable to be
> fed to exit(3)), the above is fine.
> 
> The idiom "return !!fn();" is to canonicalize the value to 0 or 1
> for the caller who asked "so, did fn() give us 0 or non-zero?",
> which can also be used as the value given to exit(3), and could be
> more appropriate under some conditions:

In editor.c, launch_editor() returns launch_specified_editor() without altering the return code.

> 
>  - we MUST know launch_editor() returns zero if and only if it is
>    successful.
launch_specified_editor() has a handful of exit points, of three kinds:
 1. return error(something)
 2. raise(sigsomething)
 3. return 0
    a. when the editor process closed happily, but the user supplied
       NULL instead of a buffer. That is, the user didn't want the
       contents of the editor given back to them in a strbuf.
    b. when the editor process closed happily and the user's supplied
       buffer was filled with the file's contents with no issue.
So I think we can check "yes" here.
> 
>  - we do not have to be confident to be in tight control of the
>    range of the returned value from launch_editor() (e.g. it could
>    return a positive upon an error).

According to 'man 3 raise' (POSIX), raise() exits the current thread. If it can't exit itself(?) it seems it returns "nonzero for failure". We've also got a mingw_raise() in compat/mingw.h which could be used instead; this one seems to return -1 or the result of raise(), presumably the one from the MSC runtime[1], which also returns "a nonzero value" if not successful.

So it's true that we aren't confident that launch_editor() returns a positive or negative value.

>  - we MUST NOT care to differenciate different error codes returned
>    from launch_editor().  IOW, we must be fine to give the invoker
>    of the program only 0 (success) or 1 (unspecified failure).

This part is a little sticking point for me; I think I'd rather give the user some hint of what's broken than no hint at all, especially in this scenario, where it's conceivable someone could say "I tried to run 'git-bugreport' and it crashed, here is the return code and I have manually collected some relevant info too".

The uncertainty coming from raise() (POSIX and MSCR both) leads me to believe if I do want to return the error on exit, I should at least check the sign before I flip it. Anybody else have opinions?

 - Emily
[1] https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/raise?view=vs-2019
Junio C Hamano· Feb 19, 2020, 22:09 UTC · re: Emily Shaffer · lore

Re: [PATCH v7 03/15] bugreport: add tool to generate debugging info

Emily Shaffer <emilyshaffer@google.com> writes:
Show 11 quoted lines
> launch_specified_editor() has a handful of exit points, of three kinds:
>  1. return error(something)
>  2. raise(sigsomething)
>  3. return 0
>     a. when the editor process closed happily, but the user supplied
>        NULL instead of a buffer. That is, the user didn't want the
>        contents of the editor given back to them in a strbuf.
>     b. when the editor process closed happily and the user's supplied
>        buffer was filled with the file's contents with no issue.
>
> So I think we can check "yes" here.

Heh. If we raised a signal to kill ourselves, then we won't be returning a value from launch_editor() anyway. That case won't affect the "between returning negation or !!, which is more appropriate?" discussion, I think.

>>  - we MUST NOT care to differenciate different error codes returned
>>    from launch_editor().  IOW, we must be fine to give the invoker
>>    of the program only 0 (success) or 1 (unspecified failure).

I actually think this holds for the codepath. A failure from start_command() returns error(), and finish_command() that waits for the spawned editor process to complete yields the exit status from the editor, but unless we re-raise the signal that killed the editor process to ourselves, we just turn any non-zero exit into "return error()", so it is safe to say launch_editor() can return either 0 or -1 and nothing else. Would we later want to tell callers of launch_editor() how/why the editor session failed by returning something else? I do not offhand think of any---we do not even differenciate between failure to start (e.g. misspelt command name for the editor) and other failures WITH the return value and consider it sufficient to tell the user with different error message right now.

So in practice returning -launch_editor() and !!launch_editor() would not make any difference, I would think.

Emily Shaffer· Feb 19, 2020, 23:06 UTC · re: Junio C Hamano · lore

Re: [PATCH v7 03/15] bugreport: add tool to generate debugging info

On Wed, Feb 19, 2020 at 02:09:48PM -0800, Junio C Hamano wrote:
Show 39 quoted lines
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> > launch_specified_editor() has a handful of exit points, of three kinds:
> >  1. return error(something)
> >  2. raise(sigsomething)
> >  3. return 0
> >     a. when the editor process closed happily, but the user supplied
> >        NULL instead of a buffer. That is, the user didn't want the
> >        contents of the editor given back to them in a strbuf.
> >     b. when the editor process closed happily and the user's supplied
> >        buffer was filled with the file's contents with no issue.
> >
> > So I think we can check "yes" here.
> 
> Heh.  If we raised a signal to kill ourselves, then we won't be
> returning a value from launch_editor() anyway.  That case won't
> affect the "between returning negation or !!, which is more
> appropriate?" discussion, I think.
> 
> >>  - we MUST NOT care to differenciate different error codes returned
> >>    from launch_editor().  IOW, we must be fine to give the invoker
> >>    of the program only 0 (success) or 1 (unspecified failure).
> 
> I actually think this holds for the codepath.  A failure from
> start_command() returns error(), and finish_command() that waits for
> the spawned editor process to complete yields the exit status from
> the editor, but unless we re-raise the signal that killed the editor
> process to ourselves, we just turn any non-zero exit into "return
> error()", so it is safe to say launch_editor() can return either 0
> or -1 and nothing else.  Would we later want to tell callers of
> launch_editor() how/why the editor session failed by returning
> something else?  I do not offhand think of any---we do not even
> differenciate between failure to start (e.g. misspelt command name
> for the editor) and other failures WITH the return value and
> consider it sufficient to tell the user with different error
> message right now.
> 
> So in practice returning -launch_editor() and !!launch_editor()
> would not make any difference, I would think.

Then, let's do the least surprising thing. I'll switch it to !! for the next reroll.

Emily Shaffer· Feb 14, 2020, 01:53 UTC · re: Emily Shaffer · lore

[PATCH v7 05/15] bugreport: add uname info

The contents of uname() can give us some insight into what sort of system the user is running on, and help us replicate their setup if need be. The domainname field is not guaranteed to be available, so don't collect it.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 13 +++++++++++++
 2 files changed, 14 insertions(+)
Show changes to 2 files +14 −0

Documentation/git-bugreport.txt, bugreport.c

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index 6ce2ae2eff..4dd72c60f5 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -26,6 +26,7 @@ The following information is requested from the user:
 The following information is captured automatically:
 
  - 'git version --build-options'
+ - uname sysname, release, version, and machine strings
 
 OPTIONS
 -------
diff --git a/bugreport.c b/bugreport.c
index b7504fe322..b76a1dfb2a 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -7,10 +7,23 @@
 
 static void get_system_info(struct strbuf *sys_info)
 {
+	struct utsname uname_info;
+
 	/* get git version from native cmd */
 	strbuf_addstr(sys_info, "git version:\n");
 	get_version_info(sys_info, 1);
 	strbuf_complete_line(sys_info);
+
+	/* system call for other version info */
+	strbuf_addstr(sys_info, "uname -a: ");
+	if (uname(&uname_info))
+		strbuf_addf(sys_info, "uname() failed with code %d\n", errno);
+	else
+		strbuf_addf(sys_info, "%s %s %s %s\n",
+			    uname_info.sysname,
+			    uname_info.release,
+			    uname_info.version,
+			    uname_info.machine);
 }
 
 static const char * const bugreport_usage[] = {
-- 
2.25.0.265.gbab2e86ba0-goog
Emily Shaffer· Feb 14, 2020, 01:53 UTC · re: Emily Shaffer · lore

[PATCH v7 06/15] bugreport: add compiler info

To help pinpoint the source of a regression, it is useful to know some info about the compiler which the user's Git client was built with. By adding a generic get_compiler_info() in 'compat/' we can choose which relevant information to share per compiler; to get started, let's demonstrate the version of glibc if the user built with 'gcc'.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     |  5 +++++
 compat/compiler.h               | 24 ++++++++++++++++++++++++
 3 files changed, 30 insertions(+)
 create mode 100644 compat/compiler.h
Show changes to 3 files +30 −0

Documentation/git-bugreport.txt, bugreport.c, compat/compiler.h

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index 4dd72c60f5..8bbc4c960c 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -27,6 +27,7 @@ The following information is captured automatically:
 
  - 'git version --build-options'
  - uname sysname, release, version, and machine strings
+ - Compiler-specific info string
 
 OPTIONS
 -------
diff --git a/bugreport.c b/bugreport.c
index b76a1dfb2a..4f9101caeb 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -4,6 +4,7 @@
 #include "strbuf.h"
 #include "time.h"
 #include "help.h"
+#include "compat/compiler.h"
 
 static void get_system_info(struct strbuf *sys_info)
 {
@@ -24,6 +25,10 @@ static void get_system_info(struct strbuf *sys_info)
 			    uname_info.release,
 			    uname_info.version,
 			    uname_info.machine);
+
+	strbuf_addstr(sys_info, "compiler info: ");
+	get_compiler_info(sys_info);
+	strbuf_complete_line(sys_info);
 }
 
 static const char * const bugreport_usage[] = {
diff --git a/compat/compiler.h b/compat/compiler.h
new file mode 100644
index 0000000000..bda5098e1b
--- /dev/null
+++ b/compat/compiler.h
@@ -0,0 +1,24 @@
+#ifndef COMPILER_H
+#define COMPILER_H
+
+#include "git-compat-util.h"
+#include "strbuf.h"
+
+#ifdef __GLIBC__
+#include <gnu/libc-version.h>
+
+static inline void get_compiler_info(struct strbuf *info)
+{
+	strbuf_addf(info, "glibc: %s", gnu_get_libc_version());
+}
+
+#else
+
+static inline void get_compiler_info(struct strbuf *info)
+{
+	strbuf_addstr(info, "get_compiler_info() not implemented");
+}
+
+#endif
+
+#endif /* COMPILER_H */
-- 
2.25.0.265.gbab2e86ba0-goog
Johannes Schindelin· Feb 19, 2020, 14:23 UTC · re: Emily Shaffer · lore

Re: [PATCH v7 06/15] bugreport: add compiler info

Hi Emily,
On Thu, 13 Feb 2020, Emily Shaffer wrote:
Show 5 quoted lines
> To help pinpoint the source of a regression, it is useful to know some
> info about the compiler which the user's Git client was built with. By
> adding a generic get_compiler_info() in 'compat/' we can choose which
> relevant information to share per compiler; to get started, let's
> demonstrate the version of glibc if the user built with 'gcc'.

I agree with the need for the compiler information, but in the patch I only see information about glibc being printed out. Shouldn't we use `__GNUC__` and `__GNUC_MINOR__` here?

Don't get me wrong, the glibc version is good and all, but the compiler information might be even more crucial. Git for Windows had to hold back compiling with GCC v8.x for a while, for example, because the stacksmasher was broken. Similar issues are not unheard of, and could help pinpoint compiler-related problems a lot quicker.

Thanks, Dscho

Show 78 quoted lines
>
> Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
> ---
>  Documentation/git-bugreport.txt |  1 +
>  bugreport.c                     |  5 +++++
>  compat/compiler.h               | 24 ++++++++++++++++++++++++
>  3 files changed, 30 insertions(+)
>  create mode 100644 compat/compiler.h
>
> diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
> index 4dd72c60f5..8bbc4c960c 100644
> --- a/Documentation/git-bugreport.txt
> +++ b/Documentation/git-bugreport.txt
> @@ -27,6 +27,7 @@ The following information is captured automatically:
>
>   - 'git version --build-options'
>   - uname sysname, release, version, and machine strings
> + - Compiler-specific info string
>
>  OPTIONS
>  -------
> diff --git a/bugreport.c b/bugreport.c
> index b76a1dfb2a..4f9101caeb 100644
> --- a/bugreport.c
> +++ b/bugreport.c
> @@ -4,6 +4,7 @@
>  #include "strbuf.h"
>  #include "time.h"
>  #include "help.h"
> +#include "compat/compiler.h"
>
>  static void get_system_info(struct strbuf *sys_info)
>  {
> @@ -24,6 +25,10 @@ static void get_system_info(struct strbuf *sys_info)
>  			    uname_info.release,
>  			    uname_info.version,
>  			    uname_info.machine);
> +
> +	strbuf_addstr(sys_info, "compiler info: ");
> +	get_compiler_info(sys_info);
> +	strbuf_complete_line(sys_info);
>  }
>
>  static const char * const bugreport_usage[] = {
> diff --git a/compat/compiler.h b/compat/compiler.h
> new file mode 100644
> index 0000000000..bda5098e1b
> --- /dev/null
> +++ b/compat/compiler.h
> @@ -0,0 +1,24 @@
> +#ifndef COMPILER_H
> +#define COMPILER_H
> +
> +#include "git-compat-util.h"
> +#include "strbuf.h"
> +
> +#ifdef __GLIBC__
> +#include <gnu/libc-version.h>
> +
> +static inline void get_compiler_info(struct strbuf *info)
> +{
> +	strbuf_addf(info, "glibc: %s", gnu_get_libc_version());
> +}
> +
> +#else
> +
> +static inline void get_compiler_info(struct strbuf *info)
> +{
> +	strbuf_addstr(info, "get_compiler_info() not implemented");
> +}
> +
> +#endif
> +
> +#endif /* COMPILER_H */
> --
> 2.25.0.265.gbab2e86ba0-goog
>
>
Emily Shaffer· Feb 19, 2020, 22:45 UTC · re: Johannes Schindelin · lore

Re: [PATCH v7 06/15] bugreport: add compiler info

On Wed, Feb 19, 2020 at 03:23:34PM +0100, Johannes Schindelin wrote:
Show 19 quoted lines
> Hi Emily,
> 
> On Thu, 13 Feb 2020, Emily Shaffer wrote:
> 
> > To help pinpoint the source of a regression, it is useful to know some
> > info about the compiler which the user's Git client was built with. By
> > adding a generic get_compiler_info() in 'compat/' we can choose which
> > relevant information to share per compiler; to get started, let's
> > demonstrate the version of glibc if the user built with 'gcc'.
> 
> I agree with the need for the compiler information, but in the patch I
> only see information about glibc being printed out. Shouldn't we use
> `__GNUC__` and `__GNUC_MINOR__` here?
> 
> Don't get me wrong, the glibc version is good and all, but the compiler
> information might be even more crucial. Git for Windows had to hold back
> compiling with GCC v8.x for a while, for example, because the stacksmasher
> was broken. Similar issues are not unheard of, and could help pinpoint
> compiler-related problems a lot quicker.
Hm, sure. Good point - thanks.

This does make me start to wonder, though - does it really make sense to have ifdef-gated redefinitions of the whole get_compiler_info() method like I do now? I wonder if it makes more sense to have only one definition, so we can write down everything we know regardless of which pieces are put together. My thinking is something like this - what if I am using glibc, but not a GNU compiler? (The GNU docs on __GCC__ indicate this is a situation that might occur - "a non-GCC compiler that claims to accept the GNU C dialects") Is there some striking reason not to implement this compat command thusly instead:

  #ifdef __GLIBC__
  #include <gnu/libc-version.h>
  #endif
  static inline void get_compiler_info(struct strbuf *info)
  {
  	#ifdef __GLIBC__
	strbuf_addf(info, "glibc: %s\n", gnu_get_libc_version());
	#endif
	#ifdef __GNUC__
	strbuf_addf(info, "gnuc: %d.%d\n", __GNUC__, __GNUC_MINOR__);
	#endif
	#ifdef _MSC_VER
	strbuf_addf(info, "msc runtime: %s\n", some_msc_info());
	#endif
  }

The thinking being - this way if I decide to use, say, LLVM + glibc, then I don't need to reimplement this command with all the glibc diagnostics again. Or, if someone else already wrote diagnostics for LLVM with some other libc, then it even Just Works for me and my new combination.

That said, I'm reasoning about these combinations of compilers and libcs and whatever else from an inexperienced viewpoint, so maybe this isn't necessary?

 - Emily
Johannes Schindelin· Feb 20, 2020, 22:33 UTC · re: Emily Shaffer · lore

Re: [PATCH v7 06/15] bugreport: add compiler info

Hi Emily,
On Wed, 19 Feb 2020, Emily Shaffer wrote:
Show 16 quoted lines
>   #ifdef __GLIBC__
>   #include <gnu/libc-version.h>
>   #endif
>
>   static inline void get_compiler_info(struct strbuf *info)
>   {
>   	#ifdef __GLIBC__
> 	strbuf_addf(info, "glibc: %s\n", gnu_get_libc_version());
> 	#endif
>
> 	#ifdef __GNUC__
> 	strbuf_addf(info, "gnuc: %d.%d\n", __GNUC__, __GNUC_MINOR__);
> 	#endif
>
> 	#ifdef _MSC_VER
> 	strbuf_addf(info, "msc runtime: %s\n", some_msc_info());
You could do it this way right away:
	strbuf_addf(info, "MSVC version: %d.%d\n",
		    _MSC_VER / 100, _MSC_VER % 100);
Note: this is _not_ the MSVC _runtime_ version, but really the compiler
version.
You could also use _MSC_FULL_VER, which is a bit more complete.
Show 12 quoted lines
> 	#endif
>   }
>
> The thinking being - this way if I decide to use, say, LLVM + glibc,
> then I don't need to reimplement this command with all the glibc
> diagnostics again. Or, if someone else already wrote diagnostics for
> LLVM with some other libc, then it even Just Works for me and my new
> combination.
>
> That said, I'm reasoning about these combinations of compilers and libcs
> and whatever else from an inexperienced viewpoint, so maybe this isn't
> necessary?

I would have hoped that the argument I made earlier about the broken GCC version would have convinced you that the answer to this question is: Yes, this is necessary.

Ciao, Dscho

>
>  - Emily
>
Emily Shaffer· Feb 20, 2020, 23:33 UTC · re: Johannes Schindelin · lore

Re: [PATCH v7 06/15] bugreport: add compiler info

On Thu, Feb 20, 2020 at 11:33:05PM +0100, Johannes Schindelin wrote:
Show 30 quoted lines
> Hi Emily,
> 
> On Wed, 19 Feb 2020, Emily Shaffer wrote:
> 
> >   #ifdef __GLIBC__
> >   #include <gnu/libc-version.h>
> >   #endif
> >
> >   static inline void get_compiler_info(struct strbuf *info)
> >   {
> >   	#ifdef __GLIBC__
> > 	strbuf_addf(info, "glibc: %s\n", gnu_get_libc_version());
> > 	#endif
> >
> > 	#ifdef __GNUC__
> > 	strbuf_addf(info, "gnuc: %d.%d\n", __GNUC__, __GNUC_MINOR__);
> > 	#endif
> >
> > 	#ifdef _MSC_VER
> > 	strbuf_addf(info, "msc runtime: %s\n", some_msc_info());
> 
> You could do it this way right away:
> 
> 	strbuf_addf(info, "MSVC version: %d.%d\n",
> 		    _MSC_VER / 100, _MSC_VER % 100);
> 
> Note: this is _not_ the MSVC _runtime_ version, but really the compiler
> version.
> 
> You could also use _MSC_FULL_VER, which is a bit more complete.

Sorry, but I'm not comfortable sending code I can't check for myself (and already muscle-memoried into my format-patch/send-email workflow). If you send a scissors I can roll it into the series with your SOB.

 - Emily
Johannes Schindelin· Feb 21, 2020, 15:22 UTC · re: Emily Shaffer · lore

Re: [PATCH v7 06/15] bugreport: add compiler info

Hi Emily,
On Thu, 20 Feb 2020, Emily Shaffer wrote:
Show 35 quoted lines
> On Thu, Feb 20, 2020 at 11:33:05PM +0100, Johannes Schindelin wrote:
> > Hi Emily,
> >
> > On Wed, 19 Feb 2020, Emily Shaffer wrote:
> >
> > >   #ifdef __GLIBC__
> > >   #include <gnu/libc-version.h>
> > >   #endif
> > >
> > >   static inline void get_compiler_info(struct strbuf *info)
> > >   {
> > >   	#ifdef __GLIBC__
> > > 	strbuf_addf(info, "glibc: %s\n", gnu_get_libc_version());
> > > 	#endif
> > >
> > > 	#ifdef __GNUC__
> > > 	strbuf_addf(info, "gnuc: %d.%d\n", __GNUC__, __GNUC_MINOR__);
> > > 	#endif
> > >
> > > 	#ifdef _MSC_VER
> > > 	strbuf_addf(info, "msc runtime: %s\n", some_msc_info());
> >
> > You could do it this way right away:
> >
> > 	strbuf_addf(info, "MSVC version: %d.%d\n",
> > 		    _MSC_VER / 100, _MSC_VER % 100);
> >
> > Note: this is _not_ the MSVC _runtime_ version, but really the compiler
> > version.
> >
> > You could also use _MSC_FULL_VER, which is a bit more complete.
>
> Sorry, but I'm not comfortable sending code I can't check for myself
> (and already muscle-memoried into my format-patch/send-email workflow).
> If you send a scissors I can roll it into the series with your SOB.

But you can check it yourself! I worked _really_ hard on that Azure Pipeline backing the PR builds at https://github.com/git/git. _REALLY_ hard. You might just as well reap the benefits so that I did not spend all of that time and sweat and stress in vain...

Ciao, Dscho

Emily Shaffer· Feb 22, 2020, 00:04 UTC · re: Johannes Schindelin · lore

Re: [PATCH v7 06/15] bugreport: add compiler info

On Fri, Feb 21, 2020 at 04:22:43PM +0100, Johannes Schindelin wrote:
Show 44 quoted lines
> Hi Emily,
> 
> On Thu, 20 Feb 2020, Emily Shaffer wrote:
> 
> > On Thu, Feb 20, 2020 at 11:33:05PM +0100, Johannes Schindelin wrote:
> > > Hi Emily,
> > >
> > > On Wed, 19 Feb 2020, Emily Shaffer wrote:
> > >
> > > >   #ifdef __GLIBC__
> > > >   #include <gnu/libc-version.h>
> > > >   #endif
> > > >
> > > >   static inline void get_compiler_info(struct strbuf *info)
> > > >   {
> > > >   	#ifdef __GLIBC__
> > > > 	strbuf_addf(info, "glibc: %s\n", gnu_get_libc_version());
> > > > 	#endif
> > > >
> > > > 	#ifdef __GNUC__
> > > > 	strbuf_addf(info, "gnuc: %d.%d\n", __GNUC__, __GNUC_MINOR__);
> > > > 	#endif
> > > >
> > > > 	#ifdef _MSC_VER
> > > > 	strbuf_addf(info, "msc runtime: %s\n", some_msc_info());
> > >
> > > You could do it this way right away:
> > >
> > > 	strbuf_addf(info, "MSVC version: %d.%d\n",
> > > 		    _MSC_VER / 100, _MSC_VER % 100);
> > >
> > > Note: this is _not_ the MSVC _runtime_ version, but really the compiler
> > > version.
> > >
> > > You could also use _MSC_FULL_VER, which is a bit more complete.
> >
> > Sorry, but I'm not comfortable sending code I can't check for myself
> > (and already muscle-memoried into my format-patch/send-email workflow).
> > If you send a scissors I can roll it into the series with your SOB.
> 
> But you can check it yourself! I worked _really_ hard on that Azure
> Pipeline backing the PR builds at https://github.com/git/git. _REALLY_
> hard. You might just as well reap the benefits so that I did not spend all
> of that time and sweat and stress in vain...

I thought a bit about this. From your Github-using point of view, "just check my Pipeline" sounds like "just look at one more thing". From my format-patch using point of view, "just check my Pipeline" sounds like "ugh, I have to add this remote again... I don't have a fork already? How do I make that? Or is my fork just 6 months behind? How do I open a PR again? Yeesh."

So I did what any good developer who's supposed to be writing semi-annual performance reviews would, and wrote a first pass at script with this brand new fancy 'gh' thing (which is, by the way, clearly not intended for scripting): https://github.com/nasamuffin/quimby. Maybe someone will find it useful.

I'll try and add your MSC snippet for the next rollup.
 - Emily

P.S. You can thank Jonathan Nieder for the name, as I asked for ideas and he said "Wikipedia tells me Chief Quimby gives Inspector Gadget something to read that later self-destructs." :P

Junio C Hamano· Feb 24, 2020, 02:55 UTC · re: Emily Shaffer · lore

Re: [PATCH v7 06/15] bugreport: add compiler info

Emily Shaffer <emilyshaffer@google.com> writes:
Show 15 quoted lines
>> > Sorry, but I'm not comfortable sending code I can't check for myself
>> > (and already muscle-memoried into my format-patch/send-email workflow).
>> > If you send a scissors I can roll it into the series with your SOB.
>> 
>> But you can check it yourself! I worked _really_ hard on that Azure
>> Pipeline backing the PR builds at https://github.com/git/git. _REALLY_
>> hard. You might just as well reap the benefits so that I did not spend all
>> of that time and sweat and stress in vain...
>
> I thought a bit about this. From your Github-using point of view, "just
> check my Pipeline" sounds like "just look at one more thing". From my
> format-patch using point of view, "just check my Pipeline" sounds like
> "ugh, I have to add this remote again... I don't have a fork already?
> How do I make that? Or is my fork just 6 months behind? How do I open a
> PR again? Yeesh."

Sorry, but this is how a typical conversation between two techies who are more intelligent than good for their social skills ;-) Each side is not extending its hand enough to reach the other side, expecting that what s/he knows should be obvious to the others.

Dscho may be frustrated for you not being aware of what he worked on, but "I worked really hard" is much less useful thing for others, who may benefit from what he worked hard on, to hear, than "here is one pager that tells you how to use it". Quite honestly, reading the above exchange from the sidelines, it is not clear to me if it is a part of, or a completely separate from, the service offered by GGG from the end-users' point of view.

Can some of you come up with a one-pager to cover the following (and related but not listed) areas, that can be added as an appendix to Documentation/SubmittingPatches? It does not have to cover the common stuff like style guidelines, writing good log messages, need for test coverage, choice of fork point, etc. "Now you have a patch or a series of patches, but you obviously do not have direct and personal access to all the platforms Git supports. But there are servies to help you test them on the more common ones."

 - How to cause TravisCI to run our test suite with your
   modification.
 - How to trigger the same on Azure Pipeline PR builds.
There may be others.
Thanks.
Emily Shaffer· Feb 14, 2020, 01:53 UTC · re: Emily Shaffer · lore

[PATCH v7 09/15] bugreport: generate config safelist based on docs

Add a new step to the build to generate a safelist of git-config variables which are appropriate to include in the output of git-bugreport. New variables can be added to the safelist by annotating their documentation in Documentation/config with the "annotate" macro, which is a no-op in AsciiDoc and AsciiDoctor.

Some configs are private in nature, and can contain remote URLs, passwords, or other sensitive information. In the event that a user doesn't notice their information while reviewing a bugreport, that user may leak their credentials to other individuals, mailing lists, or bug tracking tools inadvertently. Heuristic blocklisting of configuration keys is imperfect and prone to false negatives; given the nature of the information which can be leaked, a safelist is more reliable.

However, it's possible that in some situations, an organization may be less concerned with privacy of things like remote URLs and branch names, and more concerned with ease of diagnosis for their support staff. In those cases, it may make more sense for that organization to modify the code to use a blocklist. To that end, we should try to mark configs which are definitely safe, and configs which are definitely unsafe, and leave blank configs which are somewhere in between. To mark a config as safe, add "annotate:bugreport[include]" to the corresponding line in the config documentation; to mark it as unsafe, add "annotate:bugreport[exclude]" instead.

Generating bugreport-config-safelist.h at build time by grepping the documentation for this new macro helps us prevent staleness. The macro itself is a no-op and should not alter the appearance of the documentation in either AsciiDoc or AsciiDoctor, confirmable by running:

  cd Documentation
  ./doc-diff --asciidoctor HEAD^ HEAD
  ./doc-diff --asciidoc HEAD^ HEAD

Diffing the rendered HTML shows that only inline comments were added, which shouldn't be a problem.

Additionally, add annotations to the sendemail config documentation in order to demonstrate a proof of concept.

Helped-by: Martin Ågren <martin.agren@gmail.com>
Helped-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 .gitignore                              |  1 +
 Documentation/asciidoc.conf             |  9 ++++
 Documentation/asciidoctor-extensions.rb |  7 ++++
 Documentation/config/sendemail.txt      | 56 ++++++++++++-------------
 Makefile                                |  7 ++++
 generate-bugreport-config-safelist.sh   | 18 ++++++++
 6 files changed, 70 insertions(+), 28 deletions(-)
 create mode 100755 generate-bugreport-config-safelist.sh
Show changes to 6 files +70 −28

.gitignore, Documentation/asciidoc.conf, Documentation/asciidoctor-extensions.rb, Documentation/config/sendemail.txt, Makefile, generate-bugreport-config-safelist.sh

diff --git a/.gitignore b/.gitignore
index d89bf9e11e..bd2f49b996 100644
--- a/.gitignore
+++ b/.gitignore
@@ -192,6 +192,7 @@
 /gitweb/static/gitweb.min.*
 /config-list.h
 /command-list.h
+/bugreport-config-safelist.h
 *.tar.gz
 *.dsc
 *.deb
diff --git a/Documentation/asciidoc.conf b/Documentation/asciidoc.conf
index 8fc4b67081..663e06481f 100644
--- a/Documentation/asciidoc.conf
+++ b/Documentation/asciidoc.conf
@@ -6,9 +6,14 @@
 #
 # Show Git link as: <command>(<section>); if section is defined, else just show
 # the command.
+#
+# The annotate macro does nothing as far as rendering is
+# concerned -- we just grep for it in the sources to populate
+# things like the bugreport safelist.
 
 [macros]
 (?su)[\\]?(?P<name>linkgit):(?P<target>\S*?)\[(?P<attrlist>.*?)\]=
+(?su)[\\]?(?P<name>annotate):(?P<target>\S*?)\[(?P<attrlist>.*?)\]=
 
 [attributes]
 asterisk=&#42;
@@ -28,6 +33,8 @@ ifdef::backend-docbook[]
 {0#<citerefentry>}
 {0#<refentrytitle>{target}</refentrytitle><manvolnum>{0}</manvolnum>}
 {0#</citerefentry>}
+[annotate-inlinemacro]
+{0#}
 endif::backend-docbook[]
 
 ifdef::backend-docbook[]
@@ -94,4 +101,6 @@ ifdef::backend-xhtml11[]
 git-relative-html-prefix=
 [linkgit-inlinemacro]
 <a href="{git-relative-html-prefix}{target}.html">{target}{0?({0})}</a>
+[annotate-inlinemacro]
+<!-- -->
 endif::backend-xhtml11[]
diff --git a/Documentation/asciidoctor-extensions.rb b/Documentation/asciidoctor-extensions.rb
index d906a00803..03c80af0e5 100644
--- a/Documentation/asciidoctor-extensions.rb
+++ b/Documentation/asciidoctor-extensions.rb
@@ -39,10 +39,17 @@ module Git
         output
       end
     end
+
+    class AnnotateProcessor < Asciidoctor::Extensions::InlineMacroProcessor
+      def process(parent, target, attrs)
+        ""
+      end
+    end
   end
 end
 
 Asciidoctor::Extensions.register do
   inline_macro Git::Documentation::LinkGitProcessor, :linkgit
   postprocessor Git::Documentation::DocumentPostProcessor
+  inline_macro Git::Documentation::AnnotateProcessor, :annotate
 end
diff --git a/Documentation/config/sendemail.txt b/Documentation/config/sendemail.txt
index 0006faf800..fe27473e44 100644
--- a/Documentation/config/sendemail.txt
+++ b/Documentation/config/sendemail.txt
@@ -4,7 +4,7 @@ sendemail.identity::
 	values in the 'sendemail' section. The default identity is
 	the value of `sendemail.identity`.
 
-sendemail.smtpEncryption::
+sendemail.smtpEncryption annotate:bugreport[include] ::
 	See linkgit:git-send-email[1] for description.  Note that this
 	setting is not subject to the 'identity' mechanism.
 
@@ -15,7 +15,7 @@ sendemail.smtpsslcertpath::
 	Path to ca-certificates (either a directory or a single file).
 	Set it to an empty string to disable certificate verification.
 
-sendemail.<identity>.*::
+sendemail.<identity>.* annotate:bugreport[exclude] ::
 	Identity-specific versions of the 'sendemail.*' parameters
 	found below, taking precedence over those when this
 	identity is selected, through either the command-line or
@@ -23,41 +23,41 @@ sendemail.<identity>.*::
 
 sendemail.aliasesFile::
 sendemail.aliasFileType::
-sendemail.annotate::
-sendemail.bcc::
-sendemail.cc::
-sendemail.ccCmd::
-sendemail.chainReplyTo::
-sendemail.confirm::
-sendemail.envelopeSender::
-sendemail.from::
-sendemail.multiEdit::
-sendemail.signedoffbycc::
-sendemail.smtpPass::
-sendemail.suppresscc::
-sendemail.suppressFrom::
-sendemail.to::
-sendemail.tocmd::
-sendemail.smtpDomain::
-sendemail.smtpServer::
-sendemail.smtpServerPort::
-sendemail.smtpServerOption::
-sendemail.smtpUser::
-sendemail.thread::
-sendemail.transferEncoding::
-sendemail.validate::
-sendemail.xmailer::
+sendemail.annotate annotate:bugreport[include] ::
+sendemail.bcc annotate:bugreport[include] ::
+sendemail.cc annotate:bugreport[include] ::
+sendemail.ccCmd annotate:bugreport[include] ::
+sendemail.chainReplyTo annotate:bugreport[include] ::
+sendemail.confirm annotate:bugreport[include] ::
+sendemail.envelopeSender annotate:bugreport[include] ::
+sendemail.from annotate:bugreport[include] ::
+sendemail.multiEdit annotate:bugreport[include] ::
+sendemail.signedoffbycc annotate:bugreport[include] ::
+sendemail.smtpPass annotate:bugreport[exclude] ::
+sendemail.suppresscc annotate:bugreport[include] ::
+sendemail.suppressFrom annotate:bugreport[include] ::
+sendemail.to annotate:bugreport[include] ::
+sendemail.tocmd annotate:bugreport[include] ::
+sendemail.smtpDomain annotate:bugreport[include] ::
+sendemail.smtpServer annotate:bugreport[include] ::
+sendemail.smtpServerPort annotate:bugreport[include] ::
+sendemail.smtpServerOption annotate:bugreport[include] ::
+sendemail.smtpUser annotate:bugreport[exclude] ::
+sendemail.thread annotate:bugreport[include] ::
+sendemail.transferEncoding annotate:bugreport[include] ::
+sendemail.validate annotate:bugreport[include] ::
+sendemail.xmailer annotate:bugreport[include] ::
 	See linkgit:git-send-email[1] for description.
 
 sendemail.signedoffcc (deprecated)::
 	Deprecated alias for `sendemail.signedoffbycc`.
 
-sendemail.smtpBatchSize::
+sendemail.smtpBatchSize annotate:bugreport[include] ::
 	Number of messages to be sent per connection, after that a relogin
 	will happen.  If the value is 0 or undefined, send all messages in
 	one connection.
 	See also the `--batch-size` option of linkgit:git-send-email[1].
 
-sendemail.smtpReloginDelay::
+sendemail.smtpReloginDelay annotate:bugreport[include] ::
 	Seconds wait before reconnecting to smtp server.
 	See also the `--relogin-delay` option of linkgit:git-send-email[1].
diff --git a/Makefile b/Makefile
index a01a050aa3..2bc9f112ea 100644
--- a/Makefile
+++ b/Makefile
@@ -817,6 +817,7 @@ VCSSVN_LIB = vcs-svn/lib.a
 
 GENERATED_H += config-list.h
 GENERATED_H += command-list.h
+GENERATED_H += bugreport-config-safelist.h
 
 LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
 	$(FIND) . \
@@ -2163,6 +2164,12 @@ command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Doc
 		$(patsubst %,--exclude-program %,$(EXCLUDED_PROGRAMS)) \
 		command-list.txt >$@+ && mv $@+ $@
 
+bugreport-config-safelist.h: generate-bugreport-config-safelist.sh
+
+bugreport-config-safelist.h: Documentation/config/*.txt
+	$(QUIET_GEN)$(SHELL_PATH) ./generate-bugreport-config-safelist.sh \
+		>$@+ && mv $@+ $@
+
 SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):$(GIT_VERSION):\
 	$(localedir_SQ):$(NO_CURL):$(USE_GETTEXT_SCHEME):$(SANE_TOOL_PATH_SQ):\
 	$(gitwebdir_SQ):$(PERL_PATH_SQ):$(SANE_TEXT_GREP):$(PAGER_ENV):\
diff --git a/generate-bugreport-config-safelist.sh b/generate-bugreport-config-safelist.sh
new file mode 100755
index 0000000000..2794a8ae77
--- /dev/null
+++ b/generate-bugreport-config-safelist.sh
@@ -0,0 +1,18 @@
+#!/bin/sh
+
+cat <<EOF
+/* Automatically generated by bugreport-generate-config-safelist.sh */
+
+
+static const char *bugreport_config_safelist[] = {
+EOF
+
+# cat all regular files in Documentation/config
+find Documentation/config -type f -exec cat {} \; |
+# print the command name which matches the annotate-bugreport macro
+sed -n 's/^\([^[:blank:]]*\)[[:blank:]]\{1,\}annotate:bugreport\[include\].* ::$/  "\1",/p' \
+	| sort
+
+cat <<EOF
+};
+EOF
-- 
2.25.0.265.gbab2e86ba0-goog
Emily Shaffer· Feb 14, 2020, 01:53 UTC · re: Emily Shaffer · lore

[PATCH v7 10/15] bugreport: add config values from safelist

Teach bugreport to gather the values of config options which are present in 'bugreport-config-safelist.h', and show their origin scope.

Many config options are sensitive, and many Git add-ons use config options which git-core does not know about; it is better only to gather config options which we know to be safe, rather than excluding options which we know to be unsafe.

Rather than including the path to someone's config, which can reveal filesystem layout and project names, just name the scope (e.g. system, global, local) of the config source.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  1 +
 Makefile                        |  2 ++
 bugreport.c                     | 34 +++++++++++++++++++++++++++++++++
 3 files changed, 37 insertions(+)
Show changes to 3 files +37 −0

Documentation/git-bugreport.txt, Makefile, bugreport.c

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index 23265b0d74..4e9171d1bd 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -30,6 +30,7 @@ The following information is captured automatically:
  - Compiler-specific info string
  - 'git remote-https --build-info'
  - $SHELL
+ - Selected config values
 
 OPTIONS
 -------
diff --git a/Makefile b/Makefile
index 2bc9f112ea..7b139230ae 100644
--- a/Makefile
+++ b/Makefile
@@ -2131,6 +2131,8 @@ git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS)
 
 help.sp help.s help.o: command-list.h
 
+bugreport.sp bugreport.s bugreport.o: bugreport-config-safelist.h
+
 builtin/help.sp builtin/help.s builtin/help.o: config-list.h GIT-PREFIX
 builtin/help.sp builtin/help.s builtin/help.o: EXTRA_CPPFLAGS = \
 	'-DGIT_HTML_PATH="$(htmldir_relative_SQ)"' \
diff --git a/bugreport.c b/bugreport.c
index c1475601bf..1a9430a857 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -6,6 +6,9 @@
 #include "help.h"
 #include "compat/compiler.h"
 #include "run-command.h"
+#include "config.h"
+#include "bugreport-config-safelist.h"
+#include "khash.h"
 
 static void get_git_remote_https_version_info(struct strbuf *version_info)
 {
@@ -52,6 +55,29 @@ static void get_system_info(struct strbuf *sys_info)
 	strbuf_complete_line(sys_info);
 }
 
+static void get_safelisted_config(struct strbuf *config_info)
+{
+	size_t idx;
+	struct string_list_item *it = NULL;
+	struct key_value_info *kv_info = NULL;
+
+	for (idx = 0; idx < ARRAY_SIZE(bugreport_config_safelist); idx++) {
+		const struct string_list *list =
+			git_config_get_value_multi(bugreport_config_safelist[idx]);
+
+		if (!list)
+			continue;
+
+		strbuf_addf(config_info, "%s:\n", bugreport_config_safelist[idx]);
+		for_each_string_list_item(it, list) {
+			kv_info = it->util;
+			strbuf_addf(config_info, "  %s (%s)\n", it->string,
+				    kv_info ? config_scope_name(kv_info->scope)
+					    : "source unknown");
+		}
+	}
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
 	NULL
@@ -94,6 +120,7 @@ int cmd_main(int argc, const char **argv)
 	char *option_output = NULL;
 	char *option_suffix = "%F-%H%M";
 	struct stat statbuf;
+	int nongit_ok = 0;
 
 	const struct option bugreport_options[] = {
 		OPT_STRING('o', "output-directory", &option_output, N_("path"),
@@ -102,6 +129,10 @@ int cmd_main(int argc, const char **argv)
 			   N_("specify a strftime format suffix for the filename")),
 		OPT_END()
 	};
+
+	/* Prerequisite for hooks and config checks */
+	setup_git_directory_gently(&nongit_ok);
+
 	argc = parse_options(argc, argv, "", bugreport_options,
 			     bugreport_usage, 0);
 
@@ -132,6 +163,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "System Info");
 	get_system_info(&buffer);
 
+	get_header(&buffer, "Safelisted Config Info");
+	get_safelisted_config(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 
 	if (report == NULL) {
-- 
2.25.0.265.gbab2e86ba0-goog
Emily Shaffer· Feb 14, 2020, 01:53 UTC · re: Emily Shaffer · lore

[PATCH v7 11/15] bugreport: collect list of populated hooks

Occasionally a failure a user is seeing may be related to a specific hook which is being run, perhaps without the user realizing. While the contents of hooks can be sensitive - containing user data or process information specific to the user's organization - simply knowing that a hook is being run at a certain stage can help us to understand whether something is going wrong.

Without a definitive list of hook names within the code, we compile our own list from the documentation. This is likely prone to bitrot. To reduce the amount of code humans need to read, we turn the list into a string_list and iterate over it (as we are calling the same find_hook operation on each string). However, since bugreport should primarily be called by the user, the performance loss from massaging the string seems acceptable.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 53 +++++++++++++++++++++++++++++++++
 2 files changed, 54 insertions(+)
Show changes to 2 files +54 −0

Documentation/git-bugreport.txt, bugreport.c

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index 4e9171d1bd..779c9c7fd9 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -31,6 +31,7 @@ The following information is captured automatically:
  - 'git remote-https --build-info'
  - $SHELL
  - Selected config values
+ - A list of enabled hooks
 
 OPTIONS
 -------
diff --git a/bugreport.c b/bugreport.c
index 1a9430a857..7e6147a935 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -9,6 +9,7 @@
 #include "config.h"
 #include "bugreport-config-safelist.h"
 #include "khash.h"
+#include "run-command.h"
 
 static void get_git_remote_https_version_info(struct strbuf *version_info)
 {
@@ -78,6 +79,55 @@ static void get_safelisted_config(struct strbuf *config_info)
 	}
 }
 
+static void get_populated_hooks(struct strbuf *hook_info, int nongit)
+{
+	/*
+	 * Doesn't look like there is a list of all possible hooks; so below is
+	 * a transcription of `git help hooks`.
+	 */
+	const char *hooks = "applypatch-msg,"
+			    "pre-applypatch,"
+			    "post-applypatch,"
+			    "pre-commit,"
+			    "pre-merge-commit,"
+			    "prepare-commit-msg,"
+			    "commit-msg,"
+			    "post-commit,"
+			    "pre-rebase,"
+			    "post-checkout,"
+			    "post-merge,"
+			    "pre-push,"
+			    "pre-receive,"
+			    "update,"
+			    "post-receive,"
+			    "post-update,"
+			    "push-to-checkout,"
+			    "pre-auto-gc,"
+			    "post-rewrite,"
+			    "sendemail-validate,"
+			    "fsmonitor-watchman,"
+			    "p4-pre-submit,"
+			    "post-index-changex";
+	struct string_list hooks_list = STRING_LIST_INIT_DUP;
+	struct string_list_item *iter = NULL;
+
+
+	if (nongit) {
+		strbuf_addstr(hook_info,
+			"not run from a git repository - no hooks to show\n");
+		return;
+	}
+
+	string_list_split(&hooks_list, hooks, ',', -1);
+
+	for_each_string_list_item(iter, &hooks_list) {
+		if (find_hook(iter->string)) {
+			strbuf_addstr(hook_info, iter->string);
+			strbuf_complete_line(hook_info);
+		}
+	}
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
 	NULL
@@ -166,6 +216,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Safelisted Config Info");
 	get_safelisted_config(&buffer);
 
+	get_header(&buffer, "Enabled Hooks");
+	get_populated_hooks(&buffer, nongit_ok);
+
 	report = fopen_for_writing(report_path.buf);
 
 	if (report == NULL) {
-- 
2.25.0.265.gbab2e86ba0-goog
Emily Shaffer· Feb 14, 2020, 01:53 UTC · re: Emily Shaffer · lore

[PATCH v7 08/15] bugreport: include user interactive shell

It's possible a user may complain about the way that Git interacts with their interactive shell, e.g. autocompletion or shell prompt. In that case, it's useful for us to know which shell they're using interactively.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt | 1 +
 bugreport.c                     | 5 +++++
 2 files changed, 6 insertions(+)
Show changes to 2 files +6 −0

Documentation/git-bugreport.txt, bugreport.c

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index 33df4dec7f..23265b0d74 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -29,6 +29,7 @@ The following information is captured automatically:
  - uname sysname, release, version, and machine strings
  - Compiler-specific info string
  - 'git remote-https --build-info'
+ - $SHELL
 
 OPTIONS
 -------
diff --git a/bugreport.c b/bugreport.c
index bfdff33368..c1475601bf 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -21,6 +21,7 @@ static void get_git_remote_https_version_info(struct strbuf *version_info)
 static void get_system_info(struct strbuf *sys_info)
 {
 	struct utsname uname_info;
+	char *shell = NULL;
 
 	/* get git version from native cmd */
 	strbuf_addstr(sys_info, "git version:\n");
@@ -42,6 +43,10 @@ static void get_system_info(struct strbuf *sys_info)
 	get_compiler_info(sys_info);
 	strbuf_complete_line(sys_info);
 
+	shell = getenv("SHELL");
+	strbuf_addf(sys_info, "$SHELL (typically, interactive shell): %s\n",
+		    shell ? shell : "<unset>");
+
 	strbuf_addstr(sys_info, "git-remote-https --build-info:\n");
 	get_git_remote_https_version_info(sys_info);
 	strbuf_complete_line(sys_info);
-- 
2.25.0.265.gbab2e86ba0-goog
Emily Shaffer· Feb 14, 2020, 01:53 UTC · re: Emily Shaffer · lore

[PATCH v7 12/15] bugreport: count loose objects

The number of unpacked objects in a user's repository may help us understand the root of the problem they're seeing, especially if a command is running unusually slowly.

Helped-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 52 +++++++++++++++++++++++++++++++++
 2 files changed, 53 insertions(+)
Show changes to 2 files +53 −0

Documentation/git-bugreport.txt, bugreport.c

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index 779c9c7fd9..7e1ed44cb4 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -32,6 +32,7 @@ The following information is captured automatically:
  - $SHELL
  - Selected config values
  - A list of enabled hooks
+ - The number of loose objects in the repository
 
 OPTIONS
 -------
diff --git a/bugreport.c b/bugreport.c
index 7e6147a935..f5201ce416 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -10,6 +10,7 @@
 #include "bugreport-config-safelist.h"
 #include "khash.h"
 #include "run-command.h"
+#include "object-store.h"
 
 static void get_git_remote_https_version_info(struct strbuf *version_info)
 {
@@ -128,6 +129,54 @@ static void get_populated_hooks(struct strbuf *hook_info, int nongit)
 	}
 }
 
+static int loose_object_cb(const struct object_id *oid, const char *path,
+			   void *data) {
+	int *loose_object_count = data;
+
+	if (loose_object_count) {
+		(*loose_object_count)++;
+		return 0;
+	}
+
+	return 1;
+}
+
+static void get_loose_object_summary(struct strbuf *obj_info, int nongit) {
+
+	int local_loose_object_count = 0, total_loose_object_count = 0;
+	int local_count_questionable = 0, total_count_questionable = 0;
+
+	if (nongit) {
+		strbuf_addstr(obj_info,
+			"not run from a git repository - no objects to show\n");
+		return;
+	}
+
+	local_count_questionable = for_each_loose_object(
+					loose_object_cb,
+					&local_loose_object_count,
+					FOR_EACH_OBJECT_LOCAL_ONLY);
+
+	total_count_questionable = for_each_loose_object(
+					loose_object_cb,
+					&total_loose_object_count,
+					0);
+
+	strbuf_addf(obj_info, "%d local loose objects%s\n",
+		    local_loose_object_count,
+		    local_count_questionable ? " (problem during count)" : "");
+
+	strbuf_addf(obj_info, "%d alternate loose objects%s\n",
+		    total_loose_object_count - local_loose_object_count,
+		    (local_count_questionable || total_count_questionable)
+			? " (problem during count)"
+			: "");
+
+	strbuf_addf(obj_info, "%d total loose objects%s\n",
+		    total_loose_object_count,
+		    total_count_questionable ? " (problem during count)" : "");
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
 	NULL
@@ -219,6 +268,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Enabled Hooks");
 	get_populated_hooks(&buffer, nongit_ok);
 
+	get_header(&buffer, "Loose Object Counts");
+	get_loose_object_summary(&buffer, nongit_ok);
+
 	report = fopen_for_writing(report_path.buf);
 
 	if (report == NULL) {
-- 
2.25.0.265.gbab2e86ba0-goog
Emily Shaffer· Feb 14, 2020, 01:53 UTC · re: Emily Shaffer · lore

[PATCH v7 13/15] bugreport: add packed object summary

Alongside the loose object counts, it can be useful to show the number of packs and packed objects. This way we can check whether the repo has an appropriate ratio of packed to loose objects to help determine whether it's behaving correctly.

Add a utility to easily traverse all packfiles in a given repository. Use it in packfile.c and remove a redundant call to prepare_packed_git(), which is already called in get_all_packs().

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 30 ++++++++++++++++++++++++++++++
 object-store.h                  |  6 ++++++
 packfile.c                      |  3 +--
 4 files changed, 38 insertions(+), 2 deletions(-)
Show changes to 4 files +38 −2

Documentation/git-bugreport.txt, bugreport.c, object-store.h, packfile.c

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index 7e1ed44cb4..309b34fc42 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -33,6 +33,7 @@ The following information is captured automatically:
  - Selected config values
  - A list of enabled hooks
  - The number of loose objects in the repository
+ - The number of packs and packed objects in the repository
 
 OPTIONS
 -------
diff --git a/bugreport.c b/bugreport.c
index f5201ce416..580de13ddc 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -177,6 +177,33 @@ static void get_loose_object_summary(struct strbuf *obj_info, int nongit) {
 		    total_count_questionable ? " (problem during count)" : "");
 }
 
+static void get_packed_object_summary(struct strbuf *obj_info, int nongit)
+{
+	struct packed_git *pack = NULL;
+	int pack_count = 0;
+	int object_count = 0;
+
+	if (nongit) {
+		strbuf_addstr(obj_info,
+			"not run from a git repository - no objects to show\n");
+		return;
+	}
+
+	for_each_pack(the_repository, pack) {
+		pack_count++;
+		/*
+		 * To accurately count how many objects are packed, look inside
+		 * the packfile's index.
+		 */
+		open_pack_index(pack);
+		object_count += pack->num_objects;
+	}
+
+	strbuf_addf(obj_info, "%d total packs (%d objects)\n", pack_count,
+		    object_count);
+
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
 	NULL
@@ -271,6 +298,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Loose Object Counts");
 	get_loose_object_summary(&buffer, nongit_ok);
 
+	get_header(&buffer, "Packed Object Summary");
+	get_packed_object_summary(&buffer, nongit_ok);
+
 	report = fopen_for_writing(report_path.buf);
 
 	if (report == NULL) {
diff --git a/object-store.h b/object-store.h
index 55ee639350..518c73469d 100644
--- a/object-store.h
+++ b/object-store.h
@@ -6,6 +6,7 @@
 #include "list.h"
 #include "sha1-array.h"
 #include "strbuf.h"
+#include "packfile.h"
 
 struct object_directory {
 	struct object_directory *next;
@@ -403,4 +404,9 @@ int for_each_object_in_pack(struct packed_git *p,
 int for_each_packed_object(each_packed_object_fn, void *,
 			   enum for_each_object_flags flags);
 
+#define for_each_pack(repo, pack) 		\
+		for (pack = get_all_packs(repo);\
+		     pack;			\
+		     pack = pack->next)
+
 #endif /* OBJECT_STORE_H */
diff --git a/packfile.c b/packfile.c
index 7e7c04e4d8..6873827fb5 100644
--- a/packfile.c
+++ b/packfile.c
@@ -2061,8 +2061,7 @@ int for_each_packed_object(each_packed_object_fn cb, void *data,
 	int r = 0;
 	int pack_errors = 0;
 
-	prepare_packed_git(the_repository);
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for_each_pack(the_repository, p) {
 		if ((flags & FOR_EACH_OBJECT_LOCAL_ONLY) && !p->pack_local)
 			continue;
 		if ((flags & FOR_EACH_OBJECT_PROMISOR_ONLY) &&
-- 
2.25.0.265.gbab2e86ba0-goog
Emily Shaffer· Feb 14, 2020, 01:53 UTC · re: Emily Shaffer · lore

[PATCH v7 07/15] bugreport: add git-remote-https version

It's possible for git-remote-curl to be built separately from git; in that case we want to know what version of cURL is used by git-remote-curl, not necessarily which version was present at git-bugreport's build time. So instead, ask git-remote-curl for the version information it knows about.

Today, "git-remote-http" and "git-remote-https" are aliased to "git-remote-curl"; but in case we rely on a different library than cURL in the future, let's not explicitly reference cURL from bugreport.

For longevity purposes, invoke the alias "git-remote-https" instead of "git-remote-http".

Since it could have been built at a different time, also report the version and built-from commit of git-remote-curl alongside the cURL info.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 16 ++++++++++++++++
 remote-curl.c                   |  8 ++++++++
 3 files changed, 25 insertions(+)
Show changes to 3 files +25 −0

Documentation/git-bugreport.txt, bugreport.c, remote-curl.c

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index 8bbc4c960c..33df4dec7f 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -28,6 +28,7 @@ The following information is captured automatically:
  - 'git version --build-options'
  - uname sysname, release, version, and machine strings
  - Compiler-specific info string
+ - 'git remote-https --build-info'
 
 OPTIONS
 -------
diff --git a/bugreport.c b/bugreport.c
index 4f9101caeb..bfdff33368 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -5,6 +5,18 @@
 #include "time.h"
 #include "help.h"
 #include "compat/compiler.h"
+#include "run-command.h"
+
+static void get_git_remote_https_version_info(struct strbuf *version_info)
+{
+	struct child_process cp = CHILD_PROCESS_INIT;
+
+	argv_array_push(&cp.args, "git");
+	argv_array_push(&cp.args, "remote-https");
+	argv_array_push(&cp.args, "--build-info");
+	if (capture_command(&cp, version_info, 0))
+	    strbuf_addstr(version_info, "'git-remote-https --build-info' not supported\n");
+}
 
 static void get_system_info(struct strbuf *sys_info)
 {
@@ -29,6 +41,10 @@ static void get_system_info(struct strbuf *sys_info)
 	strbuf_addstr(sys_info, "compiler info: ");
 	get_compiler_info(sys_info);
 	strbuf_complete_line(sys_info);
+
+	strbuf_addstr(sys_info, "git-remote-https --build-info:\n");
+	get_git_remote_https_version_info(sys_info);
+	strbuf_complete_line(sys_info);
 }
 
 static const char * const bugreport_usage[] = {
diff --git a/remote-curl.c b/remote-curl.c
index 350d92a074..c590fbfae3 100644
--- a/remote-curl.c
+++ b/remote-curl.c
@@ -17,6 +17,7 @@
 #include "protocol.h"
 #include "quote.h"
 #include "transport.h"
+#include "version.h"
 
 static struct remote *remote;
 /* always ends with a trailing slash */
@@ -1374,6 +1375,13 @@ int cmd_main(int argc, const char **argv)
 	string_list_init(&options.deepen_not, 1);
 	string_list_init(&options.push_options, 1);
 
+	if (!strcmp("--build-info", argv[1])) {
+		printf("git-http-fetch version: %s\n", git_version_string);
+		printf("built from commit: %s\n", git_built_from_commit_string);
+		printf("curl version: %s\n", curl_version());
+		return 0;
+	}
+
 	/*
 	 * Just report "remote-curl" here (folding all the various aliases
 	 * ("git-remote-http", "git-remote-https", and etc.) here since they
-- 
2.25.0.265.gbab2e86ba0-goog
Johannes Schindelin· Feb 19, 2020, 14:28 UTC · re: Emily Shaffer · lore

Re: [PATCH v7 07/15] bugreport: add git-remote-https version

Hi Emily,
On Thu, 13 Feb 2020, Emily Shaffer wrote:
Show 18 quoted lines
> diff --git a/bugreport.c b/bugreport.c
> index 4f9101caeb..bfdff33368 100644
> --- a/bugreport.c
> +++ b/bugreport.c
> @@ -5,6 +5,18 @@
>  #include "time.h"
>  #include "help.h"
>  #include "compat/compiler.h"
> +#include "run-command.h"
> +
> +static void get_git_remote_https_version_info(struct strbuf *version_info)
> +{
> +	struct child_process cp = CHILD_PROCESS_INIT;
> +
> +	argv_array_push(&cp.args, "git");
> +	argv_array_push(&cp.args, "remote-https");
> +	argv_array_push(&cp.args, "--build-info");
> +	if (capture_command(&cp, version_info, 0))

Let's use RUN_GIT_CMD instead of adding `"git"` explicitly; It documents that we're interested in a Git command, and if we ever build a single-binary version of Git (as some Git for Windows users already asked for), it will make things easier.

Show 14 quoted lines
> +	    strbuf_addstr(version_info, "'git-remote-https --build-info' not supported\n");
> +}
>
>  static void get_system_info(struct strbuf *sys_info)
>  {
> diff --git a/remote-curl.c b/remote-curl.c
> index 350d92a074..c590fbfae3 100644
> --- a/remote-curl.c
> +++ b/remote-curl.c
> @@ -1374,6 +1375,13 @@ int cmd_main(int argc, const char **argv)
>  	string_list_init(&options.deepen_not, 1);
>  	string_list_init(&options.push_options, 1);
>
> +	if (!strcmp("--build-info", argv[1])) {

The context does not say this, but at this point, we already verified that `argc` is larger than 1. Good.

Also, in keeping with the existing code, we would need to use `--build-options` here (this is what `git version` calls the equivalent mode).

_However_.

I like your `--build-info` a lot more than `--build-options` (because the latter is very misleading: the commit and the date of the build are not "options" at all).

Thanks, Dscho

Show 13 quoted lines
> +		printf("git-http-fetch version: %s\n", git_version_string);
> +		printf("built from commit: %s\n", git_built_from_commit_string);
> +		printf("curl version: %s\n", curl_version());
> +		return 0;
> +	}
> +
>  	/*
>  	 * Just report "remote-curl" here (folding all the various aliases
>  	 * ("git-remote-http", "git-remote-https", and etc.) here since they
> --
> 2.25.0.265.gbab2e86ba0-goog
>
>
Emily Shaffer· Feb 19, 2020, 22:28 UTC · re: Johannes Schindelin · lore

Re: [PATCH v7 07/15] bugreport: add git-remote-https version

On Wed, Feb 19, 2020 at 03:28:35PM +0100, Johannes Schindelin wrote:
Show 27 quoted lines
> Hi Emily,
> 
> On Thu, 13 Feb 2020, Emily Shaffer wrote:
> 
> > diff --git a/bugreport.c b/bugreport.c
> > index 4f9101caeb..bfdff33368 100644
> > --- a/bugreport.c
> > +++ b/bugreport.c
> > @@ -5,6 +5,18 @@
> >  #include "time.h"
> >  #include "help.h"
> >  #include "compat/compiler.h"
> > +#include "run-command.h"
> > +
> > +static void get_git_remote_https_version_info(struct strbuf *version_info)
> > +{
> > +	struct child_process cp = CHILD_PROCESS_INIT;
> > +
> > +	argv_array_push(&cp.args, "git");
> > +	argv_array_push(&cp.args, "remote-https");
> > +	argv_array_push(&cp.args, "--build-info");
> > +	if (capture_command(&cp, version_info, 0))
> 
> Let's use RUN_GIT_CMD instead of adding `"git"` explicitly; It documents
> that we're interested in a Git command, and if we ever build a
> single-binary version of Git (as some Git for Windows users already asked
> for), it will make things easier.

Hm. RUN_GIT_CMD is an argument used for the run_command_v_opt* family of calls, but it seems that setting child_process.git_cmd has the same effect. Done.

Show 28 quoted lines
> 
> > +	    strbuf_addstr(version_info, "'git-remote-https --build-info' not supported\n");
> > +}
> >
> >  static void get_system_info(struct strbuf *sys_info)
> >  {
> > diff --git a/remote-curl.c b/remote-curl.c
> > index 350d92a074..c590fbfae3 100644
> > --- a/remote-curl.c
> > +++ b/remote-curl.c
> > @@ -1374,6 +1375,13 @@ int cmd_main(int argc, const char **argv)
> >  	string_list_init(&options.deepen_not, 1);
> >  	string_list_init(&options.push_options, 1);
> >
> > +	if (!strcmp("--build-info", argv[1])) {
> 
> The context does not say this, but at this point, we already verified that
> `argc` is larger than 1. Good.
> 
> Also, in keeping with the existing code, we would need to use
> `--build-options` here (this is what `git version` calls the equivalent
> mode).
> 
> _However_.
> 
> I like your `--build-info` a lot more than `--build-options` (because the
> latter is very misleading: the commit and the date of the build are not
> "options" at all).
Sure. Thanks for saying so.
 - Emily
Junio C Hamano· Feb 19, 2020, 22:33 UTC · re: Emily Shaffer · lore

Re: [PATCH v7 07/15] bugreport: add git-remote-https version

Emily Shaffer <emilyshaffer@google.com> writes:
Show 11 quoted lines
>> Also, in keeping with the existing code, we would need to use
>> `--build-options` here (this is what `git version` calls the equivalent
>> mode).
>> 
>> _However_.
>> 
>> I like your `--build-info` a lot more than `--build-options` (because the
>> latter is very misleading: the commit and the date of the build are not
>> "options" at all).
>
> Sure. Thanks for saying so.

I don't think anybody would mind introducing --build-info to "git version" as a synonym and deprecate --build-options from it ;-)

Johannes Schindelin· Feb 20, 2020, 22:33 UTC · re: Junio C Hamano · lore

Re: [PATCH v7 07/15] bugreport: add git-remote-https version

Hi Junio,
On Wed, 19 Feb 2020, Junio C Hamano wrote:
Show 16 quoted lines
> Emily Shaffer <emilyshaffer@google.com> writes:
>
> >> Also, in keeping with the existing code, we would need to use
> >> `--build-options` here (this is what `git version` calls the equivalent
> >> mode).
> >>
> >> _However_.
> >>
> >> I like your `--build-info` a lot more than `--build-options` (because the
> >> latter is very misleading: the commit and the date of the build are not
> >> "options" at all).
> >
> > Sure. Thanks for saying so.
>
> I don't think anybody would mind introducing --build-info to "git
> version" as a synonym and deprecate --build-options from it ;-)
Heh. I almost suggested this. Almost.

Ciao, Dscho

Emily Shaffer· Feb 14, 2020, 01:53 UTC · re: Emily Shaffer · lore

[PATCH v7 14/15] bugreport: list contents of $OBJDIR/info

Miscellaneous information used about the object store can end up in .git/objects/info; this can help us understand what may be going on with the object store when the user is reporting a bug. Otherwise, it could be difficult to track down what is going wrong with an object which isn't kept locally to .git/objects/ or .git/objects/pack. Having some understanding of where the user's objects may be kept can save us some hops during the bug reporting process.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 54 +++++++++++++++++++++++++++++++++
 2 files changed, 55 insertions(+)
Show changes to 2 files +55 −0

Documentation/git-bugreport.txt, bugreport.c

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index 309b34fc42..a21d081616 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -34,6 +34,7 @@ The following information is captured automatically:
  - A list of enabled hooks
  - The number of loose objects in the repository
  - The number of packs and packed objects in the repository
+ - A list of the contents of .git/objects/info (or equivalent)
 
 OPTIONS
 -------
diff --git a/bugreport.c b/bugreport.c
index 580de13ddc..1c67b55598 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -204,6 +204,57 @@ static void get_packed_object_summary(struct strbuf *obj_info, int nongit)
 
 }
 
+static void list_contents_of_dir_recursively(struct strbuf *contents,
+				      	     struct strbuf *dirpath)
+{
+	struct dirent *d;
+	DIR *dir;
+	size_t path_len;
+
+	dir = opendir(dirpath->buf);
+	if (!dir)
+		return;
+
+	strbuf_complete(dirpath, '/');
+	path_len = dirpath->len;
+
+	while ((d = readdir(dir))) {
+		if (!strcmp(d->d_name, ".") || !strcmp(d->d_name, ".."))
+			continue;
+
+		strbuf_addbuf(contents, dirpath);
+		strbuf_addstr(contents, d->d_name);
+		strbuf_complete_line(contents);
+
+		if (d->d_type == DT_DIR) {
+			strbuf_addstr(dirpath, d->d_name);
+			list_contents_of_dir_recursively(contents, dirpath);
+		}
+		strbuf_setlen(dirpath, path_len);
+	}
+
+	closedir(dir);
+}
+
+static void get_object_info_summary(struct strbuf *obj_info, int nongit)
+{
+	struct strbuf dirpath = STRBUF_INIT;
+
+	if (nongit) {
+		strbuf_addstr(obj_info,
+			"not run from a git repository - object info unavailable\n");
+		return;
+	}
+
+	strbuf_addstr(&dirpath, get_object_directory());
+	strbuf_complete(&dirpath, '/');
+	strbuf_addstr(&dirpath, "info/");
+
+	list_contents_of_dir_recursively(obj_info, &dirpath);
+
+	strbuf_release(&dirpath);
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
 	NULL
@@ -301,6 +352,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Packed Object Summary");
 	get_packed_object_summary(&buffer, nongit_ok);
 
+	get_header(&buffer, "Object Info Summary");
+	get_object_info_summary(&buffer, nongit_ok);
+
 	report = fopen_for_writing(report_path.buf);
 
 	if (report == NULL) {
-- 
2.25.0.265.gbab2e86ba0-goog
Junio C Hamano· Feb 14, 2020, 17:04 UTC · re: Emily Shaffer · lore

Re: [PATCH v7 14/15] bugreport: list contents of $OBJDIR/info

Emily Shaffer <emilyshaffer@google.com> writes:
Show 7 quoted lines
> Miscellaneous information used about the object store can end up in
> .git/objects/info; this can help us understand what may be going on with
> the object store when the user is reporting a bug. Otherwise, it could
> be difficult to track down what is going wrong with an object which
> isn't kept locally to .git/objects/ or .git/objects/pack. Having some
> understanding of where the user's objects may be kept can save us some
> hops during the bug reporting process.

This step seems to have a new whitespace breakage that did not exist in the previous round.

Emily Shaffer· Feb 18, 2020, 23:59 UTC · re: Junio C Hamano · lore

Re: [PATCH v7 14/15] bugreport: list contents of $OBJDIR/info

On Fri, Feb 14, 2020 at 09:04:05AM -0800, Junio C Hamano wrote:
Show 12 quoted lines
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> > Miscellaneous information used about the object store can end up in
> > .git/objects/info; this can help us understand what may be going on with
> > the object store when the user is reporting a bug. Otherwise, it could
> > be difficult to track down what is going wrong with an object which
> > isn't kept locally to .git/objects/ or .git/objects/pack. Having some
> > understanding of where the user's objects may be kept can save us some
> > hops during the bug reporting process.
> 
> This step seems to have a new whitespace breakage that did not exist
> in the previous round.

Hmph. I saw it warned at some point during the rebase-and-format-patch workflow for v7 but thought I had quashed it. Very sorry.

 - Emily
Emily Shaffer· Feb 14, 2020, 01:53 UTC · re: Emily Shaffer · lore

[PATCH v7 15/15] bugreport: summarize contents of alternates file

In some cases, it could be that the user is having a problem with an object which isn't present in their normal object directory. We can get a hint that that might be the case by examining the list of alternates where their object may be stored instead. Since paths to alternates may be sensitive, we'll instead count how many alternates have been specified and note how many of them exist or are broken.

While object-cache.h describes a function "foreach_alt_odb()", this function does not provide information on broken alternates, which are skipped over in "link_alt_odb_entry()". Since the goal is to identify missing alternates, we can gather the contents of .git/objects/info/alternates manually.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 45 +++++++++++++++++++++++++++++++++
 2 files changed, 46 insertions(+)
Show changes to 2 files +46 −0

Documentation/git-bugreport.txt, bugreport.c

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index a21d081616..e870900c80 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -35,6 +35,7 @@ The following information is captured automatically:
  - The number of loose objects in the repository
  - The number of packs and packed objects in the repository
  - A list of the contents of .git/objects/info (or equivalent)
+ - The number of valid and invalid alternates
 
 OPTIONS
 -------
diff --git a/bugreport.c b/bugreport.c
index 1c67b55598..3bc8cb3579 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -255,6 +255,48 @@ static void get_object_info_summary(struct strbuf *obj_info, int nongit)
 	strbuf_release(&dirpath);
 }
 
+static void get_alternates_summary(struct strbuf *alternates_info, int nongit)
+{
+	struct strbuf alternates_path = STRBUF_INIT;
+	struct strbuf alternate = STRBUF_INIT;
+	FILE *file;
+	size_t exists = 0, broken = 0;
+
+	if (nongit) {
+		strbuf_addstr(alternates_info,
+			"not run from a git repository - alternates unavailable\n");
+		return;
+	}
+
+	strbuf_addstr(&alternates_path, get_object_directory());
+	strbuf_complete(&alternates_path, '/');
+	strbuf_addstr(&alternates_path, "info/alternates");
+
+	file = fopen(alternates_path.buf, "r");
+	if (!file) {
+		strbuf_addstr(alternates_info, "No alternates file found.\n");
+		strbuf_release(&alternates_path);
+		return;
+	}
+
+	while (strbuf_getline(&alternate, file) != EOF) {
+		if (!access(alternate.buf, F_OK))
+			exists++;
+		else
+			broken++;
+	}
+
+	strbuf_addf(alternates_info,
+		    "%zd alternates found (%zd working, %zd broken)\n",
+		    exists + broken,
+		    exists,
+		    broken);
+
+	fclose(file);
+	strbuf_release(&alternate);
+	strbuf_release(&alternates_path);
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
 	NULL
@@ -355,6 +397,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Object Info Summary");
 	get_object_info_summary(&buffer, nongit_ok);
 
+	get_header(&buffer, "Alternates");
+	get_alternates_summary(&buffer, nongit_ok);
+
 	report = fopen_for_writing(report_path.buf);
 
 	if (report == NULL) {
-- 
2.25.0.265.gbab2e86ba0-goog
Junio C Hamano· Feb 14, 2020, 17:32 UTC · re: Emily Shaffer · lore

Re: [PATCH v7 00/15] add git-bugreport tool

Emily Shaffer <emilyshaffer@google.com> writes:
>    present(patch 9/15). This now uses POSIX character classes and {}
>    notation instead of + and has been tested on OSX; I'd love to hear if
I'd rather not to see unnecessary uses of POSIX character classes.

The interdiff of this step between the previous and this round looks to me more like "I used it, just because POSIX says I *can* use it", not "I did so because I needed to do, and it should be OK on POSIX platforms."

Instead of overly long
's/^\([^[:blank:]]*\)[[:blank:]]\{1,\}annotate:bugreport\[include\].* ::$/  "\1",/p'
just limiting ourselves to SP and saying
's/^\([^ ]*\)  *annotate:bugreport\[include\].* ::$/  "\1",/p'
would keep the result much easier to read, I would think.
Emily Shaffer· Feb 14, 2020, 22:00 UTC · re: Junio C Hamano · lore

Re: [PATCH v7 00/15] add git-bugreport tool

On Fri, Feb 14, 2020 at 09:32:08AM -0800, Junio C Hamano wrote:
Show 21 quoted lines
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> >    present(patch 9/15). This now uses POSIX character classes and {}
> >    notation instead of + and has been tested on OSX; I'd love to hear if
> 
> I'd rather not to see unnecessary uses of POSIX character classes.
> 
> The interdiff of this step between the previous and this round looks
> to me more like "I used it, just because POSIX says I *can* use it",
> not "I did so because I needed to do, and it should be OK on POSIX
> platforms."
> 
> Instead of overly long
> 
> 's/^\([^[:blank:]]*\)[[:blank:]]\{1,\}annotate:bugreport\[include\].* ::$/  "\1",/p'
> 
> just limiting ourselves to SP and saying
> 
> 's/^\([^ ]*\)  *annotate:bugreport\[include\].* ::$/  "\1",/p'
> 
> would keep the result much easier to read, I would think.

That's fine by me. I find the [[:syntax:]] extremely ugly, but I was worried about whether that was more portable somehow. Your proposal seems fine to me.

 - Emily

← back to recent threads