{"thread":{"id":"51645","subject":"[PATCH] bugreport: add tool to generate debugging info","startedAt":"2019-08-15T02:34:28Z","lastAt":"2019-12-15T17:40:44Z","messageCount":68,"participants":["Emily Shaffer","Derrick Stolee","Junio C Hamano","Johannes Schindelin","Martin Ågren","Josh Steadmon"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"380466","messageId":"20190815023418.33407-1-emilyshaffer@google.com","threadId":"51645","inReplyTo":null,"subject":"[PATCH] bugreport: add tool to generate debugging info","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2019-08-15T02:34:18Z","receivedAt":"2019-08-15T02:34:28Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"Make it easier for users who encounter a bug to send a report by\ncollecting some state information, intended to be viewed by humans\nfamiliar with Git.\n\nTeach Git how to prompt the user for a good bug report: reproduction\nsteps, expected behavior, and actual behavior. Also, teach Git to write\ndown its own version, the version of some of its dependencies, the\noperating system information, the effective gitconfig, the configured\nhooks, and the contents of $GIT_DIR/logs. Finally, make sure Git asks\nthe user to review the contents of the report after it's generated.\n\nIf users can send us a well-written bug report complete with system\ninformation, a remote we may be able to clone, and a reflog showing us\nwhere the problem occurred, we can reduce the number of questions and\nanswers which must travel between the reporter and the Git contributor.\n\nUsers may also wish to send a report like this to their local \"Git\nexpert\" if they have put their repository into a state they are confused\nby.\n\nSigned-off-by: Emily Shaffer <emilyshaffer@google.com>\n---\nThere are some things I'm not certain about from this patch, I'd\nappreciate feedback.\n\n - Do we want to advertise the Git mailing list for bug reports?\n - Which config options should we filter to avoid accidentally receiving\n   credentials?\n - At the moment, the test is trying to check \"everything we thought we\n   would populate got populated\" - that seemed to me like it would hold\n   up best to changes to the set of information being collected. But\n   maybe there's something more robust we can do.\n\nAnd of course, if there are suggestions for other things to include in\nthe report I'm interested in adding.\n\nAn example of the output can be found here:\nhttps://gist.github.com/nasamuffin/2e320892f5c2147e829cbcf5bd0759a2\n\nThanks.\n - Emily\n\n .gitignore                      |  1 +\n Documentation/git-bugreport.txt | 48 ++++++++++++++++++\n Makefile                        |  1 +\n command-list.txt                |  1 +\n git-bugreport.sh                | 86 +++++++++++++++++++++++++++++++++\n t/t0091-bugreport.sh            | 41 ++++++++++++++++\n 6 files changed, 178 insertions(+)\n create mode 100644 Documentation/git-bugreport.txt\n create mode 100755 git-bugreport.sh\n create mode 100755 t/t0091-bugreport.sh\n\ndiff --git a/.gitignore b/.gitignore\nindex 521d8f4fb4..b4f5433084 100644\n--- a/.gitignore\n+++ b/.gitignore\n@@ -25,6 +25,7 @@\n /git-bisect--helper\n /git-blame\n /git-branch\n+/git-bugreport\n /git-bundle\n /git-cat-file\n /git-check-attr\ndiff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt\nnew file mode 100644\nindex 0000000000..c5f45bbee8\n--- /dev/null\n+++ b/Documentation/git-bugreport.txt\n@@ -0,0 +1,48 @@\n+git-bugreport(1)\n+================\n+\n+NAME\n+----\n+git-bugreport - Collect information for user to file a bug report\n+\n+SYNOPSIS\n+--------\n+[verse]\n+'git bugreport' [-o | --output <path>]\n+\n+DESCRIPTION\n+-----------\n+Captures information about the user's machine, Git client, and repository state,\n+as well as a form requesting information about the behavior the user observed,\n+into a single text file which the user can then share, for example to the Git\n+mailing list, in order to report an observed bug.\n+\n+The following information is requested from the user:\n+\n+ - Reproduction steps\n+ - Expected behavior\n+ - Actual behavior\n+\n+The following information is captured automatically:\n+\n+ - Git version (`git version --build-options`)\n+ - Machine information (`uname -a`)\n+ - Versions of various dependencies\n+ - Git config contents (`git config --show-origin --list`)\n+ - The contents of all configured git-hooks in `.git/hooks/`\n+ - The contents of `.git/logs`\n+\n+OPTIONS\n+-------\n+-o [<path>]::\n+--output [<path>]::\n+\tPlace the resulting bug report file in <path> instead of the root of the\n+\tGit repository.\n+\n+NOTE\n+----\n+Bug reports can be sent to git@vger.kernel.org.\n+\n+GIT\n+---\n+Part of the linkgit:git[1] suite\ndiff --git a/Makefile b/Makefile\nindex f9255344ae..69801a1c45 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -606,6 +606,7 @@ TEST_PROGRAMS_NEED_X =\n unexport CDPATH\n \n SCRIPT_SH += git-bisect.sh\n+SCRIPT_SH += git-bugreport.sh\n SCRIPT_SH += git-difftool--helper.sh\n SCRIPT_SH += git-filter-branch.sh\n SCRIPT_SH += git-merge-octopus.sh\ndiff --git a/command-list.txt b/command-list.txt\nindex a9ac72bef4..be5a605047 100644\n--- a/command-list.txt\n+++ b/command-list.txt\n@@ -54,6 +54,7 @@ git-archive                             mainporcelain\n git-bisect                              mainporcelain           info\n git-blame                               ancillaryinterrogators          complete\n git-branch                              mainporcelain           history\n+git-bugreport                           ancillaryinterrogators\n git-bundle                              mainporcelain\n git-cat-file                            plumbinginterrogators\n git-check-attr                          purehelpers\ndiff --git a/git-bugreport.sh b/git-bugreport.sh\nnew file mode 100755\nindex 0000000000..2200703a51\n--- /dev/null\n+++ b/git-bugreport.sh\n@@ -0,0 +1,86 @@\n+#!/bin/sh\n+\n+print_filenames_and_content() {\n+while read -r line; do\n+\techo \"$line\";\n+\techo \"========\";\n+\tcat \"$line\";\n+\techo;\n+done\n+}\n+\n+generate_report_text() {\n+\n+\t# Generate a form for the user to fill out with their issue.\n+\tgettextln \"Thank you for filling out a Git bug report!\"\n+\tgettextln \"Please answer the following questions to help us understand your issue.\"\n+\techo\n+\tgettextln \"What did you do before the bug happened? (Steps to reproduce your issue)\"\n+\techo\n+\tgettextln \"What did you expect to happen? (Expected behavior)\"\n+\techo\n+\tgettextln \"What happened instead? (Actual behavior)\"\n+\techo\n+\tgettextln \"Anything else you want to add:\"\n+\techo\n+\tgettextln \"Please review the rest of the bug report below.\"\n+\tgettextln \"You can delete any lines you don't wish to send.\"\n+\techo\n+\n+\techo \"[System Information]\"\n+\tgit version --build-options\n+\tuname -a\n+\tcurl-config --version\n+\tldd --version\n+\techo\n+\n+\techo \"[Git Config]\"\n+\t# TODO: Pass this through grep -v to avoid users sending us their credentials.\n+\tgit config --show-origin --list\n+\techo\n+\n+\techo \"[Configured Hooks]\"\n+\tfind \"$GIT_DIR/hooks/\" -type f | grep -v \"\\.sample$\" | print_filenames_and_content\n+\techo\n+\n+\techo \"[Git Logs]\"\n+\tfind \"$GIT_DIR/logs\" -type f | print_filenames_and_content\n+\techo\n+\n+}\n+\n+USAGE=\"[-o | --output <path>]\"\n+\n+SUBDIRECTORY_OK=t\n+OPTIONS_SPEC=\n+. git-sh-setup\n+. git-sh-i18n\n+\n+basedir=\"$PWD\"\n+while :\n+do\n+\tcase \"$1\" in\n+\t-o|--output)\n+\t\tshift\n+\t\tbasedir=\"$1\"\n+\t\tshift\n+\t\tcontinue\n+\t\t;;\n+\t\"\")\n+\t\tbreak\n+\t\t;;\n+\t*)\n+\t\tusage\n+\t\t;;\n+\tesac\n+done\n+\n+\n+# Create bugreport file\n+BUGREPORT_FILE=\"$basedir/git-bugreport-$(whoami)-$(hostname)-$(date -Iminutes)\"\n+\n+generate_report_text >$BUGREPORT_FILE\n+\n+git_editor $BUGREPORT_FILE\n+\n+eval_gettextln \"Your new bug report is in \\$BUGREPORT_FILE.\"\ndiff --git a/t/t0091-bugreport.sh b/t/t0091-bugreport.sh\nnew file mode 100755\nindex 0000000000..6eb2ee4f66\n--- /dev/null\n+++ b/t/t0091-bugreport.sh\n@@ -0,0 +1,41 @@\n+#!/bin/bash\n+\n+test_description='git bugreport'\n+\n+. ./test-lib.sh\n+\n+# Headers \"[System Info]\" will be followed by a non-empty line if we put some\n+# information there; we can make sure all our headers were followed by some\n+# information to check if the command was successful.\n+HEADER_PATTERN=\"^\\[.*\\]$\"\n+check_all_headers_populated() {\n+\twhile read -r line; do\n+\t\tif [$(grep $HEADER_PATTERN $line)]; then\n+\t\t\tread -r nextline\n+\t\t\tif [-z $nextline]; then\n+\t\t\t\treturn 1;\n+\t\t\tfi\n+\t\tfi\n+\tdone\n+}\n+\n+test_expect_success 'creates a report with content in the right places' '\n+\tgit bugreport &&\n+\tcheck_all_headers_populated <git-bugreport-* &&\n+\trm git-bugreport-*\n+'\n+\n+test_expect_success '--output puts the report in the provided dir' '\n+\tmkdir foo/ &&\n+\tgit bugreport -o foo/ &&\n+\ttest -f foo/git-bugreport-* &&\n+\trm -fr foo/\n+'\n+\n+test_expect_success 'incorrect arguments abort with usage' '\n+\ttest_must_fail git bugreport --false 2>output &&\n+\tgrep usage output &&\n+\ttest ! -f git-bugreport-*\n+'\n+\n+test_done\n-- \n2.23.0.rc1.153.gdeed80330f-goog\n\n"},{"id":"380473","messageId":"e6d56d97-99c9-064a-71b5-2b7eb9b71e01@gmail.com","threadId":"51645","inReplyTo":"20190815023418.33407-1-emilyshaffer@google.com","subject":"Re: [PATCH] bugreport: add tool to generate debugging info","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2019-08-15T14:15:24Z","receivedAt":"2019-08-15T14:15:29Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 8/14/2019 10:34 PM, Emily Shaffer wrote:\n> Make it easier for users who encounter a bug to send a report by\n> collecting some state information, intended to be viewed by humans\n> familiar with Git.\n\nThis is an excellent idea! VFS for Git has a similar \"diagnose\" command\nthat collects logs, config, and other details (like packfile sizes and\nloose object counts). That has been a critical tool for supporting users.\n\n> Teach Git how to prompt the user for a good bug report: reproduction\n> steps, expected behavior, and actual behavior. Also, teach Git to write\n> down its own version, the version of some of its dependencies, the\n> operating system information, the effective gitconfig, the configured\n> hooks, and the contents of $GIT_DIR/logs. Finally, make sure Git asks\n> the user to review the contents of the report after it's generated.\n> \n> If users can send us a well-written bug report complete with system\n> information, a remote we may be able to clone, and a reflog showing us\n> where the problem occurred, we can reduce the number of questions and\n> answers which must travel between the reporter and the Git contributor.\n> \n> Users may also wish to send a report like this to their local \"Git\n> expert\" if they have put their repository into a state they are confused\n> by.\n> \n> Signed-off-by: Emily Shaffer <emilyshaffer@google.com>\n> ---\n> There are some things I'm not certain about from this patch, I'd\n> appreciate feedback.\n> \n>  - Do we want to advertise the Git mailing list for bug reports?\n\nThat is possible. Isn't there another mailing list for git users?\n\nI could see a patch added on top of this for git-for-windows/git that\nchanges the instructions to create issues on GitHub.\n\n>  - Which config options should we filter to avoid accidentally receiving\n>    credentials?\n\nThe remote URLs are pretty sensitive. Not only do users sometimes put passwords\nor PATs into their URLs, the literal name of the repo could be a secret.\n\n>  - At the moment, the test is trying to check \"everything we thought we\n>    would populate got populated\" - that seemed to me like it would hold\n>    up best to changes to the set of information being collected. But\n>    maybe there's something more robust we can do.\n> \n> And of course, if there are suggestions for other things to include in\n> the report I'm interested in adding.\n> \n> An example of the output can be found here:\n> https://gist.github.com/nasamuffin/2e320892f5c2147e829cbcf5bd0759a2\n> \n> Thanks.\n>  - Emily\n> \n>  .gitignore                      |  1 +\n>  Documentation/git-bugreport.txt | 48 ++++++++++++++++++\n>  Makefile                        |  1 +\n>  command-list.txt                |  1 +\n>  git-bugreport.sh                | 86 +++++++++++++++++++++++++++++++++\n>  t/t0091-bugreport.sh            | 41 ++++++++++++++++\n>  6 files changed, 178 insertions(+)\n>  create mode 100644 Documentation/git-bugreport.txt\n>  create mode 100755 git-bugreport.sh\n>  create mode 100755 t/t0091-bugreport.sh\n> \n> diff --git a/.gitignore b/.gitignore\n> index 521d8f4fb4..b4f5433084 100644\n> --- a/.gitignore\n> +++ b/.gitignore\n> @@ -25,6 +25,7 @@\n>  /git-bisect--helper\n>  /git-blame\n>  /git-branch\n> +/git-bugreport\n>  /git-bundle\n>  /git-cat-file\n>  /git-check-attr\n> diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt\n> new file mode 100644\n> index 0000000000..c5f45bbee8\n> --- /dev/null\n> +++ b/Documentation/git-bugreport.txt\n> @@ -0,0 +1,48 @@\n> +git-bugreport(1)\n> +================\n> +\n> +NAME\n> +----\n> +git-bugreport - Collect information for user to file a bug report\n> +\n> +SYNOPSIS\n> +--------\n> +[verse]\n> +'git bugreport' [-o | --output <path>]\n> +\n> +DESCRIPTION\n> +-----------\n> +Captures information about the user's machine, Git client, and repository state,\n> +as well as a form requesting information about the behavior the user observed,\n> +into a single text file which the user can then share, for example to the Git\n> +mailing list, in order to report an observed bug.\n> +\n> +The following information is requested from the user:\n> +\n> + - Reproduction steps\n> + - Expected behavior\n> + - Actual behavior\n> +\n> +The following information is captured automatically:\n> +\n> + - Git version (`git version --build-options`)\n> + - Machine information (`uname -a`)\n> + - Versions of various dependencies\n> + - Git config contents (`git config --show-origin --list`)\n> + - The contents of all configured git-hooks in `.git/hooks/`\n> + - The contents of `.git/logs`\n> +\n> +OPTIONS\n> +-------\n> +-o [<path>]::\n> +--output [<path>]::\n> +\tPlace the resulting bug report file in <path> instead of the root of the\n> +\tGit repository.\n> +\n> +NOTE\n> +----\n> +Bug reports can be sent to git@vger.kernel.org.\n> +\n> +GIT\n> +---\n> +Part of the linkgit:git[1] suite\n> diff --git a/Makefile b/Makefile\n> index f9255344ae..69801a1c45 100644\n> --- a/Makefile\n> +++ b/Makefile\n> @@ -606,6 +606,7 @@ TEST_PROGRAMS_NEED_X =\n>  unexport CDPATH\n>  \n>  SCRIPT_SH += git-bisect.sh\n> +SCRIPT_SH += git-bugreport.sh\n>  SCRIPT_SH += git-difftool--helper.sh\n>  SCRIPT_SH += git-filter-branch.sh\n>  SCRIPT_SH += git-merge-octopus.sh\n> diff --git a/command-list.txt b/command-list.txt\n> index a9ac72bef4..be5a605047 100644\n> --- a/command-list.txt\n> +++ b/command-list.txt\n> @@ -54,6 +54,7 @@ git-archive                             mainporcelain\n>  git-bisect                              mainporcelain           info\n>  git-blame                               ancillaryinterrogators          complete\n>  git-branch                              mainporcelain           history\n> +git-bugreport                           ancillaryinterrogators\n>  git-bundle                              mainporcelain\n>  git-cat-file                            plumbinginterrogators\n>  git-check-attr                          purehelpers\n> diff --git a/git-bugreport.sh b/git-bugreport.sh\n> new file mode 100755\n> index 0000000000..2200703a51\n> --- /dev/null\n> +++ b/git-bugreport.sh\n\nAt first I was alarmed by \"What? another shell script?\" but this command should\nprioritize flexibility and extensibility over speed. Running multiple processes\nshouldn't be too taxing for what we are trying to do here.\n\n> @@ -0,0 +1,86 @@\n> +#!/bin/sh\n> +\n> +print_filenames_and_content() {\n> +while read -r line; do\n> +\techo \"$line\";\n> +\techo \"========\";\n> +\tcat \"$line\";\n> +\techo;\n> +done\n> +}\n> +\n> +generate_report_text() {\n> +\n> +\t# Generate a form for the user to fill out with their issue.\n> +\tgettextln \"Thank you for filling out a Git bug report!\"\n> +\tgettextln \"Please answer the following questions to help us understand your issue.\"\n> +\techo\n> +\tgettextln \"What did you do before the bug happened? (Steps to reproduce your issue)\"\n> +\techo\n> +\tgettextln \"What did you expect to happen? (Expected behavior)\"\n> +\techo\n> +\tgettextln \"What happened instead? (Actual behavior)\"\n> +\techo\n> +\tgettextln \"Anything else you want to add:\"\n> +\techo\n> +\tgettextln \"Please review the rest of the bug report below.\"\n> +\tgettextln \"You can delete any lines you don't wish to send.\"\n> +\techo\n> +\n> +\techo \"[System Information]\"\n> +\tgit version --build-options\n> +\tuname -a\n> +\tcurl-config --version\n> +\tldd --version\n> +\techo\n> +\n> +\techo \"[Git Config]\"\n> +\t# TODO: Pass this through grep -v to avoid users sending us their credentials.\n> +\tgit config --show-origin --list\n> +\techo\n\nConfig options to consider stripping out:\n\n\t*url*\n\t*pass* (anything \"password\" but also \"sendmail.smtppass\")\n\n> +\techo \"[Configured Hooks]\"\n> +\tfind \"$GIT_DIR/hooks/\" -type f | grep -v \"\\.sample$\" | print_filenames_and_content\n> +\techo\n\nRemove the sample hooks, but focus on the others. Will this look like garbage if a hook\nis a binary file?\n\n> +\n> +\techo \"[Git Logs]\"\n> +\tfind \"$GIT_DIR/logs\" -type f | print_filenames_and_content\n> +\techo\n\nAs mentioned before, I've sometimes found it helpful to know the data shape for the object\nstore. Having a few extra steps such as the following could be nice:\n\n\techo \"[Loose Objects]\"\n\tfor objdir in $(find \"$GIT_DIR/objects/??\" -type d)\n\tdo\n\t\techo \"$objdir: $(ls $objdir | wc -l)\"\n\tdone\n\techo\n\n\techo \"[Pack Data]\"\n\tls -l \"$GIT_DIR/objects/pack\"\n\techo\n\n\techo \"[Object Info Data]\"\n\tls -lR \"$GIT_DIR/objects/info\"\n\techo\n\n\techo \"[Alternates File]\"\n\techo \"========\"\n\tcat \"$GIT_DIR/objects/info/alternates\"\n\techo\n\nThat last one will collect information on the commit-graph file, even if it is an\nincremental file.\n\n> +\n> +}\n> +\n> +USAGE=\"[-o | --output <path>]\"\n> +\n> +SUBDIRECTORY_OK=t\n> +OPTIONS_SPEC=\n> +. git-sh-setup\n> +. git-sh-i18n\n> +\n> +basedir=\"$PWD\"\n> +while :\n> +do\n> +\tcase \"$1\" in\n> +\t-o|--output)\n> +\t\tshift\n> +\t\tbasedir=\"$1\"\n> +\t\tshift\n> +\t\tcontinue\n> +\t\t;;\n> +\t\"\")\n> +\t\tbreak\n> +\t\t;;\n> +\t*)\n> +\t\tusage\n> +\t\t;;\n> +\tesac\n> +done\n> +\n> +\n> +# Create bugreport file\n> +BUGREPORT_FILE=\"$basedir/git-bugreport-$(whoami)-$(hostname)-$(date -Iminutes)\"\n> +\n> +generate_report_text >$BUGREPORT_FILE\n> +\n> +git_editor $BUGREPORT_FILE\n> +\n> +eval_gettextln \"Your new bug report is in \\$BUGREPORT_FILE.\"\n> diff --git a/t/t0091-bugreport.sh b/t/t0091-bugreport.sh\n> new file mode 100755\n> index 0000000000..6eb2ee4f66\n> --- /dev/null\n> +++ b/t/t0091-bugreport.sh\n> @@ -0,0 +1,41 @@\n> +#!/bin/bash\n> +\n> +test_description='git bugreport'\n> +\n> +. ./test-lib.sh\n> +\n> +# Headers \"[System Info]\" will be followed by a non-empty line if we put some\n> +# information there; we can make sure all our headers were followed by some\n> +# information to check if the command was successful.\n> +HEADER_PATTERN=\"^\\[.*\\]$\"\n> +check_all_headers_populated() {\n> +\twhile read -r line; do\n> +\t\tif [$(grep $HEADER_PATTERN $line)]; then\n> +\t\t\tread -r nextline\n> +\t\t\tif [-z $nextline]; then\n> +\t\t\t\treturn 1;\n> +\t\t\tfi\n> +\t\tfi\n> +\tdone\n> +}\n> +\n> +test_expect_success 'creates a report with content in the right places' '\n> +\tgit bugreport &&\n> +\tcheck_all_headers_populated <git-bugreport-* &&\n> +\trm git-bugreport-*\n> +'\n> +\n> +test_expect_success '--output puts the report in the provided dir' '\n> +\tmkdir foo/ &&\n> +\tgit bugreport -o foo/ &&\n> +\ttest -f foo/git-bugreport-* &&\n> +\trm -fr foo/\n> +'\n> +\n> +test_expect_success 'incorrect arguments abort with usage' '\n> +\ttest_must_fail git bugreport --false 2>output &&\n> +\tgrep usage output &&\n> +\ttest ! -f git-bugreport-*\n> +'\n> +\n> +test_done\n> \n\nI think this is a great start, and I'll take some time later to try it out.\n\nThanks,\n-Stolee\n"},{"id":"380474","messageId":"xmqqy2zu4hrq.fsf@gitster-ct.c.googlers.com","threadId":"51645","inReplyTo":"e6d56d97-99c9-064a-71b5-2b7eb9b71e01@gmail.com","subject":"Re: [PATCH] bugreport: add tool to generate debugging info","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2019-08-15T14:36:57Z","receivedAt":"2019-08-15T14:37:03Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Derrick Stolee <stolee@gmail.com> writes:\n\n> Config options to consider stripping out:\n>\n> \t*url*\n> \t*pass* (anything \"password\" but also \"sendmail.smtppass\")\n\nBlacklisting?  I wonder if users feel safer if these are limited to\nknown-benign ones.\n\n>> +\techo \"[Configured Hooks]\"\n>> +\tfind \"$GIT_DIR/hooks/\" -type f | grep -v \"\\.sample$\" | print_filenames_and_content\n>> +\techo\n>\n> Remove the sample hooks, but focus on the others. Will this look like garbage if a hook\n> is a binary file?\n\nThis makes me feel very nervous.  $GIT_DIR/hooks/ are private and\npeople can hardcode credentials in them; $GIT_DIR/hooks/pre-foo may\nbe written toread from $GIT_DIR/hooks/mypassword with the knowledge\nthat there won't be any \"mypassword\" hook.\n"},{"id":"380486","messageId":"xmqqzhka2tbv.fsf@gitster-ct.c.googlers.com","threadId":"51645","inReplyTo":"20190815023418.33407-1-emilyshaffer@google.com","subject":"Re: [PATCH] bugreport: add tool to generate debugging info","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2019-08-15T18:10:12Z","receivedAt":"2019-08-15T18:10:18Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Emily Shaffer <emilyshaffer@google.com> writes:\n\n> diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt\n> new file mode 100644\n> index 0000000000..c5f45bbee8\n> --- /dev/null\n> +++ b/Documentation/git-bugreport.txt\n> @@ -0,0 +1,48 @@\n> +git-bugreport(1)\n> +================\n> +\n> +NAME\n> +----\n> +git-bugreport - Collect information for user to file a bug report\n> +\n> +SYNOPSIS\n> +--------\n> +[verse]\n> +'git bugreport' [-o | --output <path>]\n> +\n> +DESCRIPTION\n> +-----------\n> +Captures information about the user's machine, Git client, and repository state,\n> +as well as a form requesting information about the behavior the user observed,\n> +into a single text file which the user can then share, for example to the Git\n> +mailing list, in order to report an observed bug.\n> +\n> +The following information is requested from the user:\n> +\n> + - Reproduction steps\n> + - Expected behavior\n> + - Actual behavior\n\n+ - How the above two are different\n\nIt is often helpful to have users explain how the expected and\nactual are different in their own words.\n\n> +NOTE\n> +----\n> +Bug reports can be sent to git@vger.kernel.org.\n\nI am not sure if this belongs here.\n\n> diff --git a/git-bugreport.sh b/git-bugreport.sh\n> new file mode 100755\n> index 0000000000..2200703a51\n> --- /dev/null\n> +++ b/git-bugreport.sh\n> @@ -0,0 +1,86 @@\n> +#!/bin/sh\n> +\n> +print_filenames_and_content() {\n> +while read -r line; do\n> +\techo \"$line\";\n> +\techo \"========\";\n> +\tcat \"$line\";\n> +\techo;\n> +done\n> +}\n\nStyle.\n\n - have SP on both sides of ()\n - one more HT indent for the function body\n - \"do\" on its own line\n - no unnecessary semicolons when LF would do\n\nYou probably are better off asking xargs to do this instead of\nrelying on \"read -r\".\n\n\tfind ... | xargs -n 1 sh -c 'echo \"$1\" && cat \"$1\"' -\n\nor something like that, perhaps.\n\n> +\n> +generate_report_text() {\n> +\n> +\t# Generate a form for the user to fill out with their issue.\n> +\tgettextln \"Thank you for filling out a Git bug report!\"\n> +\tgettextln \"Please answer the following questions to help us understand your issue.\"\n> +\techo\n> +\tgettextln \"What did you do before the bug happened? (Steps to reproduce your issue)\"\n> +\techo\n> +\tgettextln \"What did you expect to happen? (Expected behavior)\"\n> +\techo\n> +\tgettextln \"What happened instead? (Actual behavior)\"\n> +\techo\n> +\tgettextln \"Anything else you want to add:\"\n> +\techo\n> +\tgettextln \"Please review the rest of the bug report below.\"\n> +\tgettextln \"You can delete any lines you don't wish to send.\"\n> +\techo\n\nWould we on the receiving end be able to tell these section headers\nin translated to 47 different languages?  I am sure that i18n is\nused here to encourage non-C-locale users to file bugs in their own\nlanguages, but are we prepared to react to them?\n\n> +\techo \"[System Information]\"\n> +\tgit version --build-options\n> +\tuname -a\n> +\tcurl-config --version\n> +\tldd --version\n\n\"curl-config: command not found\" may be clear enough, but would\nthere be a case where errors from two or more consecutive commands\nin the above sequence would make the output confusing to the person\nsitting on the receiving end?  Would it help, as a convention, to\nalways ahve \"echo [what am I reporting]\" before each of these\ncommands?\n\n> +# Create bugreport file\n> +BUGREPORT_FILE=\"$basedir/git-bugreport-$(whoami)-$(hostname)-$(date -Iminutes)\"\n\nHow portable is -Iminutes?\n\n> diff --git a/t/t0091-bugreport.sh b/t/t0091-bugreport.sh\n> new file mode 100755\n> index 0000000000..6eb2ee4f66\n> --- /dev/null\n> +++ b/t/t0091-bugreport.sh\n> @@ -0,0 +1,41 @@\n> +#!/bin/bash\n\nMake sure /bin/sh suffices to run the test script.\n\n> +test_description='git bugreport'\n> +\n> +. ./test-lib.sh\n> +\n> +# Headers \"[System Info]\" will be followed by a non-empty line if we put some\n> +# information there; we can make sure all our headers were followed by some\n> +# information to check if the command was successful.\n> +HEADER_PATTERN=\"^\\[.*\\]$\"\n> +check_all_headers_populated() {\n> +\twhile read -r line; do\n> +\t\tif [$(grep $HEADER_PATTERN $line)]; then\n\nDocumentation/CodingGuidelines\n\nI am not sure if the traits this test script checks about the\ncontents of the output is all that interesting.  Whenever we add new\nsections to the main command because we want other kinds of\ninformation collected, we'd update this test script because\notherwise the test would fail, but would it result in quality\nbugreport tool, or is it just additional busywork?\n\nIf we decide later that a header and its body needs to be separated\nwith a blank line for better readablity, the check done here would\nalso need to be updated, but again, that does not feel like anything\nmore than just busywork to me.\n\nThe tests for \"-o\" and unknown options do make sense, though.\n\nThanks.\n\n> +# Headers \"[System Info]\" will be followed by a non-empty line if we put some\n> +# information there; we can make sure all our headers were followed by some\n> +# information to check if the command was successful.\n> +HEADER_PATTERN=\"^\\[.*\\]$\"\n> +check_all_headers_populated() {\n> +\twhile read -r line; do\n> +\t\tif [$(grep $HEADER_PATTERN $line)]; then\n> +\t\t\tread -r nextline\n> +\t\t\tif [-z $nextline]; then\n> +\t\t\t\treturn 1;\n> +\t\t\tfi\n> +\t\tfi\n> +\tdone\n> +}\n> +\n> +test_expect_success 'creates a report with content in the right places' '\n> +\tgit bugreport &&\n> +\tcheck_all_headers_populated <git-bugreport-* &&\n> +\trm git-bugreport-*\n> +'\n> +\n> +test_expect_success '--output puts the report in the provided dir' '\n> +\tmkdir foo/ &&\n> +\tgit bugreport -o foo/ &&\n> +\ttest -f foo/git-bugreport-* &&\n> +\trm -fr foo/\n> +'\n> +\n> +test_expect_success 'incorrect arguments abort with usage' '\n> +\ttest_must_fail git bugreport --false 2>output &&\n> +\tgrep usage output &&\n> +\ttest ! -f git-bugreport-*\n> +'\n> +\n> +test_done\n"},{"id":"380491","messageId":"nycvar.QRO.7.76.6.1908152205390.46@tvgsbejvaqbjf.bet","threadId":"51645","inReplyTo":"e6d56d97-99c9-064a-71b5-2b7eb9b71e01@gmail.com","subject":"Re: [PATCH] bugreport: add tool to generate debugging info","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2019-08-15T20:07:57Z","receivedAt":"2019-08-15T20:08:16Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Thu, 15 Aug 2019, Derrick Stolee wrote:\n\n> On 8/14/2019 10:34 PM, Emily Shaffer wrote:\n> > diff --git a/git-bugreport.sh b/git-bugreport.sh\n> > new file mode 100755\n> > index 0000000000..2200703a51\n> > --- /dev/null\n> > +++ b/git-bugreport.sh\n>\n> At first I was alarmed by \"What? another shell script?\" but this\n> command should prioritize flexibility and extensibility over speed.\n> Running multiple processes shouldn't be too taxing for what we are\n> trying to do here.\n\nGit for Windows sometimes receives bug reports about Bash not being able\nto start (usually it is a DLL base address problem, related to the way\nCygwin and MSYS2 emulate `fork()`).\n\nIn such a case, `git bugreport` would only be able to offer a reason for\nyet another bug report instead of adding useful metadata.\n\nSomething to keep in mind when deciding how to implement this command.\n\nCiao,\nDscho\n"},{"id":"380492","messageId":"20190815201324.GA208753@google.com","threadId":"51645","inReplyTo":"e6d56d97-99c9-064a-71b5-2b7eb9b71e01@gmail.com","subject":"Re: [PATCH] bugreport: add tool to generate debugging info","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2019-08-15T20:13:24Z","receivedAt":"2019-08-15T20:13:31Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"On Thu, Aug 15, 2019 at 10:15:24AM -0400, Derrick Stolee wrote:\n\n> >  - Do we want to advertise the Git mailing list for bug reports?\n> \n> That is possible. Isn't there another mailing list for git users?\n\nI know there's an IRC channel for Git users, I dunno about mailing list.\nI'm worried that places I name as points of contact will grow stale,\nalthough I suppose #git on freenode isn't really going anywhere, for\nexample. (But as a counterexample, I hope that nobody sends something\nlike this to #git-devel, which seems to have a lower population than\nthis mailing list.)\n\n> \n> I could see a patch added on top of this for git-for-windows/git that\n> changes the instructions to create issues on GitHub.\n\nIndeed; I imagine we'd probably like to patch it to ask for bugs in our\nbug tracker.\n\n> \n> >  - Which config options should we filter to avoid accidentally receiving\n> >    credentials?\n> \n> The remote URLs are pretty sensitive. Not only do users sometimes put passwords\n> or PATs into their URLs, the literal name of the repo could be a secret.\n\nNow here's where I start to wonder. We often debug internally by asking\nfor the remote URL and replicating the issue there, which is why I\nmentioned it explicitly in the commit message. But I hadn't considered\nfolks including the password in the URL.\n\nWell, I suppose anybody can keep a local patch to change the config\nfilter pattern. I'll try to make it easy to spot and modify.\n\n[snip]\n\n> At first I was alarmed by \"What? another shell script?\" but this command should\n> prioritize flexibility and extensibility over speed. Running multiple processes\n> shouldn't be too taxing for what we are trying to do here.\n\nIf shell scripts are entirely deprecated I can convert it, but doing it\nin C seemed like overkill when I really just wanted \"what are all the\ncommands we would ask the user to run and tell us the output?\". I\nfigured also that it would be a little more immune to bitrot to output\nthe contents of porcelain commands here.\n\n> > +\techo \"[Git Config]\"\n> > +\t# TODO: Pass this through grep -v to avoid users sending us their credentials.\n> > +\tgit config --show-origin --list\n> > +\techo\n> \n> Config options to consider stripping out:\n> \n> \t*url*\n> \t*pass* (anything \"password\" but also \"sendmail.smtppass\")\n\nDone, thanks.\n\n> \n> > +\techo \"[Configured Hooks]\"\n> > +\tfind \"$GIT_DIR/hooks/\" -type f | grep -v \"\\.sample$\" | print_filenames_and_content\n> > +\techo\n> \n> Remove the sample hooks, but focus on the others. Will this look like garbage if a hook\n> is a binary file?\n\nYeah, I'm sure it will. I'll add a check to\nprint_filenames_and_content() so it can tell us if there is a hook\ninstalled there, even if we can't see the content.\n\n> \n> > +\n> > +\techo \"[Git Logs]\"\n> > +\tfind \"$GIT_DIR/logs\" -type f | print_filenames_and_content\n> > +\techo\n> \n> As mentioned before, I've sometimes found it helpful to know the data shape for the object\n> store. Having a few extra steps such as the following could be nice:\n> \n> \techo \"[Loose Objects]\"\n> \tfor objdir in $(find \"$GIT_DIR/objects/??\" -type d)\n> \tdo\n> \t\techo \"$objdir: $(ls $objdir | wc -l)\"\n`echo \"$objdir: $(ls $objdir | wc -l) objects`\nI'll add context so we don't need to have the bugreport script memorized\nin order to read a bugreport :)\n> \tdone\n> \techo\n> \n> \techo \"[Pack Data]\"\n> \tls -l \"$GIT_DIR/objects/pack\"\n> \techo\n> \n> \techo \"[Object Info Data]\"\n> \tls -lR \"$GIT_DIR/objects/info\"\n> \techo\n> \n> \techo \"[Alternates File]\"\n> \techo \"========\"\n> \tcat \"$GIT_DIR/objects/info/alternates\"\n> \techo\n> \n> That last one will collect information on the commit-graph file, even if it is an\n> incremental file.\n\nThanks Stolee, these are awesome and exactly the kind of feedback I was\nhoping for.\n\n> \n> I think this is a great start, and I'll take some time later to try it out.\n> \n> Thanks,\n> -Stolee\n\nAwesome. I'm excited to hear how it goes.\n\n - Emily\n"},{"id":"380520","messageId":"20190815215225.GB208753@google.com","threadId":"51645","inReplyTo":"xmqqzhka2tbv.fsf@gitster-ct.c.googlers.com","subject":"Re: [PATCH] bugreport: add tool to generate debugging info","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2019-08-15T21:52:25Z","receivedAt":"2019-08-15T21:52:32Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"On Thu, Aug 15, 2019 at 11:10:12AM -0700, Junio C Hamano wrote:\n> Emily Shaffer <emilyshaffer@google.com> writes:\n> \n> > diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt\n> > new file mode 100644\n> > index 0000000000..c5f45bbee8\n> > --- /dev/null\n> > +++ b/Documentation/git-bugreport.txt\n> > @@ -0,0 +1,48 @@\n> > +git-bugreport(1)\n> > +================\n> > +\n> > +NAME\n> > +----\n> > +git-bugreport - Collect information for user to file a bug report\n> > +\n> > +SYNOPSIS\n> > +--------\n> > +[verse]\n> > +'git bugreport' [-o | --output <path>]\n> > +\n> > +DESCRIPTION\n> > +-----------\n> > +Captures information about the user's machine, Git client, and repository state,\n> > +as well as a form requesting information about the behavior the user observed,\n> > +into a single text file which the user can then share, for example to the Git\n> > +mailing list, in order to report an observed bug.\n> > +\n> > +The following information is requested from the user:\n> > +\n> > + - Reproduction steps\n> > + - Expected behavior\n> > + - Actual behavior\n> \n> + - How the above two are different\n> \n> It is often helpful to have users explain how the expected and\n> actual are different in their own words.\n\nGood point; added.\n\n> \n> > +NOTE\n> > +----\n> > +Bug reports can be sent to git@vger.kernel.org.\n> \n> I am not sure if this belongs here.\n\nSure, I wasn't certain either. Would you rather I remove the \"what to do\nwith this bugreport\" NOTE section entirely? I worry that folks will\nthink this generated text file is being magically reported behind the\nscenes, or that they won't know what to do with the file now that it's\ncreated.\n\nI suppose I did mention in the DESCRIPTION section that the user can\nshare the generated text file to report a bug. Maybe I should remove\n\"for example to the Git ML\" from there as well.\n\n> \n> > diff --git a/git-bugreport.sh b/git-bugreport.sh\n> > new file mode 100755\n> > index 0000000000..2200703a51\n> > --- /dev/null\n> > +++ b/git-bugreport.sh\n> > @@ -0,0 +1,86 @@\n> > +#!/bin/sh\n> > +\n> > +print_filenames_and_content() {\n> > +while read -r line; do\n> > +\techo \"$line\";\n> > +\techo \"========\";\n> > +\tcat \"$line\";\n> > +\techo;\n> > +done\n> > +}\n> \n> Style.\n> \n>  - have SP on both sides of ()\n>  - one more HT indent for the function body\n>  - \"do\" on its own line\n>  - no unnecessary semicolons when LF would do\n> \n> You probably are better off asking xargs to do this instead of\n> relying on \"read -r\".\n> \n> \tfind ... | xargs -n 1 sh -c 'echo \"$1\" && cat \"$1\"' -\n> \n> or something like that, perhaps.\n\nHm. In responding to Stolee's comments, I ended up replacing \"cat $line\"\nwith another function, and it seems xargs isn't happy directing to\nfunctions which haven't been exported, because it spins a new process\nfor each argument it's passed. I had originally intended to do this with\nxargs but then didn't want to pack many lines into one xargs call.\n\nThe way the code is in PS1 I think it's a reasonable replacement, but\nthe code in PS2 is less so - there's a check for whether a file given is\nbinary or whether it exists at all (the former for hooks, which I think\nyou were worried about, and the latter for the alternates file, which I\nfound I do not have.)\n\nWhy is \"read -r\" unreliable? Is it better to export a function and use\nxargs, compared to using \"read -r\" in a loop like this? I was worried\nthat exporting a function is similar to namespace pollution in C++, but\nmaybe I'm mistaken.\n\n> \n> > +\n> > +generate_report_text() {\n> > +\n> > +\t# Generate a form for the user to fill out with their issue.\n> > +\tgettextln \"Thank you for filling out a Git bug report!\"\n> > +\tgettextln \"Please answer the following questions to help us understand your issue.\"\n> > +\techo\n> > +\tgettextln \"What did you do before the bug happened? (Steps to reproduce your issue)\"\n> > +\techo\n> > +\tgettextln \"What did you expect to happen? (Expected behavior)\"\n> > +\techo\n> > +\tgettextln \"What happened instead? (Actual behavior)\"\n> > +\techo\n> > +\tgettextln \"Anything else you want to add:\"\n> > +\techo\n> > +\tgettextln \"Please review the rest of the bug report below.\"\n> > +\tgettextln \"You can delete any lines you don't wish to send.\"\n> > +\techo\n> \n> Would we on the receiving end be able to tell these section headers\n> in translated to 47 different languages?  I am sure that i18n is\n> used here to encourage non-C-locale users to file bugs in their own\n> languages, but are we prepared to react to them?\n\nI hope that we could accept reports in many languages with online\ntranslation as a guide, but practically speaking,\nmachine-learning-powered online translation may not be as good for\ntechnical topics as it is for conversational.\n\nAt the same time, it seems unnecessarily restrictive to ask for bugs to\ncome in English. My heart thinks it's better to encourage other\nlanguages too.\n\n> \n> > +\techo \"[System Information]\"\n> > +\tgit version --build-options\n> > +\tuname -a\n> > +\tcurl-config --version\n> > +\tldd --version\n> \n> \"curl-config: command not found\" may be clear enough, but would\n> there be a case where errors from two or more consecutive commands\n> in the above sequence would make the output confusing to the person\n> sitting on the receiving end?  Would it help, as a convention, to\n> always ahve \"echo [what am I reporting]\" before each of these\n> commands?\n\nYeah, I think that's a good point.\n\nIn responding to Stolee's review, I mentioned adding additional context\nto some command's output so that \"those reviewing the report don't need\nto be intimately familiar with what the bugreport script runs\". But it\nmight actually be most useful to echo the command that's being run\neverywhere.\n\nI imagine it's fine to include the [] header as well, to make it easier\nfor the user to understand what was generated.\n\n> \n> > +# Create bugreport file\n> > +BUGREPORT_FILE=\"$basedir/git-bugreport-$(whoami)-$(hostname)-$(date -Iminutes)\"\n> \n> How portable is -Iminutes?\n\nAh, it seems while it works in MingW and bash, it doesn't work in zsh.\nI'll figure something else out. Thanks.\n\n> \n> > diff --git a/t/t0091-bugreport.sh b/t/t0091-bugreport.sh\n> > new file mode 100755\n> > index 0000000000..6eb2ee4f66\n> > --- /dev/null\n> > +++ b/t/t0091-bugreport.sh\n> > @@ -0,0 +1,41 @@\n> > +#!/bin/bash\n> \n> Make sure /bin/sh suffices to run the test script.\n\nWill do. Thanks.\n\n> \n> > +test_description='git bugreport'\n> > +\n> > +. ./test-lib.sh\n> > +\n> > +# Headers \"[System Info]\" will be followed by a non-empty line if we put some\n> > +# information there; we can make sure all our headers were followed by some\n> > +# information to check if the command was successful.\n> > +HEADER_PATTERN=\"^\\[.*\\]$\"\n> > +check_all_headers_populated() {\n> > +\twhile read -r line; do\n> > +\t\tif [$(grep $HEADER_PATTERN $line)]; then\n> \n> Documentation/CodingGuidelines\n> \n> I am not sure if the traits this test script checks about the\n> contents of the output is all that interesting.  Whenever we add new\n> sections to the main command because we want other kinds of\n> information collected, we'd update this test script because\n> otherwise the test would fail, but would it result in quality\n> bugreport tool, or is it just additional busywork?\n\nI'm a little confused about this - I checked for a general [...] header\nstanding alone on a line specifically so that we wouldn't need to update\nthe test script when adding a new section. What do you see that will\ngrow stale?\n\n> \n> If we decide later that a header and its body needs to be separated\n> with a blank line for better readablity, the check done here would\n> also need to be updated, but again, that does not feel like anything\n> more than just busywork to me.\n\nSure, I agree with that.\n\nSo, what's your suggestion? Not to check the output at all? (This may\nactually be fine; it occurred to me while reading your review that if a\nuser is filing a bug report about something, one of the diagnostic\ncommands in bugreport might be what's broken for them. So perhaps it\nshould be tolerant to missing information...)\n\n> \n> The tests for \"-o\" and unknown options do make sense, though.\n> \n> Thanks.\nThanks for reviewing.\n\n - Emily\n"},{"id":"380525","messageId":"20190815222427.GC208753@google.com","threadId":"51645","inReplyTo":"nycvar.QRO.7.76.6.1908152205390.46@tvgsbejvaqbjf.bet","subject":"Re: [PATCH] bugreport: add tool to generate debugging info","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2019-08-15T22:24:27Z","receivedAt":"2019-08-15T22:24:35Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"On Thu, Aug 15, 2019 at 10:07:57PM +0200, Johannes Schindelin wrote:\n> Hi,\n> \n> On Thu, 15 Aug 2019, Derrick Stolee wrote:\n> \n> > On 8/14/2019 10:34 PM, Emily Shaffer wrote:\n> > > diff --git a/git-bugreport.sh b/git-bugreport.sh\n> > > new file mode 100755\n> > > index 0000000000..2200703a51\n> > > --- /dev/null\n> > > +++ b/git-bugreport.sh\n> >\n> > At first I was alarmed by \"What? another shell script?\" but this\n> > command should prioritize flexibility and extensibility over speed.\n> > Running multiple processes shouldn't be too taxing for what we are\n> > trying to do here.\n> \n> Git for Windows sometimes receives bug reports about Bash not being able\n> to start (usually it is a DLL base address problem, related to the way\n> Cygwin and MSYS2 emulate `fork()`).\n\nHmm. In a case like this, then, how is someone using GfW at all?\nEmbarrassingly, I don't actually have a way to try it out for myself.\nIt seems there's no GUI, and users are using it through the command line\nin mingw, so when you say \"bash doesn't start\" do you mean \"users can't\nuse the mingw prompt at all\"?\n\n> \n> In such a case, `git bugreport` would only be able to offer a reason for\n> yet another bug report instead of adding useful metadata.\n> \n> Something to keep in mind when deciding how to implement this command.\n> \n> Ciao,\n> Dscho\n\nYeah, that's an interesting point, thanks for bringing it up. So, in the\ncase when bash won't start at all, what does that failure look like? How\nmuch of Git can users still use? For example, would they be able to get\nfar enough to run a binary git-bugreport?\n\n - Emily\n"},{"id":"380526","messageId":"xmqq7e7e2hbv.fsf@gitster-ct.c.googlers.com","threadId":"51645","inReplyTo":"20190815215225.GB208753@google.com","subject":"Re: [PATCH] bugreport: add tool to generate debugging info","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2019-08-15T22:29:24Z","receivedAt":"2019-08-15T22:29:30Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Emily Shaffer <emilyshaffer@google.com> writes:\n\n>> > +NOTE\n>> > +----\n>> > +Bug reports can be sent to git@vger.kernel.org.\n>> \n>> I am not sure if this belongs here.\n>\n> Sure, I wasn't certain either. Would you rather I remove the \"what to do\n> with this bugreport\" NOTE section entirely?\n\nNot really.  You are invoking an editor to let the user edit the\npre-populated report, and I would imagine that a comment in that\nfile would be the best place to give instructions, not a manpage\nfor \"git bugreport\" command.\n\n> So, what's your suggestion? Not to check the output at all? (This may\n> actually be fine; it occurred to me while reading your review that if a\n> user is filing a bug report about something, one of the diagnostic\n> commands in bugreport might be what's broken for them. So perhaps it\n> should be tolerant to missing information...)\n\nRight.\n"},{"id":"380527","messageId":"20190815225231.GD208753@google.com","threadId":"51645","inReplyTo":"xmqqy2zu4hrq.fsf@gitster-ct.c.googlers.com","subject":"Re: [PATCH] bugreport: add tool to generate debugging info","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2019-08-15T22:52:31Z","receivedAt":"2019-08-15T22:52:38Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"On Thu, Aug 15, 2019 at 07:36:57AM -0700, Junio C Hamano wrote:\n> Derrick Stolee <stolee@gmail.com> writes:\n> \n> > Config options to consider stripping out:\n> >\n> > \t*url*\n> > \t*pass* (anything \"password\" but also \"sendmail.smtppass\")\n> \n> Blacklisting?  I wonder if users feel safer if these are limited to\n> known-benign ones.\n\nI think a whitelist of config options to print would grow stale\nimmediately, and the options we're missing would be very likely to be\nconfigs to turn on new experimental features - which is probably what we\nmost want the bugreport for.\n\n> \n> >> +\techo \"[Configured Hooks]\"\n> >> +\tfind \"$GIT_DIR/hooks/\" -type f | grep -v \"\\.sample$\" | print_filenames_and_content\n> >> +\techo\n> >\n> > Remove the sample hooks, but focus on the others. Will this look like garbage if a hook\n> > is a binary file?\n> \n> This makes me feel very nervous.  $GIT_DIR/hooks/ are private and\n> people can hardcode credentials in them; $GIT_DIR/hooks/pre-foo may\n> be written toread from $GIT_DIR/hooks/mypassword with the knowledge\n> that there won't be any \"mypassword\" hook.\n\nHmm. I think the list of valid hooks isn't one that changes often, but\nit's also not enumerated in some machine-parseable way - it exists in\nDocumentation/githooks.txt but that's all. I'd still be a little worried\nabout bitrot... I think it's probably better to list the filenames in\n$GIT_DIR/hooks but not print their contents. I'll modify it.\n\n - Emily\n"},{"id":"380528","messageId":"20190815225413.GE208753@google.com","threadId":"51645","inReplyTo":"xmqq7e7e2hbv.fsf@gitster-ct.c.googlers.com","subject":"Re: [PATCH] bugreport: add tool to generate debugging info","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2019-08-15T22:54:13Z","receivedAt":"2019-08-15T22:54:21Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"On Thu, Aug 15, 2019 at 03:29:24PM -0700, Junio C Hamano wrote:\n> Emily Shaffer <emilyshaffer@google.com> writes:\n> \n> >> > +NOTE\n> >> > +----\n> >> > +Bug reports can be sent to git@vger.kernel.org.\n> >> \n> >> I am not sure if this belongs here.\n> >\n> > Sure, I wasn't certain either. Would you rather I remove the \"what to do\n> > with this bugreport\" NOTE section entirely?\n> \n> Not really.  You are invoking an editor to let the user edit the\n> pre-populated report, and I would imagine that a comment in that\n> file would be the best place to give instructions, not a manpage\n> for \"git bugreport\" command.\n\nOh, I see! In that case, do you still want the Git mailing list address\nshown in the report text?\n\n> \n> > So, what's your suggestion? Not to check the output at all? (This may\n> > actually be fine; it occurred to me while reading your review that if a\n> > user is filing a bug report about something, one of the diagnostic\n> > commands in bugreport might be what's broken for them. So perhaps it\n> > should be tolerant to missing information...)\n> \n> Right.\n\nOk, will do.\n"},{"id":"380529","messageId":"xmqqwofe0zgd.fsf@gitster-ct.c.googlers.com","threadId":"51645","inReplyTo":"20190815225231.GD208753@google.com","subject":"Re: [PATCH] bugreport: add tool to generate debugging info","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2019-08-15T23:40:50Z","receivedAt":"2019-08-15T23:40:55Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Emily Shaffer <emilyshaffer@google.com> writes:\n\n> I think a whitelist of config options to print would grow stale\n> immediately, and the options we're missing would be very likely to be\n> configs to turn on new experimental features - which is probably what we\n> most want the bugreport for.\n\nThe implementation of your \"git bugreport\" command comes from the\nsame version of Git as such an experimental feature you want\nfeedback to is shipped with, so I am not sure where your concern\nabout staleness comes from.\n\nIf you really care about getting quality reports, you need to earn\nusers' trust (one way of doing so would be to maintain a good \"list\nof benign configuration knobs whose values help diagnosing issues\"),\nand you need to make sure we obtain relevant pieces of information.\nBoth of these things are something you need to actively work on.\n\nWe have trained ourselves to the point that we consider it a bug if\nyou add a new command git-bugreport without adding a corresponding\npattern in the .gitignore file.  And thanks to that discipline, we\nhave been fairly good at keeping .gitignore up to date.  I do not\nsee a reason why we cannot do something similar to the registry of\nconfiguration variables that we care about, which may be shipped as\npart of \"git bugreport\" command.\n\n"},{"id":"380532","messageId":"20190816012519.GF208753@google.com","threadId":"51645","inReplyTo":"xmqqwofe0zgd.fsf@gitster-ct.c.googlers.com","subject":"Re: [PATCH] bugreport: add tool to generate debugging info","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2019-08-16T01:25:19Z","receivedAt":"2019-08-16T01:25:27Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"On Thu, Aug 15, 2019 at 04:40:50PM -0700, Junio C Hamano wrote:\n> Emily Shaffer <emilyshaffer@google.com> writes:\n> \n> > I think a whitelist of config options to print would grow stale\n> > immediately, and the options we're missing would be very likely to be\n> > configs to turn on new experimental features - which is probably what we\n> > most want the bugreport for.\n> \n> The implementation of your \"git bugreport\" command comes from the\n> same version of Git as such an experimental feature you want\n> feedback to is shipped with, so I am not sure where your concern\n> about staleness comes from.\n> \n> If you really care about getting quality reports, you need to earn\n> users' trust (one way of doing so would be to maintain a good \"list\n> of benign configuration knobs whose values help diagnosing issues\"),\n> and you need to make sure we obtain relevant pieces of information.\n> Both of these things are something you need to actively work on.\n> \n> We have trained ourselves to the point that we consider it a bug if\n> you add a new command git-bugreport without adding a corresponding\n> pattern in the .gitignore file.  And thanks to that discipline, we\n> have been fairly good at keeping .gitignore up to date.  I do not\n> see a reason why we cannot do something similar to the registry of\n> configuration variables that we care about, which may be shipped as\n> part of \"git bugreport\" command.\n\nI think comparing this habit to the .gitignore isn't quite fair -\n.gitignore tells me I forgot to add my new command binary to it, when I\nrun `git status` to see what I need to add to my commit with new\ncommand.\n\nBut, I agree that it's not going to be possible to create a completely\nleakproof blacklisting heuristic here. So I'll use a whitelist for the\nnext patchset.\n\n - Emily\n"},{"id":"380538","messageId":"xmqqpnl512re.fsf@gitster-ct.c.googlers.com","threadId":"51645","inReplyTo":"20190816012519.GF208753@google.com","subject":"Re: [PATCH] bugreport: add tool to generate debugging info","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2019-08-16T16:41:41Z","receivedAt":"2019-08-16T16:41:51Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Emily Shaffer <emilyshaffer@google.com> writes:\n\n> I think comparing this habit to the .gitignore isn't quite fair -\n> .gitignore tells me I forgot to add my new command binary to it, when I\n> run `git status` to see what I need to add to my commit with new\n> command.\n\nThat is why I said that we need to actively work on, if we care\nabout getting quality reports.\n\nI do not think it is unreasonable to expect the build procedure for\n\"git bugreport\" to involve scanning in Documentation/config/ to pick\nup variable names, annotated in such a way that is invisible to\nAsciiDoc to allow us tell which ones are sensitive and which ones\nare not.  A test in t/ could even check if a documented\nconfiguration variable has such an annotation.  A commit that adds\nconfiguration variables without documentiong them does exist, but\nvariables without documentation are (1) bugs, and (2) are not worth\nserious engineering effort on until they get documented.\n\nI am not saying that you must implement the whitelist exactly like\nthe above.  I am not even saying that this must be whitelist and not\nblacklist---you'd have the same issue maintaining the list anyway.\n\nThe above is merely to illustrate the reason why I think that the\nkind of \"active work\" to make sure that the list will not go stale\nwould be feasible.\n\nThanks.\n"},{"id":"380544","messageId":"20190816190810.GG208753@google.com","threadId":"51645","inReplyTo":"xmqqpnl512re.fsf@gitster-ct.c.googlers.com","subject":"Re: [PATCH] bugreport: add tool to generate debugging info","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2019-08-16T19:08:10Z","receivedAt":"2019-08-16T19:08:17Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"On Fri, Aug 16, 2019 at 09:41:41AM -0700, Junio C Hamano wrote:\n> Emily Shaffer <emilyshaffer@google.com> writes:\n> \n> > I think comparing this habit to the .gitignore isn't quite fair -\n> > .gitignore tells me I forgot to add my new command binary to it, when I\n> > run `git status` to see what I need to add to my commit with new\n> > command.\n> \n> That is why I said that we need to actively work on, if we care\n> about getting quality reports.\n> \n> I do not think it is unreasonable to expect the build procedure for\n> \"git bugreport\" to involve scanning in Documentation/config/ to pick\n> up variable names, annotated in such a way that is invisible to\n> AsciiDoc to allow us tell which ones are sensitive and which ones\n> are not.  A test in t/ could even check if a documented\n> configuration variable has such an annotation.  A commit that adds\n> configuration variables without documentiong them does exist, but\n> variables without documentation are (1) bugs, and (2) are not worth\n> serious engineering effort on until they get documented.\n\nInteresting. I think I have an idea for a way to do this, but it ends up\nfairly large; I'll send proof of concept as a follow-on patch to this\none. Thanks for the suggestions.\n\n - Emily\n"},{"id":"380550","messageId":"nycvar.QRO.7.76.6.1908162204380.46@tvgsbejvaqbjf.bet","threadId":"51645","inReplyTo":"20190815222427.GC208753@google.com","subject":"Re: [PATCH] bugreport: add tool to generate debugging info","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2019-08-16T20:19:31Z","receivedAt":"2019-08-16T20:19:50Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Emily,\n\nOn Thu, 15 Aug 2019, Emily Shaffer wrote:\n\n> On Thu, Aug 15, 2019 at 10:07:57PM +0200, Johannes Schindelin wrote:\n> >\n> > On Thu, 15 Aug 2019, Derrick Stolee wrote:\n> >\n> > > On 8/14/2019 10:34 PM, Emily Shaffer wrote:\n> > > > diff --git a/git-bugreport.sh b/git-bugreport.sh\n> > > > new file mode 100755\n> > > > index 0000000000..2200703a51\n> > > > --- /dev/null\n> > > > +++ b/git-bugreport.sh\n> > >\n> > > At first I was alarmed by \"What? another shell script?\" but this\n> > > command should prioritize flexibility and extensibility over speed.\n> > > Running multiple processes shouldn't be too taxing for what we are\n> > > trying to do here.\n> >\n> > Git for Windows sometimes receives bug reports about Bash not being able\n> > to start (usually it is a DLL base address problem, related to the way\n> > Cygwin and MSYS2 emulate `fork()`).\n>\n> Hmm. In a case like this, then, how is someone using GfW at all?\n\nOn Windows, there are native command-line interpreters available that\nare _not_ Bash: PowerShell and CMD.\n\nThese days, you can get away with using Git _without_ a working Unix\nshell most of the time. There are only preciously few commands left that\nare still written as scripts, and most of these seem to be used less\noften than one might think:\n\n\t- submodule\n\t- bisect\n\t- filter-branch\n\t- instaweb\n\t- mergetool\n\t- some uncommon merge strategies\n\t- rebase -p (already deprecated)\n\t- several CVS/Subversion/Arch/Quilt integrations\n\t- send-email (only relevant for mailing list centric projects,\n\t  again, not very common any longer)\n\t- request-pull (I would not be surprised if this is specific for\n\t  the Linux project, as if that project was even close to the\n\t  main user of Git)\n\nSo as long as you don't use submodules, and as long as you are unaware\nof the `bisect` command, which would comprise the majority of Git users\nin my experience, you can totally use Git for Windows without using Bash\nor Perl, i.e. without using the Cygwin/MSYS2 runtime that seems to be\none of the common causes for trouble in Git for Windows.\n\n> Embarrassingly, I don't actually have a way to try it out for myself.\n> It seems there's no GUI, and users are using it through the command line\n> in mingw, so when you say \"bash doesn't start\" do you mean \"users can't\n> use the mingw prompt at all\"?\n\nThere is no MinGW prompt.\n\nMinGW stands for \"Minimal GNU on Windows\", but it really refers to a way\nto compile native Win32 programs via GCC. Meaning that some of the POSIX\nfunctionality Unix/Linux developers have come to expect is unavailable.\nFor example, `fork()`. Which means that there is no native port of Bash\nto Windows, at least not that _I_ am aware of.\n\nGit for Windows comes with a \"Git Bash\", which is essentially a Bash\nbuilt against the MSYS2 runtime (i.e. it is much more like a Cygwin\nexecutable than like a Win32 executable).\n\nThe common way to run Git for Windows is actually via `git.exe`,\nindependent of what particular command-line interpreter you prefer.\n\n> > In such a case, `git bugreport` would only be able to offer a reason for\n> > yet another bug report instead of adding useful metadata.\n> >\n> > Something to keep in mind when deciding how to implement this command.\n>\n> Yeah, that's an interesting point, thanks for bringing it up. So, in the\n> case when bash won't start at all, what does that failure look like? How\n> much of Git can users still use? For example, would they be able to get\n> far enough to run a binary git-bugreport?\n\nSee above.\n\nIf I were you, I would not write `git bugreport` as anything but a\nbuilt-in, written in C. Unless there are _really_ good reasons not to\n[*1*].  And quite honestly, I don't see any such reasons.\n\nCiao,\nDscho\n\nFootnote *1*: One good reason to use the MSYS2 Bash would be to be able\nto use MSYS2's POSIX emulation layer, e.g. to talk to the SSH agent\n(that is also an MSYS2 program, meaning that it communicates via\nemulated Unix sockets, which is a facility not available to native Win32\nprograms).\n\nAnother valid reason to use the MSYS2 Bash is to be able to talk to the\n(emulated) pseudo TTYs provided by the MSYS2 runtime, e.g. when running\ninside a MinTTY window, which is the default for Git Bash in Git for\nWindows.\n"},{"id":"380570","messageId":"20190817003926.102944-1-emilyshaffer@google.com","threadId":"51645","inReplyTo":"20190815023418.33407-1-emilyshaffer@google.com","subject":"[PATCH v2 0/2] add git-bugreport tool","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2019-08-17T00:39:24Z","receivedAt":"2019-08-17T00:39:56Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"Having read Dscho's response after already writing this reroll, I think\nit is a compelling case to rewrite as a C builtin, although I'm worried\na little about the best way to continue to call porcelain commands to\nprotect against bitrot.\n\nRegardless, I think the proof of concept in patch 2/2 still deserves\nreview; likely a reimplementation in C would see\ngit-bugreport-config-whitelist generated in a similar way, but as a\nheader instead of a textfile. So I'm still very interested in comments\non the direction and the Makefile changes there.\n\nThe patch 1/2 changes are primarily based on Stolee and Junio's\nsuggestions; thanks both. There's some rewording of manpage and commit\nmessage, as well as fixes when I realized this command did not work as\nexpected from a worktree created with `git worktree add`. I also moved\nthe reflog contents to the very bottom of the output as they're likely\nharmless, but very verbose; I didn't want the user to stop reading and\nnever examine the shorter and scarier generated output, like the config.\n\nThanks for thoughts.\n - Emily\n\nEmily Shaffer (2):\n  bugreport: add tool to generate debugging info\n  bugreport: generate config whitelist based on docs\n\n .gitignore                             |   2 +\n Documentation/config/sendemail.txt     |  68 ++++++------\n Documentation/git-bugreport.txt        |  51 +++++++++\n Makefile                               |  10 +-\n bugreport-generate-config-whitelist.sh |   4 +\n command-list.txt                       |   1 +\n git-bugreport.sh                       | 139 +++++++++++++++++++++++++\n git-sh-setup.sh                        |   2 +\n t/t0091-bugreport.sh                   |  25 +++++\n 9 files changed, 267 insertions(+), 35 deletions(-)\n create mode 100644 Documentation/git-bugreport.txt\n create mode 100755 bugreport-generate-config-whitelist.sh\n create mode 100755 git-bugreport.sh\n create mode 100755 t/t0091-bugreport.sh\n\n-- \n2.23.0.rc1.153.gdeed80330f-goog\n\n"},{"id":"380571","messageId":"20190817003926.102944-3-emilyshaffer@google.com","threadId":"51645","inReplyTo":"20190817003926.102944-1-emilyshaffer@google.com","subject":"[PATCH v2 2/2] bugreport: generate config whitelist based on docs","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2019-08-17T00:39:26Z","receivedAt":"2019-08-17T00:39:58Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"Add a new step to the build to generate a whitelist of git-config\nvariables which are appropriate to include in the output of\ngit-bugreport. New variables can be added to the whitelist by annotating\ntheir documentation in Documentation/config with the line\n\"// bugreport-include\".\n\nSome configs are private in nature, and can contain remote URLs,\npasswords, or other sensitive information. In the event that a user\ndoesn't notice their information while reviewing a bugreport, that user\nmay leak their credentials to other individuals, mailing lists, or bug\ntracking tools inadvertently. Heuristic blacklisting of configuration\nkeys is imperfect and prone to false negatives; given the nature of the\ninformation which can be leaked, a whitelist is more reliable.\n\nIn order to prevent staleness of the whitelist, add a mechanism to\ngenerate the whitelist from annotations in the config documentation,\nwhere contributors are already used to documenting their new config\nkeys.\n\nAdditionally, add annotations to the sendemail config documentation in\norder to demonstrate a proof of concept.\n\nSigned-off-by: Emily Shaffer <emilyshaffer@google.com>\n---\n .gitignore                             |  1 +\n Documentation/config/sendemail.txt     | 68 +++++++++++++-------------\n Makefile                               |  9 +++-\n bugreport-generate-config-whitelist.sh |  4 ++\n git-bugreport-config-whitelist         | 22 ---------\n 5 files changed, 47 insertions(+), 57 deletions(-)\n create mode 100755 bugreport-generate-config-whitelist.sh\n delete mode 100644 git-bugreport-config-whitelist\n\ndiff --git a/.gitignore b/.gitignore\nindex b4f5433084..65cc74c748 100644\n--- a/.gitignore\n+++ b/.gitignore\n@@ -26,6 +26,7 @@\n /git-blame\n /git-branch\n /git-bugreport\n+/git-bugreport-config-whitelist\n /git-bundle\n /git-cat-file\n /git-check-attr\ndiff --git a/Documentation/config/sendemail.txt b/Documentation/config/sendemail.txt\nindex 0006faf800..69f3e4f219 100644\n--- a/Documentation/config/sendemail.txt\n+++ b/Documentation/config/sendemail.txt\n@@ -1,63 +1,63 @@\n-sendemail.identity::\n+sendemail.identity:: // bugreport-exclude\n \tA configuration identity. When given, causes values in the\n \t'sendemail.<identity>' subsection to take precedence over\n \tvalues in the 'sendemail' section. The default identity is\n \tthe value of `sendemail.identity`.\n \n-sendemail.smtpEncryption::\n+sendemail.smtpEncryption:: // bugreport-include\n \tSee linkgit:git-send-email[1] for description.  Note that this\n \tsetting is not subject to the 'identity' mechanism.\n \n-sendemail.smtpssl (deprecated)::\n+sendemail.smtpssl (deprecated):: // bugreport-exclude\n \tDeprecated alias for 'sendemail.smtpEncryption = ssl'.\n \n-sendemail.smtpsslcertpath::\n+sendemail.smtpsslcertpath:: // bugreport-exclude\n \tPath to ca-certificates (either a directory or a single file).\n \tSet it to an empty string to disable certificate verification.\n \n-sendemail.<identity>.*::\n+sendemail.<identity>.*:: // bugreport-exclude\n \tIdentity-specific versions of the 'sendemail.*' parameters\n \tfound below, taking precedence over those when this\n \tidentity is selected, through either the command-line or\n \t`sendemail.identity`.\n \n-sendemail.aliasesFile::\n-sendemail.aliasFileType::\n-sendemail.annotate::\n-sendemail.bcc::\n-sendemail.cc::\n-sendemail.ccCmd::\n-sendemail.chainReplyTo::\n-sendemail.confirm::\n-sendemail.envelopeSender::\n-sendemail.from::\n-sendemail.multiEdit::\n-sendemail.signedoffbycc::\n-sendemail.smtpPass::\n-sendemail.suppresscc::\n-sendemail.suppressFrom::\n-sendemail.to::\n-sendemail.tocmd::\n-sendemail.smtpDomain::\n-sendemail.smtpServer::\n-sendemail.smtpServerPort::\n-sendemail.smtpServerOption::\n-sendemail.smtpUser::\n-sendemail.thread::\n-sendemail.transferEncoding::\n-sendemail.validate::\n-sendemail.xmailer::\n+sendemail.aliasesFile:: // bugreport-exclude\n+sendemail.aliasFileType:: // bugreport-exclude\n+sendemail.annotate:: // bugreport-include\n+sendemail.bcc:: // bugreport-include\n+sendemail.cc:: // bugreport-include\n+sendemail.ccCmd:: // bugreport-include\n+sendemail.chainReplyTo:: // bugreport-include\n+sendemail.confirm:: // bugreport-include\n+sendemail.envelopeSender:: // bugreport-include\n+sendemail.from:: // bugreport-include\n+sendemail.multiEdit:: // bugreport-include\n+sendemail.signedoffbycc:: // bugreport-include\n+sendemail.smtpPass:: // bugreport-exclude\n+sendemail.suppresscc:: // bugreport-include\n+sendemail.suppressFrom:: // bugreport-include\n+sendemail.to:: // bugreport-include\n+sendemail.tocmd:: // bugreport-include\n+sendemail.smtpDomain:: // bugreport-include\n+sendemail.smtpServer:: // bugreport-include\n+sendemail.smtpServerPort:: // bugreport-include\n+sendemail.smtpServerOption:: // bugreport-include\n+sendemail.smtpUser:: // bugreport-exclude\n+sendemail.thread:: // bugreport-include\n+sendemail.transferEncoding:: // bugreport-include\n+sendemail.validate:: // bugreport-include\n+sendemail.xmailer:: // bugreport-include\n \tSee linkgit:git-send-email[1] for description.\n \n-sendemail.signedoffcc (deprecated)::\n+sendemail.signedoffcc (deprecated):: // bugreport-exclude\n \tDeprecated alias for `sendemail.signedoffbycc`.\n \n-sendemail.smtpBatchSize::\n+sendemail.smtpBatchSize:: // bugreport-include\n \tNumber of messages to be sent per connection, after that a relogin\n \twill happen.  If the value is 0 or undefined, send all messages in\n \tone connection.\n \tSee also the `--batch-size` option of linkgit:git-send-email[1].\n \n-sendemail.smtpReloginDelay::\n+sendemail.smtpReloginDelay:: // bugreport-include\n \tSeconds wait before reconnecting to smtp server.\n \tSee also the `--relogin-delay` option of linkgit:git-send-email[1].\ndiff --git a/Makefile b/Makefile\nindex 69801a1c45..a62bbefb8f 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -639,6 +639,10 @@ SCRIPT_PYTHON += git-p4.py\n SCRIPT_SH_GEN = $(patsubst %.sh,%,$(SCRIPT_SH))\n SCRIPT_PERL_GEN = $(patsubst %.perl,%,$(SCRIPT_PERL))\n SCRIPT_PYTHON_GEN = $(patsubst %.py,%,$(SCRIPT_PYTHON))\n+SCRIPT_DEPENDENCIES = git-bugreport-config-whitelist\n+\n+$(SCRIPT_DEPENDENCIES): Documentation/config/*.txt\n+\tsh bugreport-generate-config-whitelist.sh\n \n # Individual rules to allow e.g.\n # \"make -C ../.. SCRIPT_PERL=contrib/foo/bar.perl build-perl-script\"\n@@ -656,17 +660,20 @@ install-perl-script: $(SCRIPT_PERL_GEN)\n install-python-script: $(SCRIPT_PYTHON_GEN)\n \t$(INSTALL) $^ '$(DESTDIR_SQ)$(gitexec_instdir_SQ)'\n \n-.PHONY: clean-perl-script clean-sh-script clean-python-script\n+.PHONY: clean-perl-script clean-sh-script clean-python-script clean-script-dependencies\n clean-sh-script:\n \t$(RM) $(SCRIPT_SH_GEN)\n clean-perl-script:\n \t$(RM) $(SCRIPT_PERL_GEN)\n clean-python-script:\n \t$(RM) $(SCRIPT_PYTHON_GEN)\n+clean-script-dependencies:\n+\t$(RM) $(SCRIPT_DEPENDENCIES)\n \n SCRIPTS = $(SCRIPT_SH_GEN) \\\n \t  $(SCRIPT_PERL_GEN) \\\n \t  $(SCRIPT_PYTHON_GEN) \\\n+\t  $(SCRIPT_DEPENDENCIES) \\\n \t  git-instaweb\n \n ETAGS_TARGET = TAGS\ndiff --git a/bugreport-generate-config-whitelist.sh b/bugreport-generate-config-whitelist.sh\nnew file mode 100755\nindex 0000000000..ca6b232024\n--- /dev/null\n+++ b/bugreport-generate-config-whitelist.sh\n@@ -0,0 +1,4 @@\n+#!/bin/sh\n+\n+grep -RhPo \".*(?=:: \\/\\/ bugreport-include)\" Documentation/config \\\n+  >git-bugreport-config-whitelist\ndiff --git a/git-bugreport-config-whitelist b/git-bugreport-config-whitelist\ndeleted file mode 100644\nindex e4f07f7175..0000000000\n--- a/git-bugreport-config-whitelist\n+++ /dev/null\n@@ -1,22 +0,0 @@\n-http.version\n-protocol.version\n-protocol.persistent-https.allow\n-protocol.rpc.allow\n-protocol.sso.allow\n-submodule.repolike\n-trace2.eventtarget\n-trace2.configparams\n-color.ui\n-core.pager\n-sendemail.smtpencryption\n-sendemail.smtpserver\n-sendemail.smtpserverport\n-sendemail.smtpsslcertpath\n-credential.helper\n-merge.tool\n-grep.linenumber\n-rerere.enabled\n-core.repositoryformatversion\n-core.filemode\n-core.bare\n-core.logallrefupdates\n-- \n2.23.0.rc1.153.gdeed80330f-goog\n\n"},{"id":"380572","messageId":"20190817003926.102944-2-emilyshaffer@google.com","threadId":"51645","inReplyTo":"20190817003926.102944-1-emilyshaffer@google.com","subject":"[PATCH v2 1/2] bugreport: add tool to generate debugging info","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2019-08-17T00:39:25Z","receivedAt":"2019-08-17T00:40:06Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"Make it easier for users who encounter a bug to send a report by\ncollecting some state information, intended to be viewed by humans\nfamiliar with Git.\n\nTeach Git how to prompt the user for a good bug report: reproduction\nsteps, expected behavior, and actual behavior. Also, teach Git to write\ndown its own version, the version of some of its dependencies, the\noperating system information, select entries from the gitconfig, the\nconfigured hooks, and the contents of $GIT_DIR/logs. Finally, make sure\nGit asks the user to review the contents of the report after it's\ngenerated.\n\nIf users can send us a well-written bug report which contains diagnostic\ninformation we would otherwise need to ask the user for, we can reduce\nthe number of question-and-answer round trips between the reporter and\nthe Git contributor.\n\nUsers may also wish to send a report like this to their local \"Git\nexpert\" if they have put their repository into a state they are confused\nby.\n\nSigned-off-by: Emily Shaffer <emilyshaffer@google.com>\n---\n .gitignore                      |   1 +\n Documentation/git-bugreport.txt |  51 ++++++++++++\n Makefile                        |   1 +\n command-list.txt                |   1 +\n git-bugreport-config-whitelist  |  22 +++++\n git-bugreport.sh                | 139 ++++++++++++++++++++++++++++++++\n git-sh-setup.sh                 |   2 +\n t/t0091-bugreport.sh            |  25 ++++++\n 8 files changed, 242 insertions(+)\n create mode 100644 Documentation/git-bugreport.txt\n create mode 100644 git-bugreport-config-whitelist\n create mode 100755 git-bugreport.sh\n create mode 100755 t/t0091-bugreport.sh\n\ndiff --git a/.gitignore b/.gitignore\nindex 521d8f4fb4..b4f5433084 100644\n--- a/.gitignore\n+++ b/.gitignore\n@@ -25,6 +25,7 @@\n /git-bisect--helper\n /git-blame\n /git-branch\n+/git-bugreport\n /git-bundle\n /git-cat-file\n /git-check-attr\ndiff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt\nnew file mode 100644\nindex 0000000000..2fc0a63192\n--- /dev/null\n+++ b/Documentation/git-bugreport.txt\n@@ -0,0 +1,51 @@\n+git-bugreport(1)\n+================\n+\n+NAME\n+----\n+git-bugreport - Collect information for user to file a bug report\n+\n+SYNOPSIS\n+--------\n+[verse]\n+'git bugreport' [-o | --output <path>]\n+\n+DESCRIPTION\n+-----------\n+Captures information about the user's machine, Git client, and repository state,\n+as well as a form requesting information about the behavior the user observed,\n+into a single text file which the user can then share, for example to the Git\n+mailing list, in order to report an observed bug.\n+\n+The following information is requested from the user:\n+\n+ - Reproduction steps\n+ - Expected behavior\n+ - Actual behavior\n+ - Difference between expected and actual behavior\n+\n+The following information is captured automatically:\n+\n+ - Git version (`git version --build-options`)\n+ - Machine information (`uname -a`)\n+ - Versions of various dependencies (curl, ldd)\n+ - Relevant user environment variables ($SHELL)\n+ - Some Git config variables, filtered by a whitelist\n+   (`git config --show-origin --get`)\n+ - A list of all configured git-hooks in `.git/hooks/` (but not their contents)\n+ - A list of all loose objects in `.git/objects/`\n+ - A list of all packs in `.git/objects/pack`\n+ - A list of all object info in `.git/objects/info`\n+ - The contents of the alternates file (`.git/objects/info/alternates`)\n+ - The contents of `.git/logs`\n+\n+OPTIONS\n+-------\n+-o [<path>]::\n+--output [<path>]::\n+\tPlace the resulting bug report file in <path> instead of the root of the\n+\tGit repository.\n+\n+GIT\n+---\n+Part of the linkgit:git[1] suite\ndiff --git a/Makefile b/Makefile\nindex f9255344ae..69801a1c45 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -606,6 +606,7 @@ TEST_PROGRAMS_NEED_X =\n unexport CDPATH\n \n SCRIPT_SH += git-bisect.sh\n+SCRIPT_SH += git-bugreport.sh\n SCRIPT_SH += git-difftool--helper.sh\n SCRIPT_SH += git-filter-branch.sh\n SCRIPT_SH += git-merge-octopus.sh\ndiff --git a/command-list.txt b/command-list.txt\nindex a9ac72bef4..be5a605047 100644\n--- a/command-list.txt\n+++ b/command-list.txt\n@@ -54,6 +54,7 @@ git-archive                             mainporcelain\n git-bisect                              mainporcelain           info\n git-blame                               ancillaryinterrogators          complete\n git-branch                              mainporcelain           history\n+git-bugreport                           ancillaryinterrogators\n git-bundle                              mainporcelain\n git-cat-file                            plumbinginterrogators\n git-check-attr                          purehelpers\ndiff --git a/git-bugreport-config-whitelist b/git-bugreport-config-whitelist\nnew file mode 100644\nindex 0000000000..e4f07f7175\n--- /dev/null\n+++ b/git-bugreport-config-whitelist\n@@ -0,0 +1,22 @@\n+http.version\n+protocol.version\n+protocol.persistent-https.allow\n+protocol.rpc.allow\n+protocol.sso.allow\n+submodule.repolike\n+trace2.eventtarget\n+trace2.configparams\n+color.ui\n+core.pager\n+sendemail.smtpencryption\n+sendemail.smtpserver\n+sendemail.smtpserverport\n+sendemail.smtpsslcertpath\n+credential.helper\n+merge.tool\n+grep.linenumber\n+rerere.enabled\n+core.repositoryformatversion\n+core.filemode\n+core.bare\n+core.logallrefupdates\ndiff --git a/git-bugreport.sh b/git-bugreport.sh\nnew file mode 100755\nindex 0000000000..0c182c62e9\n--- /dev/null\n+++ b/git-bugreport.sh\n@@ -0,0 +1,139 @@\n+#!/bin/sh\n+\n+print_file_if_exists () {\n+\tif [ -f \"$1\" ]\n+\tthen\n+\t\tif $(file --mime \"$1\" | grep -q \"charset=binary\")\n+\t\tthen\n+\t\t\techo \"<binary file>\"\n+\t\telse\n+\t\t\tcat \"$1\"\n+\t\tfi\n+\telse\n+\t\techo \"<file does not exist>\"\n+\tfi\n+}\n+\n+\n+print_filenames_and_content () {\n+\twhile read -r line\n+\tdo\n+\t\techo \"$line\"\n+\t\techo \"========\"\n+\t\tprint_file_if_exists \"$line\"\n+\t\techo\n+\tdone\n+}\n+\n+command_and_output () {\n+\techo \"$@:\"\n+\t\"$@\"\n+\techo\n+}\n+\n+generate_report_text() {\n+\n+\t# Generate a form for the user to fill out with their issue.\n+\tgettextln \"Thank you for filling out a Git bug report!\"\n+\tgettextln \"Please answer the following questions to help us understand your issue.\"\n+\techo\n+\tgettextln \"What did you do before the bug happened? (Steps to reproduce your issue)\"\n+\techo\n+\tgettextln \"What did you expect to happen? (Expected behavior)\"\n+\techo\n+\tgettextln \"What happened instead? (Actual behavior)\"\n+\techo\n+\tgettextln \"What's different between what you expected and what actually happened?\"\n+\techo\n+\tgettextln \"Anything else you want to add:\"\n+\techo\n+\tgettextln \"Please review the rest of the bug report below.\"\n+\tgettextln \"You can delete any lines you don't wish to send.\"\n+\tgettextln \\\n+\"When you're done, you should include the contents of this file in your bug \\n\\\n+report, for example by pasting it into an email to git@vger.kernel.org.\"\n+\techo\n+\n+\t# Versions of various necessary software should go here.\n+\techo \"[Version Information]\"\n+\tcommand_and_output git version --build-options\n+\tcommand_and_output uname -a\n+\tcommand_and_output curl-config --version\n+\tcommand_and_output ldd --version\n+\techo\n+\n+\t# Values of environment variables we find useful should go here.\n+\techo \"[Environment Information]\"\n+\tcommand_and_output echo $SHELL\n+\n+\techo \"[Git Config]\"\n+\txargs -n 1 -I% sh -c 'echo %: $(git config --show-origin --get %)' \\\n+\t\t<\"$(git --exec-path)/git-bugreport-config-whitelist\"\n+\techo\n+\n+\techo \"[Configured Hooks]\"\n+\tfind \"$GIT_HOOKS_DIRECTORY\" -type f | grep -v \"\\.sample$\"\n+\techo\n+\n+\techo \"[Loose Objects]\"\n+\tfor objdir in $(find \"$GIT_OBJECT_DIRECTORY\"/?? -type d);\n+\tdo\n+\t\techo \"$objdir: $(ls $objdir | wc -l) objects\"\n+\tdone\n+\techo\n+\t\n+\techo \"[Pack Data]\"\n+\tcommand_and_output ls -l \"$GIT_OBJECT_DIRECTORY/pack\"\n+\techo\n+\t\n+\techo \"[Object Info Data]\"\n+\tcommand_and_output ls -lR \"$GIT_OBJECT_DIRECTORY/info\"\n+\techo\n+\t\n+\techo \"[Alternates File]\"\n+        echo \"$GIT_OBJECT_DIRECTORY/info/alternates\" | print_filenames_and_content\n+\techo\n+\n+\t# This command is the reflog; it's fairly chatty and hard to scan\n+\t# quickly, so leave it at the bottom so that users don't miss what\n+\t# happens before it.\n+\techo \"[Git Logs]\"\n+\tfind \"$GIT_LOGS_DIRECTORY\" -type f | print_filenames_and_content\n+\techo\n+}\n+\n+USAGE=\"[-o | --output <path>]\"\n+\n+SUBDIRECTORY_OK=t\n+OPTIONS_SPEC=\n+. git-sh-setup\n+. git-sh-i18n\n+\n+basedir=\"$PWD\"\n+while :\n+do\n+\tcase \"$1\" in\n+\t-o|--output)\n+\t\tshift\n+\t\tbasedir=\"$1\"\n+\t\tshift\n+\t\tcontinue\n+\t\t;;\n+\t\"\")\n+\t\tbreak\n+\t\t;;\n+\t*)\n+\t\tusage\n+\t\t;;\n+\tesac\n+done\n+\n+\n+# Create bugreport file\n+BUGREPORT_FILE=\"$basedir/git-bugreport-$(whoami)-$(hostname)-$(date +%Y%m%d-%H%M)\"\n+\n+generate_report_text >$BUGREPORT_FILE\n+\n+git_editor $BUGREPORT_FILE\n+\n+eval_gettextln \"Your new bug report is in \\$BUGREPORT_FILE.\"\ndiff --git a/git-sh-setup.sh b/git-sh-setup.sh\nindex 10d9764185..1db8af0d76 100644\n--- a/git-sh-setup.sh\n+++ b/git-sh-setup.sh\n@@ -379,6 +379,8 @@ git_dir_init () {\n \t\texit 1\n \t}\n \t: \"${GIT_OBJECT_DIRECTORY=\"$(git rev-parse --git-path objects)\"}\"\n+        : \"${GIT_LOGS_DIRECTORY=\"$(git rev-parse --git-path logs)\"}\"\n+        : \"${GIT_HOOKS_DIRECTORY=\"$(git rev-parse --git-path hooks)\"}\"\n }\n \n if test -z \"$NONGIT_OK\"\ndiff --git a/t/t0091-bugreport.sh b/t/t0091-bugreport.sh\nnew file mode 100755\nindex 0000000000..e4e2d1a927\n--- /dev/null\n+++ b/t/t0091-bugreport.sh\n@@ -0,0 +1,25 @@\n+#!/bin/sh\n+\n+test_description='git bugreport'\n+\n+. ./test-lib.sh\n+\n+test_expect_success 'creates a report without crashing' '\n+\tgit bugreport &&\n+\trm git-bugreport-*\n+'\n+\n+test_expect_success '--output puts the report in the provided dir' '\n+\tmkdir foo/ &&\n+\tgit bugreport -o foo/ &&\n+\ttest -f foo/git-bugreport-* &&\n+\trm -fr foo/\n+'\n+\n+test_expect_success 'incorrect arguments abort with usage' '\n+\ttest_must_fail git bugreport --false 2>output &&\n+\tgrep usage output &&\n+\ttest ! -f git-bugreport-*\n+'\n+\n+test_done\n-- \n2.23.0.rc1.153.gdeed80330f-goog\n\n"},{"id":"380612","messageId":"20190817203846.31609-1-martin.agren@gmail.com","threadId":"51645","inReplyTo":"20190817003926.102944-3-emilyshaffer@google.com","subject":"Re: [PATCH v2 2/2] bugreport: generate config whitelist based on docs","fromName":"Martin Ågren","fromEmail":"martin.agren@gmail.com","sentAt":"2019-08-17T20:38:46Z","receivedAt":"2019-08-17T20:39:02Z","isPatch":true,"sender":{"key":"martin.agren@gmail.com","avatar":null},"body":"On Sat, 17 Aug 2019 at 02:42, Emily Shaffer <emilyshaffer@google.com> wrote:\n>\n> Add a new step to the build to generate a whitelist of git-config\n> variables which are appropriate to include in the output of\n> git-bugreport. New variables can be added to the whitelist by annotating\n> their documentation in Documentation/config with the line\n> \"// bugreport-include\".\n\nThese \"// bugreport-include\" show up in the rendered manpages, both with\nAsciiDoc and Asciidoctor. :-(\n\n> --- a/Documentation/config/sendemail.txt\n> +++ b/Documentation/config/sendemail.txt\n> @@ -1,63 +1,63 @@\n> -sendemail.identity::\n> +sendemail.identity:: // bugreport-exclude\n>         A configuration identity. When given, causes values in the\n\nIf I put each comment on a line of its own (after the config option, but\nI suppose before would work the same way), Asciidoctor truly ignores\nthem and everything's fine. And AsciiDoc renders this one and others\nlike it ok.\n\n> -sendemail.aliasesFile::\n> -sendemail.aliasFileType::\n> -sendemail.annotate::\n> +sendemail.aliasesFile:: // bugreport-exclude\n> +sendemail.aliasFileType:: // bugreport-exclude\n> +sendemail.annotate:: // bugreport-include\n\nHowever, AsciiDoc (version 8.6.10) seems to effectively replace those\ncomments with an empty line during processing, and it makes quite the\ndifference here. Instead of these appearing in a compact comma-separated\nlist, they are treated as individual items in the description list with\nno supporting content.\n\nFWIW, I like the idea of annotating things here to make it harder to\nforget this whitelisting when adding a new config item.\n\nBelow is what I came up with as an alternative approach. Feel free to\nsteal, squash and/or ignore as you see fit.\n\nBTW, I'm not sure the grep invocation is portable (-R ? -h ? -P ? -o ?).\nWe should probably also do the usual \"create a candidate output file,\nthen move it into place\" dance for robustness.\n\nI do think we should test the generated whitelist in some minimal way,\ne.g., to check that it does contain something which objectively belongs\nin the whitelist and -- more importantly IMHO -- does *not* contain\nsomething that shouldn't be there, such as sendemail.smtpPass.\n\nMartin\n\n-- >8 --\nSubject: [PATCH] Use a bugreport:include/exclude macro instead\n\nImplement a no-op \"bugreport\" macro for use as \"bugreport:include[x]\" to\nannotate the config keys that should be included in the automatically\ngenerated whitelist. Use \"exclude\" for the others.\n\nWith Asciidoctor, it's ok to say \"bugreport:include[]\", but AsciiDoc\nseems to want something between the brackets. A bit unfortunate, but\nnot a huge problem -- we'll just provide an \"x\".\n\n\"doc-diff\" reports that this macro doesn't render at all. That is,\nthese are both empty after this commit:\n\n  cd Documentation\n  ./doc-diff --asciidoctor :/\"bugreport: add tool\" HEAD\n  ./doc-diff --asciidoc    :/\"bugreport: add tool\" HEAD\n\nDiffing the rendered HTML shows that there is some small amount of\nwhitespace and comments added. That shouldn't be a problem.\n\nWe could perhaps let the implementation verify that the \"action\" is one\nof \"include\" and \"exclude\". For the Asciidoctor implementation that\nshould be straightforward, but for AsciiDoc I don't immediately know how\nto do it. Anyway, if someone stumbles on the keyboard and writes\n\"bugreport:icndule\", they'll \"only\" miss out on the config key being\nincluded in the whitelist. If this were a blacklist, the consequences of\na misspelled target could be a lot more severe.\n\nDisclaimer: This is the outcome of a combination of copy-paste and\ntrial and error -- better solutions might be possible...\n\nSigned-off-by: Martin Ågren <martin.agren@gmail.com>\n---\n I suppose this could also be \"annotate:bugreport[ignore]\" to avoid the\n \"[x]\" and to place all of this under a more general \"annotate\" macro.\n\n Documentation/asciidoc.conf             |  8 +++\n Documentation/asciidoctor-extensions.rb |  8 +++\n Documentation/config/sendemail.txt      | 68 ++++++++++++-------------\n bugreport-generate-config-whitelist.sh  |  2 +-\n 4 files changed, 51 insertions(+), 35 deletions(-)\n\ndiff --git a/Documentation/asciidoc.conf b/Documentation/asciidoc.conf\nindex 2c16c536ba..5a3c5ef028 100644\n--- a/Documentation/asciidoc.conf\n+++ b/Documentation/asciidoc.conf\n@@ -6,9 +6,13 @@\n #\n # Show Git link as: <command>(<section>); if section is defined, else just show\n # the command.\n+#\n+# The bugreport macro does nothing as far as rendering is\n+# concerned -- we just grep for it in the sources.\n \n [macros]\n (?su)[\\\\]?(?P<name>linkgit):(?P<target>\\S*?)\\[(?P<attrlist>.*?)\\]=\n+(?su)[\\\\]?(?P<name>bugreport):(?P<action>\\S*?)\\[(?P<attrlist>.*?)\\]=\n \n [attributes]\n asterisk=&#42;\n@@ -28,6 +32,8 @@ ifdef::backend-docbook[]\n {0#<citerefentry>}\n {0#<refentrytitle>{target}</refentrytitle><manvolnum>{0}</manvolnum>}\n {0#</citerefentry>}\n+[bugreport-inlinemacro]\n+{0#}\n endif::backend-docbook[]\n \n ifdef::backend-docbook[]\n@@ -94,4 +100,6 @@ ifdef::backend-xhtml11[]\n git-relative-html-prefix=\n [linkgit-inlinemacro]\n <a href=\"{git-relative-html-prefix}{target}.html\">{target}{0?({0})}</a>\n+[bugreport-inlinemacro]\n+<!-- -->\n endif::backend-xhtml11[]\ndiff --git a/Documentation/asciidoctor-extensions.rb b/Documentation/asciidoctor-extensions.rb\nindex 0089e0cfb8..54cbd5ddaf 100644\n--- a/Documentation/asciidoctor-extensions.rb\n+++ b/Documentation/asciidoctor-extensions.rb\n@@ -20,9 +20,17 @@ module Git\n         end\n       end\n     end\n+    class BugReportProcessor < Asciidoctor::Extensions::InlineMacroProcessor\n+      def process(parent, action, attrs)\n+        \"\"\n+      end\n+    end\n   end\n end\n \n Asciidoctor::Extensions.register do\n   inline_macro Git::Documentation::LinkGitProcessor, :linkgit\n+  # The bugreport macro does nothing as far as rendering is\n+  # concerned -- we just grep for it in the sources.\n+  inline_macro Git::Documentation::BugReportProcessor, :bugreport\n end\ndiff --git a/Documentation/config/sendemail.txt b/Documentation/config/sendemail.txt\nindex 69f3e4f219..92f5082013 100644\n--- a/Documentation/config/sendemail.txt\n+++ b/Documentation/config/sendemail.txt\n@@ -1,63 +1,63 @@\n-sendemail.identity:: // bugreport-exclude\n+sendemail.identity bugreport:exclude[x] ::\n \tA configuration identity. When given, causes values in the\n \t'sendemail.<identity>' subsection to take precedence over\n \tvalues in the 'sendemail' section. The default identity is\n \tthe value of `sendemail.identity`.\n \n-sendemail.smtpEncryption:: // bugreport-include\n+sendemail.smtpEncryption bugreport:include[x] ::\n \tSee linkgit:git-send-email[1] for description.  Note that this\n \tsetting is not subject to the 'identity' mechanism.\n \n-sendemail.smtpssl (deprecated):: // bugreport-exclude\n+sendemail.smtpssl (deprecated) bugreport:exclude[x] ::\n \tDeprecated alias for 'sendemail.smtpEncryption = ssl'.\n \n-sendemail.smtpsslcertpath:: // bugreport-exclude\n+sendemail.smtpsslcertpath bugreport:exclude[x] ::\n \tPath to ca-certificates (either a directory or a single file).\n \tSet it to an empty string to disable certificate verification.\n \n-sendemail.<identity>.*:: // bugreport-exclude\n+sendemail.<identity>.* bugreport:exclude[x] ::\n \tIdentity-specific versions of the 'sendemail.*' parameters\n \tfound below, taking precedence over those when this\n \tidentity is selected, through either the command-line or\n \t`sendemail.identity`.\n \n-sendemail.aliasesFile:: // bugreport-exclude\n-sendemail.aliasFileType:: // bugreport-exclude\n-sendemail.annotate:: // bugreport-include\n-sendemail.bcc:: // bugreport-include\n-sendemail.cc:: // bugreport-include\n-sendemail.ccCmd:: // bugreport-include\n-sendemail.chainReplyTo:: // bugreport-include\n-sendemail.confirm:: // bugreport-include\n-sendemail.envelopeSender:: // bugreport-include\n-sendemail.from:: // bugreport-include\n-sendemail.multiEdit:: // bugreport-include\n-sendemail.signedoffbycc:: // bugreport-include\n-sendemail.smtpPass:: // bugreport-exclude\n-sendemail.suppresscc:: // bugreport-include\n-sendemail.suppressFrom:: // bugreport-include\n-sendemail.to:: // bugreport-include\n-sendemail.tocmd:: // bugreport-include\n-sendemail.smtpDomain:: // bugreport-include\n-sendemail.smtpServer:: // bugreport-include\n-sendemail.smtpServerPort:: // bugreport-include\n-sendemail.smtpServerOption:: // bugreport-include\n-sendemail.smtpUser:: // bugreport-exclude\n-sendemail.thread:: // bugreport-include\n-sendemail.transferEncoding:: // bugreport-include\n-sendemail.validate:: // bugreport-include\n-sendemail.xmailer:: // bugreport-include\n+sendemail.aliasesFile bugreport:exclude[x] ::\n+sendemail.aliasFileType bugreport:exclude[x] ::\n+sendemail.annotate bugreport:include[x] ::\n+sendemail.bcc bugreport:include[x] ::\n+sendemail.cc bugreport:include[x] ::\n+sendemail.ccCmd bugreport:include[x] ::\n+sendemail.chainReplyTo bugreport:include[x] ::\n+sendemail.confirm bugreport:include[x] ::\n+sendemail.envelopeSender bugreport:include[x] ::\n+sendemail.from bugreport:include[x] ::\n+sendemail.multiEdit bugreport:include[x] ::\n+sendemail.signedoffbycc bugreport:include[x] ::\n+sendemail.smtpPass bugreport:exclude[x] ::\n+sendemail.suppresscc bugreport:include[x] ::\n+sendemail.suppressFrom bugreport:include[x] ::\n+sendemail.to bugreport:include[x] ::\n+sendemail.tocmd bugreport:include[x] ::\n+sendemail.smtpDomain bugreport:include[x] ::\n+sendemail.smtpServer bugreport:include[x] ::\n+sendemail.smtpServerPort bugreport:include[x] ::\n+sendemail.smtpServerOption bugreport:include[x] ::\n+sendemail.smtpUser bugreport:exclude[x] ::\n+sendemail.thread bugreport:include[x] ::\n+sendemail.transferEncoding bugreport:include[x] ::\n+sendemail.validate bugreport:include[x] ::\n+sendemail.xmailer bugreport:include[x] ::\n \tSee linkgit:git-send-email[1] for description.\n \n-sendemail.signedoffcc (deprecated):: // bugreport-exclude\n+sendemail.signedoffcc (deprecated) bugreport:exclude[x] ::\n \tDeprecated alias for `sendemail.signedoffbycc`.\n \n-sendemail.smtpBatchSize:: // bugreport-include\n+sendemail.smtpBatchSize bugreport:include[x] ::\n \tNumber of messages to be sent per connection, after that a relogin\n \twill happen.  If the value is 0 or undefined, send all messages in\n \tone connection.\n \tSee also the `--batch-size` option of linkgit:git-send-email[1].\n \n-sendemail.smtpReloginDelay:: // bugreport-include\n+sendemail.smtpReloginDelay bugreport:include[x] ::\n \tSeconds wait before reconnecting to smtp server.\n \tSee also the `--relogin-delay` option of linkgit:git-send-email[1].\ndiff --git a/bugreport-generate-config-whitelist.sh b/bugreport-generate-config-whitelist.sh\nindex ca6b232024..896b838cfa 100755\n--- a/bugreport-generate-config-whitelist.sh\n+++ b/bugreport-generate-config-whitelist.sh\n@@ -1,4 +1,4 @@\n #!/bin/sh\n \n-grep -RhPo \".*(?=:: \\/\\/ bugreport-include)\" Documentation/config \\\n+grep -RhPo \".*(?= +bugreport:include.* ::)\" Documentation/config \\\n   >git-bugreport-config-whitelist\n-- \n2.23.0\n\n"},{"id":"380896","messageId":"20190821174013.GA35435@google.com","threadId":"51645","inReplyTo":"20190817203846.31609-1-martin.agren@gmail.com","subject":"Re: [PATCH v2 2/2] bugreport: generate config whitelist based on docs","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2019-08-21T17:40:13Z","receivedAt":"2019-08-21T17:40:23Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"On Sat, Aug 17, 2019 at 10:38:46PM +0200, Martin Ågren wrote:\n> On Sat, 17 Aug 2019 at 02:42, Emily Shaffer <emilyshaffer@google.com> wrote:\n> >\n> > Add a new step to the build to generate a whitelist of git-config\n> > variables which are appropriate to include in the output of\n> > git-bugreport. New variables can be added to the whitelist by annotating\n> > their documentation in Documentation/config with the line\n> > \"// bugreport-include\".\n> \n> These \"// bugreport-include\" show up in the rendered manpages, both with\n> AsciiDoc and Asciidoctor. :-(\n\nHmm, I don't see it in the troff or the HTML with asciidoc using `make`\n- but I do see with asciidoctor, or `asciidoc -d html5`. Nice catch,\nthanks.\n\n> \n> > --- a/Documentation/config/sendemail.txt\n> > +++ b/Documentation/config/sendemail.txt\n> > @@ -1,63 +1,63 @@\n> > -sendemail.identity::\n> > +sendemail.identity:: // bugreport-exclude\n> >         A configuration identity. When given, causes values in the\n> \n> If I put each comment on a line of its own (after the config option, but\n> I suppose before would work the same way), Asciidoctor truly ignores\n> them and everything's fine. And AsciiDoc renders this one and others\n> like it ok.\n> \n> > -sendemail.aliasesFile::\n> > -sendemail.aliasFileType::\n> > -sendemail.annotate::\n> > +sendemail.aliasesFile:: // bugreport-exclude\n> > +sendemail.aliasFileType:: // bugreport-exclude\n> > +sendemail.annotate:: // bugreport-include\n> \n> However, AsciiDoc (version 8.6.10) seems to effectively replace those\n> comments with an empty line during processing, and it makes quite the\n> difference here. Instead of these appearing in a compact comma-separated\n> list, they are treated as individual items in the description list with\n> no supporting content.\n> \n> FWIW, I like the idea of annotating things here to make it harder to\n> forget this whitelisting when adding a new config item.\n> \n> Below is what I came up with as an alternative approach. Feel free to\n> steal, squash and/or ignore as you see fit.\n\nThis is cool - I'll add this to the commit chain and build on top of it.\nThanks!\n\n> \n> BTW, I'm not sure the grep invocation is portable (-R ? -h ? -P ? -o ?).\n\nYeah, I'll see what I can do about that (recursive, no filename,\nPerl-compatible regex, match only) - I can probably replace this with\nsomething like `find | xargs pgrep` - I'll keep digging. Thanks.\n\n> We should probably also do the usual \"create a candidate output file,\n> then move it into place\" dance for robustness.\n> \n> I do think we should test the generated whitelist in some minimal way,\n> e.g., to check that it does contain something which objectively belongs\n> in the whitelist and -- more importantly IMHO -- does *not* contain\n> something that shouldn't be there, such as sendemail.smtpPass.\n\nGood point, I'll add a test for this.\n\nThanks very much for this!\n - Emily\n"},{"id":"384827","messageId":"20191025025129.250049-1-emilyshaffer@google.com","threadId":"51645","inReplyTo":"20190817003926.102944-1-emilyshaffer@google.com","subject":"[PATCH v3 0/9] add git-bugreport tool","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2019-10-25T02:51:20Z","receivedAt":"2019-10-25T02:51:39Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"Thanks for the patience with the long wait, all. Here's an attempt at\nthe rewrite in C; I think it does verbatim what the sh version did\nexcept that this doesn't print the reflog - Jonathan Nieder was good\nenough to point out to me that folks probably don't want to share their\ncommit subjects all willy-nilly if they're working on something\nsecretive.\n\nFor the most part I hoped to do this stuff stupidly - that is to say,\nindependent of the libraries within Git - in an effort to both avoid\nbitrot, and see what is (as opposed to what the libraries think should\nbe).\n\nThe one common patch with v2 is \"generate config whitelist based on\ndocs\", which I have not changed at all.\n\nI look forward to everyone's suggestions.\n\n - Emily\n\nEmily Shaffer (9):\n  bugreport: add tool to generate debugging info\n  bugreport: generate config whitelist based on docs\n  bugreport: add version and system information\n  bugreport: add config values from whitelist\n  bugreport: collect list of populated hooks\n  bugreport: count loose objects\n  bugreport: add packed object summary\n  bugreport: list contents of $OBJDIR/info\n  bugreport: print contents of alternates file\n\n .gitignore                             |   1 +\n Documentation/config/sendemail.txt     |  68 +++---\n Makefile                               |  11 +-\n bugreport-generate-config-whitelist.sh |   4 +\n bugreport.c                            | 314 +++++++++++++++++++++++++\n bugreport.h                            |  44 ++++\n builtin.h                              |   1 +\n builtin/bugreport.c                    |  90 +++++++\n git.c                                  |   1 +\n 9 files changed, 499 insertions(+), 35 deletions(-)\n create mode 100755 bugreport-generate-config-whitelist.sh\n create mode 100644 bugreport.c\n create mode 100644 bugreport.h\n create mode 100644 builtin/bugreport.c\n\n-- \n2.24.0.rc0.303.g954a862665-goog\n\n"},{"id":"384828","messageId":"20191025025129.250049-2-emilyshaffer@google.com","threadId":"51645","inReplyTo":"20191025025129.250049-1-emilyshaffer@google.com","subject":"[PATCH v3 1/9] bugreport: add tool to generate debugging info","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2019-10-25T02:51:21Z","receivedAt":"2019-10-25T02:51:43Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"Teach Git how to prompt the user for a good bug report: reproduction\nsteps, expected behavior, and actual behavior. Later, Git can learn how\nto collect some diagnostic information from the repository.\n\nIf users can send us a well-written bug report which contains diagnostic\ninformation we would otherwise need to ask the user for, we can reduce\nthe number of question-and-answer round trips between the reporter and\nthe Git contributor.\n\nUsers may also wish to send a report like this to their local \"Git\nexpert\" if they have put their repository into a state they are confused\nby.\n\nSigned-off-by: Emily Shaffer <emilyshaffer@google.com>\n---\n Makefile            |  1 +\n builtin.h           |  1 +\n builtin/bugreport.c | 50 +++++++++++++++++++++++++++++++++++++++++++++\n git.c               |  1 +\n 4 files changed, 53 insertions(+)\n create mode 100644 builtin/bugreport.c\n\ndiff --git a/Makefile b/Makefile\nindex 58b92af54b..132e2a52da 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -1039,6 +1039,7 @@ BUILTIN_OBJS += builtin/archive.o\n BUILTIN_OBJS += builtin/bisect--helper.o\n BUILTIN_OBJS += builtin/blame.o\n BUILTIN_OBJS += builtin/branch.o\n+BUILTIN_OBJS += builtin/bugreport.o\n BUILTIN_OBJS += builtin/bundle.o\n BUILTIN_OBJS += builtin/cat-file.o\n BUILTIN_OBJS += builtin/check-attr.o\ndiff --git a/builtin.h b/builtin.h\nindex 5cf5df69f7..c6373d3289 100644\n--- a/builtin.h\n+++ b/builtin.h\n@@ -135,6 +135,7 @@ int cmd_archive(int argc, const char **argv, const char *prefix);\n int cmd_bisect__helper(int argc, const char **argv, const char *prefix);\n int cmd_blame(int argc, const char **argv, const char *prefix);\n int cmd_branch(int argc, const char **argv, const char *prefix);\n+int cmd_bugreport(int argc, const char **argv, const char *prefix);\n int cmd_bundle(int argc, const char **argv, const char *prefix);\n int cmd_cat_file(int argc, const char **argv, const char *prefix);\n int cmd_checkout(int argc, const char **argv, const char *prefix);\ndiff --git a/builtin/bugreport.c b/builtin/bugreport.c\nnew file mode 100644\nindex 0000000000..2ef16440a0\n--- /dev/null\n+++ b/builtin/bugreport.c\n@@ -0,0 +1,50 @@\n+#include \"builtin.h\"\n+#include \"stdio.h\"\n+#include \"strbuf.h\"\n+#include \"time.h\"\n+\n+int get_bug_template(struct strbuf *template)\n+{\n+\tconst char template_text[] =\n+\"Thank you for filling out a Git bug report!\\n\"\n+\"Please answer the following questions to help us understand your issue.\\n\"\n+\"\\n\"\n+\"What did you do before the bug happened? (Steps to reproduce your issue)\\n\"\n+\"\\n\"\n+\"What did you expect to happen? (Expected behavior)\\n\"\n+\"\\n\"\n+\"What happened instead? (Actual behavior)\\n\"\n+\"\\n\"\n+\"What's different between what you expected and what actually happened?\\n\"\n+\"\\n\"\n+\"Anything else you want to add:\\n\"\n+\"\\n\"\n+\"Please review the rest of the bug report below.\\n\"\n+\"You can delete any lines you don't wish to send.\\n\";\n+\n+\tstrbuf_reset(template);\n+\tstrbuf_add(template, template_text, strlen(template_text));\n+\treturn 0;\n+}\n+\n+int cmd_bugreport(int argc, const char **argv, const char *prefix)\n+{\n+\tstruct strbuf buffer = STRBUF_INIT;\n+\tstruct strbuf report_path = STRBUF_INIT;\n+\tFILE *report;\n+\ttime_t now = time(NULL);\n+\n+\tstrbuf_addstr(&report_path, \"git-bugreport-\");\n+\tstrbuf_addftime(&report_path, \"%F\", gmtime(&now), 0, 0);\n+\tstrbuf_addstr(&report_path, \".txt\");\n+\n+\treport = fopen_for_writing(report_path.buf);\n+\n+\tget_bug_template(&buffer);\n+\tstrbuf_write(&buffer, report);\n+\n+\tfclose(report);\n+\n+\tlaunch_editor(report_path.buf, NULL, NULL);\n+\treturn 0;\n+}\ndiff --git a/git.c b/git.c\nindex ce6ab0ece2..2d6a64f019 100644\n--- a/git.c\n+++ b/git.c\n@@ -473,6 +473,7 @@ static struct cmd_struct commands[] = {\n \t{ \"bisect--helper\", cmd_bisect__helper, RUN_SETUP },\n \t{ \"blame\", cmd_blame, RUN_SETUP },\n \t{ \"branch\", cmd_branch, RUN_SETUP | DELAY_PAGER_CONFIG },\n+\t{ \"bugreport\", cmd_bugreport, RUN_SETUP },\n \t{ \"bundle\", cmd_bundle, RUN_SETUP_GENTLY | NO_PARSEOPT },\n \t{ \"cat-file\", cmd_cat_file, RUN_SETUP },\n \t{ \"check-attr\", cmd_check_attr, RUN_SETUP },\n-- \n2.24.0.rc0.303.g954a862665-goog\n\n"},{"id":"384829","messageId":"20191025025129.250049-3-emilyshaffer@google.com","threadId":"51645","inReplyTo":"20191025025129.250049-1-emilyshaffer@google.com","subject":"[PATCH v3 2/9] bugreport: generate config whitelist based on docs","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2019-10-25T02:51:22Z","receivedAt":"2019-10-25T02:51:47Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"Add a new step to the build to generate a whitelist of git-config\nvariables which are appropriate to include in the output of\ngit-bugreport. New variables can be added to the whitelist by annotating\ntheir documentation in Documentation/config with the line\n\"// bugreport-include\".\n\nSome configs are private in nature, and can contain remote URLs,\npasswords, or other sensitive information. In the event that a user\ndoesn't notice their information while reviewing a bugreport, that user\nmay leak their credentials to other individuals, mailing lists, or bug\ntracking tools inadvertently. Heuristic blacklisting of configuration\nkeys is imperfect and prone to false negatives; given the nature of the\ninformation which can be leaked, a whitelist is more reliable.\n\nIn order to prevent staleness of the whitelist, add a mechanism to\ngenerate the whitelist from annotations in the config documentation,\nwhere contributors are already used to documenting their new config\nkeys.\n\nAdditionally, add annotations to the sendemail config documentation in\norder to demonstrate a proof of concept.\n\nSigned-off-by: Emily Shaffer <emilyshaffer@google.com>\n---\n .gitignore                             |  1 +\n Documentation/config/sendemail.txt     | 68 +++++++++++++-------------\n Makefile                               |  9 +++-\n bugreport-generate-config-whitelist.sh |  4 ++\n 4 files changed, 47 insertions(+), 35 deletions(-)\n create mode 100755 bugreport-generate-config-whitelist.sh\n\ndiff --git a/.gitignore b/.gitignore\nindex 89b3b79c1a..055a84c4a8 100644\n--- a/.gitignore\n+++ b/.gitignore\n@@ -25,6 +25,7 @@\n /git-bisect--helper\n /git-blame\n /git-branch\n+/git-bugreport\n /git-bundle\n /git-cat-file\n /git-check-attr\ndiff --git a/Documentation/config/sendemail.txt b/Documentation/config/sendemail.txt\nindex 0006faf800..69f3e4f219 100644\n--- a/Documentation/config/sendemail.txt\n+++ b/Documentation/config/sendemail.txt\n@@ -1,63 +1,63 @@\n-sendemail.identity::\n+sendemail.identity:: // bugreport-exclude\n \tA configuration identity. When given, causes values in the\n \t'sendemail.<identity>' subsection to take precedence over\n \tvalues in the 'sendemail' section. The default identity is\n \tthe value of `sendemail.identity`.\n \n-sendemail.smtpEncryption::\n+sendemail.smtpEncryption:: // bugreport-include\n \tSee linkgit:git-send-email[1] for description.  Note that this\n \tsetting is not subject to the 'identity' mechanism.\n \n-sendemail.smtpssl (deprecated)::\n+sendemail.smtpssl (deprecated):: // bugreport-exclude\n \tDeprecated alias for 'sendemail.smtpEncryption = ssl'.\n \n-sendemail.smtpsslcertpath::\n+sendemail.smtpsslcertpath:: // bugreport-exclude\n \tPath to ca-certificates (either a directory or a single file).\n \tSet it to an empty string to disable certificate verification.\n \n-sendemail.<identity>.*::\n+sendemail.<identity>.*:: // bugreport-exclude\n \tIdentity-specific versions of the 'sendemail.*' parameters\n \tfound below, taking precedence over those when this\n \tidentity is selected, through either the command-line or\n \t`sendemail.identity`.\n \n-sendemail.aliasesFile::\n-sendemail.aliasFileType::\n-sendemail.annotate::\n-sendemail.bcc::\n-sendemail.cc::\n-sendemail.ccCmd::\n-sendemail.chainReplyTo::\n-sendemail.confirm::\n-sendemail.envelopeSender::\n-sendemail.from::\n-sendemail.multiEdit::\n-sendemail.signedoffbycc::\n-sendemail.smtpPass::\n-sendemail.suppresscc::\n-sendemail.suppressFrom::\n-sendemail.to::\n-sendemail.tocmd::\n-sendemail.smtpDomain::\n-sendemail.smtpServer::\n-sendemail.smtpServerPort::\n-sendemail.smtpServerOption::\n-sendemail.smtpUser::\n-sendemail.thread::\n-sendemail.transferEncoding::\n-sendemail.validate::\n-sendemail.xmailer::\n+sendemail.aliasesFile:: // bugreport-exclude\n+sendemail.aliasFileType:: // bugreport-exclude\n+sendemail.annotate:: // bugreport-include\n+sendemail.bcc:: // bugreport-include\n+sendemail.cc:: // bugreport-include\n+sendemail.ccCmd:: // bugreport-include\n+sendemail.chainReplyTo:: // bugreport-include\n+sendemail.confirm:: // bugreport-include\n+sendemail.envelopeSender:: // bugreport-include\n+sendemail.from:: // bugreport-include\n+sendemail.multiEdit:: // bugreport-include\n+sendemail.signedoffbycc:: // bugreport-include\n+sendemail.smtpPass:: // bugreport-exclude\n+sendemail.suppresscc:: // bugreport-include\n+sendemail.suppressFrom:: // bugreport-include\n+sendemail.to:: // bugreport-include\n+sendemail.tocmd:: // bugreport-include\n+sendemail.smtpDomain:: // bugreport-include\n+sendemail.smtpServer:: // bugreport-include\n+sendemail.smtpServerPort:: // bugreport-include\n+sendemail.smtpServerOption:: // bugreport-include\n+sendemail.smtpUser:: // bugreport-exclude\n+sendemail.thread:: // bugreport-include\n+sendemail.transferEncoding:: // bugreport-include\n+sendemail.validate:: // bugreport-include\n+sendemail.xmailer:: // bugreport-include\n \tSee linkgit:git-send-email[1] for description.\n \n-sendemail.signedoffcc (deprecated)::\n+sendemail.signedoffcc (deprecated):: // bugreport-exclude\n \tDeprecated alias for `sendemail.signedoffbycc`.\n \n-sendemail.smtpBatchSize::\n+sendemail.smtpBatchSize:: // bugreport-include\n \tNumber of messages to be sent per connection, after that a relogin\n \twill happen.  If the value is 0 or undefined, send all messages in\n \tone connection.\n \tSee also the `--batch-size` option of linkgit:git-send-email[1].\n \n-sendemail.smtpReloginDelay::\n+sendemail.smtpReloginDelay:: // bugreport-include\n \tSeconds wait before reconnecting to smtp server.\n \tSee also the `--relogin-delay` option of linkgit:git-send-email[1].\ndiff --git a/Makefile b/Makefile\nindex 132e2a52da..78767ecdde 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -634,6 +634,10 @@ SCRIPT_PYTHON += git-p4.py\n SCRIPT_SH_GEN = $(patsubst %.sh,%,$(SCRIPT_SH))\n SCRIPT_PERL_GEN = $(patsubst %.perl,%,$(SCRIPT_PERL))\n SCRIPT_PYTHON_GEN = $(patsubst %.py,%,$(SCRIPT_PYTHON))\n+SCRIPT_DEPENDENCIES = git-bugreport-config-whitelist\n+\n+$(SCRIPT_DEPENDENCIES): Documentation/config/*.txt\n+\tsh bugreport-generate-config-whitelist.sh\n \n # Individual rules to allow e.g.\n # \"make -C ../.. SCRIPT_PERL=contrib/foo/bar.perl build-perl-script\"\n@@ -651,17 +655,20 @@ install-perl-script: $(SCRIPT_PERL_GEN)\n install-python-script: $(SCRIPT_PYTHON_GEN)\n \t$(INSTALL) $^ '$(DESTDIR_SQ)$(gitexec_instdir_SQ)'\n \n-.PHONY: clean-perl-script clean-sh-script clean-python-script\n+.PHONY: clean-perl-script clean-sh-script clean-python-script clean-script-dependencies\n clean-sh-script:\n \t$(RM) $(SCRIPT_SH_GEN)\n clean-perl-script:\n \t$(RM) $(SCRIPT_PERL_GEN)\n clean-python-script:\n \t$(RM) $(SCRIPT_PYTHON_GEN)\n+clean-script-dependencies:\n+\t$(RM) $(SCRIPT_DEPENDENCIES)\n \n SCRIPTS = $(SCRIPT_SH_GEN) \\\n \t  $(SCRIPT_PERL_GEN) \\\n \t  $(SCRIPT_PYTHON_GEN) \\\n+\t  $(SCRIPT_DEPENDENCIES) \\\n \t  git-instaweb\n \n ETAGS_TARGET = TAGS\ndiff --git a/bugreport-generate-config-whitelist.sh b/bugreport-generate-config-whitelist.sh\nnew file mode 100755\nindex 0000000000..ca6b232024\n--- /dev/null\n+++ b/bugreport-generate-config-whitelist.sh\n@@ -0,0 +1,4 @@\n+#!/bin/sh\n+\n+grep -RhPo \".*(?=:: \\/\\/ bugreport-include)\" Documentation/config \\\n+  >git-bugreport-config-whitelist\n-- \n2.24.0.rc0.303.g954a862665-goog\n\n"},{"id":"384830","messageId":"20191025025129.250049-4-emilyshaffer@google.com","threadId":"51645","inReplyTo":"20191025025129.250049-1-emilyshaffer@google.com","subject":"[PATCH v3 3/9] bugreport: add version and system information","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2019-10-25T02:51:23Z","receivedAt":"2019-10-25T02:51:50Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"Teach bugreport how to collect the Git, curl, and ldd versions, as well\nas the uname string.\n\nLearning the uname and versions in the user's environment will enable us\nto reproduce behavior reported by the user; the versions will also give\nus a good starting place while bisecting a newly introduced bug.\n\nPut this functionality in a library, so that other commands can easily\ninclude this debug info if necessary.\n\nSigned-off-by: Emily Shaffer <emilyshaffer@google.com>\n---\n Makefile            |  1 +\n bugreport.c         | 55 +++++++++++++++++++++++++++++++++++++++++++++\n bugreport.h         |  7 ++++++\n builtin/bugreport.c | 13 +++++++++++\n 4 files changed, 76 insertions(+)\n create mode 100644 bugreport.c\n create mode 100644 bugreport.h\n\ndiff --git a/Makefile b/Makefile\nindex 78767ecdde..045b22cb67 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -844,6 +844,7 @@ LIB_OBJS += bisect.o\n LIB_OBJS += blame.o\n LIB_OBJS += blob.o\n LIB_OBJS += branch.o\n+LIB_OBJS += bugreport.o\n LIB_OBJS += bulk-checkin.o\n LIB_OBJS += bundle.o\n LIB_OBJS += cache-tree.o\ndiff --git a/bugreport.c b/bugreport.c\nnew file mode 100644\nindex 0000000000..ada54fe583\n--- /dev/null\n+++ b/bugreport.c\n@@ -0,0 +1,55 @@\n+#include \"cache.h\"\n+\n+#include \"bugreport.h\"\n+#include \"help.h\"\n+#include \"run-command.h\"\n+#include \"version.h\"\n+\n+void get_system_info(struct strbuf *sys_info)\n+{\n+\tstruct child_process cp = CHILD_PROCESS_INIT;\n+\tstruct strbuf std_out = STRBUF_INIT;\n+\n+\tstrbuf_reset(sys_info);\n+\n+\t// get git version from native cmd\n+\tstrbuf_addstr(sys_info, \"git version: \");\n+\tstrbuf_addstr(sys_info, git_version_string);\n+\tstrbuf_complete_line(sys_info);\n+\n+\t// system call for other version info\n+\targv_array_push(&cp.args, \"uname\");\n+\targv_array_push(&cp.args, \"-a\");\n+\tcapture_command(&cp, &std_out, 0);\n+\n+\tstrbuf_addstr(sys_info, \"uname -a: \");\n+\tstrbuf_addbuf(sys_info, &std_out);\n+\tstrbuf_complete_line(sys_info);\n+\n+\targv_array_clear(&cp.args);\n+\tstrbuf_reset(&std_out);\n+\n+\n+\targv_array_push(&cp.args, \"curl-config\");\n+\targv_array_push(&cp.args, \"--version\");\n+\tcapture_command(&cp, &std_out, 0);\n+\n+\tstrbuf_addstr(sys_info, \"curl-config --version: \");\n+\tstrbuf_addbuf(sys_info, &std_out);\n+\tstrbuf_complete_line(sys_info);\n+\n+\targv_array_clear(&cp.args);\n+\tstrbuf_reset(&std_out);\n+\n+\n+\targv_array_push(&cp.args, \"ldd\");\n+\targv_array_push(&cp.args, \"--version\");\n+\tcapture_command(&cp, &std_out, 0);\n+\n+\tstrbuf_addstr(sys_info, \"ldd --version: \");\n+\tstrbuf_addbuf(sys_info, &std_out);\n+\tstrbuf_complete_line(sys_info);\n+\n+\targv_array_clear(&cp.args);\n+\tstrbuf_reset(&std_out);\n+}\ndiff --git a/bugreport.h b/bugreport.h\nnew file mode 100644\nindex 0000000000..ba216acf3f\n--- /dev/null\n+++ b/bugreport.h\n@@ -0,0 +1,7 @@\n+#include \"strbuf.h\"\n+\n+/**\n+ * Adds the Git version, `uname -a`, and `curl-config --version` to sys_info.\n+ * The previous contents of sys_info will be discarded.\n+ */\n+void get_system_info(struct strbuf *sys_info);\ndiff --git a/builtin/bugreport.c b/builtin/bugreport.c\nindex 2ef16440a0..7232d31be7 100644\n--- a/builtin/bugreport.c\n+++ b/builtin/bugreport.c\n@@ -1,4 +1,5 @@\n #include \"builtin.h\"\n+#include \"bugreport.h\"\n #include \"stdio.h\"\n #include \"strbuf.h\"\n #include \"time.h\"\n@@ -27,6 +28,13 @@ int get_bug_template(struct strbuf *template)\n \treturn 0;\n }\n \n+void add_header(FILE *report, const char *title)\n+{\n+\tstruct strbuf buffer = STRBUF_INIT;\n+\tstrbuf_addf(&buffer, \"\\n\\n[%s]\\n\", title);\n+\tstrbuf_write(&buffer, report);\n+}\n+\n int cmd_bugreport(int argc, const char **argv, const char *prefix)\n {\n \tstruct strbuf buffer = STRBUF_INIT;\n@@ -43,6 +51,11 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix)\n \tget_bug_template(&buffer);\n \tstrbuf_write(&buffer, report);\n \n+\t// add other contents\n+\tadd_header(report, \"System Info\");\n+\tget_system_info(&buffer);\n+\tstrbuf_write(&buffer, report);\n+\n \tfclose(report);\n \n \tlaunch_editor(report_path.buf, NULL, NULL);\n-- \n2.24.0.rc0.303.g954a862665-goog\n\n"},{"id":"384831","messageId":"20191025025129.250049-5-emilyshaffer@google.com","threadId":"51645","inReplyTo":"20191025025129.250049-1-emilyshaffer@google.com","subject":"[PATCH v3 4/9] bugreport: add config values from whitelist","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2019-10-25T02:51:24Z","receivedAt":"2019-10-25T02:51:54Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"Teach bugreport to gather the values of config options which are present\nin 'git-bugreport-config-whitelist'.\n\nMany config options are sensitive, and many Git add-ons use config\noptions which git-core does not know about; it is better only to gather\nconfig options which we know to be safe, rather than excluding options\nwhich we know to be unsafe.\n\nReading the whitelist into memory and sorting it saves us time -\nsince git_config_bugreport() is called for every option the user has\nconfigured, limiting the file IO to one open/read/close and performing\noption lookup in sublinear time is a useful optimization.\n\nSigned-off-by: Emily Shaffer <emilyshaffer@google.com>\n---\n bugreport.c         | 50 +++++++++++++++++++++++++++++++++++++++++++++\n bugreport.h         |  7 +++++++\n builtin/bugreport.c |  4 ++++\n 3 files changed, 61 insertions(+)\n\ndiff --git a/bugreport.c b/bugreport.c\nindex ada54fe583..afa4836ab1 100644\n--- a/bugreport.c\n+++ b/bugreport.c\n@@ -1,10 +1,24 @@\n #include \"cache.h\"\n \n #include \"bugreport.h\"\n+#include \"config.h\"\n+#include \"exec-cmd.h\"\n #include \"help.h\"\n #include \"run-command.h\"\n #include \"version.h\"\n \n+/**\n+ * A sorted list of config options which we will add to the bugreport. Managed\n+ * by 'gather_whitelist(...)'.\n+ */\n+struct string_list whitelist = STRING_LIST_INIT_DUP;\n+struct strbuf configs_and_values = STRBUF_INIT;\n+\n+// git version --build-options\n+// uname -a\n+// curl-config --version\n+// ldd --version\n+// echo $SHELL\n void get_system_info(struct strbuf *sys_info)\n {\n \tstruct child_process cp = CHILD_PROCESS_INIT;\n@@ -53,3 +67,39 @@ void get_system_info(struct strbuf *sys_info)\n \targv_array_clear(&cp.args);\n \tstrbuf_reset(&std_out);\n }\n+\n+void gather_whitelist(struct strbuf *path)\n+{\n+\tstruct strbuf tmp = STRBUF_INIT;\n+\tstrbuf_read_file(&tmp, path->buf, 0);\n+\tstring_list_init(&whitelist, 1);\n+\tstring_list_split(&whitelist, tmp.buf, '\\n', -1);\n+\tstring_list_sort(&whitelist);\n+}\n+\n+int git_config_bugreport(const char *var, const char *value, void *cb)\n+{\n+\tif (string_list_has_string(&whitelist, var)) {\n+\t\tstrbuf_addf(&configs_and_values,\n+\t\t\t    \"%s : %s\\n\",\n+\t\t\t    var, value);\n+\t}\n+\n+\treturn 0;\n+}\n+\n+void get_whitelisted_config(struct strbuf *config_info)\n+{\n+\tstruct strbuf path = STRBUF_INIT;\n+\n+\tstrbuf_addstr(&path, git_exec_path());\n+\tstrbuf_addstr(&path, \"/git-bugreport-config-whitelist\");\n+\n+\tgather_whitelist(&path);\n+\tstrbuf_init(&configs_and_values, whitelist.nr);\n+\n+\tgit_config(git_config_bugreport, NULL);\n+\n+\tstrbuf_reset(config_info);\n+\tstrbuf_addbuf(config_info, &configs_and_values);\n+}\ndiff --git a/bugreport.h b/bugreport.h\nindex ba216acf3f..7413e7e1be 100644\n--- a/bugreport.h\n+++ b/bugreport.h\n@@ -5,3 +5,10 @@\n  * The previous contents of sys_info will be discarded.\n  */\n void get_system_info(struct strbuf *sys_info);\n+\n+/**\n+ * Adds the values of the config items listed in\n+ * 'git-bugreport-config-whitelist' to config_info. The previous contents of\n+ * config_info will be discarded.\n+ */\n+void get_whitelisted_config(struct strbuf *sys_info);\ndiff --git a/builtin/bugreport.c b/builtin/bugreport.c\nindex 7232d31be7..70fe0d2b85 100644\n--- a/builtin/bugreport.c\n+++ b/builtin/bugreport.c\n@@ -56,6 +56,10 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix)\n \tget_system_info(&buffer);\n \tstrbuf_write(&buffer, report);\n \n+\tadd_header(report, \"Whitelisted Config\");\n+\tget_whitelisted_config(&buffer);\n+\tstrbuf_write(&buffer, report);\n+\n \tfclose(report);\n \n \tlaunch_editor(report_path.buf, NULL, NULL);\n-- \n2.24.0.rc0.303.g954a862665-goog\n\n"},{"id":"384832","messageId":"20191025025129.250049-6-emilyshaffer@google.com","threadId":"51645","inReplyTo":"20191025025129.250049-1-emilyshaffer@google.com","subject":"[PATCH v3 5/9] bugreport: collect list of populated hooks","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2019-10-25T02:51:25Z","receivedAt":"2019-10-25T02:51:58Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"Occasionally a failure a user is seeing may be related to a specific\nhook which is being run, perhaps without the user realizing. While the\ncontents of hooks can be sensitive - containing user data or process\ninformation specific to the user's organization - simply knowing that a\nhook is being run at a certain stage can help us to understand whether\nsomething is going wrong.\n\nWithout a definitive list of hook names within the code, we compile our\nown list from the documentation. This is likely prone to bitrot. To\nreduce the amount of code humans need to read, we turn the list into a\nstring_list and iterate over it (as we are calling the same find_hook\noperation on each string). However, since bugreport should primarily be\ncalled by the user, the performance loss from massaging the string\nseems acceptable.\n\nSigned-off-by: Emily Shaffer <emilyshaffer@google.com>\n---\n bugreport.c         | 44 ++++++++++++++++++++++++++++++++++++++++++++\n bugreport.h         |  6 ++++++\n builtin/bugreport.c |  4 ++++\n 3 files changed, 54 insertions(+)\n\ndiff --git a/bugreport.c b/bugreport.c\nindex afa4836ab1..9d7f44ff28 100644\n--- a/bugreport.c\n+++ b/bugreport.c\n@@ -103,3 +103,47 @@ void get_whitelisted_config(struct strbuf *config_info)\n \tstrbuf_reset(config_info);\n \tstrbuf_addbuf(config_info, &configs_and_values);\n }\n+\n+void get_populated_hooks(struct strbuf *hook_info)\n+{\n+\t/*\n+\t * Doesn't look like there is a list of all possible hooks; so below is\n+\t * a transcription of `git help hook`.\n+\t */\n+\tconst char *hooks = \"applypatch-msg,\"\n+\t\t\t    \"pre-applypatch,\"\n+\t\t\t    \"post-applypatch,\"\n+\t\t\t    \"pre-commit,\"\n+\t\t\t    \"pre-merge-commit,\"\n+\t\t\t    \"prepare-commit-msg,\"\n+\t\t\t    \"commit-msg,\"\n+\t\t\t    \"post-commit,\"\n+\t\t\t    \"pre-rebase,\"\n+\t\t\t    \"post-checkout,\"\n+\t\t\t    \"post-merge,\"\n+\t\t\t    \"pre-push,\"\n+\t\t\t    \"pre-receive,\"\n+\t\t\t    \"update,\"\n+\t\t\t    \"post-receive,\"\n+\t\t\t    \"post-update,\"\n+\t\t\t    \"push-to-checkout,\"\n+\t\t\t    \"pre-auto-gc,\"\n+\t\t\t    \"post-rewrite,\"\n+\t\t\t    \"sendemail-validate,\"\n+\t\t\t    \"fsmonitor-watchman,\"\n+\t\t\t    \"p4-pre-submit,\"\n+\t\t\t    \"post-index-changex\";\n+\tstruct string_list hooks_list = STRING_LIST_INIT_DUP;\n+\tstruct string_list_item *iter = NULL;\n+\n+\tstrbuf_reset(hook_info);\n+\n+\tstring_list_split(&hooks_list, hooks, ',', -1);\n+\n+\tfor_each_string_list_item(iter, &hooks_list) {\n+\t\tif (find_hook(iter->string)) {\n+\t\t\tstrbuf_addstr(hook_info, iter->string);\n+\t\t\tstrbuf_complete_line(hook_info);\n+\t\t}\n+\t}\n+}\ndiff --git a/bugreport.h b/bugreport.h\nindex 7413e7e1be..942a5436e3 100644\n--- a/bugreport.h\n+++ b/bugreport.h\n@@ -12,3 +12,9 @@ void get_system_info(struct strbuf *sys_info);\n  * config_info will be discarded.\n  */\n void get_whitelisted_config(struct strbuf *sys_info);\n+\n+/**\n+ * Adds the paths to all configured hooks (but not their contents). The previous\n+ * contents of hook_info will be discarded.\n+ */\n+void get_populated_hooks(struct strbuf *hook_info);\ndiff --git a/builtin/bugreport.c b/builtin/bugreport.c\nindex 70fe0d2b85..a0eefba498 100644\n--- a/builtin/bugreport.c\n+++ b/builtin/bugreport.c\n@@ -60,6 +60,10 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix)\n \tget_whitelisted_config(&buffer);\n \tstrbuf_write(&buffer, report);\n \n+\tadd_header(report, \"Populated Hooks\");\n+\tget_populated_hooks(&buffer);\n+\tstrbuf_write(&buffer, report);\n+\n \tfclose(report);\n \n \tlaunch_editor(report_path.buf, NULL, NULL);\n-- \n2.24.0.rc0.303.g954a862665-goog\n\n"},{"id":"384833","messageId":"20191025025129.250049-7-emilyshaffer@google.com","threadId":"51645","inReplyTo":"20191025025129.250049-1-emilyshaffer@google.com","subject":"[PATCH v3 6/9] bugreport: count loose objects","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2019-10-25T02:51:26Z","receivedAt":"2019-10-25T02:52:02Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"The number of unpacked objects in a user's repository may help us\nunderstand the root of the problem they're seeing, especially if a\ncommand is running unusually slowly.\n\nRather than directly invoking 'git-count-objects', which may sometimes\nfail unexpectedly on Git for Windows, manually count the contents of\n.git/objects. Additionally, since we may wish to inspect other\ndirectories' contents for bugreport in the future, put the directory\nlisting into a helper function.\n\nSigned-off-by: Emily Shaffer <emilyshaffer@google.com>\n---\n bugreport.c         | 72 +++++++++++++++++++++++++++++++++++++++++++++\n bugreport.h         |  6 ++++\n builtin/bugreport.c |  4 +++\n 3 files changed, 82 insertions(+)\n\ndiff --git a/bugreport.c b/bugreport.c\nindex 9d7f44ff28..54e1d47103 100644\n--- a/bugreport.c\n+++ b/bugreport.c\n@@ -5,8 +5,11 @@\n #include \"exec-cmd.h\"\n #include \"help.h\"\n #include \"run-command.h\"\n+#include \"strbuf.h\"\n #include \"version.h\"\n \n+#include \"dirent.h\"\n+\n /**\n  * A sorted list of config options which we will add to the bugreport. Managed\n  * by 'gather_whitelist(...)'.\n@@ -147,3 +150,72 @@ void get_populated_hooks(struct strbuf *hook_info)\n \t\t}\n \t}\n }\n+\n+/**\n+ * Fill 'contents' with the contents of the dir at 'dirpath'.\n+ * If 'filter' is nonzero, the contents are filtered on d_type as 'type' - see\n+ * 'man readdir'. opendir() doesn't take string length as an arg, so don't\n+ * bother passing it in.\n+ */\n+void list_contents_of_dir(struct string_list *contents, struct strbuf *dirpath,\n+\t\t\t  int filter, unsigned char type)\n+{\n+\tstruct dirent *dir = NULL;\n+\tDIR *dh = NULL;\n+\n+\tdh = opendir(dirpath->buf);\n+\twhile (dh && (dir = readdir(dh))) {\n+\t\tif (!filter || type == dir->d_type) {\n+\t\t\tstring_list_append(contents, dir->d_name);\n+\t\t}\n+\t}\n+}\n+\n+\n+void get_object_counts(struct strbuf *obj_info)\n+{\n+\tstruct child_process cp = CHILD_PROCESS_INIT;\n+\tstruct strbuf std_out = STRBUF_INIT;\n+\n+\targv_array_push(&cp.args, \"count-objects\");\n+\targv_array_push(&cp.args, \"-vH\");\n+\tcp.git_cmd = 1;\n+\tcapture_command(&cp, &std_out, 0);\n+\n+\tstrbuf_reset(obj_info);\n+\tstrbuf_addstr(obj_info, \"git-count-objects -vH:\\n\");\n+\tstrbuf_addbuf(obj_info, &std_out);\n+}\n+\n+void get_loose_object_summary(struct strbuf *obj_info)\n+{\n+\tstruct strbuf dirpath = STRBUF_INIT;\n+\tstruct string_list subdirs = STRING_LIST_INIT_DUP;\n+\tstruct string_list_item *subdir;\n+\n+\tstrbuf_reset(obj_info);\n+\n+\tstrbuf_addstr(&dirpath, get_object_directory());\n+\tstrbuf_complete(&dirpath, '/');\n+\n+\tlist_contents_of_dir(&subdirs, &dirpath, 1, DT_DIR);\n+\n+\tfor_each_string_list_item(subdir, &subdirs)\n+\t{\n+\t\tstruct strbuf subdir_buf = STRBUF_INIT;\n+\t\tstruct string_list objects = STRING_LIST_INIT_DUP;\n+\n+\t\t/*\n+\t\t * Only interested in loose objects - so dirs named with the\n+\t\t * first byte of the object ID\n+\t\t */\n+\t\tif (strlen(subdir->string) != 2 || !strcmp(subdir->string, \"..\"))\n+\t\t\tcontinue;\n+\n+\t\tstrbuf_addbuf(&subdir_buf, &dirpath);\n+\t\tstrbuf_addstr(&subdir_buf, subdir->string);\n+\t\tlist_contents_of_dir(&objects, &subdir_buf, 0, 0);\n+\t\tstrbuf_addf(obj_info, \"%s: %d objects\\n\", subdir->string,\n+\t\t\t    objects.nr);\n+\t}\n+}\ndiff --git a/bugreport.h b/bugreport.h\nindex 942a5436e3..09ad0c2599 100644\n--- a/bugreport.h\n+++ b/bugreport.h\n@@ -18,3 +18,9 @@ void get_whitelisted_config(struct strbuf *sys_info);\n  * contents of hook_info will be discarded.\n  */\n void get_populated_hooks(struct strbuf *hook_info);\n+\n+/**\n+ * Adds the output of `git count-object -vH`. The previous contents of hook_info\n+ * will be discarded.\n+ */\n+void get_loose_object_summary(struct strbuf *obj_info);\ndiff --git a/builtin/bugreport.c b/builtin/bugreport.c\nindex a0eefba498..b2ab194207 100644\n--- a/builtin/bugreport.c\n+++ b/builtin/bugreport.c\n@@ -64,6 +64,10 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix)\n \tget_populated_hooks(&buffer);\n \tstrbuf_write(&buffer, report);\n \n+\tadd_header(report, \"Object Counts\");\n+\tget_loose_object_summary(&buffer);\n+\tstrbuf_write(&buffer, report);\n+\n \tfclose(report);\n \n \tlaunch_editor(report_path.buf, NULL, NULL);\n-- \n2.24.0.rc0.303.g954a862665-goog\n\n"},{"id":"384834","messageId":"20191025025129.250049-8-emilyshaffer@google.com","threadId":"51645","inReplyTo":"20191025025129.250049-1-emilyshaffer@google.com","subject":"[PATCH v3 7/9] bugreport: add packed object summary","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2019-10-25T02:51:27Z","receivedAt":"2019-10-25T02:52:05Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"Alongside the list of loose objects, it's useful to see the list of\nobject packs as well. It can help us to examine what Git did and did not\npack.\n\nSigned-off-by: Emily Shaffer <emilyshaffer@google.com>\n---\n bugreport.c         | 21 +++++++++++++++++++++\n bugreport.h         |  6 ++++++\n builtin/bugreport.c |  4 ++++\n 3 files changed, 31 insertions(+)\n\ndiff --git a/bugreport.c b/bugreport.c\nindex 54e1d47103..79ddb8baaa 100644\n--- a/bugreport.c\n+++ b/bugreport.c\n@@ -219,3 +219,24 @@ void get_loose_object_summary(struct strbuf *obj_info)\n \t\t\t    objects.nr);\n \t}\n }\n+\n+void get_packed_object_summary(struct strbuf *obj_info)\n+{\n+\tstruct strbuf dirpath = STRBUF_INIT;\n+\tstruct string_list contents = STRING_LIST_INIT_DUP;\n+\tstruct string_list_item *entry;\n+\n+\tstrbuf_reset(obj_info);\n+\n+\tstrbuf_addstr(&dirpath, get_object_directory());\n+\tstrbuf_complete(&dirpath, '/');\n+\tstrbuf_addstr(&dirpath, \"pack/\");\n+\tlist_contents_of_dir(&contents, &dirpath, 0, 0);\n+\n+\t// list full contents of $GIT_OBJECT_DIRECTORY/pack/\n+\tfor_each_string_list_item(entry, &contents) {\n+\t\tstrbuf_addbuf(obj_info, &dirpath);\n+\t\tstrbuf_addstr(obj_info, entry->string);\n+\t\tstrbuf_complete_line(obj_info);\n+\t}\n+}\ndiff --git a/bugreport.h b/bugreport.h\nindex 09ad0c2599..11ff7df41b 100644\n--- a/bugreport.h\n+++ b/bugreport.h\n@@ -24,3 +24,9 @@ void get_populated_hooks(struct strbuf *hook_info);\n  * will be discarded.\n  */\n void get_loose_object_summary(struct strbuf *obj_info);\n+\n+/**\n+ * Adds a list of the contents of '.git/objects/pack'. The previous contents of\n+ * hook_info will be discarded.\n+ */\n+void get_packed_object_summary(struct strbuf *obj_info);\ndiff --git a/builtin/bugreport.c b/builtin/bugreport.c\nindex b2ab194207..da91a3944e 100644\n--- a/builtin/bugreport.c\n+++ b/builtin/bugreport.c\n@@ -68,6 +68,10 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix)\n \tget_loose_object_summary(&buffer);\n \tstrbuf_write(&buffer, report);\n \n+\tadd_header(report, \"Packed Object Summary\");\n+\tget_packed_object_summary(&buffer);\n+\tstrbuf_write(&buffer, report);\n+\n \tfclose(report);\n \n \tlaunch_editor(report_path.buf, NULL, NULL);\n-- \n2.24.0.rc0.303.g954a862665-goog\n\n"},{"id":"384835","messageId":"20191025025129.250049-9-emilyshaffer@google.com","threadId":"51645","inReplyTo":"20191025025129.250049-1-emilyshaffer@google.com","subject":"[PATCH v3 8/9] bugreport: list contents of $OBJDIR/info","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2019-10-25T02:51:28Z","receivedAt":"2019-10-25T02:52:11Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"Miscellaneous information used about the object store can end up in\n.git/objects/info; this can help us understand what may be going on with\nthe object store when the user is reporting a bug. Otherwise, it could\nbe difficult to track down what is going wrong with an object which\nisn't kept locally to .git/objects/ or .git/objects/pack. Having some\nunderstanding of where the user's objects may be kept can save us some\nhops during the bug reporting process.\n\nSigned-off-by: Emily Shaffer <emilyshaffer@google.com>\n---\n bugreport.c         | 58 +++++++++++++++++++++++++++++++++++++++++++++\n bugreport.h         |  6 +++++\n builtin/bugreport.c |  7 ++++++\n 3 files changed, 71 insertions(+)\n\ndiff --git a/bugreport.c b/bugreport.c\nindex 79ddb8baaa..ce15904fec 100644\n--- a/bugreport.c\n+++ b/bugreport.c\n@@ -6,6 +6,7 @@\n #include \"help.h\"\n #include \"run-command.h\"\n #include \"strbuf.h\"\n+#include \"string-list.h\"\n #include \"version.h\"\n \n #include \"dirent.h\"\n@@ -171,6 +172,41 @@ void list_contents_of_dir(struct string_list *contents, struct strbuf *dirpath,\n \t}\n }\n \n+/**\n+ * Fills 'contents' with a list of all directories within the provided\n+ * directory, recursing into each directory.\n+ */\n+void list_contents_of_dir_recursively(struct string_list *contents,\n+\t\t\t\t      struct strbuf *dirpath)\n+{\n+\tstruct string_list current_contents = STRING_LIST_INIT_DUP;\n+\tstruct string_list current_subdirs = STRING_LIST_INIT_DUP;\n+\tstruct string_list_item *it;\n+\tstruct strbuf buf = STRBUF_INIT;\n+\n+\tlist_contents_of_dir(&current_contents, dirpath, 0, 0);\n+\tfor_each_string_list_item(it, &current_contents) {\n+\t\tstrbuf_reset(&buf);\n+\t\tstrbuf_addbuf(&buf, dirpath);\n+\t\tstrbuf_complete(&buf, '/');\n+\t\tstrbuf_addstr(&buf, it->string);\n+\n+\t\tstring_list_append(contents, buf.buf);\n+\t}\n+\n+\tlist_contents_of_dir(&current_subdirs, dirpath, 1, DT_DIR);\n+\tfor_each_string_list_item(it, &current_subdirs) {\n+\t\tif (strcmp(it->string, \".\") == 0\n+\t\t    || strcmp(it->string, \"..\") == 0)\n+\t\t\tcontinue;\n+\t\tstrbuf_reset(&buf);\n+\t\tstrbuf_addbuf(&buf, dirpath);\n+\t\tstrbuf_complete(&buf, '/');\n+\t\tstrbuf_addstr(&buf, it->string);\n+\n+\t\tlist_contents_of_dir_recursively(contents, &buf);\n+\t}\n+}\n \n void get_object_counts(struct strbuf *obj_info)\n {\n@@ -240,3 +276,25 @@ void get_packed_object_summary(struct strbuf *obj_info)\n \t\tstrbuf_complete_line(obj_info);\n \t}\n }\n+\n+void get_object_info_summary(struct strbuf *obj_info)\n+{\n+\t// strbuf += GITDIR/info/:\n+\t// recursively list contents of $GIT_OBJECT_DIRECTORY/info\n+\tstruct strbuf dirpath = STRBUF_INIT;\n+\tstruct string_list contents = STRING_LIST_INIT_DUP;\n+\tstruct string_list_item *entry;\n+\n+\tstrbuf_reset(obj_info);\n+\n+\tstrbuf_addstr(&dirpath, get_object_directory());\n+\tstrbuf_complete(&dirpath, '/');\n+\tstrbuf_addstr(&dirpath, \"info/\");\n+\n+\tlist_contents_of_dir_recursively(&contents, &dirpath);\n+\n+\tfor_each_string_list_item(entry, &contents) {\n+\t\tstrbuf_addstr(obj_info, entry->string);\n+\t\tstrbuf_complete_line(obj_info);\n+\t}\n+}\ndiff --git a/bugreport.h b/bugreport.h\nindex 11ff7df41b..4f5e2d1b9a 100644\n--- a/bugreport.h\n+++ b/bugreport.h\n@@ -30,3 +30,9 @@ void get_loose_object_summary(struct strbuf *obj_info);\n  * hook_info will be discarded.\n  */\n void get_packed_object_summary(struct strbuf *obj_info);\n+\n+/**\n+ * Adds a list of all contents (recursively) of '.git/objects/info'. The\n+ * previous contents of hook_info will be discarded.\n+ */\n+void get_object_info_summary(struct strbuf *obj_info);\ndiff --git a/builtin/bugreport.c b/builtin/bugreport.c\nindex da91a3944e..8aad33a9b0 100644\n--- a/builtin/bugreport.c\n+++ b/builtin/bugreport.c\n@@ -72,6 +72,13 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix)\n \tget_packed_object_summary(&buffer);\n \tstrbuf_write(&buffer, report);\n \n+\tadd_header(report, \"Object Info Data\");\n+\tget_object_info_summary(&buffer);\n+\tstrbuf_write(&buffer, report);\n+\n+\t// Close file\n+\t// open file in editor\n+\tlaunch_editor(report_path, NULL, NULL);\n \tfclose(report);\n \n \tlaunch_editor(report_path.buf, NULL, NULL);\n-- \n2.24.0.rc0.303.g954a862665-goog\n\n"},{"id":"384836","messageId":"20191025025129.250049-10-emilyshaffer@google.com","threadId":"51645","inReplyTo":"20191025025129.250049-1-emilyshaffer@google.com","subject":"[PATCH v3 9/9] bugreport: print contents of alternates file","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2019-10-25T02:51:29Z","receivedAt":"2019-10-25T02:52:14Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"In some cases, it could be that the user is having a problem with an\nobject which isn't present in their normal object directory. We can get\na hint that that might be the case by examining the list of alternates\nwhere their object may be stored instead.\n\nSigned-off-by: Emily Shaffer <emilyshaffer@google.com>\n---\n bugreport.c         | 14 ++++++++++++++\n bugreport.h         |  6 ++++++\n builtin/bugreport.c |  4 ++++\n 3 files changed, 24 insertions(+)\n\ndiff --git a/bugreport.c b/bugreport.c\nindex ce15904fec..a7bdc72b7f 100644\n--- a/bugreport.c\n+++ b/bugreport.c\n@@ -298,3 +298,17 @@ void get_object_info_summary(struct strbuf *obj_info)\n \t\tstrbuf_complete_line(obj_info);\n \t}\n }\n+\n+void get_alternates_file(struct strbuf *alternates_info)\n+{\n+\tstruct strbuf alternates_path = STRBUF_INIT;\n+\n+\tstrbuf_addstr(&alternates_path, get_object_directory());\n+\tstrbuf_complete(&alternates_path, '/');\n+\tstrbuf_addstr(&alternates_path, \"info/alternates\");\n+\n+\tstrbuf_reset(alternates_info);\n+\tstrbuf_addbuf(alternates_info, &alternates_path);\n+\tstrbuf_complete_line(alternates_info);\n+\tstrbuf_read_file(alternates_info, alternates_path.buf, 0);\n+}\ndiff --git a/bugreport.h b/bugreport.h\nindex 4f5e2d1b9a..74d1f79960 100644\n--- a/bugreport.h\n+++ b/bugreport.h\n@@ -36,3 +36,9 @@ void get_packed_object_summary(struct strbuf *obj_info);\n  * previous contents of hook_info will be discarded.\n  */\n void get_object_info_summary(struct strbuf *obj_info);\n+\n+/**\n+ * Adds the contents of '.git/info/alternates'. The previous contents of\n+ * alternates_info will be discarded.\n+ */\n+void get_alternates_file(struct strbuf *alt_info);\ndiff --git a/builtin/bugreport.c b/builtin/bugreport.c\nindex 8aad33a9b0..0784bdc42a 100644\n--- a/builtin/bugreport.c\n+++ b/builtin/bugreport.c\n@@ -76,6 +76,10 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix)\n \tget_object_info_summary(&buffer);\n \tstrbuf_write(&buffer, report);\n \n+\tadd_header(report, \"Alternates File\");\n+\tget_alternates_file(&buffer);\n+\tstrbuf_write(&buffer, report);\n+\n \t// Close file\n \t// open file in editor\n \tlaunch_editor(report_path, NULL, NULL);\n-- \n2.24.0.rc0.303.g954a862665-goog\n\n"},{"id":"384991","messageId":"nycvar.QRO.7.76.6.1910281425360.46@tvgsbejvaqbjf.bet","threadId":"51645","inReplyTo":"20191025025129.250049-3-emilyshaffer@google.com","subject":"Re: [PATCH v3 2/9] bugreport: generate config whitelist based on docs","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2019-10-28T13:27:32Z","receivedAt":"2019-10-28T13:27:52Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Emily,\n\nOn Thu, 24 Oct 2019, Emily Shaffer wrote:\n\n> diff --git a/bugreport-generate-config-whitelist.sh b/bugreport-generate-config-whitelist.sh\n> new file mode 100755\n> index 0000000000..ca6b232024\n> --- /dev/null\n> +++ b/bugreport-generate-config-whitelist.sh\n> @@ -0,0 +1,4 @@\n> +#!/bin/sh\n> +\n> +grep -RhPo \".*(?=:: \\/\\/ bugreport-include)\" Documentation/config \\\n\nI am rather certain that `-P` is not supported by BSD grep (see\nhttps://man.openbsd.org/grep).\n\nWhy not something portable, e.g.\n\n\tfind Documentation/config -type f -exec cat {} \\; |\n\tsed -n 's/^\\(.*\\):: \\/\\/ bugreport-include$/\\1/p' \\\n\n?\n\nCiao,\nDscho\n\n> +  >git-bugreport-config-whitelist\n> --\n> 2.24.0.rc0.303.g954a862665-goog\n>\n>\n"},{"id":"384992","messageId":"nycvar.QRO.7.76.6.1910281432180.46@tvgsbejvaqbjf.bet","threadId":"51645","inReplyTo":"20191025025129.250049-4-emilyshaffer@google.com","subject":"Re: [PATCH v3 3/9] bugreport: add version and system information","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2019-10-28T13:49:29Z","receivedAt":"2019-10-28T13:49:50Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Emily,\n\nOn Thu, 24 Oct 2019, Emily Shaffer wrote:\n\n> diff --git a/bugreport.c b/bugreport.c\n> new file mode 100644\n> index 0000000000..ada54fe583\n> --- /dev/null\n> +++ b/bugreport.c\n> @@ -0,0 +1,55 @@\n> +#include \"cache.h\"\n> +\n> +#include \"bugreport.h\"\n> +#include \"help.h\"\n> +#include \"run-command.h\"\n> +#include \"version.h\"\n> +\n> +void get_system_info(struct strbuf *sys_info)\n> +{\n> +\tstruct child_process cp = CHILD_PROCESS_INIT;\n> +\tstruct strbuf std_out = STRBUF_INIT;\n> +\n> +\tstrbuf_reset(sys_info);\n> +\n> +\t// get git version from native cmd\n\nPlease use C-style comments instead of C++ ones.\n\n> +\tstrbuf_addstr(sys_info, \"git version: \");\n> +\tstrbuf_addstr(sys_info, git_version_string);\n> +\tstrbuf_complete_line(sys_info);\n> +\n> +\t// system call for other version info\n> +\targv_array_push(&cp.args, \"uname\");\n> +\targv_array_push(&cp.args, \"-a\");\n> +\tcapture_command(&cp, &std_out, 0);\n\nMmmkay. I am not too much of a fan of relying on the `uname` executable,\nas it can very well be a 32-bit `uname.exe` on Windows, which obviously\nwould _not_ report the architecture of the machine, but something\nmisleading.\n\nWhy not use the `uname()` function (that we even define in\n`compat/mingw.c`)?\n\nAlso, why not include the same information as `git version\n--build-options`, most importantly `GIT_HOST_CPU`?\n\nIn any case, if you are capturing the output of `uname -a`, you\ndefinitely will want to pass the `RUN_COMMAND_STDOUT_TO_STDERR` flag (in\ncase, say, the MSYS2 `uname.exe` fails with those dreaded Cygwin error\nmessages like \"*** fatal error - add_item\n(\"\\??\\D:\\git\\installation\\Git\", \"/\", ...) failed, errno 1\").\n\n> +\n> +\tstrbuf_addstr(sys_info, \"uname -a: \");\n> +\tstrbuf_addbuf(sys_info, &std_out);\n> +\tstrbuf_complete_line(sys_info);\n> +\n> +\targv_array_clear(&cp.args);\n> +\tstrbuf_reset(&std_out);\n> +\n> +\n> +\targv_array_push(&cp.args, \"curl-config\");\n> +\targv_array_push(&cp.args, \"--version\");\n> +\tcapture_command(&cp, &std_out, 0);\n\nThis will be almost certainly be incorrect, as most _regular_ users\nwon't have `curl-config`. I know, it is easy to forget that most Git\nusers are no hard-core C developers ;-)\n\nA better alternative would be to use `curl_version()`, guarded, of\ncourse, by `#ifndef NO_CURL`...\n\n> +\n> +\tstrbuf_addstr(sys_info, \"curl-config --version: \");\n> +\tstrbuf_addbuf(sys_info, &std_out);\n> +\tstrbuf_complete_line(sys_info);\n> +\n> +\targv_array_clear(&cp.args);\n> +\tstrbuf_reset(&std_out);\n> +\n> +\n> +\targv_array_push(&cp.args, \"ldd\");\n> +\targv_array_push(&cp.args, \"--version\");\n> +\tcapture_command(&cp, &std_out, 0);\n\nAgain, this command will only be present in few setups. I am not\nactually sure that the output of this is interesting to begin with.\n\nWhat I _do_ think is that a much more interesting piece of information\nwould be the exact GLIBC version, the OS name and the OS version.\n\n> +\n> +\tstrbuf_addstr(sys_info, \"ldd --version: \");\n> +\tstrbuf_addbuf(sys_info, &std_out);\n> +\tstrbuf_complete_line(sys_info);\n> +\n> +\targv_array_clear(&cp.args);\n> +\tstrbuf_reset(&std_out);\n> +}\n> diff --git a/bugreport.h b/bugreport.h\n> new file mode 100644\n> index 0000000000..ba216acf3f\n> --- /dev/null\n> +++ b/bugreport.h\n> @@ -0,0 +1,7 @@\n> +#include \"strbuf.h\"\n> +\n> +/**\n> + * Adds the Git version, `uname -a`, and `curl-config --version` to sys_info.\n> + * The previous contents of sys_info will be discarded.\n> + */\n> +void get_system_info(struct strbuf *sys_info);\n> diff --git a/builtin/bugreport.c b/builtin/bugreport.c\n> index 2ef16440a0..7232d31be7 100644\n> --- a/builtin/bugreport.c\n> +++ b/builtin/bugreport.c\n> @@ -1,4 +1,5 @@\n>  #include \"builtin.h\"\n> +#include \"bugreport.h\"\n>  #include \"stdio.h\"\n>  #include \"strbuf.h\"\n>  #include \"time.h\"\n> @@ -27,6 +28,13 @@ int get_bug_template(struct strbuf *template)\n>  \treturn 0;\n>  }\n>\n> +void add_header(FILE *report, const char *title)\n> +{\n> +\tstruct strbuf buffer = STRBUF_INIT;\n> +\tstrbuf_addf(&buffer, \"\\n\\n[%s]\\n\", title);\n> +\tstrbuf_write(&buffer, report);\n\nThis leaks `buffer`. Why not write into `report` via `fprintf()`\ndirectly?\n\nCiao,\nDscho\n\n> +}\n> +\n>  int cmd_bugreport(int argc, const char **argv, const char *prefix)\n>  {\n>  \tstruct strbuf buffer = STRBUF_INIT;\n> @@ -43,6 +51,11 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix)\n>  \tget_bug_template(&buffer);\n>  \tstrbuf_write(&buffer, report);\n>\n> +\t// add other contents\n> +\tadd_header(report, \"System Info\");\n> +\tget_system_info(&buffer);\n> +\tstrbuf_write(&buffer, report);\n> +\n>  \tfclose(report);\n>\n>  \tlaunch_editor(report_path.buf, NULL, NULL);\n> --\n> 2.24.0.rc0.303.g954a862665-goog\n>\n>\n"},{"id":"384993","messageId":"nycvar.QRO.7.76.6.1910281506040.46@tvgsbejvaqbjf.bet","threadId":"51645","inReplyTo":"20191025025129.250049-5-emilyshaffer@google.com","subject":"Re: [PATCH v3 4/9] bugreport: add config values from whitelist","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2019-10-28T14:14:35Z","receivedAt":"2019-10-28T14:15:08Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Emily,\n\nOn Thu, 24 Oct 2019, Emily Shaffer wrote:\n\n> Teach bugreport to gather the values of config options which are present\n> in 'git-bugreport-config-whitelist'.\n>\n> Many config options are sensitive, and many Git add-ons use config\n> options which git-core does not know about; it is better only to gather\n> config options which we know to be safe, rather than excluding options\n> which we know to be unsafe.\n\nShould we still have the `// bugreport-exclude` comments, then?\n\n>\n> Reading the whitelist into memory and sorting it saves us time -\n> since git_config_bugreport() is called for every option the user has\n> configured, limiting the file IO to one open/read/close and performing\n> option lookup in sublinear time is a useful optimization.\n\nMaybe we even want a hashmap? That would reduce the time complexity even\nfurther.\n\n> diff --git a/bugreport.c b/bugreport.c\n> index ada54fe583..afa4836ab1 100644\n> --- a/bugreport.c\n> +++ b/bugreport.c\n> @@ -1,10 +1,24 @@\n>  #include \"cache.h\"\n>\n>  #include \"bugreport.h\"\n> +#include \"config.h\"\n> +#include \"exec-cmd.h\"\n>  #include \"help.h\"\n>  #include \"run-command.h\"\n>  #include \"version.h\"\n>\n> +/**\n> + * A sorted list of config options which we will add to the bugreport. Managed\n> + * by 'gather_whitelist(...)'.\n> + */\n> +struct string_list whitelist = STRING_LIST_INIT_DUP;\n> +struct strbuf configs_and_values = STRBUF_INIT;\n> +\n> +// git version --build-options\n> +// uname -a\n> +// curl-config --version\n> +// ldd --version\n> +// echo $SHELL\n\nThese comments probably want to move to a single, C style comment, and\nthey probably want to be introduced together with `get_system_info()`.\n\nI also have to admit that I might have missed where `$SHELL` was added\nto the output...\n\n>  void get_system_info(struct strbuf *sys_info)\n>  {\n>  \tstruct child_process cp = CHILD_PROCESS_INIT;\n> @@ -53,3 +67,39 @@ void get_system_info(struct strbuf *sys_info)\n>  \targv_array_clear(&cp.args);\n>  \tstrbuf_reset(&std_out);\n>  }\n> +\n> +void gather_whitelist(struct strbuf *path)\n> +{\n> +\tstruct strbuf tmp = STRBUF_INIT;\n> +\tstrbuf_read_file(&tmp, path->buf, 0);\n> +\tstring_list_init(&whitelist, 1);\n> +\tstring_list_split(&whitelist, tmp.buf, '\\n', -1);\n> +\tstring_list_sort(&whitelist);\n> +}\n> +\n> +int git_config_bugreport(const char *var, const char *value, void *cb)\n> +{\n> +\tif (string_list_has_string(&whitelist, var)) {\n> +\t\tstrbuf_addf(&configs_and_values,\n> +\t\t\t    \"%s : %s\\n\",\n> +\t\t\t    var, value);\n\nA quite useful piece of information would be the config source. Not sure\nwhether we can do that outside of `config.c` yet...\n\n> +\t}\n> +\n> +\treturn 0;\n> +}\n> +\n> +void get_whitelisted_config(struct strbuf *config_info)\n> +{\n> +\tstruct strbuf path = STRBUF_INIT;\n> +\n> +\tstrbuf_addstr(&path, git_exec_path());\n> +\tstrbuf_addstr(&path, \"/git-bugreport-config-whitelist\");\n\nHmm. I would have expected this patch to come directly after the patch\n2/9 that generates that white-list, and I would also have expected that\nto be pre-sorted, and compiled in.\n\nDo you want users to _edit_ the file in the exec path? In general, that\npath will be write-protected, though. A better alternative would\nprobably be to compile in a hard-coded list, and to allow including more\nvalues e.g. by offering command-line options to specify config setting\npatterns. But if we allow patterns, we might actually want to have those\nexclusions to prevent sensitive data from being included.\n\n> +\n> +\tgather_whitelist(&path);\n> +\tstrbuf_init(&configs_and_values, whitelist.nr);\n> +\n> +\tgit_config(git_config_bugreport, NULL);\n> +\n> +\tstrbuf_reset(config_info);\n> +\tstrbuf_addbuf(config_info, &configs_and_values);\n> +}\n> diff --git a/bugreport.h b/bugreport.h\n> index ba216acf3f..7413e7e1be 100644\n> --- a/bugreport.h\n> +++ b/bugreport.h\n> @@ -5,3 +5,10 @@\n>   * The previous contents of sys_info will be discarded.\n>   */\n>  void get_system_info(struct strbuf *sys_info);\n> +\n> +/**\n\nI also frequently use JavaDoc-style `/**`, but I am not sure that this\nis actually desired in Git's source code ;-)\n\n> + * Adds the values of the config items listed in\n> + * 'git-bugreport-config-whitelist' to config_info. The previous contents of\n> + * config_info will be discarded.\n> + */\n> +void get_whitelisted_config(struct strbuf *sys_info);\n> diff --git a/builtin/bugreport.c b/builtin/bugreport.c\n> index 7232d31be7..70fe0d2b85 100644\n> --- a/builtin/bugreport.c\n> +++ b/builtin/bugreport.c\n> @@ -56,6 +56,10 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix)\n>  \tget_system_info(&buffer);\n>  \tstrbuf_write(&buffer, report);\n>\n> +\tadd_header(report, \"Whitelisted Config\");\n\nQuite honestly, I would like to avoid the term \"whitelist\" for good. How\nabout \"Selected config settings\" instead?\n\nThanks,\nDscho\n\n> +\tget_whitelisted_config(&buffer);\n> +\tstrbuf_write(&buffer, report);\n> +\n>  \tfclose(report);\n>\n>  \tlaunch_editor(report_path.buf, NULL, NULL);\n> --\n> 2.24.0.rc0.303.g954a862665-goog\n>\n>\n"},{"id":"384997","messageId":"nycvar.QRO.7.76.6.1910281522200.46@tvgsbejvaqbjf.bet","threadId":"51645","inReplyTo":"20191025025129.250049-6-emilyshaffer@google.com","subject":"Re: [PATCH v3 5/9] bugreport: collect list of populated hooks","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2019-10-28T14:31:36Z","receivedAt":"2019-10-28T14:31:56Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Emily,\n\nOn Thu, 24 Oct 2019, Emily Shaffer wrote:\n\n> Occasionally a failure a user is seeing may be related to a specific\n> hook which is being run, perhaps without the user realizing. While the\n> contents of hooks can be sensitive - containing user data or process\n> information specific to the user's organization - simply knowing that a\n> hook is being run at a certain stage can help us to understand whether\n> something is going wrong.\n>\n> Without a definitive list of hook names within the code, we compile our\n> own list from the documentation. This is likely prone to bitrot. To\n> reduce the amount of code humans need to read, we turn the list into a\n> string_list and iterate over it (as we are calling the same find_hook\n> operation on each string). However, since bugreport should primarily be\n> called by the user, the performance loss from massaging the string\n> seems acceptable.\n\nGood idea!\n\n>\n> Signed-off-by: Emily Shaffer <emilyshaffer@google.com>\n> ---\n>  bugreport.c         | 44 ++++++++++++++++++++++++++++++++++++++++++++\n>  bugreport.h         |  6 ++++++\n>  builtin/bugreport.c |  4 ++++\n>  3 files changed, 54 insertions(+)\n>\n> diff --git a/bugreport.c b/bugreport.c\n> index afa4836ab1..9d7f44ff28 100644\n> --- a/bugreport.c\n> +++ b/bugreport.c\n> @@ -103,3 +103,47 @@ void get_whitelisted_config(struct strbuf *config_info)\n>  \tstrbuf_reset(config_info);\n>  \tstrbuf_addbuf(config_info, &configs_and_values);\n>  }\n> +\n> +void get_populated_hooks(struct strbuf *hook_info)\n> +{\n> +\t/*\n> +\t * Doesn't look like there is a list of all possible hooks; so below is\n> +\t * a transcription of `git help hook`.\n> +\t */\n> +\tconst char *hooks = \"applypatch-msg,\"\n> +\t\t\t    \"pre-applypatch,\"\n> +\t\t\t    \"post-applypatch,\"\n> +\t\t\t    \"pre-commit,\"\n> +\t\t\t    \"pre-merge-commit,\"\n> +\t\t\t    \"prepare-commit-msg,\"\n> +\t\t\t    \"commit-msg,\"\n> +\t\t\t    \"post-commit,\"\n> +\t\t\t    \"pre-rebase,\"\n> +\t\t\t    \"post-checkout,\"\n> +\t\t\t    \"post-merge,\"\n> +\t\t\t    \"pre-push,\"\n> +\t\t\t    \"pre-receive,\"\n> +\t\t\t    \"update,\"\n> +\t\t\t    \"post-receive,\"\n> +\t\t\t    \"post-update,\"\n> +\t\t\t    \"push-to-checkout,\"\n> +\t\t\t    \"pre-auto-gc,\"\n> +\t\t\t    \"post-rewrite,\"\n> +\t\t\t    \"sendemail-validate,\"\n> +\t\t\t    \"fsmonitor-watchman,\"\n> +\t\t\t    \"p4-pre-submit,\"\n> +\t\t\t    \"post-index-changex\";\n\nWell, this is disappointing: I tried to come up with a scripted way to\nextract the commit names from our source code, and I failed! This is\nwhere I gave up:\n\n\tgit grep run_hook |\n\tsed -n 's/.*run_hook[^(]*([^,]*, *\\([^,]*\\).*/\\1/p' |\n\tsed -e '/^name$/d' -e '/^const char \\*name$/d' \\\n\t\t-e 's/push_to_checkout_hook/\"push-to-checkout\"/' |\n\tsort |\n\tuniq\n\nThis prints only the following hook names:\n\n\t\"applypatch-msg\"\n\t\"post-applypatch\"\n\t\"post-checkout\"\n\t\"post-index-change\"\n\t\"post-merge\"\n\t\"pre-applypatch\"\n\t\"pre-auto-gc\"\n\t\"pre-rebase\"\n\t\"prepare-commit-msg\"\n\t\"push-to-checkout\"\n\nBut at least it was easy to script the extracting from the\nDocumentation:\n\n\tsed -n '/^[a-z]/{N;/\\n~~~/{s/\\n.*//;p}}' \\\n\t\tDocumentation/githooks.txt\n\n> +\tstruct string_list hooks_list = STRING_LIST_INIT_DUP;\n> +\tstruct string_list_item *iter = NULL;\n> +\n> +\tstrbuf_reset(hook_info);\n> +\n> +\tstring_list_split(&hooks_list, hooks, ',', -1);\n> +\n> +\tfor_each_string_list_item(iter, &hooks_list) {\n\nThis should definitely be done at compile time, I think. We should be\nable to generate an appropriate header via something like this:\n\n\tcat >hook-names.h <<-EOF\n\tstatic const char *hook_names[] = {\n\t$(sed -n '/^[a-z]/{N;/\\n~~~/{s/\\n.*/\",/;s/^/\t\"/;p}}' \\\n\t\tDocumentation/githooks.txt)\n\t};\n\tEOF\n\nThen you would use a simple `for()` loop using `ARRAY_SIZE()` to\niterate over the hook names.\n\n> +\t\tif (find_hook(iter->string)) {\n> +\t\t\tstrbuf_addstr(hook_info, iter->string);\n> +\t\t\tstrbuf_complete_line(hook_info);\n> +\t\t}\n> +\t}\n> +}\n> diff --git a/bugreport.h b/bugreport.h\n> index 7413e7e1be..942a5436e3 100644\n> --- a/bugreport.h\n> +++ b/bugreport.h\n> @@ -12,3 +12,9 @@ void get_system_info(struct strbuf *sys_info);\n>   * config_info will be discarded.\n>   */\n>  void get_whitelisted_config(struct strbuf *sys_info);\n> +\n> +/**\n> + * Adds the paths to all configured hooks (but not their contents). The previous\n> + * contents of hook_info will be discarded.\n> + */\n> +void get_populated_hooks(struct strbuf *hook_info);\n> diff --git a/builtin/bugreport.c b/builtin/bugreport.c\n> index 70fe0d2b85..a0eefba498 100644\n> --- a/builtin/bugreport.c\n> +++ b/builtin/bugreport.c\n> @@ -60,6 +60,10 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix)\n>  \tget_whitelisted_config(&buffer);\n>  \tstrbuf_write(&buffer, report);\n>\n> +\tadd_header(report, \"Populated Hooks\");\n\nWait! I should have stumbled over this in an earlier patch. The\n`add_header()` function should not take a `FILE *` parameter at all, but\ninstead an `struct strbuf *` one!\n\nCiao,\nDscho\n\n> +\tget_populated_hooks(&buffer);\n> +\tstrbuf_write(&buffer, report);\n> +\n>  \tfclose(report);\n>\n>  \tlaunch_editor(report_path.buf, NULL, NULL);\n> --\n> 2.24.0.rc0.303.g954a862665-goog\n>\n>\n"},{"id":"385001","messageId":"nycvar.QRO.7.76.6.1910281540550.46@tvgsbejvaqbjf.bet","threadId":"51645","inReplyTo":"20191025025129.250049-7-emilyshaffer@google.com","subject":"Re: [PATCH v3 6/9] bugreport: count loose objects","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2019-10-28T15:07:40Z","receivedAt":"2019-10-28T15:08:10Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Emily,\n\nOn Thu, 24 Oct 2019, Emily Shaffer wrote:\n\n> The number of unpacked objects in a user's repository may help us\n> understand the root of the problem they're seeing, especially if a\n> command is running unusually slowly.\n>\n> Rather than directly invoking 'git-count-objects', which may sometimes\n> fail unexpectedly on Git for Windows, manually count the contents of\n> .git/objects. Additionally, since we may wish to inspect other\n> directories' contents for bugreport in the future, put the directory\n> listing into a helper function.\n\nThank you, much appreciated!\n\nI guess the next step is to count the number of packs, and the number of\nsubmodules ;-)\n\n>\n> Signed-off-by: Emily Shaffer <emilyshaffer@google.com>\n> ---\n>  bugreport.c         | 72 +++++++++++++++++++++++++++++++++++++++++++++\n>  bugreport.h         |  6 ++++\n>  builtin/bugreport.c |  4 +++\n>  3 files changed, 82 insertions(+)\n>\n> diff --git a/bugreport.c b/bugreport.c\n> index 9d7f44ff28..54e1d47103 100644\n> --- a/bugreport.c\n> +++ b/bugreport.c\n> @@ -5,8 +5,11 @@\n>  #include \"exec-cmd.h\"\n>  #include \"help.h\"\n>  #include \"run-command.h\"\n> +#include \"strbuf.h\"\n\nWhy not append this to the end of the `#include` list, as is common in\nGit's commit history?\n\n>  #include \"version.h\"\n>\n> +#include \"dirent.h\"\n\nThis header (although with pointy brackets instead of double quotes) is\nalready included in `git-compat-util.h`\n\n> +\n>  /**\n>   * A sorted list of config options which we will add to the bugreport. Managed\n>   * by 'gather_whitelist(...)'.\n> @@ -147,3 +150,72 @@ void get_populated_hooks(struct strbuf *hook_info)\n>  \t\t}\n>  \t}\n>  }\n> +\n> +/**\n> + * Fill 'contents' with the contents of the dir at 'dirpath'.\n\nSince you start this comment in JavaDoc style, there should be an almost\nempty line after this one (\"almost\" because it still contains the\nasterisk, of course).\n\n> + * If 'filter' is nonzero, the contents are filtered on d_type as 'type' - see\n> + * 'man readdir'. opendir() doesn't take string length as an arg, so don't\n> + * bother passing it in.\n> + */\n> +void list_contents_of_dir(struct string_list *contents, struct strbuf *dirpath,\n\nShouldn't this be `static`?\n\n> +\t\t\t  int filter, unsigned char type)\n> +{\n> +\tstruct dirent *dir = NULL;\n> +\tDIR *dh = NULL;\n> +\n> +\tdh = opendir(dirpath->buf);\n> +\twhile (dh && (dir = readdir(dh))) {\n> +\t\tif (!filter || type == dir->d_type) {\n> +\t\t\tstring_list_append(contents, dir->d_name);\n> +\t\t}\n> +\t}\n> +}\n> +\n> +\n> +void get_object_counts(struct strbuf *obj_info)\n\nOops. This function is no longer used.\n\n> +{\n> +\tstruct child_process cp = CHILD_PROCESS_INIT;\n> +\tstruct strbuf std_out = STRBUF_INIT;\n> +\n> +\targv_array_push(&cp.args, \"count-objects\");\n> +\targv_array_push(&cp.args, \"-vH\");\n> +\tcp.git_cmd = 1;\n> +\tcapture_command(&cp, &std_out, 0);\n> +\n> +\tstrbuf_reset(obj_info);\n> +\tstrbuf_addstr(obj_info, \"git-count-objects -vH:\\n\");\n> +\tstrbuf_addbuf(obj_info, &std_out);\n> +}\n> +\n> +void get_loose_object_summary(struct strbuf *obj_info)\n> +{\n> +\tstruct strbuf dirpath = STRBUF_INIT;\n> +\tstruct string_list subdirs = STRING_LIST_INIT_DUP;\n> +\tstruct string_list_item *subdir;\n> +\n> +\tstrbuf_reset(obj_info);\n> +\n> +\tstrbuf_addstr(&dirpath, get_object_directory());\n> +\tstrbuf_complete(&dirpath, '/');\n> +\n> +\tlist_contents_of_dir(&subdirs, &dirpath, 1, DT_DIR);\n> +\n> +\tfor_each_string_list_item(subdir, &subdirs)\n> +\t{\n> +\t\tstruct strbuf subdir_buf = STRBUF_INIT;\n> +\t\tstruct string_list objects = STRING_LIST_INIT_DUP;\n> +\n> +\t\t/*\n> +\t\t * Only interested in loose objects - so dirs named with the\n> +\t\t * first byte of the object ID\n> +\t\t */\n> +\t\tif (strlen(subdir->string) != 2 || !strcmp(subdir->string, \"..\"))\n> +\t\t\tcontinue;\n> +\n> +\t\tstrbuf_addbuf(&subdir_buf, &dirpath);\n> +\t\tstrbuf_addstr(&subdir_buf, subdir->string);\n> +\t\tlist_contents_of_dir(&objects, &subdir_buf, 0, 0);\n> +\t\tstrbuf_addf(obj_info, \"%s: %d objects\\n\", subdir->string,\n> +\t\t\t    objects.nr);\n\nHmm. Not only does this leak `objects`, it also throws away the contents\nthat we so painfully constructed.\n\nWouldn't it make more sense to do something like this instead?\n\nstatic int is_hex(const char *string, size_t count)\n{\n\tfor (; count; string++, count--)\n\t\tif (hexval(*string) < 0)\n\t\t\treturn 0;\n\treturn 1;\n}\n\nstatic ssize_t count_loose_objects(struct strbuf *objects_path)\n{\n\tssize_t ret = 0;\n\tsize_t len;\n\tstruct dirent *d;\n\tDIR *dir, *subdir;\n\n\tdir = opendir(objects_path->buf);\n\tif (!dir)\n\t\treturn -1;\n\n\tstrbuf_complete(objects_path, '/');\n\tlen = objects_path->len;\n\twhile ((d = readdir(dir))) {\n\t\tif (d->d_type != DT_DIR)\n\t\t\tcontinue;\n\t\tstrbuf_setlen(objects_path, len);\n\t\tstrbuf_addstr(objects_path, d->d_name);\n\t\tsubdir = opendir(objects_path->buf);\n\t\tif (!subdir)\n\t\t\tcontinue;\n\t\twhile ((d = readdir(subdir)))\n\t\t\tif (d->dt_type == DT_REG &&\n\t\t\t    is_hex(dir->d_name, the_repository->hash_algo->hexsz))\n\t\t\t\tret++;\n\t\tclosedir(subdir);\n\t}\n\tclosedir(dir);\n\tstrbuf_reset(objects_path, len);\n\treturn ret;\n}\n\nCiao,\nDscho\n\n> +\t}\n> +}\n> diff --git a/bugreport.h b/bugreport.h\n> index 942a5436e3..09ad0c2599 100644\n> --- a/bugreport.h\n> +++ b/bugreport.h\n> @@ -18,3 +18,9 @@ void get_whitelisted_config(struct strbuf *sys_info);\n>   * contents of hook_info will be discarded.\n>   */\n>  void get_populated_hooks(struct strbuf *hook_info);\n> +\n> +/**\n> + * Adds the output of `git count-object -vH`. The previous contents of hook_info\n> + * will be discarded.\n> + */\n> +void get_loose_object_summary(struct strbuf *obj_info);\n> diff --git a/builtin/bugreport.c b/builtin/bugreport.c\n> index a0eefba498..b2ab194207 100644\n> --- a/builtin/bugreport.c\n> +++ b/builtin/bugreport.c\n> @@ -64,6 +64,10 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix)\n>  \tget_populated_hooks(&buffer);\n>  \tstrbuf_write(&buffer, report);\n>\n> +\tadd_header(report, \"Object Counts\");\n> +\tget_loose_object_summary(&buffer);\n> +\tstrbuf_write(&buffer, report);\n> +\n>  \tfclose(report);\n>\n>  \tlaunch_editor(report_path.buf, NULL, NULL);\n> --\n> 2.24.0.rc0.303.g954a862665-goog\n>\n>\n"},{"id":"385003","messageId":"nycvar.QRO.7.76.6.1910281608170.46@tvgsbejvaqbjf.bet","threadId":"51645","inReplyTo":"20191025025129.250049-8-emilyshaffer@google.com","subject":"Re: [PATCH v3 7/9] bugreport: add packed object summary","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2019-10-28T15:43:50Z","receivedAt":"2019-10-28T15:44:10Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Emily,\n\nOn Thu, 24 Oct 2019, Emily Shaffer wrote:\n\n> Alongside the list of loose objects, it's useful to see the list of\n> object packs as well. It can help us to examine what Git did and did not\n> pack.\n\nYes, I was write! Packs are next ;-)\n\n>\n> Signed-off-by: Emily Shaffer <emilyshaffer@google.com>\n> ---\n>  bugreport.c         | 21 +++++++++++++++++++++\n>  bugreport.h         |  6 ++++++\n>  builtin/bugreport.c |  4 ++++\n>  3 files changed, 31 insertions(+)\n>\n> diff --git a/bugreport.c b/bugreport.c\n> index 54e1d47103..79ddb8baaa 100644\n> --- a/bugreport.c\n> +++ b/bugreport.c\n> @@ -219,3 +219,24 @@ void get_loose_object_summary(struct strbuf *obj_info)\n>  \t\t\t    objects.nr);\n>  \t}\n>  }\n> +\n> +void get_packed_object_summary(struct strbuf *obj_info)\n> +{\n> +\tstruct strbuf dirpath = STRBUF_INIT;\n> +\tstruct string_list contents = STRING_LIST_INIT_DUP;\n> +\tstruct string_list_item *entry;\n> +\n> +\tstrbuf_reset(obj_info);\n> +\n> +\tstrbuf_addstr(&dirpath, get_object_directory());\n> +\tstrbuf_complete(&dirpath, '/');\n> +\tstrbuf_addstr(&dirpath, \"pack/\");\n> +\tlist_contents_of_dir(&contents, &dirpath, 0, 0);\n> +\n> +\t// list full contents of $GIT_OBJECT_DIRECTORY/pack/\n> +\tfor_each_string_list_item(entry, &contents) {\n> +\t\tstrbuf_addbuf(obj_info, &dirpath);\n> +\t\tstrbuf_addstr(obj_info, entry->string);\n> +\t\tstrbuf_complete_line(obj_info);\n> +\t}\n> +}\n\nOkay, but I think that you will want to discern between regular `.pack`\nfiles, `.idx` files and `tmp_*` files.\n\n> diff --git a/bugreport.h b/bugreport.h\n> index 09ad0c2599..11ff7df41b 100644\n> --- a/bugreport.h\n> +++ b/bugreport.h\n> @@ -24,3 +24,9 @@ void get_populated_hooks(struct strbuf *hook_info);\n>   * will be discarded.\n>   */\n>  void get_loose_object_summary(struct strbuf *obj_info);\n> +\n> +/**\n> + * Adds a list of the contents of '.git/objects/pack'. The previous contents of\n> + * hook_info will be discarded.\n> + */\n> +void get_packed_object_summary(struct strbuf *obj_info);\n> diff --git a/builtin/bugreport.c b/builtin/bugreport.c\n> index b2ab194207..da91a3944e 100644\n> --- a/builtin/bugreport.c\n> +++ b/builtin/bugreport.c\n> @@ -68,6 +68,10 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix)\n>  \tget_loose_object_summary(&buffer);\n>  \tstrbuf_write(&buffer, report);\n>\n> +\tadd_header(report, \"Packed Object Summary\");\n> +\tget_packed_object_summary(&buffer);\n> +\tstrbuf_write(&buffer, report);\n> +\n\nHmm. At this point, I am unclear whether you want to write into an\n`strbuf`, or directly into a `FILE *`? I would rather have only one, not\na mix.\n\nCiao,\nDscho\n\n>  \tfclose(report);\n>\n>  \tlaunch_editor(report_path.buf, NULL, NULL);\n> --\n> 2.24.0.rc0.303.g954a862665-goog\n>\n>\n"},{"id":"385004","messageId":"nycvar.QRO.7.76.6.1910281645580.46@tvgsbejvaqbjf.bet","threadId":"51645","inReplyTo":"20191025025129.250049-9-emilyshaffer@google.com","subject":"Re: [PATCH v3 8/9] bugreport: list contents of $OBJDIR/info","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2019-10-28T15:51:52Z","receivedAt":"2019-10-28T15:52:12Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Emily,\n\nOn Thu, 24 Oct 2019, Emily Shaffer wrote:\n\n> Miscellaneous information used about the object store can end up in\n> .git/objects/info; this can help us understand what may be going on with\n> the object store when the user is reporting a bug. Otherwise, it could\n> be difficult to track down what is going wrong with an object which\n> isn't kept locally to .git/objects/ or .git/objects/pack. Having some\n> understanding of where the user's objects may be kept can save us some\n> hops during the bug reporting process.\n\nMakes sense.\n\n>\n> Signed-off-by: Emily Shaffer <emilyshaffer@google.com>\n> ---\n>  bugreport.c         | 58 +++++++++++++++++++++++++++++++++++++++++++++\n>  bugreport.h         |  6 +++++\n>  builtin/bugreport.c |  7 ++++++\n>  3 files changed, 71 insertions(+)\n>\n> diff --git a/bugreport.c b/bugreport.c\n> index 79ddb8baaa..ce15904fec 100644\n> --- a/bugreport.c\n> +++ b/bugreport.c\n> @@ -6,6 +6,7 @@\n>  #include \"help.h\"\n>  #include \"run-command.h\"\n>  #include \"strbuf.h\"\n> +#include \"string-list.h\"\n>  #include \"version.h\"\n>\n>  #include \"dirent.h\"\n> @@ -171,6 +172,41 @@ void list_contents_of_dir(struct string_list *contents, struct strbuf *dirpath,\n>  \t}\n>  }\n>\n> +/**\n> + * Fills 'contents' with a list of all directories within the provided\n> + * directory, recursing into each directory.\n> + */\n> +void list_contents_of_dir_recursively(struct string_list *contents,\n> +\t\t\t\t      struct strbuf *dirpath)\n> +{\n> +\tstruct string_list current_contents = STRING_LIST_INIT_DUP;\n> +\tstruct string_list current_subdirs = STRING_LIST_INIT_DUP;\n\nRather than populating those string lists, why not write into the\n`contents` directly, _while_ traversing the directories?\n\nBut if you really, really want to use string lists (hey, I'm a fan, I\nintroduced them to Git's source code, after all), then please make sure\nto release them in the end, too.\n\n> +\tstruct string_list_item *it;\n> +\tstruct strbuf buf = STRBUF_INIT;\n> +\n> +\tlist_contents_of_dir(&current_contents, dirpath, 0, 0);\n> +\tfor_each_string_list_item(it, &current_contents) {\n> +\t\tstrbuf_reset(&buf);\n> +\t\tstrbuf_addbuf(&buf, dirpath);\n> +\t\tstrbuf_complete(&buf, '/');\n> +\t\tstrbuf_addstr(&buf, it->string);\n\nYou don't need a separate `strbuf` for that: just complete `dirpath`\nwith a trailing slash, remember the length, then in each iteration reset\nthe length and append the current subdirectory name.\n\n> +\n> +\t\tstring_list_append(contents, buf.buf);\n> +\t}\n> +\n> +\tlist_contents_of_dir(&current_subdirs, dirpath, 1, DT_DIR);\n> +\tfor_each_string_list_item(it, &current_subdirs) {\n> +\t\tif (strcmp(it->string, \".\") == 0\n> +\t\t    || strcmp(it->string, \"..\") == 0)\n> +\t\t\tcontinue;\n> +\t\tstrbuf_reset(&buf);\n> +\t\tstrbuf_addbuf(&buf, dirpath);\n> +\t\tstrbuf_complete(&buf, '/');\n> +\t\tstrbuf_addstr(&buf, it->string);\n> +\n> +\t\tlist_contents_of_dir_recursively(contents, &buf);\n> +\t}\n> +}\n>\n>  void get_object_counts(struct strbuf *obj_info)\n>  {\n> @@ -240,3 +276,25 @@ void get_packed_object_summary(struct strbuf *obj_info)\n>  \t\tstrbuf_complete_line(obj_info);\n>  \t}\n>  }\n> +\n> +void get_object_info_summary(struct strbuf *obj_info)\n> +{\n> +\t// strbuf += GITDIR/info/:\n> +\t// recursively list contents of $GIT_OBJECT_DIRECTORY/info\n> +\tstruct strbuf dirpath = STRBUF_INIT;\n> +\tstruct string_list contents = STRING_LIST_INIT_DUP;\n> +\tstruct string_list_item *entry;\n> +\n> +\tstrbuf_reset(obj_info);\n> +\n> +\tstrbuf_addstr(&dirpath, get_object_directory());\n> +\tstrbuf_complete(&dirpath, '/');\n> +\tstrbuf_addstr(&dirpath, \"info/\");\n> +\n> +\tlist_contents_of_dir_recursively(&contents, &dirpath);\n> +\n> +\tfor_each_string_list_item(entry, &contents) {\n> +\t\tstrbuf_addstr(obj_info, entry->string);\n> +\t\tstrbuf_complete_line(obj_info);\n> +\t}\n\nAs mentioned earlier, I would rather write things into `obj_info`\ndirectly, without detouring to a string list.\n\nOr, if you decide that you want to write to a `FILE *` anyway, I'd\nrather avoid the detour to the `strbuf` altogether.\n\nCiao,\nDscho\n\n> +}\n> diff --git a/bugreport.h b/bugreport.h\n> index 11ff7df41b..4f5e2d1b9a 100644\n> --- a/bugreport.h\n> +++ b/bugreport.h\n> @@ -30,3 +30,9 @@ void get_loose_object_summary(struct strbuf *obj_info);\n>   * hook_info will be discarded.\n>   */\n>  void get_packed_object_summary(struct strbuf *obj_info);\n> +\n> +/**\n> + * Adds a list of all contents (recursively) of '.git/objects/info'. The\n> + * previous contents of hook_info will be discarded.\n> + */\n> +void get_object_info_summary(struct strbuf *obj_info);\n> diff --git a/builtin/bugreport.c b/builtin/bugreport.c\n> index da91a3944e..8aad33a9b0 100644\n> --- a/builtin/bugreport.c\n> +++ b/builtin/bugreport.c\n> @@ -72,6 +72,13 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix)\n>  \tget_packed_object_summary(&buffer);\n>  \tstrbuf_write(&buffer, report);\n>\n> +\tadd_header(report, \"Object Info Data\");\n> +\tget_object_info_summary(&buffer);\n> +\tstrbuf_write(&buffer, report);\n> +\n> +\t// Close file\n> +\t// open file in editor\n> +\tlaunch_editor(report_path, NULL, NULL);\n>  \tfclose(report);\n>\n>  \tlaunch_editor(report_path.buf, NULL, NULL);\n> --\n> 2.24.0.rc0.303.g954a862665-goog\n>\n>\n"},{"id":"385005","messageId":"nycvar.QRO.7.76.6.1910281655420.46@tvgsbejvaqbjf.bet","threadId":"51645","inReplyTo":"20191025025129.250049-10-emilyshaffer@google.com","subject":"Re: [PATCH v3 9/9] bugreport: print contents of alternates file","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2019-10-28T15:57:01Z","receivedAt":"2019-10-28T15:57:20Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Emily,\n\nOn Thu, 24 Oct 2019, Emily Shaffer wrote:\n\n> In some cases, it could be that the user is having a problem with an\n> object which isn't present in their normal object directory. We can get\n> a hint that that might be the case by examining the list of alternates\n> where their object may be stored instead.\n\nDoesn't this open the possibility of leaking project's (possibly NDA'ed) names?\n\nI could imagine that we might rather want to count the alternates, and\nmaybe separate into those alternates that actually exist and alternates\nthat do not exist (which would produce a warning that the user might\nhave trained themselves to ignore).\n\nCiao,\nDscho\n\n>\n> Signed-off-by: Emily Shaffer <emilyshaffer@google.com>\n> ---\n>  bugreport.c         | 14 ++++++++++++++\n>  bugreport.h         |  6 ++++++\n>  builtin/bugreport.c |  4 ++++\n>  3 files changed, 24 insertions(+)\n>\n> diff --git a/bugreport.c b/bugreport.c\n> index ce15904fec..a7bdc72b7f 100644\n> --- a/bugreport.c\n> +++ b/bugreport.c\n> @@ -298,3 +298,17 @@ void get_object_info_summary(struct strbuf *obj_info)\n>  \t\tstrbuf_complete_line(obj_info);\n>  \t}\n>  }\n> +\n> +void get_alternates_file(struct strbuf *alternates_info)\n> +{\n> +\tstruct strbuf alternates_path = STRBUF_INIT;\n> +\n> +\tstrbuf_addstr(&alternates_path, get_object_directory());\n> +\tstrbuf_complete(&alternates_path, '/');\n> +\tstrbuf_addstr(&alternates_path, \"info/alternates\");\n> +\n> +\tstrbuf_reset(alternates_info);\n> +\tstrbuf_addbuf(alternates_info, &alternates_path);\n> +\tstrbuf_complete_line(alternates_info);\n> +\tstrbuf_read_file(alternates_info, alternates_path.buf, 0);\n> +}\n> diff --git a/bugreport.h b/bugreport.h\n> index 4f5e2d1b9a..74d1f79960 100644\n> --- a/bugreport.h\n> +++ b/bugreport.h\n> @@ -36,3 +36,9 @@ void get_packed_object_summary(struct strbuf *obj_info);\n>   * previous contents of hook_info will be discarded.\n>   */\n>  void get_object_info_summary(struct strbuf *obj_info);\n> +\n> +/**\n> + * Adds the contents of '.git/info/alternates'. The previous contents of\n> + * alternates_info will be discarded.\n> + */\n> +void get_alternates_file(struct strbuf *alt_info);\n> diff --git a/builtin/bugreport.c b/builtin/bugreport.c\n> index 8aad33a9b0..0784bdc42a 100644\n> --- a/builtin/bugreport.c\n> +++ b/builtin/bugreport.c\n> @@ -76,6 +76,10 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix)\n>  \tget_object_info_summary(&buffer);\n>  \tstrbuf_write(&buffer, report);\n>\n> +\tadd_header(report, \"Alternates File\");\n> +\tget_alternates_file(&buffer);\n> +\tstrbuf_write(&buffer, report);\n> +\n>  \t// Close file\n>  \t// open file in editor\n>  \tlaunch_editor(report_path, NULL, NULL);\n> --\n> 2.24.0.rc0.303.g954a862665-goog\n>\n>\n"},{"id":"385034","messageId":"xmqqd0eg8geu.fsf@gitster-ct.c.googlers.com","threadId":"51645","inReplyTo":"20191025025129.250049-1-emilyshaffer@google.com","subject":"Re: [PATCH v3 0/9] add git-bugreport tool","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2019-10-29T01:54:49Z","receivedAt":"2019-10-29T01:54:54Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Emily Shaffer <emilyshaffer@google.com> writes:\n\n> Thanks for the patience with the long wait, all. Here's an attempt at\n> the rewrite in C; I think it does verbatim what the sh version did\n> except that this doesn't print the reflog - Jonathan Nieder was good\n> enough to point out to me that folks probably don't want to share their\n> commit subjects all willy-nilly if they're working on something\n> secretive.\n\nIs the goal to give a tool the end users can type \"git\nbugreport<RET>\" and automatically send the result to this mailing\nlist (or some bug tracker)?  Or is this only about producing a\npre-filled bug report template to be slurped into their MUA and then\nfurther manually edited before sending it out?\n\nIt probably is controversial if we exposed contents of hooks scripts\n(I can imagine some people may trigger external process by pinging a\nURL that includes credential material out of laziness), so the\npresence test you have is probably a good stopping point.  I do not\nknow how much it helps to know which hooks exist in order to\ndiagnose an anomaly without knowing what their contents are, but it\nis a start.\n\nBy the way, I doubt that it is the best use of developer time to\nwrite these in C---taking various pieces of information from\ndifferent places to prepare a text file sounds more suited to\nscripting languages, especially while we are still learning what\nkind of information we want to collect and how we want to present\nthe final result (iow, for the first handful of years after\ndeploying this command).\n\nThanks.\n"},{"id":"385068","messageId":"nycvar.QRO.7.76.6.1910291202160.46@tvgsbejvaqbjf.bet","threadId":"51645","inReplyTo":"xmqqd0eg8geu.fsf@gitster-ct.c.googlers.com","subject":"Re: [PATCH v3 0/9] add git-bugreport tool","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2019-10-29T11:13:27Z","receivedAt":"2019-10-29T11:13:52Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Junio,\n\nOn Tue, 29 Oct 2019, Junio C Hamano wrote:\n\n> Emily Shaffer <emilyshaffer@google.com> writes:\n>\n> > Thanks for the patience with the long wait, all. Here's an attempt at\n> > the rewrite in C; I think it does verbatim what the sh version did\n> > except that this doesn't print the reflog - Jonathan Nieder was good\n> > enough to point out to me that folks probably don't want to share their\n> > commit subjects all willy-nilly if they're working on something\n> > secretive.\n>\n> Is the goal to give a tool the end users can type \"git\n> bugreport<RET>\" and automatically send the result to this mailing\n> list (or some bug tracker)?  Or is this only about producing a\n> pre-filled bug report template to be slurped into their MUA and then\n> further manually edited before sending it out?\n>\n> It probably is controversial if we exposed contents of hooks scripts\n> (I can imagine some people may trigger external process by pinging a\n> URL that includes credential material out of laziness), so the\n> presence test you have is probably a good stopping point.  I do not\n> know how much it helps to know which hooks exist in order to\n> diagnose an anomaly without knowing what their contents are, but it\n> is a start.\n>\n> By the way, I doubt that it is the best use of developer time to\n> write these in C---taking various pieces of information from\n> different places to prepare a text file sounds more suited to\n> scripting languages, especially while we are still learning what\n> kind of information we want to collect and how we want to present\n> the final result (iow, for the first handful of years after\n> deploying this command).\n\nIf you want to limit Git to Linux, then relying on a specific scripting\nlanguage like Unix shell scripting sounds okay.\n\nIf you target more platforms, then a range of scripting languages sounds\na lot more sensible, my top choice would be node.js.\n\nI understand that that is not an option here, and there is a strong\npreference for Unix shell scripting in Git. This is a bit unfortunate:\n\nA good chunk of the bug reports I see in Git for Windows are _about_\nproblems with Unix shell scripting. There are fork problems, problems\nwhere `add_item` failed (which is one of the least helpful error\nmessages I saw in my entire life), there are problems with anti-malware\ncausing segmentation faults in the Unix shell script interpreter (the\nMSYS2 Bash), and there are even (believe it or not) problems with\n_display drivers_. Yes, that is right, display drivers can interfere\nwith Unix shell script execution on Windows. (This is really rare, but\nhey, bug reports are rare in Git for Windows to begin with...)\n\nAll of these problems _prevent_ Unix shell scripts from being\ninterpreted as expected on Windows. Which would ridicule the idea of\nreporting any of these bugs using a tool that is implemented in Unix\nshell.\n\nAnd what is worse than a bug in an application? A bug in the\napplication's very own bug reporting tool.\n\nI pointed this out to Emily, and that is the reason why this amount of\nwork was put in to _improve_ the tool, precisely by _not_ implementing\nit as a Unix shell script, but instead in plain C (which is a _lot_ more\nrobust).\n\nCiao,\nDscho\n"},{"id":"385107","messageId":"20191029202940.GA12243@google.com","threadId":"51645","inReplyTo":"20191025025129.250049-2-emilyshaffer@google.com","subject":"Re: [PATCH v3 1/9] bugreport: add tool to generate debugging info","fromName":"Josh Steadmon","fromEmail":"steadmon@google.com","sentAt":"2019-10-29T20:29:40Z","receivedAt":"2019-10-29T20:29:51Z","isPatch":true,"sender":{"key":"steadmon@google.com","avatar":"https://avatars.githubusercontent.com/u/2654920?v=4"},"body":"On 2019.10.24 19:51, Emily Shaffer wrote:\n> Teach Git how to prompt the user for a good bug report: reproduction\n> steps, expected behavior, and actual behavior. Later, Git can learn how\n> to collect some diagnostic information from the repository.\n> \n> If users can send us a well-written bug report which contains diagnostic\n> information we would otherwise need to ask the user for, we can reduce\n> the number of question-and-answer round trips between the reporter and\n> the Git contributor.\n> \n> Users may also wish to send a report like this to their local \"Git\n> expert\" if they have put their repository into a state they are confused\n> by.\n> \n> Signed-off-by: Emily Shaffer <emilyshaffer@google.com>\n> ---\n>  Makefile            |  1 +\n>  builtin.h           |  1 +\n>  builtin/bugreport.c | 50 +++++++++++++++++++++++++++++++++++++++++++++\n>  git.c               |  1 +\n>  4 files changed, 53 insertions(+)\n>  create mode 100644 builtin/bugreport.c\n> \n> diff --git a/Makefile b/Makefile\n> index 58b92af54b..132e2a52da 100644\n> --- a/Makefile\n> +++ b/Makefile\n> @@ -1039,6 +1039,7 @@ BUILTIN_OBJS += builtin/archive.o\n>  BUILTIN_OBJS += builtin/bisect--helper.o\n>  BUILTIN_OBJS += builtin/blame.o\n>  BUILTIN_OBJS += builtin/branch.o\n> +BUILTIN_OBJS += builtin/bugreport.o\n>  BUILTIN_OBJS += builtin/bundle.o\n>  BUILTIN_OBJS += builtin/cat-file.o\n>  BUILTIN_OBJS += builtin/check-attr.o\n> diff --git a/builtin.h b/builtin.h\n> index 5cf5df69f7..c6373d3289 100644\n> --- a/builtin.h\n> +++ b/builtin.h\n> @@ -135,6 +135,7 @@ int cmd_archive(int argc, const char **argv, const char *prefix);\n>  int cmd_bisect__helper(int argc, const char **argv, const char *prefix);\n>  int cmd_blame(int argc, const char **argv, const char *prefix);\n>  int cmd_branch(int argc, const char **argv, const char *prefix);\n> +int cmd_bugreport(int argc, const char **argv, const char *prefix);\n>  int cmd_bundle(int argc, const char **argv, const char *prefix);\n>  int cmd_cat_file(int argc, const char **argv, const char *prefix);\n>  int cmd_checkout(int argc, const char **argv, const char *prefix);\n> diff --git a/builtin/bugreport.c b/builtin/bugreport.c\n> new file mode 100644\n> index 0000000000..2ef16440a0\n> --- /dev/null\n> +++ b/builtin/bugreport.c\n> @@ -0,0 +1,50 @@\n> +#include \"builtin.h\"\n> +#include \"stdio.h\"\n> +#include \"strbuf.h\"\n> +#include \"time.h\"\n> +\n> +int get_bug_template(struct strbuf *template)\n\nCompilation fails here for me with:\n  builtin/bugreport.c:6:5: error: no previous prototype\n      for ‘get_bug_template’ [-Werror=missing-prototypes]\n\nCan you make this function static?\n\n\n> +{\n> +\tconst char template_text[] =\n> +\"Thank you for filling out a Git bug report!\\n\"\n> +\"Please answer the following questions to help us understand your issue.\\n\"\n> +\"\\n\"\n> +\"What did you do before the bug happened? (Steps to reproduce your issue)\\n\"\n> +\"\\n\"\n> +\"What did you expect to happen? (Expected behavior)\\n\"\n> +\"\\n\"\n> +\"What happened instead? (Actual behavior)\\n\"\n> +\"\\n\"\n> +\"What's different between what you expected and what actually happened?\\n\"\n> +\"\\n\"\n> +\"Anything else you want to add:\\n\"\n> +\"\\n\"\n> +\"Please review the rest of the bug report below.\\n\"\n> +\"You can delete any lines you don't wish to send.\\n\";\n> +\n> +\tstrbuf_reset(template);\n> +\tstrbuf_add(template, template_text, strlen(template_text));\n> +\treturn 0;\n> +}\n> +\n> +int cmd_bugreport(int argc, const char **argv, const char *prefix)\n> +{\n> +\tstruct strbuf buffer = STRBUF_INIT;\n> +\tstruct strbuf report_path = STRBUF_INIT;\n> +\tFILE *report;\n> +\ttime_t now = time(NULL);\n> +\n> +\tstrbuf_addstr(&report_path, \"git-bugreport-\");\n> +\tstrbuf_addftime(&report_path, \"%F\", gmtime(&now), 0, 0);\n> +\tstrbuf_addstr(&report_path, \".txt\");\n> +\n> +\treport = fopen_for_writing(report_path.buf);\n> +\n> +\tget_bug_template(&buffer);\n> +\tstrbuf_write(&buffer, report);\n> +\n> +\tfclose(report);\n> +\n> +\tlaunch_editor(report_path.buf, NULL, NULL);\n> +\treturn 0;\n> +}\n> diff --git a/git.c b/git.c\n> index ce6ab0ece2..2d6a64f019 100644\n> --- a/git.c\n> +++ b/git.c\n> @@ -473,6 +473,7 @@ static struct cmd_struct commands[] = {\n>  \t{ \"bisect--helper\", cmd_bisect__helper, RUN_SETUP },\n>  \t{ \"blame\", cmd_blame, RUN_SETUP },\n>  \t{ \"branch\", cmd_branch, RUN_SETUP | DELAY_PAGER_CONFIG },\n> +\t{ \"bugreport\", cmd_bugreport, RUN_SETUP },\n>  \t{ \"bundle\", cmd_bundle, RUN_SETUP_GENTLY | NO_PARSEOPT },\n>  \t{ \"cat-file\", cmd_cat_file, RUN_SETUP },\n>  \t{ \"check-attr\", cmd_check_attr, RUN_SETUP },\n> -- \n> 2.24.0.rc0.303.g954a862665-goog\n\nCan you also add /git-bugreport to .gitignore?\n\n"},{"id":"385113","messageId":"20191029204355.GB12243@google.com","threadId":"51645","inReplyTo":"20191025025129.250049-4-emilyshaffer@google.com","subject":"Re: [PATCH v3 3/9] bugreport: add version and system information","fromName":"Josh Steadmon","fromEmail":"steadmon@google.com","sentAt":"2019-10-29T20:43:55Z","receivedAt":"2019-10-29T20:44:04Z","isPatch":true,"sender":{"key":"steadmon@google.com","avatar":"https://avatars.githubusercontent.com/u/2654920?v=4"},"body":"On 2019.10.24 19:51, Emily Shaffer wrote:\n[snip]\n> diff --git a/builtin/bugreport.c b/builtin/bugreport.c\n> index 2ef16440a0..7232d31be7 100644\n> --- a/builtin/bugreport.c\n> +++ b/builtin/bugreport.c\n> @@ -1,4 +1,5 @@\n>  #include \"builtin.h\"\n> +#include \"bugreport.h\"\n>  #include \"stdio.h\"\n>  #include \"strbuf.h\"\n>  #include \"time.h\"\n> @@ -27,6 +28,13 @@ int get_bug_template(struct strbuf *template)\n>  \treturn 0;\n>  }\n>  \n> +void add_header(FILE *report, const char *title)\n\nI get the same compilation error here, can you make add_header() static\nas well?\n"},{"id":"385114","messageId":"20191029205823.GC12243@google.com","threadId":"51645","inReplyTo":"20191025025129.250049-5-emilyshaffer@google.com","subject":"Re: [PATCH v3 4/9] bugreport: add config values from whitelist","fromName":"Josh Steadmon","fromEmail":"steadmon@google.com","sentAt":"2019-10-29T20:58:23Z","receivedAt":"2019-10-29T20:58:31Z","isPatch":true,"sender":{"key":"steadmon@google.com","avatar":"https://avatars.githubusercontent.com/u/2654920?v=4"},"body":"On 2019.10.24 19:51, Emily Shaffer wrote:\n[snip]\n> diff --git a/bugreport.c b/bugreport.c\n> index ada54fe583..afa4836ab1 100644\n> --- a/bugreport.c\n> +++ b/bugreport.c\n> @@ -1,10 +1,24 @@\n>  #include \"cache.h\"\n>  \n>  #include \"bugreport.h\"\n> +#include \"config.h\"\n> +#include \"exec-cmd.h\"\n>  #include \"help.h\"\n>  #include \"run-command.h\"\n>  #include \"version.h\"\n>  \n> +/**\n> + * A sorted list of config options which we will add to the bugreport. Managed\n> + * by 'gather_whitelist(...)'.\n> + */\n> +struct string_list whitelist = STRING_LIST_INIT_DUP;\n> +struct strbuf configs_and_values = STRBUF_INIT;\n> +\n> +// git version --build-options\n> +// uname -a\n> +// curl-config --version\n> +// ldd --version\n> +// echo $SHELL\n>  void get_system_info(struct strbuf *sys_info)\n>  {\n>  \tstruct child_process cp = CHILD_PROCESS_INIT;\n> @@ -53,3 +67,39 @@ void get_system_info(struct strbuf *sys_info)\n>  \targv_array_clear(&cp.args);\n>  \tstrbuf_reset(&std_out);\n>  }\n> +\n> +void gather_whitelist(struct strbuf *path)\n\nThis and git_config_bugreport() below should both be static as well.\nRather than repeating advice on the later patches, I'll just note that\nany new functions that don't show up in the corresponding .h file should\nbe marked static.\n"},{"id":"385115","messageId":"20191029211853.GD12243@google.com","threadId":"51645","inReplyTo":"20191025025129.250049-7-emilyshaffer@google.com","subject":"Re: [PATCH v3 6/9] bugreport: count loose objects","fromName":"Josh Steadmon","fromEmail":"steadmon@google.com","sentAt":"2019-10-29T21:18:53Z","receivedAt":"2019-10-29T21:19:00Z","isPatch":true,"sender":{"key":"steadmon@google.com","avatar":"https://avatars.githubusercontent.com/u/2654920?v=4"},"body":"On 2019.10.24 19:51, Emily Shaffer wrote:\n[snip]\n> diff --git a/bugreport.h b/bugreport.h\n> index 942a5436e3..09ad0c2599 100644\n> --- a/bugreport.h\n> +++ b/bugreport.h\n> @@ -18,3 +18,9 @@ void get_whitelisted_config(struct strbuf *sys_info);\n>   * contents of hook_info will be discarded.\n>   */\n>  void get_populated_hooks(struct strbuf *hook_info);\n> +\n> +/**\n> + * Adds the output of `git count-object -vH`. The previous contents of hook_info\n> + * will be discarded.\n> + */\n> +void get_loose_object_summary(struct strbuf *obj_info);\n\nLooks like a copy/paste typo here, shouldn't \"hook_info\" be \"obj_info\"\nin the comment?\n"},{"id":"385126","messageId":"xmqqwocn3tfa.fsf@gitster-ct.c.googlers.com","threadId":"51645","inReplyTo":"20191029205823.GC12243@google.com","subject":"Re: [PATCH v3 4/9] bugreport: add config values from whitelist","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2019-10-30T01:37:13Z","receivedAt":"2019-10-30T01:37:22Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Josh Steadmon <steadmon@google.com> writes:\n\n> This and git_config_bugreport() below should both be static as well.\n> Rather than repeating advice on the later patches, I'll just note that\n> any new functions that don't show up in the corresponding .h file should\n> be marked static.\n\nGood advice.  \n\nMore importantly, given that \"git bugreport\" itself has no service\nof itself to offer other existing parts of Git (it is just a \"gather\nvarious pieces of information from different places, and then\nproduce a text file output\" application), I do not see much point in\nit having its own header file that others would #include (i.e. the\ninclude file is to define services that are offered by it).  If\nthere are common enough service routines invented to support the\nneed of bugreport.c (e.g. perhaps it wants to give more info than\nwhat is currently available via the existing API on the contents of\nin-core index), by definition of being them common enough, they\nshould be added to the header that can be used by both bugreport.c\nand other existing users of the same subsystem (e.g. if it is about\nin-core index, perhaps cache.h).\n\nIt makes perfect sense for bugreport.c to #include header files for\nthe Git internals to collect pieces of information from inside Git,\nthough.\n\n"},{"id":"385816","messageId":"20191108214757.GB22855@google.com","threadId":"51645","inReplyTo":"nycvar.QRO.7.76.6.1910281432180.46@tvgsbejvaqbjf.bet","subject":"Re: [PATCH v3 3/9] bugreport: add version and system information","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2019-11-08T21:48:37Z","receivedAt":"2019-11-08T21:48:46Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"On Mon, Oct 28, 2019 at 02:49:29PM +0100, Johannes Schindelin wrote:\n> Hi Emily,\n> \n> On Thu, 24 Oct 2019, Emily Shaffer wrote:\n> \n> > diff --git a/bugreport.c b/bugreport.c\n> > new file mode 100644\n> > index 0000000000..ada54fe583\n> > --- /dev/null\n> > +++ b/bugreport.c\n> > @@ -0,0 +1,55 @@\n> > +#include \"cache.h\"\n> > +\n> > +#include \"bugreport.h\"\n> > +#include \"help.h\"\n> > +#include \"run-command.h\"\n> > +#include \"version.h\"\n> > +\n> > +void get_system_info(struct strbuf *sys_info)\n> > +{\n> > +\tstruct child_process cp = CHILD_PROCESS_INIT;\n> > +\tstruct strbuf std_out = STRBUF_INIT;\n> > +\n> > +\tstrbuf_reset(sys_info);\n> > +\n> > +\t// get git version from native cmd\n> \n> Please use C-style comments instead of C++ ones.\n> \n> > +\tstrbuf_addstr(sys_info, \"git version: \");\n> > +\tstrbuf_addstr(sys_info, git_version_string);\n> > +\tstrbuf_complete_line(sys_info);\n> > +\n> > +\t// system call for other version info\n> > +\targv_array_push(&cp.args, \"uname\");\n> > +\targv_array_push(&cp.args, \"-a\");\n> > +\tcapture_command(&cp, &std_out, 0);\n> \n> Mmmkay. I am not too much of a fan of relying on the `uname` executable,\n> as it can very well be a 32-bit `uname.exe` on Windows, which obviously\n> would _not_ report the architecture of the machine, but something\n> misleading.\n> \n> Why not use the `uname()` function (that we even define in\n> `compat/mingw.c`)?\n\nVery glad to have your review and point this kind of thing out. :) It\nsimplifies the code, too. I wasn't sure about these system calls, so I\nappreciate you suggesting alternatives.\n\n> \n> Also, why not include the same information as `git version\n> --build-options`, most importantly `GIT_HOST_CPU`?\n> \n> In any case, if you are capturing the output of `uname -a`, you\n> definitely will want to pass the `RUN_COMMAND_STDOUT_TO_STDERR` flag (in\n> case, say, the MSYS2 `uname.exe` fails with those dreaded Cygwin error\n> messages like \"*** fatal error - add_item\n> (\"\\??\\D:\\git\\installation\\Git\", \"/\", ...) failed, errno 1\").\n> \n> > +\n> > +\tstrbuf_addstr(sys_info, \"uname -a: \");\n> > +\tstrbuf_addbuf(sys_info, &std_out);\n> > +\tstrbuf_complete_line(sys_info);\n> > +\n> > +\targv_array_clear(&cp.args);\n> > +\tstrbuf_reset(&std_out);\n> > +\n> > +\n> > +\targv_array_push(&cp.args, \"curl-config\");\n> > +\targv_array_push(&cp.args, \"--version\");\n> > +\tcapture_command(&cp, &std_out, 0);\n> \n> This will be almost certainly be incorrect, as most _regular_ users\n> won't have `curl-config`. I know, it is easy to forget that most Git\n> users are no hard-core C developers ;-)\n\nHeresy! :)\n\n> \n> A better alternative would be to use `curl_version()`, guarded, of\n> course, by `#ifndef NO_CURL`...\n> \n> > +\n> > +\tstrbuf_addstr(sys_info, \"curl-config --version: \");\n> > +\tstrbuf_addbuf(sys_info, &std_out);\n> > +\tstrbuf_complete_line(sys_info);\n> > +\n> > +\targv_array_clear(&cp.args);\n> > +\tstrbuf_reset(&std_out);\n> > +\n> > +\n> > +\targv_array_push(&cp.args, \"ldd\");\n> > +\targv_array_push(&cp.args, \"--version\");\n> > +\tcapture_command(&cp, &std_out, 0);\n> \n> Again, this command will only be present in few setups. I am not\n> actually sure that the output of this is interesting to begin with.\n\nIt was a suggestion, I believe, from Jonathan Nieder.\n\n> \n> What I _do_ think is that a much more interesting piece of information\n> would be the exact GLIBC version, the OS name and the OS version.\n\nThe glibc version is easy; I've done that. It certainly looks nicer than\nthe ldd call.\n\nI guess I may be missing something, because as I start to look into how\nto the OS info, I fall down a hole of many, many preprocessor defines to\ncheck. If that's the approach you want me to take, just say the word,\nbut it will be ugly :) I suppose I had hoped the uname info would give us\na close enough idea that full OS info isn't necessary.\n\n> \n> > +\n> > +\tstrbuf_addstr(sys_info, \"ldd --version: \");\n> > +\tstrbuf_addbuf(sys_info, &std_out);\n> > +\tstrbuf_complete_line(sys_info);\n> > +\n> > +\targv_array_clear(&cp.args);\n> > +\tstrbuf_reset(&std_out);\n> > +}\n> > diff --git a/bugreport.h b/bugreport.h\n> > new file mode 100644\n> > index 0000000000..ba216acf3f\n> > --- /dev/null\n> > +++ b/bugreport.h\n> > @@ -0,0 +1,7 @@\n> > +#include \"strbuf.h\"\n> > +\n> > +/**\n> > + * Adds the Git version, `uname -a`, and `curl-config --version` to sys_info.\n> > + * The previous contents of sys_info will be discarded.\n> > + */\n> > +void get_system_info(struct strbuf *sys_info);\n> > diff --git a/builtin/bugreport.c b/builtin/bugreport.c\n> > index 2ef16440a0..7232d31be7 100644\n> > --- a/builtin/bugreport.c\n> > +++ b/builtin/bugreport.c\n> > @@ -1,4 +1,5 @@\n> >  #include \"builtin.h\"\n> > +#include \"bugreport.h\"\n> >  #include \"stdio.h\"\n> >  #include \"strbuf.h\"\n> >  #include \"time.h\"\n> > @@ -27,6 +28,13 @@ int get_bug_template(struct strbuf *template)\n> >  \treturn 0;\n> >  }\n> >\n> > +void add_header(FILE *report, const char *title)\n> > +{\n> > +\tstruct strbuf buffer = STRBUF_INIT;\n> > +\tstrbuf_addf(&buffer, \"\\n\\n[%s]\\n\", title);\n> > +\tstrbuf_write(&buffer, report);\n> \n> This leaks `buffer`. Why not write into `report` via `fprintf()`\n> directly?\n\nRather, to match the style of the rest of the builtin, modified\nget_header to add the header to a passed-in strbuf instead of\nmodifying the file directly.\n\n> \n> Ciao,\n> Dscho\n> \n> > +}\n> > +\n> >  int cmd_bugreport(int argc, const char **argv, const char *prefix)\n> >  {\n> >  \tstruct strbuf buffer = STRBUF_INIT;\n> > @@ -43,6 +51,11 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix)\n> >  \tget_bug_template(&buffer);\n> >  \tstrbuf_write(&buffer, report);\n> >\n> > +\t// add other contents\n> > +\tadd_header(report, \"System Info\");\n> > +\tget_system_info(&buffer);\n> > +\tstrbuf_write(&buffer, report);\n> > +\n> >  \tfclose(report);\n> >\n> >  \tlaunch_editor(report_path.buf, NULL, NULL);\n> > --\n> > 2.24.0.rc0.303.g954a862665-goog\n> >\n> >\n\n\nBased on Dscho's comments, each piece of system info is gathered\ndifferently enough that I will do each in an independent commit now.\n"},{"id":"385905","messageId":"nycvar.QRO.7.76.6.1911111439510.46@tvgsbejvaqbjf.bet","threadId":"51645","inReplyTo":"20191108214757.GB22855@google.com","subject":"Re: [PATCH v3 3/9] bugreport: add version and system information","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2019-11-11T13:48:13Z","receivedAt":"2019-11-11T13:48:37Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Emily,\n\nOn Fri, 8 Nov 2019, Emily Shaffer wrote:\n\n> On Mon, Oct 28, 2019 at 02:49:29PM +0100, Johannes Schindelin wrote:\n>\n> > On Thu, 24 Oct 2019, Emily Shaffer wrote:\n> >\n> > > diff --git a/bugreport.c b/bugreport.c\n> > > new file mode 100644\n> > > index 0000000000..ada54fe583\n> > > --- /dev/null\n> > > +++ b/bugreport.c\n> > > [...]\n> > > +\tstrbuf_addstr(sys_info, \"curl-config --version: \");\n> > > +\tstrbuf_addbuf(sys_info, &std_out);\n> > > +\tstrbuf_complete_line(sys_info);\n> > > +\n> > > +\targv_array_clear(&cp.args);\n> > > +\tstrbuf_reset(&std_out);\n> > > +\n> > > +\n> > > +\targv_array_push(&cp.args, \"ldd\");\n> > > +\targv_array_push(&cp.args, \"--version\");\n> > > +\tcapture_command(&cp, &std_out, 0);\n> >\n> > Again, this command will only be present in few setups. I am not\n> > actually sure that the output of this is interesting to begin with.\n>\n> It was a suggestion, I believe, from Jonathan Nieder.\n\nYes, I guess Jonathan builds their Git locally, too.\n\nIt _is_ easy to forget that most users find this too involved to even\ntry.\n\nNothing like reading through a bug tracker quite frequently to learn\nabout the actual troubles actual users have :-)\n\n> > What I _do_ think is that a much more interesting piece of information\n> > would be the exact GLIBC version, the OS name and the OS version.\n>\n> The glibc version is easy; I've done that. It certainly looks nicer than\n> the ldd call.\n>\n> I guess I may be missing something, because as I start to look into how\n> to the OS info, I fall down a hole of many, many preprocessor defines to\n> check. If that's the approach you want me to take, just say the word,\n> but it will be ugly :) I suppose I had hoped the uname info would give us\n> a close enough idea that full OS info isn't necessary.\n\nWe could go down the pre-processor route, but that would give us the OS\nname and version at build time, not at run time. I think we will be\nmostly interested in the latter, though.\n\nWe might need to enhance our `uname()` emulation in `compat/mingw.c`,\nbut I think we already have enough information there.\n\nWhen it comes to glibc, I think `gnu_get_libc_version()` would get us\nwhat we want. A trickier thing might be to determine whether we're\nactually linking against glibc; I do not want to break musl builds\nagain, I already did that inadvertently when requiring `REG_STARTEND`\nback in the days.\n\n> > > diff --git a/builtin/bugreport.c b/builtin/bugreport.c\n> > > index 2ef16440a0..7232d31be7 100644\n> > > --- a/builtin/bugreport.c\n> > > +++ b/builtin/bugreport.c\n> > > @@ -1,4 +1,5 @@\n> > >  #include \"builtin.h\"\n> > > +#include \"bugreport.h\"\n> > >  #include \"stdio.h\"\n> > >  #include \"strbuf.h\"\n> > >  #include \"time.h\"\n> > > @@ -27,6 +28,13 @@ int get_bug_template(struct strbuf *template)\n> > >  \treturn 0;\n> > >  }\n> > >\n> > > +void add_header(FILE *report, const char *title)\n> > > +{\n> > > +\tstruct strbuf buffer = STRBUF_INIT;\n> > > +\tstrbuf_addf(&buffer, \"\\n\\n[%s]\\n\", title);\n> > > +\tstrbuf_write(&buffer, report);\n> >\n> > This leaks `buffer`. Why not write into `report` via `fprintf()`\n> > directly?\n>\n> Rather, to match the style of the rest of the builtin, modified\n> get_header to add the header to a passed-in strbuf instead of\n> modifying the file directly.\n\nHmm. It makes the code less elegant in my opinion. I would rather either\nrender the entire bug report into a single `strbuf` and then write it\nvia `write_in_full()`, or use `fprintf()` directly.\n\nCiao,\nDscho\n"},{"id":"386212","messageId":"20191114214230.GA36377@google.com","threadId":"51645","inReplyTo":"nycvar.QRO.7.76.6.1911111439510.46@tvgsbejvaqbjf.bet","subject":"Re: [PATCH v3 3/9] bugreport: add version and system information","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2019-11-14T21:42:30Z","receivedAt":"2019-11-14T21:42:40Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"On Mon, Nov 11, 2019 at 02:48:13PM +0100, Johannes Schindelin wrote:\n> > > > diff --git a/builtin/bugreport.c b/builtin/bugreport.c\n> > > > index 2ef16440a0..7232d31be7 100644\n> > > > --- a/builtin/bugreport.c\n> > > > +++ b/builtin/bugreport.c\n> > > > @@ -1,4 +1,5 @@\n> > > >  #include \"builtin.h\"\n> > > > +#include \"bugreport.h\"\n> > > >  #include \"stdio.h\"\n> > > >  #include \"strbuf.h\"\n> > > >  #include \"time.h\"\n> > > > @@ -27,6 +28,13 @@ int get_bug_template(struct strbuf *template)\n> > > >  \treturn 0;\n> > > >  }\n> > > >\n> > > > +void add_header(FILE *report, const char *title)\n> > > > +{\n> > > > +\tstruct strbuf buffer = STRBUF_INIT;\n> > > > +\tstrbuf_addf(&buffer, \"\\n\\n[%s]\\n\", title);\n> > > > +\tstrbuf_write(&buffer, report);\n> > >\n> > > This leaks `buffer`. Why not write into `report` via `fprintf()`\n> > > directly?\n> >\n> > Rather, to match the style of the rest of the builtin, modified\n> > get_header to add the header to a passed-in strbuf instead of\n> > modifying the file directly.\n> \n> Hmm. It makes the code less elegant in my opinion. I would rather either\n> render the entire bug report into a single `strbuf` and then write it\n> via `write_in_full()`, or use `fprintf()` directly.\n\nYeah, that sounds good. I will do that. :)\n"},{"id":"386213","messageId":"20191114215548.GC38770@google.com","threadId":"51645","inReplyTo":"xmqqwocn3tfa.fsf@gitster-ct.c.googlers.com","subject":"Re: [PATCH v3 4/9] bugreport: add config values from whitelist","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2019-11-14T21:55:48Z","receivedAt":"2019-11-14T21:55:56Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"On Wed, Oct 30, 2019 at 10:37:13AM +0900, Junio C Hamano wrote:\n> Josh Steadmon <steadmon@google.com> writes:\n> \n> > This and git_config_bugreport() below should both be static as well.\n> > Rather than repeating advice on the later patches, I'll just note that\n> > any new functions that don't show up in the corresponding .h file should\n> > be marked static.\n> \n> Good advice.  \n> \n> More importantly, given that \"git bugreport\" itself has no service\n> of itself to offer other existing parts of Git (it is just a \"gather\n> various pieces of information from different places, and then\n> produce a text file output\" application), I do not see much point in\n> it having its own header file that others would #include (i.e. the\n> include file is to define services that are offered by it).  If\n> there are common enough service routines invented to support the\n> need of bugreport.c (e.g. perhaps it wants to give more info than\n> what is currently available via the existing API on the contents of\n> in-core index), by definition of being them common enough, they\n> should be added to the header that can be used by both bugreport.c\n> and other existing users of the same subsystem (e.g. if it is about\n> in-core index, perhaps cache.h).\n\nAre you asking me to have only builtin/bugreport.c and implement each\nfunction there, and eliminate both <basedir>/bugreport.[ch]?\n\nIt may be true that deciding to put this functionality into a library\n\"in case someone needs it later\" was premature, so I can do that - I\njust want to make sure I understand you.\n\n - Emily\n"},{"id":"386359","messageId":"xmqqr2284is5.fsf@gitster-ct.c.googlers.com","threadId":"51645","inReplyTo":"20191025025129.250049-2-emilyshaffer@google.com","subject":"Re: [PATCH v3 1/9] bugreport: add tool to generate debugging info","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2019-11-16T03:11:38Z","receivedAt":"2019-11-16T03:11:43Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Emily Shaffer <emilyshaffer@google.com> writes:\n\n> Teach Git how to prompt the user for a good bug report: reproduction\n> steps, expected behavior, and actual behavior. Later, Git can learn how\n> to collect some diagnostic information from the repository.\n\nIt makes sense, but I do not think of any good reason why this\nshould be implemented as a builtin.  I'd expect it would probably\nneed to collect more info on the running environment than otherwise\nnecessary for the regular Git operation, and perhaps you'd want to\neven link with libraries that are not needed for the regular Git\noperation to achieve that.\n\nCan you make it a standalone binary instead to avoid bloat of the\nmain \"git\" binary?\n\nThanks.\n\n\n"},{"id":"386550","messageId":"20191119202542.GB36377@google.com","threadId":"51645","inReplyTo":"xmqqr2284is5.fsf@gitster-ct.c.googlers.com","subject":"Re: [PATCH v3 1/9] bugreport: add tool to generate debugging info","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2019-11-19T20:25:42Z","receivedAt":"2019-11-19T20:25:50Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"On Sat, Nov 16, 2019 at 12:11:38PM +0900, Junio C Hamano wrote:\n> Emily Shaffer <emilyshaffer@google.com> writes:\n> \n> > Teach Git how to prompt the user for a good bug report: reproduction\n> > steps, expected behavior, and actual behavior. Later, Git can learn how\n> > to collect some diagnostic information from the repository.\n> \n> It makes sense, but I do not think of any good reason why this\n> should be implemented as a builtin.  I'd expect it would probably\n> need to collect more info on the running environment than otherwise\n> necessary for the regular Git operation, and perhaps you'd want to\n> even link with libraries that are not needed for the regular Git\n> operation to achieve that.\n> \n> Can you make it a standalone binary instead to avoid bloat of the\n> main \"git\" binary?\n\nSure. This would fix some other issues (needing to link against curl to\nget the curl version, which is exactly what you implied). I wasn't\ncertain which circumstances a standalone binary was preferred, but I\nagree with your reasoning here for sure.\n\n - Emily\n"},{"id":"386551","messageId":"20191119204013.GC36377@google.com","threadId":"51645","inReplyTo":"nycvar.QRO.7.76.6.1910281655420.46@tvgsbejvaqbjf.bet","subject":"Re: [PATCH v3 9/9] bugreport: print contents of alternates file","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2019-11-19T20:40:13Z","receivedAt":"2019-11-19T20:40:21Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"On Mon, Oct 28, 2019 at 04:57:01PM +0100, Johannes Schindelin wrote:\n> Hi Emily,\n> \n> On Thu, 24 Oct 2019, Emily Shaffer wrote:\n> \n> > In some cases, it could be that the user is having a problem with an\n> > object which isn't present in their normal object directory. We can get\n> > a hint that that might be the case by examining the list of alternates\n> > where their object may be stored instead.\n> \n> Doesn't this open the possibility of leaking project's (possibly NDA'ed) names?\n> \n> I could imagine that we might rather want to count the alternates, and\n> maybe separate into those alternates that actually exist and alternates\n> that do not exist (which would produce a warning that the user might\n> have trained themselves to ignore).\n\nSounds reasonable. Will do.\n\n - Emily\n"},{"id":"386565","messageId":"nycvar.QRO.7.76.6.1911200018470.15956@tvgsbejvaqbjf.bet","threadId":"51645","inReplyTo":"20191119202542.GB36377@google.com","subject":"Re: [PATCH v3 1/9] bugreport: add tool to generate debugging info","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2019-11-19T23:24:44Z","receivedAt":"2019-11-19T23:25:10Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Tue, 19 Nov 2019, Emily Shaffer wrote:\n\n> On Sat, Nov 16, 2019 at 12:11:38PM +0900, Junio C Hamano wrote:\n> > Emily Shaffer <emilyshaffer@google.com> writes:\n> >\n> > > Teach Git how to prompt the user for a good bug report: reproduction\n> > > steps, expected behavior, and actual behavior. Later, Git can learn how\n> > > to collect some diagnostic information from the repository.\n> >\n> > It makes sense, but I do not think of any good reason why this\n> > should be implemented as a builtin.  I'd expect it would probably\n> > need to collect more info on the running environment than otherwise\n> > necessary for the regular Git operation, and perhaps you'd want to\n> > even link with libraries that are not needed for the regular Git\n> > operation to achieve that.\n> >\n> > Can you make it a standalone binary instead to avoid bloat of the\n> > main \"git\" binary?\n>\n> Sure. This would fix some other issues (needing to link against curl to\n> get the curl version, which is exactly what you implied). I wasn't\n> certain which circumstances a standalone binary was preferred, but I\n> agree with your reasoning here for sure.\n\nFWIW I disagree with the idea that a tiny built-in command like\n`bugreport` would \"bloat\" the main `git` binary.\n\nIn contrast, I think that stand-alone commands _do_ bloat. Look here:\n\n$ ls -lh git-daemon.exe git-credential-store.exe\n-rwxr-xr-x 1 me 4096 1.8M Nov  6 13:43 git-credential-store.exe*\n-rwxr-xr-x 1 me 4096 1.8M Nov  6 13:43 git-daemon.exe*\n\nIn other words, even a super simple stand-alone like `credential-store`\n(the `credential-store.c` file has only 198 lines!) weighs in with almost\ntwo megabytes.\n\nSo I fear that the claim that a stand-alone command would add less bloat\nthan a built-in one, especially for a relatively small thing like\n`bugreport` has it exactly backwards.\n\nCiao,\nDscho\n"},{"id":"386566","messageId":"nycvar.QRO.7.76.6.1911200027460.15956@tvgsbejvaqbjf.bet","threadId":"51645","inReplyTo":"20191119202542.GB36377@google.com","subject":"Re: [PATCH v3 1/9] bugreport: add tool to generate debugging info","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2019-11-19T23:31:42Z","receivedAt":"2019-11-19T23:32:07Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Emily,\n\nOn Tue, 19 Nov 2019, Emily Shaffer wrote:\n\n> [...] some other issues (needing to link against curl to get the curl\n> version, which is exactly what you implied) [...]\n\nI did suggest on IRC to teach `git-remote-https` an option where it prints\nthe cURL version (and build options) and exits.\n\nThis would have the further advantage of making sure that the correct\ninformation is included. If you have two separate binaries that both link\nto cURL, they could still be linked statically, to different versions of\ncURL (this could happen e.g. if you have a `git-remote-https` from a\ndifferent build in your path).\n\nIn short: I still think that it would make for a much better idea to query\nthe `git-remote-https` executable for the version information than to link\n`bugreport` to libcurl.\n\nCiao,\nDscho\n"},{"id":"386579","messageId":"xmqq8sobfkvl.fsf@gitster-ct.c.googlers.com","threadId":"51645","inReplyTo":"20191119202542.GB36377@google.com","subject":"Re: [PATCH v3 1/9] bugreport: add tool to generate debugging info","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2019-11-20T00:32:14Z","receivedAt":"2019-11-20T00:32:21Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Emily Shaffer <emilyshaffer@google.com> writes:\n\n>> Can you make it a standalone binary instead to avoid bloat of the\n>> main \"git\" binary?\n>\n> Sure. This would fix some other issues (needing to link against curl to\n> get the curl version, which is exactly what you implied).\n\nHmph, I actually was not thinking about the cURL library.  \n\nI imagined that your tool may need to learn more about the system in\norder to make the report and for that there may be libraries that\nmakes it easy than say reading directly from the /proc filesystem\netc. that you may end up using.  Unlike cURL, such a library would\nhave no use in the rest of Git anywhere.\n"},{"id":"386581","messageId":"xmqq4kyzfknc.fsf@gitster-ct.c.googlers.com","threadId":"51645","inReplyTo":"nycvar.QRO.7.76.6.1911200018470.15956@tvgsbejvaqbjf.bet","subject":"Re: [PATCH v3 1/9] bugreport: add tool to generate debugging info","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2019-11-20T00:37:11Z","receivedAt":"2019-11-20T00:37:19Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:\n\n> FWIW I disagree with the idea that a tiny built-in command like\n> `bugreport` would \"bloat\" the main `git` binary.\n>\n> In contrast, I think that stand-alone commands _do_ bloat. Look here:\n\nI probably should have clarified that \"bloat\" in the context of this\ndiscussion is not about on-disk space.  It is about resources needed\nto run \"git status/diff/etc\" that are everyday local commands that\nare expected to be lightweight, i.e. the same criteria applied when\nmaking the networking part (which the user expects to be coffee time)\nseparate from them.\n\n"},{"id":"386582","messageId":"xmqqzhgre5yo.fsf@gitster-ct.c.googlers.com","threadId":"51645","inReplyTo":"nycvar.QRO.7.76.6.1911200027460.15956@tvgsbejvaqbjf.bet","subject":"Re: [PATCH v3 1/9] bugreport: add tool to generate debugging info","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2019-11-20T00:39:43Z","receivedAt":"2019-11-20T00:39:48Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:\n\n> I did suggest on IRC to teach `git-remote-https` an option where it prints\n> the cURL version (and build options) and exits.\n\nI like that.  You ask the exact binary what (it thinks) it uses, so\nthat there won't be skew between the view by \"git remote-http\" and\n\"git bugreport\" on the world.\n"},{"id":"386599","messageId":"20191120020933.GD36377@google.com","threadId":"51645","inReplyTo":"nycvar.QRO.7.76.6.1911200027460.15956@tvgsbejvaqbjf.bet","subject":"Re: [PATCH v3 1/9] bugreport: add tool to generate debugging info","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2019-11-20T02:09:33Z","receivedAt":"2019-11-20T02:09:41Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"On Wed, Nov 20, 2019 at 12:31:42AM +0100, Johannes Schindelin wrote:\n> Hi Emily,\n> \n> On Tue, 19 Nov 2019, Emily Shaffer wrote:\n> \n> > [...] some other issues (needing to link against curl to get the curl\n> > version, which is exactly what you implied) [...]\n> \n> I did suggest on IRC to teach `git-remote-https` an option where it prints\n> the cURL version (and build options) and exits.\n> \n> This would have the further advantage of making sure that the correct\n> information is included. If you have two separate binaries that both link\n> to cURL, they could still be linked statically, to different versions of\n> cURL (this could happen e.g. if you have a `git-remote-https` from a\n> different build in your path).\n\nYeah, it's a good point and an angle I hadn't thought of. Thanks.\n\n> In short: I still think that it would make for a much better idea to query\n> the `git-remote-https` executable for the version information than to link\n> `bugreport` to libcurl.\n\nWill do - the git-bugreport standalone will invoke the git-remote-https\nstandalone to ask for version info.\n\n - Emily\n"},{"id":"386630","messageId":"nycvar.QRO.7.76.6.1911201150260.15956@tvgsbejvaqbjf.bet","threadId":"51645","inReplyTo":"xmqq4kyzfknc.fsf@gitster-ct.c.googlers.com","subject":"Re: [PATCH v3 1/9] bugreport: add tool to generate debugging info","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2019-11-20T10:51:51Z","receivedAt":"2019-11-20T10:52:14Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Junio,\n\nOn Wed, 20 Nov 2019, Junio C Hamano wrote:\n\n> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:\n>\n> > FWIW I disagree with the idea that a tiny built-in command like\n> > `bugreport` would \"bloat\" the main `git` binary.\n> >\n> > In contrast, I think that stand-alone commands _do_ bloat. Look here:\n>\n> I probably should have clarified that \"bloat\" in the context of this\n> discussion is not about on-disk space.  It is about resources needed\n> to run \"git status/diff/etc\" that are everyday local commands that\n> are expected to be lightweight, i.e. the same criteria applied when\n> making the networking part (which the user expects to be coffee time)\n> separate from them.\n\nI guess I still do not understand.\n\nAre you suggesting that `bugreport` would be unwelcome as a built-in? If\nso, I would really like to know why. Because I think it would make for a\nvery fine built-in. I interact with users all the time, and a good tool to\nprovide good information to accompany a bug report is definitely something\nthat would improve the current situation.\n\nCiao,\nDscho\n"},{"id":"387923","messageId":"20191210223457.GA107889@google.com","threadId":"51645","inReplyTo":"nycvar.QRO.7.76.6.1910281540550.46@tvgsbejvaqbjf.bet","subject":"Re: [PATCH v3 6/9] bugreport: count loose objects","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2019-12-10T22:34:57Z","receivedAt":"2019-12-10T22:35:07Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"On Mon, Oct 28, 2019 at 04:07:40PM +0100, Johannes Schindelin wrote:\n> Hi Emily,\n> \n> On Thu, 24 Oct 2019, Emily Shaffer wrote:\n> \n> > The number of unpacked objects in a user's repository may help us\n> > understand the root of the problem they're seeing, especially if a\n> > command is running unusually slowly.\n> >\n> > Rather than directly invoking 'git-count-objects', which may sometimes\n> > fail unexpectedly on Git for Windows, manually count the contents of\n> > .git/objects. Additionally, since we may wish to inspect other\n> > directories' contents for bugreport in the future, put the directory\n> > listing into a helper function.\n> \n> Thank you, much appreciated!\n> \n> I guess the next step is to count the number of packs, and the number of\n> submodules ;-)\n> \n> >\n> > Signed-off-by: Emily Shaffer <emilyshaffer@google.com>\n> > ---\n> >  bugreport.c         | 72 +++++++++++++++++++++++++++++++++++++++++++++\n> >  bugreport.h         |  6 ++++\n> >  builtin/bugreport.c |  4 +++\n> >  3 files changed, 82 insertions(+)\n> >\n> > diff --git a/bugreport.c b/bugreport.c\n> > index 9d7f44ff28..54e1d47103 100644\n> > --- a/bugreport.c\n> > +++ b/bugreport.c\n> > @@ -5,8 +5,11 @@\n> >  #include \"exec-cmd.h\"\n> >  #include \"help.h\"\n> >  #include \"run-command.h\"\n> > +#include \"strbuf.h\"\n> \n> Why not append this to the end of the `#include` list, as is common in\n> Git's commit history?\n> \n> >  #include \"version.h\"\n> >\n> > +#include \"dirent.h\"\n> \n> This header (although with pointy brackets instead of double quotes) is\n> already included in `git-compat-util.h`\n> \n> > +\n> >  /**\n> >   * A sorted list of config options which we will add to the bugreport. Managed\n> >   * by 'gather_whitelist(...)'.\n> > @@ -147,3 +150,72 @@ void get_populated_hooks(struct strbuf *hook_info)\n> >  \t\t}\n> >  \t}\n> >  }\n> > +\n> > +/**\n> > + * Fill 'contents' with the contents of the dir at 'dirpath'.\n> \n> Since you start this comment in JavaDoc style, there should be an almost\n> empty line after this one (\"almost\" because it still contains the\n> asterisk, of course).\n> \n> > + * If 'filter' is nonzero, the contents are filtered on d_type as 'type' - see\n> > + * 'man readdir'. opendir() doesn't take string length as an arg, so don't\n> > + * bother passing it in.\n> > + */\n> > +void list_contents_of_dir(struct string_list *contents, struct strbuf *dirpath,\n> \n> Shouldn't this be `static`?\n> \n> > +\t\t\t  int filter, unsigned char type)\n> > +{\n> > +\tstruct dirent *dir = NULL;\n> > +\tDIR *dh = NULL;\n> > +\n> > +\tdh = opendir(dirpath->buf);\n> > +\twhile (dh && (dir = readdir(dh))) {\n> > +\t\tif (!filter || type == dir->d_type) {\n> > +\t\t\tstring_list_append(contents, dir->d_name);\n> > +\t\t}\n> > +\t}\n> > +}\n> > +\n> > +\n> > +void get_object_counts(struct strbuf *obj_info)\n> \n> Oops. This function is no longer used.\n> \n> > +{\n> > +\tstruct child_process cp = CHILD_PROCESS_INIT;\n> > +\tstruct strbuf std_out = STRBUF_INIT;\n> > +\n> > +\targv_array_push(&cp.args, \"count-objects\");\n> > +\targv_array_push(&cp.args, \"-vH\");\n> > +\tcp.git_cmd = 1;\n> > +\tcapture_command(&cp, &std_out, 0);\n> > +\n> > +\tstrbuf_reset(obj_info);\n> > +\tstrbuf_addstr(obj_info, \"git-count-objects -vH:\\n\");\n> > +\tstrbuf_addbuf(obj_info, &std_out);\n> > +}\n> > +\n> > +void get_loose_object_summary(struct strbuf *obj_info)\n> > +{\n> > +\tstruct strbuf dirpath = STRBUF_INIT;\n> > +\tstruct string_list subdirs = STRING_LIST_INIT_DUP;\n> > +\tstruct string_list_item *subdir;\n> > +\n> > +\tstrbuf_reset(obj_info);\n> > +\n> > +\tstrbuf_addstr(&dirpath, get_object_directory());\n> > +\tstrbuf_complete(&dirpath, '/');\n> > +\n> > +\tlist_contents_of_dir(&subdirs, &dirpath, 1, DT_DIR);\n> > +\n> > +\tfor_each_string_list_item(subdir, &subdirs)\n> > +\t{\n> > +\t\tstruct strbuf subdir_buf = STRBUF_INIT;\n> > +\t\tstruct string_list objects = STRING_LIST_INIT_DUP;\n> > +\n> > +\t\t/*\n> > +\t\t * Only interested in loose objects - so dirs named with the\n> > +\t\t * first byte of the object ID\n> > +\t\t */\n> > +\t\tif (strlen(subdir->string) != 2 || !strcmp(subdir->string, \"..\"))\n> > +\t\t\tcontinue;\n> > +\n> > +\t\tstrbuf_addbuf(&subdir_buf, &dirpath);\n> > +\t\tstrbuf_addstr(&subdir_buf, subdir->string);\n> > +\t\tlist_contents_of_dir(&objects, &subdir_buf, 0, 0);\n> > +\t\tstrbuf_addf(obj_info, \"%s: %d objects\\n\", subdir->string,\n> > +\t\t\t    objects.nr);\n> \n> Hmm. Not only does this leak `objects`, it also throws away the contents\n> that we so painfully constructed.\n> \n> Wouldn't it make more sense to do something like this instead?\n> \n> static int is_hex(const char *string, size_t count)\n> {\n> \tfor (; count; string++, count--)\n> \t\tif (hexval(*string) < 0)\n> \t\t\treturn 0;\n> \treturn 1;\n> }\n\nTrue if the string matches [0-9a-fA-F]*, false otherwise.\n\n> \n> static ssize_t count_loose_objects(struct strbuf *objects_path)\n> {\n> \tssize_t ret = 0;\n> \tsize_t len;\n> \tstruct dirent *d;\n> \tDIR *dir, *subdir;\n> \n> \tdir = opendir(objects_path->buf);\n> \tif (!dir)\n> \t\treturn -1;\n> \n> \tstrbuf_complete(objects_path, '/');\n\nstarting with .git/objects/...\n\n> \tlen = objects_path->len;\n> \twhile ((d = readdir(dir))) {\nFor all contents of dir...\n> \t\tif (d->d_type != DT_DIR)\n> \t\t\tcontinue;\n..which are directories...\n> \t\tstrbuf_setlen(objects_path, len);\n> \t\tstrbuf_addstr(objects_path, d->d_name);\n> \t\tsubdir = opendir(objects_path->buf);\nshow all the contents of that subdirectory.\n> \t\tif (!subdir)\n> \t\t\tcontinue;\n> \t\twhile ((d = readdir(subdir)))\nfor each regular file there,\n> \t\t\tif (d->dt_type == DT_REG &&\nif it's a regular file,\n> \t\t\t    is_hex(dir->d_name, the_repository->hash_algo->hexsz))\nand the directory is named like an object prefix,\n> \t\t\t\tret++;\nincrease the total count of numbers of loose objects.\n> \t\tclosedir(subdir);\n> \t}\n> \tclosedir(dir);\n> \tstrbuf_reset(objects_path, len);\n> \treturn ret;\n> }\n\nThe foremost difference here is that the loose object count previously\nwas not given in total - instead, it was divided up by object prefix. I\ncan't speak to whether that's actually very important to know about, but\nthe original request from stolee was to have the summary by dirname.\nThat's certainly still possible with a light modification to this code.\n\n(Suggestion from Stolee, some months ago earlier in this thread:)\n\n  As mentioned before, I've sometimes found it helpful to know the data shape for the object\n  store. Having a few extra steps such as the following could be nice:\n  \n          echo \"[Loose Objects]\"\n          for objdir in $(find \"$GIT_DIR/objects/??\" -type d)\n          do\n                  echo \"$objdir: $(ls $objdir | wc -l)\"\n          done\n          echo\n  ...\n\nChecking that the directory we're about to inspect is hexval rather than\nthat it's only two characters long is also an interesting point. I'd\nprobably rather do both, though, since I think we both missed\nfutureproofing by a little bit.\n\nDropping the many string_list is fine by me - call it object-oriented\nhabits dying hard.\n\nI worry somewhat on delving into every directory and only afterwards\nchecking whether the directory is one we care about, but that's an easy\nmodification too to your basic suggestion (\"don't use a string list for\nthat, silly\").\n\nThanks. I'll make the changes with the modifications I mentioned and add\nyour Helped-by line on this commit.\n\n - Emily\n\n> \n> Ciao,\n> Dscho\n> \n> > +\t}\n> > +}\n> > diff --git a/bugreport.h b/bugreport.h\n> > index 942a5436e3..09ad0c2599 100644\n> > --- a/bugreport.h\n> > +++ b/bugreport.h\n> > @@ -18,3 +18,9 @@ void get_whitelisted_config(struct strbuf *sys_info);\n> >   * contents of hook_info will be discarded.\n> >   */\n> >  void get_populated_hooks(struct strbuf *hook_info);\n> > +\n> > +/**\n> > + * Adds the output of `git count-object -vH`. The previous contents of hook_info\n> > + * will be discarded.\n> > + */\n> > +void get_loose_object_summary(struct strbuf *obj_info);\n> > diff --git a/builtin/bugreport.c b/builtin/bugreport.c\n> > index a0eefba498..b2ab194207 100644\n> > --- a/builtin/bugreport.c\n> > +++ b/builtin/bugreport.c\n> > @@ -64,6 +64,10 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix)\n> >  \tget_populated_hooks(&buffer);\n> >  \tstrbuf_write(&buffer, report);\n> >\n> > +\tadd_header(report, \"Object Counts\");\n> > +\tget_loose_object_summary(&buffer);\n> > +\tstrbuf_write(&buffer, report);\n> > +\n> >  \tfclose(report);\n> >\n> >  \tlaunch_editor(report_path.buf, NULL, NULL);\n> > --\n> > 2.24.0.rc0.303.g954a862665-goog\n> >\n> >\n"},{"id":"387925","messageId":"20191211002934.GB107889@google.com","threadId":"51645","inReplyTo":"nycvar.QRO.7.76.6.1910281608170.46@tvgsbejvaqbjf.bet","subject":"Re: [PATCH v3 7/9] bugreport: add packed object summary","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2019-12-11T00:29:34Z","receivedAt":"2019-12-11T00:29:41Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"On Mon, Oct 28, 2019 at 04:43:50PM +0100, Johannes Schindelin wrote:\n> Hi Emily,\n> \n> On Thu, 24 Oct 2019, Emily Shaffer wrote:\n> \n> > Alongside the list of loose objects, it's useful to see the list of\n> > object packs as well. It can help us to examine what Git did and did not\n> > pack.\n> \n> Yes, I was write! Packs are next ;-)\n> \n> >\n> > Signed-off-by: Emily Shaffer <emilyshaffer@google.com>\n> > ---\n> >  bugreport.c         | 21 +++++++++++++++++++++\n> >  bugreport.h         |  6 ++++++\n> >  builtin/bugreport.c |  4 ++++\n> >  3 files changed, 31 insertions(+)\n> >\n> > diff --git a/bugreport.c b/bugreport.c\n> > index 54e1d47103..79ddb8baaa 100644\n> > --- a/bugreport.c\n> > +++ b/bugreport.c\n> > @@ -219,3 +219,24 @@ void get_loose_object_summary(struct strbuf *obj_info)\n> >  \t\t\t    objects.nr);\n> >  \t}\n> >  }\n> > +\n> > +void get_packed_object_summary(struct strbuf *obj_info)\n> > +{\n> > +\tstruct strbuf dirpath = STRBUF_INIT;\n> > +\tstruct string_list contents = STRING_LIST_INIT_DUP;\n> > +\tstruct string_list_item *entry;\n> > +\n> > +\tstrbuf_reset(obj_info);\n> > +\n> > +\tstrbuf_addstr(&dirpath, get_object_directory());\n> > +\tstrbuf_complete(&dirpath, '/');\n> > +\tstrbuf_addstr(&dirpath, \"pack/\");\n> > +\tlist_contents_of_dir(&contents, &dirpath, 0, 0);\n> > +\n> > +\t// list full contents of $GIT_OBJECT_DIRECTORY/pack/\n> > +\tfor_each_string_list_item(entry, &contents) {\n> > +\t\tstrbuf_addbuf(obj_info, &dirpath);\n> > +\t\tstrbuf_addstr(obj_info, entry->string);\n> > +\t\tstrbuf_complete_line(obj_info);\n> > +\t}\n> > +}\n> \n> Okay, but I think that you will want to discern between regular `.pack`\n> files, `.idx` files and `tmp_*` files.\n\nDiscern in what way? How would you like to see them treated separately?\nThey're all being listed here, not just counted, so it seems to me like\nI can read the generated bugreport and see which files are index, pack,\nor temporary here.\n\n> \n> > diff --git a/bugreport.h b/bugreport.h\n> > index 09ad0c2599..11ff7df41b 100644\n> > --- a/bugreport.h\n> > +++ b/bugreport.h\n> > @@ -24,3 +24,9 @@ void get_populated_hooks(struct strbuf *hook_info);\n> >   * will be discarded.\n> >   */\n> >  void get_loose_object_summary(struct strbuf *obj_info);\n> > +\n> > +/**\n> > + * Adds a list of the contents of '.git/objects/pack'. The previous contents of\n> > + * hook_info will be discarded.\n> > + */\n> > +void get_packed_object_summary(struct strbuf *obj_info);\n> > diff --git a/builtin/bugreport.c b/builtin/bugreport.c\n> > index b2ab194207..da91a3944e 100644\n> > --- a/builtin/bugreport.c\n> > +++ b/builtin/bugreport.c\n> > @@ -68,6 +68,10 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix)\n> >  \tget_loose_object_summary(&buffer);\n> >  \tstrbuf_write(&buffer, report);\n> >\n> > +\tadd_header(report, \"Packed Object Summary\");\n> > +\tget_packed_object_summary(&buffer);\n> > +\tstrbuf_write(&buffer, report);\n> > +\n> \n> Hmm. At this point, I am unclear whether you want to write into an\n> `strbuf`, or directly into a `FILE *`? I would rather have only one, not\n> a mix.\n> \n> Ciao,\n> Dscho\n> \n> >  \tfclose(report);\n> >\n> >  \tlaunch_editor(report_path.buf, NULL, NULL);\n> > --\n> > 2.24.0.rc0.303.g954a862665-goog\n> >\n> >\n"},{"id":"387940","messageId":"nycvar.QRO.7.76.6.1912111436290.46@tvgsbejvaqbjf.bet","threadId":"51645","inReplyTo":"20191211002934.GB107889@google.com","subject":"Re: [PATCH v3 7/9] bugreport: add packed object summary","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2019-12-11T13:37:10Z","receivedAt":"2019-12-11T13:37:35Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Emily,\n\nOn Tue, 10 Dec 2019, Emily Shaffer wrote:\n\n> On Mon, Oct 28, 2019 at 04:43:50PM +0100, Johannes Schindelin wrote:\n> > Hi Emily,\n> >\n> > On Thu, 24 Oct 2019, Emily Shaffer wrote:\n> >\n> > > Alongside the list of loose objects, it's useful to see the list of\n> > > object packs as well. It can help us to examine what Git did and did not\n> > > pack.\n> >\n> > Yes, I was write! Packs are next ;-)\n> >\n> > >\n> > > Signed-off-by: Emily Shaffer <emilyshaffer@google.com>\n> > > ---\n> > >  bugreport.c         | 21 +++++++++++++++++++++\n> > >  bugreport.h         |  6 ++++++\n> > >  builtin/bugreport.c |  4 ++++\n> > >  3 files changed, 31 insertions(+)\n> > >\n> > > diff --git a/bugreport.c b/bugreport.c\n> > > index 54e1d47103..79ddb8baaa 100644\n> > > --- a/bugreport.c\n> > > +++ b/bugreport.c\n> > > @@ -219,3 +219,24 @@ void get_loose_object_summary(struct strbuf *obj_info)\n> > >  \t\t\t    objects.nr);\n> > >  \t}\n> > >  }\n> > > +\n> > > +void get_packed_object_summary(struct strbuf *obj_info)\n> > > +{\n> > > +\tstruct strbuf dirpath = STRBUF_INIT;\n> > > +\tstruct string_list contents = STRING_LIST_INIT_DUP;\n> > > +\tstruct string_list_item *entry;\n> > > +\n> > > +\tstrbuf_reset(obj_info);\n> > > +\n> > > +\tstrbuf_addstr(&dirpath, get_object_directory());\n> > > +\tstrbuf_complete(&dirpath, '/');\n> > > +\tstrbuf_addstr(&dirpath, \"pack/\");\n> > > +\tlist_contents_of_dir(&contents, &dirpath, 0, 0);\n> > > +\n> > > +\t// list full contents of $GIT_OBJECT_DIRECTORY/pack/\n> > > +\tfor_each_string_list_item(entry, &contents) {\n> > > +\t\tstrbuf_addbuf(obj_info, &dirpath);\n> > > +\t\tstrbuf_addstr(obj_info, entry->string);\n> > > +\t\tstrbuf_complete_line(obj_info);\n> > > +\t}\n> > > +}\n> >\n> > Okay, but I think that you will want to discern between regular `.pack`\n> > files, `.idx` files and `tmp_*` files.\n>\n> Discern in what way? How would you like to see them treated separately?\n> They're all being listed here, not just counted, so it seems to me like\n> I can read the generated bugreport and see which files are index, pack,\n> or temporary here.\n\nI take your word for it (sorry, it's been half an eternity since I wrapped\nmy head around the diff, I forgotten pretty much all about it ;-))\n\nCiao,\nDscho\n\n>\n> >\n> > > diff --git a/bugreport.h b/bugreport.h\n> > > index 09ad0c2599..11ff7df41b 100644\n> > > --- a/bugreport.h\n> > > +++ b/bugreport.h\n> > > @@ -24,3 +24,9 @@ void get_populated_hooks(struct strbuf *hook_info);\n> > >   * will be discarded.\n> > >   */\n> > >  void get_loose_object_summary(struct strbuf *obj_info);\n> > > +\n> > > +/**\n> > > + * Adds a list of the contents of '.git/objects/pack'. The previous contents of\n> > > + * hook_info will be discarded.\n> > > + */\n> > > +void get_packed_object_summary(struct strbuf *obj_info);\n> > > diff --git a/builtin/bugreport.c b/builtin/bugreport.c\n> > > index b2ab194207..da91a3944e 100644\n> > > --- a/builtin/bugreport.c\n> > > +++ b/builtin/bugreport.c\n> > > @@ -68,6 +68,10 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix)\n> > >  \tget_loose_object_summary(&buffer);\n> > >  \tstrbuf_write(&buffer, report);\n> > >\n> > > +\tadd_header(report, \"Packed Object Summary\");\n> > > +\tget_packed_object_summary(&buffer);\n> > > +\tstrbuf_write(&buffer, report);\n> > > +\n> >\n> > Hmm. At this point, I am unclear whether you want to write into an\n> > `strbuf`, or directly into a `FILE *`? I would rather have only one, not\n> > a mix.\n> >\n> > Ciao,\n> > Dscho\n> >\n> > >  \tfclose(report);\n> > >\n> > >  \tlaunch_editor(report_path.buf, NULL, NULL);\n> > > --\n> > > 2.24.0.rc0.303.g954a862665-goog\n> > >\n> > >\n>\n"},{"id":"387975","messageId":"20191211204811.GC107889@google.com","threadId":"51645","inReplyTo":"nycvar.QRO.7.76.6.1910281506040.46@tvgsbejvaqbjf.bet","subject":"Re: [PATCH v3 4/9] bugreport: add config values from whitelist","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2019-12-11T20:48:11Z","receivedAt":"2019-12-11T20:48:23Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"On Mon, Oct 28, 2019 at 03:14:35PM +0100, Johannes Schindelin wrote:\n> Hi Emily,\n\nSorry for the delay in replying. This work has been backburnered and\nthis mail slipped through the cracks.\n\n> \n> On Thu, 24 Oct 2019, Emily Shaffer wrote:\n> \n> > Teach bugreport to gather the values of config options which are present\n> > in 'git-bugreport-config-whitelist'.\n> >\n> > Many config options are sensitive, and many Git add-ons use config\n> > options which git-core does not know about; it is better only to gather\n> > config options which we know to be safe, rather than excluding options\n> > which we know to be unsafe.\n> \n> Should we still have the `// bugreport-exclude` comments, then?\n\nThey were optional (useless) before too. I can remove them if you want;\nI suppose I like the idea of having precedent if someone wants to build\ntheir own internal version with opt-out configs rather than opt-in. I\ncan remove them if we want; it doesn't matter very much to me either\nway.\n\n> \n> >\n> > Reading the whitelist into memory and sorting it saves us time -\n> > since git_config_bugreport() is called for every option the user has\n> > configured, limiting the file IO to one open/read/close and performing\n> > option lookup in sublinear time is a useful optimization.\n> \n> Maybe we even want a hashmap? That would reduce the time complexity even\n> further.\n\nSure, we can do it. I'll make that change.\n\n> \n> > diff --git a/bugreport.c b/bugreport.c\n> > index ada54fe583..afa4836ab1 100644\n> > --- a/bugreport.c\n> > +++ b/bugreport.c\n> > @@ -1,10 +1,24 @@\n> >  #include \"cache.h\"\n> >\n> >  #include \"bugreport.h\"\n> > +#include \"config.h\"\n> > +#include \"exec-cmd.h\"\n> >  #include \"help.h\"\n> >  #include \"run-command.h\"\n> >  #include \"version.h\"\n> >\n> > +/**\n> > + * A sorted list of config options which we will add to the bugreport. Managed\n> > + * by 'gather_whitelist(...)'.\n> > + */\n> > +struct string_list whitelist = STRING_LIST_INIT_DUP;\n> > +struct strbuf configs_and_values = STRBUF_INIT;\n> > +\n> > +// git version --build-options\n> > +// uname -a\n> > +// curl-config --version\n> > +// ldd --version\n> > +// echo $SHELL\n> \n> These comments probably want to move to a single, C style comment, and\n> they probably want to be introduced together with `get_system_info()`.\n\nYeah, it's stale and has been removed now. It was less commentary and\nmore todo list for author ;)\n\n> \n> I also have to admit that I might have missed where `$SHELL` was added\n> to the output...\n\nI skipped it entirely since bugreport doesn't run in shell anymore. If\nyou have advice for gathering the user's shell I can try to add it; is\nthere such a difference between, say, a Debian user using bash and a\nDebian user using zsh? I suppose it could be useful if someone has an\nissue with GIT_PS1, or with autocompletion. I'll look into gathering it.\n\n> \n> >  void get_system_info(struct strbuf *sys_info)\n> >  {\n> >  \tstruct child_process cp = CHILD_PROCESS_INIT;\n> > @@ -53,3 +67,39 @@ void get_system_info(struct strbuf *sys_info)\n> >  \targv_array_clear(&cp.args);\n> >  \tstrbuf_reset(&std_out);\n> >  }\n> > +\n> > +void gather_whitelist(struct strbuf *path)\n> > +{\n> > +\tstruct strbuf tmp = STRBUF_INIT;\n> > +\tstrbuf_read_file(&tmp, path->buf, 0);\n> > +\tstring_list_init(&whitelist, 1);\n> > +\tstring_list_split(&whitelist, tmp.buf, '\\n', -1);\n> > +\tstring_list_sort(&whitelist);\n> > +}\n> > +\n> > +int git_config_bugreport(const char *var, const char *value, void *cb)\n> > +{\n> > +\tif (string_list_has_string(&whitelist, var)) {\n> > +\t\tstrbuf_addf(&configs_and_values,\n> > +\t\t\t    \"%s : %s\\n\",\n> > +\t\t\t    var, value);\n> \n> A quite useful piece of information would be the config source. Not sure\n> whether we can do that outside of `config.c` yet...\n\nIt's possible. I can add it.\n\n> \n> > +\t}\n> > +\n> > +\treturn 0;\n> > +}\n> > +\n> > +void get_whitelisted_config(struct strbuf *config_info)\n> > +{\n> > +\tstruct strbuf path = STRBUF_INIT;\n> > +\n> > +\tstrbuf_addstr(&path, git_exec_path());\n> > +\tstrbuf_addstr(&path, \"/git-bugreport-config-whitelist\");\n> \n> Hmm. I would have expected this patch to come directly after the patch\n> 2/9 that generates that white-list, and I would also have expected that\n> to be pre-sorted, and compiled in.\n> \n> Do you want users to _edit_ the file in the exec path? In general, that\n> path will be write-protected, though. A better alternative would\n> probably be to compile in a hard-coded list, and to allow including more\n> values e.g. by offering command-line options to specify config setting\n> patterns. But if we allow patterns, we might actually want to have those\n> exclusions to prevent sensitive data from being included.\n\nHm, interesting. Do we have precedent for compiling in a header\ngenerated during the build process? I think I saw one when I was adding\nthis script - I'll take a look.\n\n> \n> > +\n> > +\tgather_whitelist(&path);\n> > +\tstrbuf_init(&configs_and_values, whitelist.nr);\n> > +\n> > +\tgit_config(git_config_bugreport, NULL);\n> > +\n> > +\tstrbuf_reset(config_info);\n> > +\tstrbuf_addbuf(config_info, &configs_and_values);\n> > +}\n> > diff --git a/bugreport.h b/bugreport.h\n> > index ba216acf3f..7413e7e1be 100644\n> > --- a/bugreport.h\n> > +++ b/bugreport.h\n> > @@ -5,3 +5,10 @@\n> >   * The previous contents of sys_info will be discarded.\n> >   */\n> >  void get_system_info(struct strbuf *sys_info);\n> > +\n> > +/**\n> \n> I also frequently use JavaDoc-style `/**`, but I am not sure that this\n> is actually desired in Git's source code ;-)\n> \n> > + * Adds the values of the config items listed in\n> > + * 'git-bugreport-config-whitelist' to config_info. The previous contents of\n> > + * config_info will be discarded.\n> > + */\n> > +void get_whitelisted_config(struct strbuf *sys_info);\n> > diff --git a/builtin/bugreport.c b/builtin/bugreport.c\n> > index 7232d31be7..70fe0d2b85 100644\n> > --- a/builtin/bugreport.c\n> > +++ b/builtin/bugreport.c\n> > @@ -56,6 +56,10 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix)\n> >  \tget_system_info(&buffer);\n> >  \tstrbuf_write(&buffer, report);\n> >\n> > +\tadd_header(report, \"Whitelisted Config\");\n> \n> Quite honestly, I would like to avoid the term \"whitelist\" for good. How\n> about \"Selected config settings\" instead?\n\nWill do - thanks for the callout.\n\n> \n> Thanks,\n> Dscho\n> \n> > +\tget_whitelisted_config(&buffer);\n> > +\tstrbuf_write(&buffer, report);\n> > +\n> >  \tfclose(report);\n> >\n> >  \tlaunch_editor(report_path.buf, NULL, NULL);\n> > --\n> > 2.24.0.rc0.303.g954a862665-goog\n> >\n> >\n"},{"id":"387976","messageId":"20191211205114.GD107889@google.com","threadId":"51645","inReplyTo":"nycvar.QRO.7.76.6.1910281522200.46@tvgsbejvaqbjf.bet","subject":"Re: [PATCH v3 5/9] bugreport: collect list of populated hooks","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2019-12-11T20:51:14Z","receivedAt":"2019-12-11T20:51:22Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"On Mon, Oct 28, 2019 at 03:31:36PM +0100, Johannes Schindelin wrote:\n> Hi Emily,\n> \n> On Thu, 24 Oct 2019, Emily Shaffer wrote:\n> \n> > Occasionally a failure a user is seeing may be related to a specific\n> > hook which is being run, perhaps without the user realizing. While the\n> > contents of hooks can be sensitive - containing user data or process\n> > information specific to the user's organization - simply knowing that a\n> > hook is being run at a certain stage can help us to understand whether\n> > something is going wrong.\n> >\n> > Without a definitive list of hook names within the code, we compile our\n> > own list from the documentation. This is likely prone to bitrot. To\n> > reduce the amount of code humans need to read, we turn the list into a\n> > string_list and iterate over it (as we are calling the same find_hook\n> > operation on each string). However, since bugreport should primarily be\n> > called by the user, the performance loss from massaging the string\n> > seems acceptable.\n> \n> Good idea!\n> \n> >\n> > Signed-off-by: Emily Shaffer <emilyshaffer@google.com>\n> > ---\n> >  bugreport.c         | 44 ++++++++++++++++++++++++++++++++++++++++++++\n> >  bugreport.h         |  6 ++++++\n> >  builtin/bugreport.c |  4 ++++\n> >  3 files changed, 54 insertions(+)\n> >\n> > diff --git a/bugreport.c b/bugreport.c\n> > index afa4836ab1..9d7f44ff28 100644\n> > --- a/bugreport.c\n> > +++ b/bugreport.c\n> > @@ -103,3 +103,47 @@ void get_whitelisted_config(struct strbuf *config_info)\n> >  \tstrbuf_reset(config_info);\n> >  \tstrbuf_addbuf(config_info, &configs_and_values);\n> >  }\n> > +\n> > +void get_populated_hooks(struct strbuf *hook_info)\n> > +{\n> > +\t/*\n> > +\t * Doesn't look like there is a list of all possible hooks; so below is\n> > +\t * a transcription of `git help hook`.\n> > +\t */\n> > +\tconst char *hooks = \"applypatch-msg,\"\n> > +\t\t\t    \"pre-applypatch,\"\n> > +\t\t\t    \"post-applypatch,\"\n> > +\t\t\t    \"pre-commit,\"\n> > +\t\t\t    \"pre-merge-commit,\"\n> > +\t\t\t    \"prepare-commit-msg,\"\n> > +\t\t\t    \"commit-msg,\"\n> > +\t\t\t    \"post-commit,\"\n> > +\t\t\t    \"pre-rebase,\"\n> > +\t\t\t    \"post-checkout,\"\n> > +\t\t\t    \"post-merge,\"\n> > +\t\t\t    \"pre-push,\"\n> > +\t\t\t    \"pre-receive,\"\n> > +\t\t\t    \"update,\"\n> > +\t\t\t    \"post-receive,\"\n> > +\t\t\t    \"post-update,\"\n> > +\t\t\t    \"push-to-checkout,\"\n> > +\t\t\t    \"pre-auto-gc,\"\n> > +\t\t\t    \"post-rewrite,\"\n> > +\t\t\t    \"sendemail-validate,\"\n> > +\t\t\t    \"fsmonitor-watchman,\"\n> > +\t\t\t    \"p4-pre-submit,\"\n> > +\t\t\t    \"post-index-changex\";\n> \n> Well, this is disappointing: I tried to come up with a scripted way to\n> extract the commit names from our source code, and I failed! This is\n> where I gave up:\n> \n> \tgit grep run_hook |\n> \tsed -n 's/.*run_hook[^(]*([^,]*, *\\([^,]*\\).*/\\1/p' |\n> \tsed -e '/^name$/d' -e '/^const char \\*name$/d' \\\n> \t\t-e 's/push_to_checkout_hook/\"push-to-checkout\"/' |\n> \tsort |\n> \tuniq\n> \n> This prints only the following hook names:\n> \n> \t\"applypatch-msg\"\n> \t\"post-applypatch\"\n> \t\"post-checkout\"\n> \t\"post-index-change\"\n> \t\"post-merge\"\n> \t\"pre-applypatch\"\n> \t\"pre-auto-gc\"\n> \t\"pre-rebase\"\n> \t\"prepare-commit-msg\"\n> \t\"push-to-checkout\"\n> \n> But at least it was easy to script the extracting from the\n> Documentation:\n> \n> \tsed -n '/^[a-z]/{N;/\\n~~~/{s/\\n.*//;p}}' \\\n> \t\tDocumentation/githooks.txt\n> \n\nTo be totally frank, I'm not keen on spending a lot of time with\nscripting this. My parallel work with config-based hooks will also\ninvolve an in-code source of truth for available hooknames; I'd like to\npunt on some scripting, putting it in the makefile, etc blah since I\nknow I'm planning to fix this particular annoyance at the source - and\nsince it looks like bugreport will stay in the review phase for at least\na little longer.\n\n> > +\tstruct string_list hooks_list = STRING_LIST_INIT_DUP;\n> > +\tstruct string_list_item *iter = NULL;\n> > +\n> > +\tstrbuf_reset(hook_info);\n> > +\n> > +\tstring_list_split(&hooks_list, hooks, ',', -1);\n> > +\n> > +\tfor_each_string_list_item(iter, &hooks_list) {\n> \n> This should definitely be done at compile time, I think. We should be\n> able to generate an appropriate header via something like this:\n> \n> \tcat >hook-names.h <<-EOF\n> \tstatic const char *hook_names[] = {\n> \t$(sed -n '/^[a-z]/{N;/\\n~~~/{s/\\n.*/\",/;s/^/\t\"/;p}}' \\\n> \t\tDocumentation/githooks.txt)\n> \t};\n> \tEOF\n> \n> Then you would use a simple `for()` loop using `ARRAY_SIZE()` to\n> iterate over the hook names.\n> \n> > +\t\tif (find_hook(iter->string)) {\n> > +\t\t\tstrbuf_addstr(hook_info, iter->string);\n> > +\t\t\tstrbuf_complete_line(hook_info);\n> > +\t\t}\n> > +\t}\n> > +}\n> > diff --git a/bugreport.h b/bugreport.h\n> > index 7413e7e1be..942a5436e3 100644\n> > --- a/bugreport.h\n> > +++ b/bugreport.h\n> > @@ -12,3 +12,9 @@ void get_system_info(struct strbuf *sys_info);\n> >   * config_info will be discarded.\n> >   */\n> >  void get_whitelisted_config(struct strbuf *sys_info);\n> > +\n> > +/**\n> > + * Adds the paths to all configured hooks (but not their contents). The previous\n> > + * contents of hook_info will be discarded.\n> > + */\n> > +void get_populated_hooks(struct strbuf *hook_info);\n> > diff --git a/builtin/bugreport.c b/builtin/bugreport.c\n> > index 70fe0d2b85..a0eefba498 100644\n> > --- a/builtin/bugreport.c\n> > +++ b/builtin/bugreport.c\n> > @@ -60,6 +60,10 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix)\n> >  \tget_whitelisted_config(&buffer);\n> >  \tstrbuf_write(&buffer, report);\n> >\n> > +\tadd_header(report, \"Populated Hooks\");\n> \n> Wait! I should have stumbled over this in an earlier patch. The\n> `add_header()` function should not take a `FILE *` parameter at all, but\n> instead an `struct strbuf *` one!\n> \n> Ciao,\n> Dscho\n> \n> > +\tget_populated_hooks(&buffer);\n> > +\tstrbuf_write(&buffer, report);\n> > +\n> >  \tfclose(report);\n> >\n> >  \tlaunch_editor(report_path.buf, NULL, NULL);\n> > --\n> > 2.24.0.rc0.303.g954a862665-goog\n> >\n> >\n"},{"id":"387977","messageId":"20191211205251.GE107889@google.com","threadId":"51645","inReplyTo":"nycvar.QRO.7.76.6.1912111436290.46@tvgsbejvaqbjf.bet","subject":"Re: [PATCH v3 7/9] bugreport: add packed object summary","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2019-12-11T20:52:51Z","receivedAt":"2019-12-11T20:53:00Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"On Wed, Dec 11, 2019 at 02:37:10PM +0100, Johannes Schindelin wrote:\n> Hi Emily,\n> \n> On Tue, 10 Dec 2019, Emily Shaffer wrote:\n> \n> > On Mon, Oct 28, 2019 at 04:43:50PM +0100, Johannes Schindelin wrote:\n> > > Hi Emily,\n> > >\n> > > On Thu, 24 Oct 2019, Emily Shaffer wrote:\n> > >\n> > > > Alongside the list of loose objects, it's useful to see the list of\n> > > > object packs as well. It can help us to examine what Git did and did not\n> > > > pack.\n> > >\n> > > Yes, I was write! Packs are next ;-)\n> > >\n> > > >\n> > > > Signed-off-by: Emily Shaffer <emilyshaffer@google.com>\n> > > > ---\n> > > >  bugreport.c         | 21 +++++++++++++++++++++\n> > > >  bugreport.h         |  6 ++++++\n> > > >  builtin/bugreport.c |  4 ++++\n> > > >  3 files changed, 31 insertions(+)\n> > > >\n> > > > diff --git a/bugreport.c b/bugreport.c\n> > > > index 54e1d47103..79ddb8baaa 100644\n> > > > --- a/bugreport.c\n> > > > +++ b/bugreport.c\n> > > > @@ -219,3 +219,24 @@ void get_loose_object_summary(struct strbuf *obj_info)\n> > > >  \t\t\t    objects.nr);\n> > > >  \t}\n> > > >  }\n> > > > +\n> > > > +void get_packed_object_summary(struct strbuf *obj_info)\n> > > > +{\n> > > > +\tstruct strbuf dirpath = STRBUF_INIT;\n> > > > +\tstruct string_list contents = STRING_LIST_INIT_DUP;\n> > > > +\tstruct string_list_item *entry;\n> > > > +\n> > > > +\tstrbuf_reset(obj_info);\n> > > > +\n> > > > +\tstrbuf_addstr(&dirpath, get_object_directory());\n> > > > +\tstrbuf_complete(&dirpath, '/');\n> > > > +\tstrbuf_addstr(&dirpath, \"pack/\");\n> > > > +\tlist_contents_of_dir(&contents, &dirpath, 0, 0);\n> > > > +\n> > > > +\t// list full contents of $GIT_OBJECT_DIRECTORY/pack/\n> > > > +\tfor_each_string_list_item(entry, &contents) {\n> > > > +\t\tstrbuf_addbuf(obj_info, &dirpath);\n> > > > +\t\tstrbuf_addstr(obj_info, entry->string);\n> > > > +\t\tstrbuf_complete_line(obj_info);\n> > > > +\t}\n> > > > +}\n> > >\n> > > Okay, but I think that you will want to discern between regular `.pack`\n> > > files, `.idx` files and `tmp_*` files.\n> >\n> > Discern in what way? How would you like to see them treated separately?\n> > They're all being listed here, not just counted, so it seems to me like\n> > I can read the generated bugreport and see which files are index, pack,\n> > or temporary here.\n> \n> I take your word for it (sorry, it's been half an eternity since I wrapped\n> my head around the diff, I forgotten pretty much all about it ;-))\n\nThat makes two of us. :)\n\n> \n> Ciao,\n> Dscho\n> \n> >\n> > >\n> > > > diff --git a/bugreport.h b/bugreport.h\n> > > > index 09ad0c2599..11ff7df41b 100644\n> > > > --- a/bugreport.h\n> > > > +++ b/bugreport.h\n> > > > @@ -24,3 +24,9 @@ void get_populated_hooks(struct strbuf *hook_info);\n> > > >   * will be discarded.\n> > > >   */\n> > > >  void get_loose_object_summary(struct strbuf *obj_info);\n> > > > +\n> > > > +/**\n> > > > + * Adds a list of the contents of '.git/objects/pack'. The previous contents of\n> > > > + * hook_info will be discarded.\n> > > > + */\n> > > > +void get_packed_object_summary(struct strbuf *obj_info);\n> > > > diff --git a/builtin/bugreport.c b/builtin/bugreport.c\n> > > > index b2ab194207..da91a3944e 100644\n> > > > --- a/builtin/bugreport.c\n> > > > +++ b/builtin/bugreport.c\n> > > > @@ -68,6 +68,10 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix)\n> > > >  \tget_loose_object_summary(&buffer);\n> > > >  \tstrbuf_write(&buffer, report);\n> > > >\n> > > > +\tadd_header(report, \"Packed Object Summary\");\n> > > > +\tget_packed_object_summary(&buffer);\n> > > > +\tstrbuf_write(&buffer, report);\n> > > > +\n> > >\n> > > Hmm. At this point, I am unclear whether you want to write into an\n> > > `strbuf`, or directly into a `FILE *`? I would rather have only one, not\n> > > a mix.\n> > >\n> > > Ciao,\n> > > Dscho\n> > >\n> > > >  \tfclose(report);\n> > > >\n> > > >  \tlaunch_editor(report_path.buf, NULL, NULL);\n> > > > --\n> > > > 2.24.0.rc0.303.g954a862665-goog\n> > > >\n> > > >\n> >\n"},{"id":"388231","messageId":"nycvar.QRO.7.76.6.1912151826380.46@tvgsbejvaqbjf.bet","threadId":"51645","inReplyTo":"20191211204811.GC107889@google.com","subject":"Re: [PATCH v3 4/9] bugreport: add config values from whitelist","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2019-12-15T17:30:59Z","receivedAt":"2019-12-15T17:31:22Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Emily,\n\nOn Wed, 11 Dec 2019, Emily Shaffer wrote:\n\n> On Mon, Oct 28, 2019 at 03:14:35PM +0100, Johannes Schindelin wrote:\n>\n> >\n> > On Thu, 24 Oct 2019, Emily Shaffer wrote:\n> >\n> > > Teach bugreport to gather the values of config options which are\n> > > present in 'git-bugreport-config-whitelist'.\n> > >\n> > > Many config options are sensitive, and many Git add-ons use config\n> > > options which git-core does not know about; it is better only to\n> > > gather config options which we know to be safe, rather than\n> > > excluding options which we know to be unsafe.\n> >\n> > Should we still have the `// bugreport-exclude` comments, then?\n>\n> They were optional (useless) before too. I can remove them if you want;\n> I suppose I like the idea of having precedent if someone wants to build\n> their own internal version with opt-out configs rather than opt-in. I\n> can remove them if we want; it doesn't matter very much to me either\n> way.\n\nHow are you guaranteeing that this information does not become stale,\nlike, immediately?\n\nIf you _still_ insist on keeping those comments even after answering this\nquestion, at least please turn them into C-style comments: we have no\nC++-style comments in Git and want to keep it this way.\n\n> > > diff --git a/bugreport.c b/bugreport.c\n> > > [...]\n> > > +\t}\n> > > +\n> > > +\treturn 0;\n> > > +}\n> > > +\n> > > +void get_whitelisted_config(struct strbuf *config_info)\n> > > +{\n> > > +\tstruct strbuf path = STRBUF_INIT;\n> > > +\n> > > +\tstrbuf_addstr(&path, git_exec_path());\n> > > +\tstrbuf_addstr(&path, \"/git-bugreport-config-whitelist\");\n> >\n> > Hmm. I would have expected this patch to come directly after the patch\n> > 2/9 that generates that white-list, and I would also have expected that\n> > to be pre-sorted, and compiled in.\n> >\n> > Do you want users to _edit_ the file in the exec path? In general, that\n> > path will be write-protected, though. A better alternative would\n> > probably be to compile in a hard-coded list, and to allow including more\n> > values e.g. by offering command-line options to specify config setting\n> > patterns. But if we allow patterns, we might actually want to have those\n> > exclusions to prevent sensitive data from being included.\n>\n> Hm, interesting. Do we have precedent for compiling in a header\n> generated during the build process? I think I saw one when I was adding\n> this script - I'll take a look.\n\nYes, we do. The entire `command-list.h` is generated during the build.\nLook for `GENERATED_H` in the `Makefile`.\n\nCiao,\nDscho\n"},{"id":"388233","messageId":"nycvar.QRO.7.76.6.1912151839280.46@tvgsbejvaqbjf.bet","threadId":"51645","inReplyTo":"20191211205114.GD107889@google.com","subject":"Re: [PATCH v3 5/9] bugreport: collect list of populated hooks","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2019-12-15T17:40:22Z","receivedAt":"2019-12-15T17:40:44Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Emily,\n\nOn Wed, 11 Dec 2019, Emily Shaffer wrote:\n\n> On Mon, Oct 28, 2019 at 03:31:36PM +0100, Johannes Schindelin wrote:\n>\n> > On Thu, 24 Oct 2019, Emily Shaffer wrote:\n> >\n> > > diff --git a/bugreport.c b/bugreport.c\n> > > index afa4836ab1..9d7f44ff28 100644\n> > > --- a/bugreport.c\n> > > +++ b/bugreport.c\n> > > @@ -103,3 +103,47 @@ void get_whitelisted_config(struct strbuf *config_info)\n> > >  \tstrbuf_reset(config_info);\n> > >  \tstrbuf_addbuf(config_info, &configs_and_values);\n> > >  }\n> > > +\n> > > +void get_populated_hooks(struct strbuf *hook_info)\n> > > +{\n> > > +\t/*\n> > > +\t * Doesn't look like there is a list of all possible hooks; so below is\n> > > +\t * a transcription of `git help hook`.\n> > > +\t */\n> > > +\tconst char *hooks = \"applypatch-msg,\"\n> > > +\t\t\t    \"pre-applypatch,\"\n> > > +\t\t\t    \"post-applypatch,\"\n> > > +\t\t\t    \"pre-commit,\"\n> > > +\t\t\t    \"pre-merge-commit,\"\n> > > +\t\t\t    \"prepare-commit-msg,\"\n> > > +\t\t\t    \"commit-msg,\"\n> > > +\t\t\t    \"post-commit,\"\n> > > +\t\t\t    \"pre-rebase,\"\n> > > +\t\t\t    \"post-checkout,\"\n> > > +\t\t\t    \"post-merge,\"\n> > > +\t\t\t    \"pre-push,\"\n> > > +\t\t\t    \"pre-receive,\"\n> > > +\t\t\t    \"update,\"\n> > > +\t\t\t    \"post-receive,\"\n> > > +\t\t\t    \"post-update,\"\n> > > +\t\t\t    \"push-to-checkout,\"\n> > > +\t\t\t    \"pre-auto-gc,\"\n> > > +\t\t\t    \"post-rewrite,\"\n> > > +\t\t\t    \"sendemail-validate,\"\n> > > +\t\t\t    \"fsmonitor-watchman,\"\n> > > +\t\t\t    \"p4-pre-submit,\"\n> > > +\t\t\t    \"post-index-changex\";\n> >\n> > Well, this is disappointing: I tried to come up with a scripted way to\n> > extract the commit names from our source code, and I failed! This is\n> > where I gave up:\n> >\n> > \tgit grep run_hook |\n> > \tsed -n 's/.*run_hook[^(]*([^,]*, *\\([^,]*\\).*/\\1/p' |\n> > \tsed -e '/^name$/d' -e '/^const char \\*name$/d' \\\n> > \t\t-e 's/push_to_checkout_hook/\"push-to-checkout\"/' |\n> > \tsort |\n> > \tuniq\n> >\n> > This prints only the following hook names:\n> >\n> > \t\"applypatch-msg\"\n> > \t\"post-applypatch\"\n> > \t\"post-checkout\"\n> > \t\"post-index-change\"\n> > \t\"post-merge\"\n> > \t\"pre-applypatch\"\n> > \t\"pre-auto-gc\"\n> > \t\"pre-rebase\"\n> > \t\"prepare-commit-msg\"\n> > \t\"push-to-checkout\"\n> >\n> > But at least it was easy to script the extracting from the\n> > Documentation:\n> >\n> > \tsed -n '/^[a-z]/{N;/\\n~~~/{s/\\n.*//;p}}' \\\n> > \t\tDocumentation/githooks.txt\n> >\n>\n> To be totally frank, I'm not keen on spending a lot of time with\n> scripting this. My parallel work with config-based hooks will also\n> involve an in-code source of truth for available hooknames; I'd like to\n> punt on some scripting, putting it in the makefile, etc blah since I\n> know I'm planning to fix this particular annoyance at the source - and\n> since it looks like bugreport will stay in the review phase for at least\n> a little longer.\n\nFair enough, especially if it addresses my complaint about the\nscriptability.\n\nCiao,\nDscho\n"}]}