{"thread":{"id":"47496","subject":"[PATCH v2 0/7] Codespeed perf results","startedAt":"2017-12-26T21:59:18Z","lastAt":"2017-12-28T19:47:39Z","messageCount":12,"participants":["Christian Couder","Junio C Hamano"],"isPatch":true,"patchVersion":2,"patchTotal":7},"messages":[{"id":"335332","messageId":"20171226215908.425-1-chriscool@tuxfamily.org","threadId":"47496","inReplyTo":null,"subject":"[PATCH v2 0/7] Codespeed perf results","fromName":"Christian Couder","fromEmail":"christian.couder@gmail.com","sentAt":"2017-12-26T21:59:01Z","receivedAt":"2017-12-26T21:59:18Z","isPatch":true,"sender":{"key":"christian.couder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/208954?v=4"},"body":"This patch series is built on top of cc/perf-run-config which recently\ngraduated to master.\n\nIt makes it possible to send perf results to a Codespeed server. See\nhttps://github.com/tobami/codespeed/ and web sites like\nhttp://speed.pypy.org/ which are using Codespeed.\n\nThe end goal would be to have such a server always available to track\nhow the different git commands perform over time on different kind of\nrepos (small, medium, large, ...) with different optimizations on and\noff (split-index, libpcre2, BLK_SHA1, ...)\n\nWith this series and a config file like:\n\n$ cat perf.conf\n[perf]\n        dirsOrRevs = v2.12.0 v2.13.0\n        repeatCount = 10\n\tsendToCodespeed = http://localhost:8000\n\trepoName = Git repo\n[perf \"with libpcre\"]\n        makeOpts = \"DEVELOPER=1 USE_LIBPCRE=YesPlease\"\n[perf \"without libpcre\"]\n        makeOpts = \"DEVELOPER=1\"\n\nOne should be able to just launch:\n\n$ ./run --config perf.conf p7810-grep.sh\n\nand then get nice graphs in a Codespeed instance running on\nhttp://localhost:8000.\n\nCaveat\n~~~~~~\n\nFor now one has to create the \"Git repo\" environment in the Codespeed\nadmin interface. (We send the perf.repoName config variable in the\n\"environment\" Codespeed field.) This is because Codespeed requires the\nenvironment fields to be created and does not provide a simple way to\ncreate these fields programmatically.\n\nThere are discussions on a Codespeed issue\n(https://github.com/tobami/codespeed/issues/232) about creating a\nproper API for Codespeed that could address this problem in the\nfuture.\n\nChanges since previous version\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\n  - Fixed the way the 'executable' field sent to Codespeed is set in\n    `perf/aggregate.perl` in patch 3/7. We now use `uname -s -m` to\n    which we concatenate the perf.repoName config value if it is set,\n    as suggested by Junio and Eric.\n\n  - Fixed the name of the GIT_PERF_REPO_NAME variable in patches 3/7\n    and 7/7. It was GIT_TEST_REPO_NAME in some places.\n\n  - Fixed how the conf_opt argument is added to the\n    get_var_from_env_or_config() function in patch 4/7. It was added\n    as the last, so fifth, argument, but it is simpler and makes more\n    sense to keep the default value argument as the last argument, so\n    now the conf_opt argument is added as the fourth argument.\n\n  - What patch 4/7 did was previously done in 2 patches in the\n    previous version (patches 4/8 and 5/8), but as we are not doing\n    exactly the same thing (see the above item) it is simpler to do it\n    in only one patch instead of two.\n\n  - We now use the --int type specifier when getting the\n    perf.repeatCount config variable in patch 4/7.\n\nLinks\n~~~~~\n\nThis patch series:\n\nhttps://github.com/chriscool/git/commits/codespeed\n\nPrevious version (v1):\n\nhttps://github.com/chriscool/git/commits/codespeed1\n\nDiscussions about v1:\n\nhttps://public-inbox.org/git/CAP8UFD3Q4h-aYBDABSPOW948LQYVydWZ1hLPAD+kr9ZpXVZiaQ@mail.gmail.com/\n\nDiscussions about the cc/perf-run-config patch series:\n\nv1: https://public-inbox.org/git/20170713065050.19215-1-chriscool@tuxfamily.org/\nv2: https://public-inbox.org/git/CAP8UFD2j-UFh+9awz91gtZ-jusq7EUOExMgURO59vpf29jXS4A@mail.gmail.com/\n\nChristian Couder (7):\n  perf/aggregate: fix checking ENV{GIT_PERF_SUBSECTION}\n  perf/aggregate: refactor printing results\n  perf/aggregate: implement codespeed JSON output\n  perf/run: add conf_opts argument to get_var_from_env_or_config()\n  perf/run: learn about perf.codespeedOutput\n  perf/run: learn to send output to codespeed server\n  perf/run: read GIT_TEST_REPO_NAME from perf.repoName\n\n t/perf/aggregate.perl | 163 +++++++++++++++++++++++++++++++++++---------------\n t/perf/run            |  31 ++++++++--\n 2 files changed, 140 insertions(+), 54 deletions(-)\n\n-- \n2.15.1.361.g8b07d831d0\n\n"},{"id":"335333","messageId":"20171226215908.425-3-chriscool@tuxfamily.org","threadId":"47496","inReplyTo":"20171226215908.425-1-chriscool@tuxfamily.org","subject":"[PATCH v2 2/7] perf/aggregate: refactor printing results","fromName":"Christian Couder","fromEmail":"christian.couder@gmail.com","sentAt":"2017-12-26T21:59:03Z","receivedAt":"2017-12-26T21:59:21Z","isPatch":true,"sender":{"key":"christian.couder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/208954?v=4"},"body":"As we want to implement another kind of output than\nthe current output for the perf test results, let's\nrefactor the existing code that outputs the results\nin its own print_default_results() function.\n\nSigned-off-by: Christian Couder <chriscool@tuxfamily.org>\n---\n t/perf/aggregate.perl | 96 +++++++++++++++++++++++++++------------------------\n 1 file changed, 50 insertions(+), 46 deletions(-)\n\ndiff --git a/t/perf/aggregate.perl b/t/perf/aggregate.perl\nindex 769d418708..3609cb5dc3 100755\n--- a/t/perf/aggregate.perl\n+++ b/t/perf/aggregate.perl\n@@ -100,13 +100,6 @@ sub read_descr {\n \treturn $line;\n }\n \n-my %descrs;\n-my $descrlen = 4; # \"Test\"\n-for my $t (@subtests) {\n-\t$descrs{$t} = $shorttests{$t}.\": \".read_descr(\"$resultsdir/$t.descr\");\n-\t$descrlen = length $descrs{$t} if length $descrs{$t}>$descrlen;\n-}\n-\n sub have_duplicate {\n \tmy %seen;\n \tfor (@_) {\n@@ -122,54 +115,65 @@ sub have_slash {\n \treturn 0;\n }\n \n-my %newdirabbrevs = %dirabbrevs;\n-while (!have_duplicate(values %newdirabbrevs)) {\n-\t%dirabbrevs = %newdirabbrevs;\n-\tlast if !have_slash(values %dirabbrevs);\n-\t%newdirabbrevs = %dirabbrevs;\n-\tfor (values %newdirabbrevs) {\n-\t\ts{^[^/]*/}{};\n+sub print_default_results {\n+\tmy %descrs;\n+\tmy $descrlen = 4; # \"Test\"\n+\tfor my $t (@subtests) {\n+\t\t$descrs{$t} = $shorttests{$t}.\": \".read_descr(\"$resultsdir/$t.descr\");\n+\t\t$descrlen = length $descrs{$t} if length $descrs{$t}>$descrlen;\n \t}\n-}\n \n-my %times;\n-my @colwidth = ((0)x@dirs);\n-for my $i (0..$#dirs) {\n-\tmy $d = $dirs[$i];\n-\tmy $w = length (exists $dirabbrevs{$d} ? $dirabbrevs{$d} : $dirnames{$d});\n-\t$colwidth[$i] = $w if $w > $colwidth[$i];\n-}\n-for my $t (@subtests) {\n-\tmy $firstr;\n+\tmy %newdirabbrevs = %dirabbrevs;\n+\twhile (!have_duplicate(values %newdirabbrevs)) {\n+\t\t%dirabbrevs = %newdirabbrevs;\n+\t\tlast if !have_slash(values %dirabbrevs);\n+\t\t%newdirabbrevs = %dirabbrevs;\n+\t\tfor (values %newdirabbrevs) {\n+\t\t\ts{^[^/]*/}{};\n+\t\t}\n+\t}\n+\n+\tmy %times;\n+\tmy @colwidth = ((0)x@dirs);\n \tfor my $i (0..$#dirs) {\n \t\tmy $d = $dirs[$i];\n-\t\t$times{$prefixes{$d}.$t} = [get_times(\"$resultsdir/$prefixes{$d}$t.times\")];\n-\t\tmy ($r,$u,$s) = @{$times{$prefixes{$d}.$t}};\n-\t\tmy $w = length format_times($r,$u,$s,$firstr);\n+\t\tmy $w = length (exists $dirabbrevs{$d} ? $dirabbrevs{$d} : $dirnames{$d});\n \t\t$colwidth[$i] = $w if $w > $colwidth[$i];\n-\t\t$firstr = $r unless defined $firstr;\n \t}\n-}\n-my $totalwidth = 3*@dirs+$descrlen;\n-$totalwidth += $_ for (@colwidth);\n-\n-binmode STDOUT, \":utf8\" or die \"PANIC on binmode: $!\";\n+\tfor my $t (@subtests) {\n+\t\tmy $firstr;\n+\t\tfor my $i (0..$#dirs) {\n+\t\t\tmy $d = $dirs[$i];\n+\t\t\t$times{$prefixes{$d}.$t} = [get_times(\"$resultsdir/$prefixes{$d}$t.times\")];\n+\t\t\tmy ($r,$u,$s) = @{$times{$prefixes{$d}.$t}};\n+\t\t\tmy $w = length format_times($r,$u,$s,$firstr);\n+\t\t\t$colwidth[$i] = $w if $w > $colwidth[$i];\n+\t\t\t$firstr = $r unless defined $firstr;\n+\t\t}\n+\t}\n+\tmy $totalwidth = 3*@dirs+$descrlen;\n+\t$totalwidth += $_ for (@colwidth);\n \n-printf \"%-${descrlen}s\", \"Test\";\n-for my $i (0..$#dirs) {\n-\tmy $d = $dirs[$i];\n-\tprintf \"   %-$colwidth[$i]s\", (exists $dirabbrevs{$d} ? $dirabbrevs{$d} : $dirnames{$d});\n-}\n-print \"\\n\";\n-print \"-\"x$totalwidth, \"\\n\";\n-for my $t (@subtests) {\n-\tprintf \"%-${descrlen}s\", $descrs{$t};\n-\tmy $firstr;\n+\tprintf \"%-${descrlen}s\", \"Test\";\n \tfor my $i (0..$#dirs) {\n \t\tmy $d = $dirs[$i];\n-\t\tmy ($r,$u,$s) = @{$times{$prefixes{$d}.$t}};\n-\t\tprintf \"   %-$colwidth[$i]s\", format_times($r,$u,$s,$firstr);\n-\t\t$firstr = $r unless defined $firstr;\n+\t\tprintf \"   %-$colwidth[$i]s\", (exists $dirabbrevs{$d} ? $dirabbrevs{$d} : $dirnames{$d});\n \t}\n \tprint \"\\n\";\n+\tprint \"-\"x$totalwidth, \"\\n\";\n+\tfor my $t (@subtests) {\n+\t\tprintf \"%-${descrlen}s\", $descrs{$t};\n+\t\tmy $firstr;\n+\t\tfor my $i (0..$#dirs) {\n+\t\t\tmy $d = $dirs[$i];\n+\t\t\tmy ($r,$u,$s) = @{$times{$prefixes{$d}.$t}};\n+\t\t\tprintf \"   %-$colwidth[$i]s\", format_times($r,$u,$s,$firstr);\n+\t\t\t$firstr = $r unless defined $firstr;\n+\t\t}\n+\t\tprint \"\\n\";\n+\t}\n }\n+\n+binmode STDOUT, \":utf8\" or die \"PANIC on binmode: $!\";\n+\n+print_default_results();\n-- \n2.15.1.361.g8b07d831d0\n\n"},{"id":"335334","messageId":"20171226215908.425-6-chriscool@tuxfamily.org","threadId":"47496","inReplyTo":"20171226215908.425-1-chriscool@tuxfamily.org","subject":"[PATCH v2 5/7] perf/run: learn about perf.codespeedOutput","fromName":"Christian Couder","fromEmail":"christian.couder@gmail.com","sentAt":"2017-12-26T21:59:06Z","receivedAt":"2017-12-26T21:59:24Z","isPatch":true,"sender":{"key":"christian.couder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/208954?v=4"},"body":"Let's make it possible to set in a config file the output\nformat (regular or codespeed) of the perf tests.\n\nSigned-off-by: Christian Couder <chriscool@tuxfamily.org>\n---\n t/perf/run | 7 ++++++-\n 1 file changed, 6 insertions(+), 1 deletion(-)\n\ndiff --git a/t/perf/run b/t/perf/run\nindex 214d658172..4e62d6bb3f 100755\n--- a/t/perf/run\n+++ b/t/perf/run\n@@ -144,10 +144,15 @@ run_subsection () {\n \t\tset -- . \"$@\"\n \tfi\n \n+\tcodespeed_opt=\n+\ttest \"$GIT_PERF_CODESPEED_OUTPUT\" = \"true\" && codespeed_opt=\"--codespeed\"\n+\n \trun_dirs \"$@\"\n-\t./aggregate.perl \"$@\"\n+\t./aggregate.perl $codespeed_opt \"$@\"\n }\n \n+get_var_from_env_or_config \"GIT_PERF_CODESPEED_OUTPUT\" \"perf\" \"codespeedOutput\" \"--bool\"\n+\n cd \"$(dirname $0)\"\n . ../../GIT-BUILD-OPTIONS\n \n-- \n2.15.1.361.g8b07d831d0\n\n"},{"id":"335335","messageId":"20171226215908.425-8-chriscool@tuxfamily.org","threadId":"47496","inReplyTo":"20171226215908.425-1-chriscool@tuxfamily.org","subject":"[PATCH v2 7/7] perf/run: read GIT_TEST_REPO_NAME from perf.repoName","fromName":"Christian Couder","fromEmail":"christian.couder@gmail.com","sentAt":"2017-12-26T21:59:08Z","receivedAt":"2017-12-26T21:59:26Z","isPatch":true,"sender":{"key":"christian.couder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/208954?v=4"},"body":"The GIT_PERF_REPO_NAME env variable is used in\nthe `aggregate.perl` script to set the 'environment'\nfield in the JSON Codespeed output.\n\nLet's make it easy to set this variable by setting it\nin a config file.\n\nSigned-off-by: Christian Couder <chriscool@tuxfamily.org>\n---\n t/perf/run | 3 +++\n 1 file changed, 3 insertions(+)\n\ndiff --git a/t/perf/run b/t/perf/run\nindex ef56396546..1a100d6134 100755\n--- a/t/perf/run\n+++ b/t/perf/run\n@@ -137,6 +137,9 @@ run_subsection () {\n \tget_var_from_env_or_config \"GIT_PERF_MAKE_COMMAND\" \"perf\" \"makeCommand\"\n \tget_var_from_env_or_config \"GIT_PERF_MAKE_OPTS\" \"perf\" \"makeOpts\"\n \n+\tget_var_from_env_or_config \"GIT_PERF_REPO_NAME\" \"perf\" \"repoName\"\n+\texport GIT_PERF_REPO_NAME\n+\n \tGIT_PERF_AGGREGATING_LATER=t\n \texport GIT_PERF_AGGREGATING_LATER\n \n-- \n2.15.1.361.g8b07d831d0\n\n"},{"id":"335336","messageId":"20171226215908.425-2-chriscool@tuxfamily.org","threadId":"47496","inReplyTo":"20171226215908.425-1-chriscool@tuxfamily.org","subject":"[PATCH v2 1/7] perf/aggregate: fix checking ENV{GIT_PERF_SUBSECTION}","fromName":"Christian Couder","fromEmail":"christian.couder@gmail.com","sentAt":"2017-12-26T21:59:02Z","receivedAt":"2017-12-26T21:59:29Z","isPatch":true,"sender":{"key":"christian.couder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/208954?v=4"},"body":"Signed-off-by: Christian Couder <chriscool@tuxfamily.org>\n---\n t/perf/aggregate.perl | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/t/perf/aggregate.perl b/t/perf/aggregate.perl\nindex e401208488..769d418708 100755\n--- a/t/perf/aggregate.perl\n+++ b/t/perf/aggregate.perl\n@@ -70,7 +70,7 @@ if (not @tests) {\n }\n \n my $resultsdir = \"test-results\";\n-if ($ENV{GIT_PERF_SUBSECTION} ne \"\") {\n+if (exists $ENV{GIT_PERF_SUBSECTION} and $ENV{GIT_PERF_SUBSECTION} ne \"\") {\n \t$resultsdir .= \"/\" . $ENV{GIT_PERF_SUBSECTION};\n }\n \n-- \n2.15.1.361.g8b07d831d0\n\n"},{"id":"335337","messageId":"20171226215908.425-4-chriscool@tuxfamily.org","threadId":"47496","inReplyTo":"20171226215908.425-1-chriscool@tuxfamily.org","subject":"[PATCH v2 3/7] perf/aggregate: implement codespeed JSON output","fromName":"Christian Couder","fromEmail":"christian.couder@gmail.com","sentAt":"2017-12-26T21:59:04Z","receivedAt":"2017-12-26T21:59:32Z","isPatch":true,"sender":{"key":"christian.couder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/208954?v=4"},"body":"Codespeed (https://github.com/tobami/codespeed/) is an open source\nproject that can be used to track how some software performs over\ntime. It stores performance test results in a database and can show\nnice graphs and charts on a web interface.\n\nAs it can be interesting to Codespeed to see how Git performance\nevolves over time and releases, let's implement a Codespeed output\nin \"perf/aggregate.perl\".\n\nHelped-by: Eric Sunshine <sunshine@sunshineco.com>\nSigned-off-by: Christian Couder <chriscool@tuxfamily.org>\n---\n t/perf/aggregate.perl | 67 +++++++++++++++++++++++++++++++++++++++++++++++++--\n 1 file changed, 65 insertions(+), 2 deletions(-)\n\ndiff --git a/t/perf/aggregate.perl b/t/perf/aggregate.perl\nindex 3609cb5dc3..34d74fc015 100755\n--- a/t/perf/aggregate.perl\n+++ b/t/perf/aggregate.perl\n@@ -35,10 +35,15 @@ sub format_times {\n \treturn $out;\n }\n \n-my (@dirs, %dirnames, %dirabbrevs, %prefixes, @tests);\n+my (@dirs, %dirnames, %dirabbrevs, %prefixes, @tests, $codespeed);\n while (scalar @ARGV) {\n \tmy $arg = $ARGV[0];\n \tmy $dir;\n+\tif ($arg eq \"--codespeed\") {\n+\t\t$codespeed = 1;\n+\t\tshift @ARGV;\n+\t\tnext;\n+\t}\n \tlast if -f $arg or $arg eq \"--\";\n \tif (! -d $arg) {\n \t\tmy $rev = Git::command_oneline(qw(rev-parse --verify), $arg);\n@@ -70,8 +75,10 @@ if (not @tests) {\n }\n \n my $resultsdir = \"test-results\";\n+my $results_section = \"\";\n if (exists $ENV{GIT_PERF_SUBSECTION} and $ENV{GIT_PERF_SUBSECTION} ne \"\") {\n \t$resultsdir .= \"/\" . $ENV{GIT_PERF_SUBSECTION};\n+\t$results_section = $ENV{GIT_PERF_SUBSECTION};\n }\n \n my @subtests;\n@@ -174,6 +181,62 @@ sub print_default_results {\n \t}\n }\n \n+sub print_codespeed_results {\n+\tmy ($results_section) = @_;\n+\n+\tmy $project = \"Git\";\n+\n+\tmy $executable = `uname -s -m`;\n+\tchomp $executable;\n+\n+\tif ($results_section ne \"\") {\n+\t\t$executable .= \", \" . $results_section;\n+\t}\n+\n+\tmy $environment;\n+\tif (exists $ENV{GIT_PERF_REPO_NAME} and $ENV{GIT_PERF_REPO_NAME} ne \"\") {\n+\t\t$environment = $ENV{GIT_PERF_REPO_NAME};\n+\t} elsif (exists $ENV{GIT_TEST_INSTALLED} and $ENV{GIT_TEST_INSTALLED} ne \"\") {\n+\t\t$environment = $ENV{GIT_TEST_INSTALLED};\n+\t\t$environment =~ s|/bin-wrappers$||;\n+\t} else {\n+\t\t$environment = `uname -r`;\n+\t\tchomp $environment;\n+\t}\n+\n+\tmy @data;\n+\n+\tfor my $t (@subtests) {\n+\t\tfor my $d (@dirs) {\n+\t\t\tmy $commitid = $prefixes{$d};\n+\t\t\t$commitid =~ s/^build_//;\n+\t\t\t$commitid =~ s/\\.$//;\n+\t\t\tmy ($result_value, $u, $s) = get_times(\"$resultsdir/$prefixes{$d}$t.times\");\n+\n+\t\t\tmy %vals = (\n+\t\t\t\t\"commitid\" => $commitid,\n+\t\t\t\t\"project\" => $project,\n+\t\t\t\t\"branch\" => $dirnames{$d},\n+\t\t\t\t\"executable\" => $executable,\n+\t\t\t\t\"benchmark\" => $shorttests{$t} . \" \" . read_descr(\"$resultsdir/$t.descr\"),\n+\t\t\t\t\"environment\" => $environment,\n+\t\t\t\t\"result_value\" => $result_value,\n+\t\t\t    );\n+\t\t\tpush @data, \\%vals;\n+\t\t}\n+\t}\n+\n+\t#use Data::Dumper qw/ Dumper /;\n+\t#print Dumper(\\@data);\n+\n+\tuse JSON;\n+\tprint to_json(\\@data, {utf8 => 1, pretty => 1}), \"\\n\";\n+}\n+\n binmode STDOUT, \":utf8\" or die \"PANIC on binmode: $!\";\n \n-print_default_results();\n+if ($codespeed) {\n+\tprint_codespeed_results($results_section);\n+} else {\n+\tprint_default_results();\n+}\n-- \n2.15.1.361.g8b07d831d0\n\n"},{"id":"335338","messageId":"20171226215908.425-5-chriscool@tuxfamily.org","threadId":"47496","inReplyTo":"20171226215908.425-1-chriscool@tuxfamily.org","subject":"[PATCH v2 4/7] perf/run: add conf_opts argument to get_var_from_env_or_config()","fromName":"Christian Couder","fromEmail":"christian.couder@gmail.com","sentAt":"2017-12-26T21:59:05Z","receivedAt":"2017-12-26T21:59:34Z","isPatch":true,"sender":{"key":"christian.couder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/208954?v=4"},"body":"Let's make it possible to use `git config` type specifiers like\n`--int` or `--bool`, so that config values are converted to the\ncanonical form and easier to use.\n\nThis additional argument is now the fourth argument of\nget_var_from_env_or_config() instead of the fifth because we\nwant the default value argument to be unset if it is not\npassed, and this is simpler if it is the last argument.\n\nSigned-off-by: Christian Couder <chriscool@tuxfamily.org>\n---\n t/perf/run | 11 ++++++-----\n 1 file changed, 6 insertions(+), 5 deletions(-)\n\ndiff --git a/t/perf/run b/t/perf/run\nindex 43e4de49ef..214d658172 100755\n--- a/t/perf/run\n+++ b/t/perf/run\n@@ -105,7 +105,8 @@ get_var_from_env_or_config () {\n \tenv_var=\"$1\"\n \tconf_sec=\"$2\"\n \tconf_var=\"$3\"\n-\t# $4 can be set to a default value\n+\tconf_opts=\"$4\" # optional\n+\t# $5 can be set to a default value\n \n \t# Do nothing if the env variable is already set\n \teval \"test -z \\\"\\${$env_var+x}\\\"\" || return\n@@ -116,18 +117,18 @@ get_var_from_env_or_config () {\n \tif test -n \"$GIT_PERF_SUBSECTION\"\n \tthen\n \t\tvar=\"$conf_sec.$GIT_PERF_SUBSECTION.$conf_var\"\n-\t\tconf_value=$(git config -f \"$GIT_PERF_CONFIG_FILE\" \"$var\") &&\n+\t\tconf_value=$(git config $conf_opts -f \"$GIT_PERF_CONFIG_FILE\" \"$var\") &&\n \t\teval \"$env_var=\\\"$conf_value\\\"\" && return\n \tfi\n \tvar=\"$conf_sec.$conf_var\"\n-\tconf_value=$(git config -f \"$GIT_PERF_CONFIG_FILE\" \"$var\") &&\n+\tconf_value=$(git config $conf_opts -f \"$GIT_PERF_CONFIG_FILE\" \"$var\") &&\n \teval \"$env_var=\\\"$conf_value\\\"\" && return\n \n-\ttest -n \"${4+x}\" && eval \"$env_var=\\\"$4\\\"\"\n+\ttest -n \"${5+x}\" && eval \"$env_var=\\\"$5\\\"\"\n }\n \n run_subsection () {\n-\tget_var_from_env_or_config \"GIT_PERF_REPEAT_COUNT\" \"perf\" \"repeatCount\" 3\n+\tget_var_from_env_or_config \"GIT_PERF_REPEAT_COUNT\" \"perf\" \"repeatCount\" \"--int\" 3\n \texport GIT_PERF_REPEAT_COUNT\n \n \tget_var_from_env_or_config \"GIT_PERF_DIRS_OR_REVS\" \"perf\" \"dirsOrRevs\"\n-- \n2.15.1.361.g8b07d831d0\n\n"},{"id":"335339","messageId":"20171226215908.425-7-chriscool@tuxfamily.org","threadId":"47496","inReplyTo":"20171226215908.425-1-chriscool@tuxfamily.org","subject":"[PATCH v2 6/7] perf/run: learn to send output to codespeed server","fromName":"Christian Couder","fromEmail":"christian.couder@gmail.com","sentAt":"2017-12-26T21:59:07Z","receivedAt":"2017-12-26T21:59:36Z","isPatch":true,"sender":{"key":"christian.couder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/208954?v=4"},"body":"Let's make it possible to set in a config file the URL of\na codespeed server. And then let's make the `run` script\nsend the perf test results to this URL at the end of the\ntests.\n\nThis should make is possible to easily automate the process\nof running perf tests and having their results available in\nCodespeed.\n\nSigned-off-by: Christian Couder <chriscool@tuxfamily.org>\n---\n t/perf/run | 12 +++++++++++-\n 1 file changed, 11 insertions(+), 1 deletion(-)\n\ndiff --git a/t/perf/run b/t/perf/run\nindex 4e62d6bb3f..ef56396546 100755\n--- a/t/perf/run\n+++ b/t/perf/run\n@@ -148,10 +148,20 @@ run_subsection () {\n \ttest \"$GIT_PERF_CODESPEED_OUTPUT\" = \"true\" && codespeed_opt=\"--codespeed\"\n \n \trun_dirs \"$@\"\n-\t./aggregate.perl $codespeed_opt \"$@\"\n+\n+\tif test -z \"$GIT_PERF_SEND_TO_CODESPEED\"\n+\tthen\n+\t\t./aggregate.perl $codespeed_opt \"$@\"\n+\telse\n+\t\tjson_res_file=\"test-results/$GIT_PERF_SUBSECTION/aggregate.json\"\n+\t\t./aggregate.perl --codespeed \"$@\" | tee \"$json_res_file\"\n+\t\tsend_data_url=\"$GIT_PERF_SEND_TO_CODESPEED/result/add/json/\"\n+\t\tcurl -v --request POST --data-urlencode \"json=$(cat \"$json_res_file\")\" \"$send_data_url\"\n+\tfi\n }\n \n get_var_from_env_or_config \"GIT_PERF_CODESPEED_OUTPUT\" \"perf\" \"codespeedOutput\" \"--bool\"\n+get_var_from_env_or_config \"GIT_PERF_SEND_TO_CODESPEED\" \"perf\" \"sendToCodespeed\"\n \n cd \"$(dirname $0)\"\n . ../../GIT-BUILD-OPTIONS\n-- \n2.15.1.361.g8b07d831d0\n\n"},{"id":"335340","messageId":"CAP8UFD2ryn_qHDGeoqSKhH+j-EosCPmMezXHDQTP9Us-3rfUyw@mail.gmail.com","threadId":"47496","inReplyTo":"20171226215908.425-1-chriscool@tuxfamily.org","subject":"Re: [PATCH v2 0/7] Codespeed perf results","fromName":"Christian Couder","fromEmail":"christian.couder@gmail.com","sentAt":"2017-12-26T22:07:45Z","receivedAt":"2017-12-26T22:07:53Z","isPatch":true,"sender":{"key":"christian.couder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/208954?v=4"},"body":"On Tue, Dec 26, 2017 at 10:59 PM, Christian Couder\n<christian.couder@gmail.com> wrote:\n\n> Changes since previous version\n> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n>\n>   - Fixed the way the 'executable' field sent to Codespeed is set in\n>     `perf/aggregate.perl` in patch 3/7. We now use `uname -s -m` to\n>     which we concatenate the perf.repoName config value if it is set,\n>     as suggested by Junio and Eric.\n>\n>   - Fixed the name of the GIT_PERF_REPO_NAME variable in patches 3/7\n>     and 7/7. It was GIT_TEST_REPO_NAME in some places.\n>\n>   - Fixed how the conf_opt argument is added to the\n>     get_var_from_env_or_config() function in patch 4/7. It was added\n>     as the last, so fifth, argument, but it is simpler and makes more\n>     sense to keep the default value argument as the last argument, so\n>     now the conf_opt argument is added as the fourth argument.\n>\n>   - What patch 4/7 did was previously done in 2 patches in the\n>     previous version (patches 4/8 and 5/8), but as we are not doing\n>     exactly the same thing (see the above item) it is simpler to do it\n>     in only one patch instead of two.\n>\n>   - We now use the --int type specifier when getting the\n>     perf.repeatCount config variable in patch 4/7.\n\nHere is the diff with v1:\n\ndiff --git a/t/perf/aggregate.perl b/t/perf/aggregate.perl\nindex 6e15f62a9e..34d74fc015 100755\n--- a/t/perf/aggregate.perl\n+++ b/t/perf/aggregate.perl\n@@ -186,17 +186,16 @@ sub print_codespeed_results {\n\n     my $project = \"Git\";\n\n-    my $executable;\n-    if ($results_section eq \"\") {\n-        $executable = `uname -o -p`;\n-    } else {\n-        $executable = $results_section;\n-        chomp $executable;\n+    my $executable = `uname -s -m`;\n+    chomp $executable;\n+\n+    if ($results_section ne \"\") {\n+        $executable .= \", \" . $results_section;\n     }\n\n     my $environment;\n-    if (exists $ENV{GIT_TEST_REPO_NAME} and $ENV{GIT_TEST_REPO_NAME} ne \"\") {\n-        $environment = $ENV{GIT_TEST_REPO_NAME};\n+    if (exists $ENV{GIT_PERF_REPO_NAME} and $ENV{GIT_PERF_REPO_NAME} ne \"\") {\n+        $environment = $ENV{GIT_PERF_REPO_NAME};\n     } elsif (exists $ENV{GIT_TEST_INSTALLED} and\n$ENV{GIT_TEST_INSTALLED} ne \"\") {\n         $environment = $ENV{GIT_TEST_INSTALLED};\n         $environment =~ s|/bin-wrappers$||;\ndiff --git a/t/perf/run b/t/perf/run\nindex 279c2d41f6..1a100d6134 100755\n--- a/t/perf/run\n+++ b/t/perf/run\n@@ -105,8 +105,8 @@ get_var_from_env_or_config () {\n     env_var=\"$1\"\n     conf_sec=\"$2\"\n     conf_var=\"$3\"\n-    default_value=\"$4\" # optional\n-    conf_opts=\"$5\" # optional\n+    conf_opts=\"$4\" # optional\n+    # $5 can be set to a default value\n\n     # Do nothing if the env variable is already set\n     eval \"test -z \\\"\\${$env_var+x}\\\"\" || return\n@@ -124,11 +124,11 @@ get_var_from_env_or_config () {\n     conf_value=$(git config $conf_opts -f \"$GIT_PERF_CONFIG_FILE\" \"$var\") &&\n     eval \"$env_var=\\\"$conf_value\\\"\" && return\n\n-    test -n \"${default_value+x}\" && eval \"$env_var=\\\"$default_value\\\"\"\n+    test -n \"${5+x}\" && eval \"$env_var=\\\"$5\\\"\"\n }\n\n run_subsection () {\n-    get_var_from_env_or_config \"GIT_PERF_REPEAT_COUNT\" \"perf\" \"repeatCount\" 3\n+    get_var_from_env_or_config \"GIT_PERF_REPEAT_COUNT\" \"perf\"\n\"repeatCount\" \"--int\" 3\n     export GIT_PERF_REPEAT_COUNT\n\n     get_var_from_env_or_config \"GIT_PERF_DIRS_OR_REVS\" \"perf\" \"dirsOrRevs\"\n@@ -137,7 +137,7 @@ run_subsection () {\n     get_var_from_env_or_config \"GIT_PERF_MAKE_COMMAND\" \"perf\" \"makeCommand\"\n     get_var_from_env_or_config \"GIT_PERF_MAKE_OPTS\" \"perf\" \"makeOpts\"\n\n-    get_var_from_env_or_config \"GIT_TEST_REPO_NAME\" \"perf\" \"repoName\"\n+    get_var_from_env_or_config \"GIT_PERF_REPO_NAME\" \"perf\" \"repoName\"\n     export GIT_PERF_REPO_NAME\n\n     GIT_PERF_AGGREGATING_LATER=t\n@@ -163,7 +163,7 @@ run_subsection () {\n     fi\n }\n\n-get_var_from_env_or_config \"GIT_PERF_CODESPEED_OUTPUT\" \"perf\"\n\"codespeedOutput\" \"\" --bool\n+get_var_from_env_or_config \"GIT_PERF_CODESPEED_OUTPUT\" \"perf\"\n\"codespeedOutput\" \"--bool\"\n get_var_from_env_or_config \"GIT_PERF_SEND_TO_CODESPEED\" \"perf\"\n\"sendToCodespeed\"\n\n cd \"$(dirname $0)\"\n"},{"id":"335482","messageId":"xmqqtvwak5fn.fsf@gitster.mtv.corp.google.com","threadId":"47496","inReplyTo":"20171226215908.425-1-chriscool@tuxfamily.org","subject":"Re: [PATCH v2 0/7] Codespeed perf results","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-12-28T19:39:40Z","receivedAt":"2017-12-28T19:39:47Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Christian Couder <christian.couder@gmail.com> writes:\n\n> This patch series is built on top of cc/perf-run-config which recently\n> graduated to master.\n>\n> It makes it possible to send perf results to a Codespeed server. See\n> https://github.com/tobami/codespeed/ and web sites like\n> http://speed.pypy.org/ which are using Codespeed.\n\nThanks for an update.  Will queue.\n"},{"id":"335483","messageId":"xmqqpo6yk5dh.fsf@gitster.mtv.corp.google.com","threadId":"47496","inReplyTo":"20171226215908.425-2-chriscool@tuxfamily.org","subject":"Re: [PATCH v2 1/7] perf/aggregate: fix checking ENV{GIT_PERF_SUBSECTION}","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-12-28T19:40:58Z","receivedAt":"2017-12-28T19:41:05Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Christian Couder <christian.couder@gmail.com> writes:\n\n> Signed-off-by: Christian Couder <chriscool@tuxfamily.org>\n> ---\n>  t/perf/aggregate.perl | 2 +-\n>  1 file changed, 1 insertion(+), 1 deletion(-)\n>\n> diff --git a/t/perf/aggregate.perl b/t/perf/aggregate.perl\n> index e401208488..769d418708 100755\n> --- a/t/perf/aggregate.perl\n> +++ b/t/perf/aggregate.perl\n> @@ -70,7 +70,7 @@ if (not @tests) {\n>  }\n>  \n>  my $resultsdir = \"test-results\";\n> -if ($ENV{GIT_PERF_SUBSECTION} ne \"\") {\n> +if (exists $ENV{GIT_PERF_SUBSECTION} and $ENV{GIT_PERF_SUBSECTION} ne \"\") {\n>  \t$resultsdir .= \"/\" . $ENV{GIT_PERF_SUBSECTION};\n>  }\n\nIt appears that this is the only remaining environment variable\nreference that could trigger comparison between undef and \"\" that\nmay be flagged by use of strict & warnings?  A good change.\n\n"},{"id":"335484","messageId":"xmqqlghmk52j.fsf@gitster.mtv.corp.google.com","threadId":"47496","inReplyTo":"20171226215908.425-4-chriscool@tuxfamily.org","subject":"Re: [PATCH v2 3/7] perf/aggregate: implement codespeed JSON output","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-12-28T19:47:32Z","receivedAt":"2017-12-28T19:47:39Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Christian Couder <christian.couder@gmail.com> writes:\n\n> ...\n> +sub print_codespeed_results {\n> +\tmy ($results_section) = @_;\n> +\n> +\t#use Data::Dumper qw/ Dumper /;\n> +\t#print Dumper(\\@data);\n\nPerhaps lose these.  It is OK to keep the code live and hide it\nbehind --debug or something, but it seems to me that you are done\ndebugging it pretty much so...\n\n> +\tuse JSON;\n\nHave this at the top, perhaps?  It's not like this would let us\navoid loading JSON module when not doing codespeed (you need\n'require' for that, right?).\n\n> +\tprint to_json(\\@data, {utf8 => 1, pretty => 1}), \"\\n\";\n> +}\n> +\n>  binmode STDOUT, \":utf8\" or die \"PANIC on binmode: $!\";\n>  \n> -print_default_results();\n> +if ($codespeed) {\n> +\tprint_codespeed_results($results_section);\n> +} else {\n> +\tprint_default_results();\n> +}\n"}]}