{"thread":{"id":"48148","subject":"[PATCH v2 0/2] perf/aggregate: sort results by regression","startedAt":"2018-03-26T07:25:01Z","lastAt":"2018-03-26T07:25:06Z","messageCount":3,"participants":["Christian Couder"],"isPatch":true,"patchVersion":2,"patchTotal":2},"messages":[{"id":"342967","messageId":"20180326072431.30771-1-chriscool@tuxfamily.org","threadId":"48148","inReplyTo":null,"subject":"[PATCH v2 0/2] perf/aggregate: sort results by regression","fromName":"Christian Couder","fromEmail":"christian.couder@gmail.com","sentAt":"2018-03-26T07:24:29Z","receivedAt":"2018-03-26T07:25:01Z","isPatch":true,"sender":{"key":"christian.couder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/208954?v=4"},"body":"This small patch series makes it easy to spot big performance\nregressions, so that they can later be investigated.\n\nFor example:\n\n$ ./aggregate.perl --sort-by=regression --subsection \"without libpcre\" v2.14.3 v2.15.1 v2.16.2 p4220-log-grep-engines.sh \n+5.0% p4220-log-grep-engines.2 0.60(0.58+0.02) 0.63(0.59+0.04) v2.14.3 v2.15.1\n+4.5% p4220-log-grep-engines.10 0.67(0.64+0.03) 0.70(0.67+0.02) v2.14.3 v2.15.1\n+1.7% p4220-log-grep-engines.5 0.58(0.57+0.01) 0.59(0.59+0.00) v2.14.3 v2.15.1\n+1.7% p4220-log-grep-engines.6 0.58(0.58+0.00) 0.59(0.56+0.02) v2.14.3 v2.15.1\n+1.7% p4220-log-grep-engines.17 0.58(0.57+0.01) 0.59(0.56+0.02) v2.14.3 v2.15.1\n+1.7% p4220-log-grep-engines.1 0.60(0.58+0.01) 0.61(0.60+0.01) v2.14.3 v2.15.1\n+1.6% p4220-log-grep-engines.13 0.64(0.63+0.02) 0.65(0.63+0.01) v2.14.3 v2.15.1\n+1.5% p4220-log-grep-engines.9 0.67(0.66+0.01) 0.68(0.67+0.01) v2.14.3 v2.15.1\n+0.0% p4220-log-grep-engines.14 0.65(0.62+0.02) 0.65(0.63+0.02) v2.14.3 v2.15.1\n+0.0% p4220-log-grep-engines.18 0.58(0.57+0.00) 0.58(0.56+0.02) v2.14.3 v2.15.1\n-1.5% p4220-log-grep-engines.13 0.65(0.63+0.01) 0.64(0.62+0.01) v2.15.1 v2.16.2\n-1.5% p4220-log-grep-engines.14 0.65(0.63+0.02) 0.64(0.60+0.03) v2.15.1 v2.16.2\n-1.6% p4220-log-grep-engines.1 0.61(0.60+0.01) 0.60(0.58+0.02) v2.15.1 v2.16.2\n-1.7% p4220-log-grep-engines.5 0.59(0.59+0.00) 0.58(0.55+0.02) v2.15.1 v2.16.2\n-1.7% p4220-log-grep-engines.6 0.59(0.56+0.02) 0.58(0.55+0.02) v2.15.1 v2.16.2\n-1.7% p4220-log-grep-engines.18 0.58(0.56+0.02) 0.57(0.55+0.02) v2.15.1 v2.16.2\n-2.9% p4220-log-grep-engines.9 0.68(0.67+0.01) 0.66(0.64+0.02) v2.15.1 v2.16.2\n-3.4% p4220-log-grep-engines.17 0.59(0.56+0.02) 0.57(0.55+0.01) v2.15.1 v2.16.2\n-4.3% p4220-log-grep-engines.10 0.70(0.67+0.02) 0.67(0.66+0.01) v2.15.1 v2.16.2\n-4.8% p4220-log-grep-engines.2 0.63(0.59+0.04) 0.60(0.57+0.03) v2.15.1 v2.16.2\n\nSince V1, the indent issues have been fixed and the name of the option\nhas been changed to --sort-by=regression (from --sortbyregression) as\nsuggested by Junio.\n\nChristian Couder (2):\n  perf/aggregate: add display_dir()\n  perf/aggregate: add --sort-by=regression option\n\n t/perf/aggregate.perl | 70 +++++++++++++++++++++++++++++++++++++++----\n 1 file changed, 65 insertions(+), 5 deletions(-)\n\n-- \n2.17.0.rc1.36.g098d832c9.dirty\n\n"},{"id":"342968","messageId":"20180326072431.30771-2-chriscool@tuxfamily.org","threadId":"48148","inReplyTo":"20180326072431.30771-1-chriscool@tuxfamily.org","subject":"[PATCH v2 1/2] perf/aggregate: add display_dir()","fromName":"Christian Couder","fromEmail":"christian.couder@gmail.com","sentAt":"2018-03-26T07:24:30Z","receivedAt":"2018-03-26T07:25:04Z","isPatch":true,"sender":{"key":"christian.couder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/208954?v=4"},"body":"This new helper function will be reused in a subsequent\ncommit.\n\nSigned-off-by: Christian Couder <chriscool@tuxfamily.org>\n---\n t/perf/aggregate.perl | 11 +++++++----\n 1 file changed, 7 insertions(+), 4 deletions(-)\n\ndiff --git a/t/perf/aggregate.perl b/t/perf/aggregate.perl\nindex 821cf1498b..890d85fa7b 100755\n--- a/t/perf/aggregate.perl\n+++ b/t/perf/aggregate.perl\n@@ -147,6 +147,11 @@ sub have_slash {\n \treturn 0;\n }\n \n+sub display_dir {\n+\tmy ($d) = @_;\n+\treturn exists $dirabbrevs{$d} ? $dirabbrevs{$d} : $dirnames{$d};\n+}\n+\n sub print_default_results {\n \tmy %descrs;\n \tmy $descrlen = 4; # \"Test\"\n@@ -168,8 +173,7 @@ sub print_default_results {\n \tmy %times;\n \tmy @colwidth = ((0)x@dirs);\n \tfor my $i (0..$#dirs) {\n-\t\tmy $d = $dirs[$i];\n-\t\tmy $w = length (exists $dirabbrevs{$d} ? $dirabbrevs{$d} : $dirnames{$d});\n+\t\tmy $w = length display_dir($dirs[$i]);\n \t\t$colwidth[$i] = $w if $w > $colwidth[$i];\n \t}\n \tfor my $t (@subtests) {\n@@ -188,8 +192,7 @@ sub print_default_results {\n \n \tprintf \"%-${descrlen}s\", \"Test\";\n \tfor my $i (0..$#dirs) {\n-\t\tmy $d = $dirs[$i];\n-\t\tprintf \"   %-$colwidth[$i]s\", (exists $dirabbrevs{$d} ? $dirabbrevs{$d} : $dirnames{$d});\n+\t\tprintf \"   %-$colwidth[$i]s\", display_dir($dirs[$i]);\n \t}\n \tprint \"\\n\";\n \tprint \"-\"x$totalwidth, \"\\n\";\n-- \n2.17.0.rc1.36.g098d832c9.dirty\n\n"},{"id":"342969","messageId":"20180326072431.30771-3-chriscool@tuxfamily.org","threadId":"48148","inReplyTo":"20180326072431.30771-1-chriscool@tuxfamily.org","subject":"[PATCH v2 2/2] perf/aggregate: add --sort-by=regression option","fromName":"Christian Couder","fromEmail":"christian.couder@gmail.com","sentAt":"2018-03-26T07:24:31Z","receivedAt":"2018-03-26T07:25:06Z","isPatch":true,"sender":{"key":"christian.couder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/208954?v=4"},"body":"One of the most interesting thing one can be interested in when\nlooking at performance test results is possible performance\nregressions.\n\nThis new option makes it easy to spot such possible regressions.\n\nThis new option is named '--sort-by=regression' to make it\npossible and easy to add other ways to sort the results, like for\nexample '--sort-by=utime'.\n\nIf we would like to sort according to how much the stime regressed\nwe could also add a new option called '--sort-by=regression:stime'.\nThen '--sort-by=regression' could become a synonym for\n'--sort-by=regression:rtime'.\n\nSigned-off-by: Christian Couder <chriscool@tuxfamily.org>\n---\n t/perf/aggregate.perl | 59 ++++++++++++++++++++++++++++++++++++++++++-\n 1 file changed, 58 insertions(+), 1 deletion(-)\n\ndiff --git a/t/perf/aggregate.perl b/t/perf/aggregate.perl\nindex 890d85fa7b..48637ef64b 100755\n--- a/t/perf/aggregate.perl\n+++ b/t/perf/aggregate.perl\n@@ -37,7 +37,7 @@ sub format_times {\n }\n \n my (@dirs, %dirnames, %dirabbrevs, %prefixes, @tests,\n-    $codespeed, $subsection, $reponame);\n+    $codespeed, $sortby, $subsection, $reponame);\n while (scalar @ARGV) {\n \tmy $arg = $ARGV[0];\n \tmy $dir;\n@@ -46,6 +46,18 @@ while (scalar @ARGV) {\n \t\tshift @ARGV;\n \t\tnext;\n \t}\n+\tif ($arg =~ /--sort-by(?:=(.*))?/) {\n+\t\tshift @ARGV;\n+\t\tif (defined $1) {\n+\t\t\t$sortby = $1;\n+\t\t} else {\n+\t\t\t$sortby = shift @ARGV;\n+\t\t\tif (! defined $sortby) {\n+\t\t\t\tdie \"'--sort-by' requires an argument\";\n+\t\t\t}\n+\t\t}\n+\t\tnext;\n+\t}\n \tif ($arg eq \"--subsection\") {\n \t\tshift @ARGV;\n \t\t$subsection = $ARGV[0];\n@@ -209,6 +221,49 @@ sub print_default_results {\n \t}\n }\n \n+sub print_sorted_results {\n+\tmy ($sortby) = @_;\n+\n+\tif ($sortby ne \"regression\") {\n+\t\tdie \"only 'regression' is supported as '--sort-by' argument\";\n+\t}\n+\n+\tmy @evolutions;\n+\tfor my $t (@subtests) {\n+\t\tmy ($prevr, $prevu, $prevs, $prevrev);\n+\t\tfor my $i (0..$#dirs) {\n+\t\t\tmy $d = $dirs[$i];\n+\t\t\tmy ($r, $u, $s) = get_times(\"$resultsdir/$prefixes{$d}$t.times\");\n+\t\t\tif ($i > 0 and defined $r and defined $prevr and $prevr > 0) {\n+\t\t\t\tmy $percent = 100.0 * ($r - $prevr) / $prevr;\n+\t\t\t\tpush @evolutions, { \"percent\"  => $percent,\n+\t\t\t\t\t\t    \"test\"     => $t,\n+\t\t\t\t\t\t    \"prevrev\"  => $prevrev,\n+\t\t\t\t\t\t    \"rev\"      => $d,\n+\t\t\t\t\t\t    \"prevr\"    => $prevr,\n+\t\t\t\t\t\t    \"r\"        => $r,\n+\t\t\t\t\t\t    \"prevu\"    => $prevu,\n+\t\t\t\t\t\t    \"u\"        => $u,\n+\t\t\t\t\t\t    \"prevs\"    => $prevs,\n+\t\t\t\t\t\t    \"s\"        => $s};\n+\t\t\t}\n+\t\t\t($prevr, $prevu, $prevs, $prevrev) = ($r, $u, $s, $d);\n+\t\t}\n+\t}\n+\n+\tmy @sorted_evolutions = sort { $b->{percent} <=> $a->{percent} } @evolutions;\n+\n+\tfor my $e (@sorted_evolutions) {\n+\t\tprintf \"%+.1f%%\", $e->{percent};\n+\t\tprint \" \" . $e->{test};\n+\t\tprint \" \" . format_times($e->{prevr}, $e->{prevu}, $e->{prevs});\n+\t\tprint \" \" . format_times($e->{r}, $e->{u}, $e->{s});\n+\t\tprint \" \" . display_dir($e->{prevrev});\n+\t\tprint \" \" . display_dir($e->{rev});\n+\t\tprint \"\\n\";\n+\t}\n+}\n+\n sub print_codespeed_results {\n \tmy ($subsection) = @_;\n \n@@ -263,6 +318,8 @@ binmode STDOUT, \":utf8\" or die \"PANIC on binmode: $!\";\n \n if ($codespeed) {\n \tprint_codespeed_results($subsection);\n+} elsif (defined $sortby) {\n+\tprint_sorted_results($sortby);\n } else {\n \tprint_default_results();\n }\n-- \n2.17.0.rc1.36.g098d832c9.dirty\n\n"}]}