{"thread":{"id":"45954","subject":"[PATCH 0/5] p0004: support being called by t/perf/run","startedAt":"2017-05-13T15:55:22Z","lastAt":"2017-05-15T13:14:44Z","messageCount":7,"participants":["René Scharfe","Jeff Hostetler"],"isPatch":true,"patchVersion":1,"patchTotal":5},"messages":[{"id":"319619","messageId":"9b574d73-8dfb-f0b3-dc3e-d593726a8f0e@web.de","threadId":"45954","inReplyTo":null,"subject":"[PATCH 0/5] p0004: support being called by t/perf/run","fromName":"René Scharfe","fromEmail":"l.s.r@web.de","sentAt":"2017-05-13T15:55:05Z","receivedAt":"2017-05-13T15:55:22Z","isPatch":true,"sender":{"key":"l.s.r@web.de","avatar":"https://avatars.githubusercontent.com/u/26122331?v=4"},"body":"p0004-lazy-init-name-hash.sh errors out if the test repo is too small,\nand doesn't generate any perf test results even if it finishes\nsuccessfully.  That prevents t/perf/run from running the whole test\nsuite.  This series tries to address these issues.\n\n  p0004: simplify calls of test-lazy-init-name-hash\n  p0004: avoid using pipes\n  p0004: use test_perf\n  p0004: don't abort if multi-threaded is too slow\n  p0004: don't error out if test repo is too small\n\n t/perf/p0004-lazy-init-name-hash.sh | 47 +++++++++++++++++++++++++++++++++----\n 1 file changed, 42 insertions(+), 5 deletions(-)\n\n-- \n2.12.2\n"},{"id":"319620","messageId":"02f4218d-857a-3615-cc3a-d03ecbf35e67@web.de","threadId":"45954","inReplyTo":"9b574d73-8dfb-f0b3-dc3e-d593726a8f0e@web.de","subject":"[PATCH 1/5] p0004: simplify calls of test-lazy-init-name-hash","fromName":"René Scharfe","fromEmail":"l.s.r@web.de","sentAt":"2017-05-13T15:59:46Z","receivedAt":"2017-05-13T16:00:01Z","isPatch":true,"sender":{"key":"l.s.r@web.de","avatar":"https://avatars.githubusercontent.com/u/26122331?v=4"},"body":"The test library puts helpers into $PATH, so we can simply call them\nwithout specifying their location.\n\nThe suffix $X is also not necessary because .exe files on Windows can be\nstarted without specifying their extension, and on other platforms it's\nempty anyway.\n\nSigned-off-by: Rene Scharfe <l.s.r@web.de>\n---\n t/perf/p0004-lazy-init-name-hash.sh | 6 +++---\n 1 file changed, 3 insertions(+), 3 deletions(-)\n\ndiff --git a/t/perf/p0004-lazy-init-name-hash.sh b/t/perf/p0004-lazy-init-name-hash.sh\nindex 5afa8c8df3..716c951553 100755\n--- a/t/perf/p0004-lazy-init-name-hash.sh\n+++ b/t/perf/p0004-lazy-init-name-hash.sh\n@@ -7,13 +7,13 @@ test_perf_large_repo\n test_checkout_worktree\n \n test_expect_success 'verify both methods build the same hashmaps' '\n-\t$GIT_BUILD_DIR/t/helper/test-lazy-init-name-hash$X --dump --single | sort >out.single &&\n-\t$GIT_BUILD_DIR/t/helper/test-lazy-init-name-hash$X --dump --multi  | sort >out.multi  &&\n+\ttest-lazy-init-name-hash --dump --single | sort >out.single &&\n+\ttest-lazy-init-name-hash --dump --multi  | sort >out.multi  &&\n \ttest_cmp out.single out.multi\n '\n \n test_expect_success 'multithreaded should be faster' '\n-\t$GIT_BUILD_DIR/t/helper/test-lazy-init-name-hash$X --perf >out.perf\n+\ttest-lazy-init-name-hash --perf >out.perf\n '\n \n test_done\n-- \n2.12.2\n\n"},{"id":"319621","messageId":"f6d4d434-1f55-ef7c-3c7e-55b56fe96361@web.de","threadId":"45954","inReplyTo":"9b574d73-8dfb-f0b3-dc3e-d593726a8f0e@web.de","subject":"[PATCH 2/5] p0004: avoid using pipes","fromName":"René Scharfe","fromEmail":"l.s.r@web.de","sentAt":"2017-05-13T16:00:15Z","receivedAt":"2017-05-13T16:00:29Z","isPatch":true,"sender":{"key":"l.s.r@web.de","avatar":"https://avatars.githubusercontent.com/u/26122331?v=4"},"body":"The return code of commands on the producing end of a pipe is ignored.\nEvaluate the outcome of test-lazy-init-name-hash by calling sort\nseparately.\n\nSigned-off-by: Rene Scharfe <l.s.r@web.de>\n---\n t/perf/p0004-lazy-init-name-hash.sh | 8 +++++---\n 1 file changed, 5 insertions(+), 3 deletions(-)\n\ndiff --git a/t/perf/p0004-lazy-init-name-hash.sh b/t/perf/p0004-lazy-init-name-hash.sh\nindex 716c951553..576bdc3c4e 100755\n--- a/t/perf/p0004-lazy-init-name-hash.sh\n+++ b/t/perf/p0004-lazy-init-name-hash.sh\n@@ -7,9 +7,11 @@ test_perf_large_repo\n test_checkout_worktree\n \n test_expect_success 'verify both methods build the same hashmaps' '\n-\ttest-lazy-init-name-hash --dump --single | sort >out.single &&\n-\ttest-lazy-init-name-hash --dump --multi  | sort >out.multi  &&\n-\ttest_cmp out.single out.multi\n+\ttest-lazy-init-name-hash --dump --single >out.single &&\n+\ttest-lazy-init-name-hash --dump --multi >out.multi &&\n+\tsort <out.single >sorted.single &&\n+\tsort <out.multi >sorted.multi &&\n+\ttest_cmp sorted.single sorted.multi\n '\n \n test_expect_success 'multithreaded should be faster' '\n-- \n2.12.2\n\n"},{"id":"319622","messageId":"5d8cff82-c695-ea6a-4d82-448dc10b8b63@web.de","threadId":"45954","inReplyTo":"9b574d73-8dfb-f0b3-dc3e-d593726a8f0e@web.de","subject":"[PATCH 3/5] p0004: use test_perf","fromName":"René Scharfe","fromEmail":"l.s.r@web.de","sentAt":"2017-05-13T16:02:32Z","receivedAt":"2017-05-13T16:02:46Z","isPatch":true,"sender":{"key":"l.s.r@web.de","avatar":"https://avatars.githubusercontent.com/u/26122331?v=4"},"body":"The perf test suite (more specifically: t/perf/aggregate.perl) requires\neach test script to write test results into a file, otherwise it aborts\nwhen aggregating.  Add actual performance tests with test_perf to allow\np0004 to be run together with other perf scripts.\n\nCalibrate the value for the parameter --count based on the size of the\ntest repository, in order to get meaningful results with smaller repos\nyet still be able to finish the script against huge ones without having\nto wait for hours.\n\nSigned-off-by: Rene Scharfe <l.s.r@web.de>\n---\nThe numbers are just guesses; I didn't actually test all ranges.\n\n t/perf/p0004-lazy-init-name-hash.sh | 36 ++++++++++++++++++++++++++++++++++++\n 1 file changed, 36 insertions(+)\n\ndiff --git a/t/perf/p0004-lazy-init-name-hash.sh b/t/perf/p0004-lazy-init-name-hash.sh\nindex 576bdc3c4e..d30c32f97b 100755\n--- a/t/perf/p0004-lazy-init-name-hash.sh\n+++ b/t/perf/p0004-lazy-init-name-hash.sh\n@@ -18,4 +18,40 @@ test_expect_success 'multithreaded should be faster' '\n \ttest-lazy-init-name-hash --perf >out.perf\n '\n \n+test_expect_success 'calibrate' '\n+\tentries=$(wc -l <out.single) &&\n+\n+\tcase $entries in\n+\t?) count=1000000 ;;\n+\t??) count=100000 ;;\n+\t???) count=10000 ;;\n+\t????) count=1000 ;;\n+\t?????) count=100 ;;\n+\t??????) count=10 ;;\n+\t*) count=1 ;;\n+\tesac &&\n+\texport count &&\n+\n+\tcase $entries in\n+\t1) entries_desc=\"1 entry\" ;;\n+\t*) entries_desc=\"$entries entries\" ;;\n+\tesac &&\n+\n+\tcase $count in\n+\t1) count_desc=\"1 round\" ;;\n+\t*) count_desc=\"$count rounds\" ;;\n+\tesac &&\n+\n+\tdesc=\"$entries_desc, $count_desc\" &&\n+\texport desc\n+'\n+\n+test_perf \"single-threaded, $desc\" \"\n+\ttest-lazy-init-name-hash --single --count=$count\n+\"\n+\n+test_perf \"multi-threaded, $desc\" \"\n+\ttest-lazy-init-name-hash --multi --count=$count\n+\"\n+\n test_done\n-- \n2.12.2\n\n"},{"id":"319623","messageId":"871f5f29-feb3-5c69-7ddd-f92819a76cce@web.de","threadId":"45954","inReplyTo":"9b574d73-8dfb-f0b3-dc3e-d593726a8f0e@web.de","subject":"[PATCH 4/5] p0004: don't abort if multi-threaded is too slow","fromName":"René Scharfe","fromEmail":"l.s.r@web.de","sentAt":"2017-05-13T16:03:22Z","receivedAt":"2017-05-13T16:03:40Z","isPatch":true,"sender":{"key":"l.s.r@web.de","avatar":"https://avatars.githubusercontent.com/u/26122331?v=4"},"body":"If the single-threaded variant beats the multi-threaded one then we may\nhave a performance bug, but that doesn't justify aborting the test.\nDrop that check; we can compare the results for --single and --multi\nusing the actual performance tests.\n\nSigned-off-by: Rene Scharfe <l.s.r@web.de>\n---\n t/perf/p0004-lazy-init-name-hash.sh | 4 ----\n 1 file changed, 4 deletions(-)\n\ndiff --git a/t/perf/p0004-lazy-init-name-hash.sh b/t/perf/p0004-lazy-init-name-hash.sh\nindex d30c32f97b..3c2135a185 100755\n--- a/t/perf/p0004-lazy-init-name-hash.sh\n+++ b/t/perf/p0004-lazy-init-name-hash.sh\n@@ -14,10 +14,6 @@ test_expect_success 'verify both methods build the same hashmaps' '\n \ttest_cmp sorted.single sorted.multi\n '\n \n-test_expect_success 'multithreaded should be faster' '\n-\ttest-lazy-init-name-hash --perf >out.perf\n-'\n-\n test_expect_success 'calibrate' '\n \tentries=$(wc -l <out.single) &&\n \n-- \n2.12.2\n\n"},{"id":"319624","messageId":"ca844223-bb04-8885-2f33-2c3e9c5a5e3c@web.de","threadId":"45954","inReplyTo":"9b574d73-8dfb-f0b3-dc3e-d593726a8f0e@web.de","subject":"[PATCH 5/5] p0004: don't error out if test repo is too small","fromName":"René Scharfe","fromEmail":"l.s.r@web.de","sentAt":"2017-05-13T16:03:50Z","receivedAt":"2017-05-13T16:04:04Z","isPatch":true,"sender":{"key":"l.s.r@web.de","avatar":"https://avatars.githubusercontent.com/u/26122331?v=4"},"body":"Repositories with less than 4000 entries are always handled using a\nsingle thread, causing test-lazy-init-name-hash --multi to error out.\nDon't abort the whole test script in that case, but simply skip the\nmulti-threaded performance check.  We can still use it to compare the\nsingle-threaded speed of different versions in that case.\n\nSigned-off-by: Rene Scharfe <l.s.r@web.de>\n---\n t/perf/p0004-lazy-init-name-hash.sh | 13 ++++++++-----\n 1 file changed, 8 insertions(+), 5 deletions(-)\n\ndiff --git a/t/perf/p0004-lazy-init-name-hash.sh b/t/perf/p0004-lazy-init-name-hash.sh\nindex 3c2135a185..8de5a98cfc 100755\n--- a/t/perf/p0004-lazy-init-name-hash.sh\n+++ b/t/perf/p0004-lazy-init-name-hash.sh\n@@ -8,10 +8,13 @@ test_checkout_worktree\n \n test_expect_success 'verify both methods build the same hashmaps' '\n \ttest-lazy-init-name-hash --dump --single >out.single &&\n-\ttest-lazy-init-name-hash --dump --multi >out.multi &&\n-\tsort <out.single >sorted.single &&\n-\tsort <out.multi >sorted.multi &&\n-\ttest_cmp sorted.single sorted.multi\n+\tif test-lazy-init-name-hash --dump --multi >out.multi\n+\tthen\n+\t\ttest_set_prereq REPO_BIG_ENOUGH_FOR_MULTI &&\n+\t\tsort <out.single >sorted.single &&\n+\t\tsort <out.multi >sorted.multi &&\n+\t\ttest_cmp sorted.single sorted.multi\n+\tfi\n '\n \n test_expect_success 'calibrate' '\n@@ -46,7 +49,7 @@ test_perf \"single-threaded, $desc\" \"\n \ttest-lazy-init-name-hash --single --count=$count\n \"\n \n-test_perf \"multi-threaded, $desc\" \"\n+test_perf REPO_BIG_ENOUGH_FOR_MULTI \"multi-threaded, $desc\" \"\n \ttest-lazy-init-name-hash --multi --count=$count\n \"\n \n-- \n2.12.2\n\n"},{"id":"319783","messageId":"543cd794-273f-9d11-b4cd-4d8dc8b6df96@jeffhostetler.com","threadId":"45954","inReplyTo":"9b574d73-8dfb-f0b3-dc3e-d593726a8f0e@web.de","subject":"Re: [PATCH 0/5] p0004: support being called by t/perf/run","fromName":"Jeff Hostetler","fromEmail":"git@jeffhostetler.com","sentAt":"2017-05-15T13:14:37Z","receivedAt":"2017-05-15T13:14:44Z","isPatch":true,"sender":{"key":"git@jeffhostetler.com","avatar":null},"body":"\n\nOn 5/13/2017 11:55 AM, René Scharfe wrote:\n> p0004-lazy-init-name-hash.sh errors out if the test repo is too small,\n> and doesn't generate any perf test results even if it finishes\n> successfully.  That prevents t/perf/run from running the whole test\n> suite.  This series tries to address these issues.\n>\n>   p0004: simplify calls of test-lazy-init-name-hash\n>   p0004: avoid using pipes\n>   p0004: use test_perf\n>   p0004: don't abort if multi-threaded is too slow\n>   p0004: don't error out if test repo is too small\n>\n>  t/perf/p0004-lazy-init-name-hash.sh | 47 +++++++++++++++++++++++++++++++++----\n>  1 file changed, 42 insertions(+), 5 deletions(-)\n>\n\n\nNicely done!  Thanks for cleaning this up.\nJeff\n"}]}