{"thread":{"id":"45928","subject":"[PATCH v3 1/2] perf: add function to setup a fresh test repo","startedAt":"2017-05-11T07:44:30Z","lastAt":"2017-05-11T19:18:59Z","messageCount":7,"participants":["Ævar Arnfjörð Bjarmason","Jonathan Nieder"],"isPatch":true,"patchVersion":3,"patchTotal":2},"messages":[{"id":"319329","messageId":"20170511074354.11336-2-avarab@gmail.com","threadId":"45928","inReplyTo":"20170511074354.11336-1-avarab@gmail.com","subject":"[PATCH v3 1/2] perf: add function to setup a fresh test repo","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2017-05-11T07:43:53Z","receivedAt":"2017-05-11T07:44:30Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Add a function to setup a fresh test repo via 'git init' to compliment\nthe existing functions to copy over a normal & large repo.\n\nSome performance tests don't need any existing repository data at all\nto be significant, e.g. tests which stress glob matches against single\npathological revisions or files, which I'm about to add in a\nsubsequent commit.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n t/perf/README | 1 +\n t/perf/perf-lib.sh | 17 +++++++++++++----\n 2 files changed, 14 insertions(+), 4 deletions(-)\n\ndiff --git a/t/perf/README b/t/perf/README\nindex 49ea4349be..de2fe15696 100644\n--- a/t/perf/README\n+++ b/t/perf/README\n@@ -106,6 +106,7 @@ sources perf-lib.sh:\n \n After that you will want to use some of the following:\n \n+\ttest_perf_fresh_repo # sets up an empty repository\n \ttest_perf_default_repo # sets up a \"normal\" repository\n \ttest_perf_large_repo # sets up a \"large\" repository\n \ndiff --git a/t/perf/perf-lib.sh b/t/perf/perf-lib.sh\nindex ab4b8b06ae..f51fc773e8 100644\n--- a/t/perf/perf-lib.sh\n+++ b/t/perf/perf-lib.sh\n@@ -78,6 +78,10 @@ if test -z \"$GIT_PERF_LARGE_REPO\"; then\n \tGIT_PERF_LARGE_REPO=$TEST_DIRECTORY/..\n fi\n \n+test_perf_do_repo_symlink_config_ () {\n+\ttest_have_prereq SYMLINKS || git config core.symlinks false\n+}\n+\n test_perf_create_repo_from () {\n \ttest \"$#\" = 2 ||\n \terror \"bug in the test script: not 2 parameters to test-create-repo\"\n@@ -102,15 +106,20 @@ test_perf_create_repo_from () {\n \t) &&\n \t(\n \t\tcd \"$repo\" &&\n-\t\t\"$MODERN_GIT\" init -q && {\n-\t\t\ttest_have_prereq SYMLINKS ||\n-\t\t\tgit config core.symlinks false\n-\t\t} &&\n+\t\t\"$MODERN_GIT\" init -q &&\n+\t\ttest_perf_do_repo_symlink_config_ &&\n \t\tmv .git/hooks .git/hooks-disabled 2>/dev/null\n \t) || error \"failed to copy repository '$source' to '$repo'\"\n }\n \n # call at least one of these to establish an appropriately-sized repository\n+test_perf_fresh_repo () {\n+\trepo=\"${1:-$TRASH_DIRECTORY}\"\n+\t\"$MODERN_GIT\" init -q \"$repo\" &&\n+\tcd \"$repo\" &&\n+\ttest_perf_do_repo_symlink_config_\n+}\n+\n test_perf_default_repo () {\n \ttest_perf_create_repo_from \"${1:-$TRASH_DIRECTORY}\" \"$GIT_PERF_REPO\"\n }\n-- \n2.11.0\n\n"},{"id":"319330","messageId":"20170511074354.11336-3-avarab@gmail.com","threadId":"45928","inReplyTo":"20170511074354.11336-1-avarab@gmail.com","subject":"[PATCH v3 2/2] perf: add test showing exponential growth in path globbing","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2017-05-11T07:43:54Z","receivedAt":"2017-05-11T07:44:34Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Add a test showing that runtimes of the wildmatch() function used for\nglobbing in git grow exponentially in the face of some pathological\nglobs.\n\nThis issue affects both globs matching filenames via e.g. ls-files,\nand globs matching refnames via e.g. for-each-ref.\n\nAs noted in the test description this is a test to see whether Git\nsuffers from the issue noted in an article Russ Cox posted today about\ncommon bugs in various glob implementations:\nhttps://research.swtch.com/glob\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n t/perf/p0100-globbing.sh | 43 +++++++++++++++++++++++++++++++++++++++++++\n 1 file changed, 43 insertions(+)\n create mode 100755 t/perf/p0100-globbing.sh\n\ndiff --git a/t/perf/p0100-globbing.sh b/t/perf/p0100-globbing.sh\nnew file mode 100755\nindex 0000000000..dd18a9ce2b\n--- /dev/null\n+++ b/t/perf/p0100-globbing.sh\n@@ -0,0 +1,43 @@\n+#!/bin/sh\n+\n+test_description=\"Tests pathological globbing performance\n+\n+Shows how Git's globbing performance performs when given the sort of\n+pathological patterns described in at https://research.swtch.com/glob\n+\"\n+\n+. ./perf-lib.sh\n+\n+test_globs_big='10 25 50 75 100'\n+test_globs_small='1 2 3 4 5 6'\n+\n+test_perf_fresh_repo\n+\n+test_expect_success 'setup' '\n+\tfor i in $(test_seq 1 100)\n+\tdo\n+\t\tprintf \"a\" >>refname &&\n+\t\tfor j in $(test_seq 1 $i)\n+\t\tdo\n+\t\t\tprintf \"a*\" >>refglob.$i\n+\t\tdone &&\n+\t\techo b >>refglob.$i\n+\tdone &&\n+\ttest_commit test $(cat refname).t \"\" $(cat refname).t\n+'\n+\n+for i in $test_globs_small\n+do\n+\ttest_perf \"refglob((a*)^nb) against tag (a^100).t; n = $i\" '\n+\t\tgit for-each-ref \"refs/tags/$(cat refglob.'$i')b\"\n+\t'\n+done\n+\n+for i in $test_globs_small\n+do\n+\ttest_perf \"fileglob((a*)^nb) against file (a^100).t; n = $i\" '\n+\t\tgit ls-files \"$(cat refglob.'$i')b\"\n+\t'\n+done\n+\n+test_done\n-- \n2.11.0\n\n"},{"id":"319331","messageId":"20170511074354.11336-1-avarab@gmail.com","threadId":"45928","inReplyTo":null,"subject":"[PATCH v3 0/2] perf: show that wildmatch() regressed for pathological cases in v2.0","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2017-05-11T07:43:52Z","receivedAt":"2017-05-11T07:44:49Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Fixes the issues noted in v3, see\n<20170510225316.31680-1-avarab@gmail.com>\n(https://public-inbox.org/git/20170510225316.31680-1-avarab@gmail.com/).\n\nIn addition I was wrong about for-each-ref not being subjected to this\nslowdown, I was just screwing up the testcase. Fix that. Now:\n\n $ GIT_PERF_REPEAT_COUNT=1 GIT_PERF_MAKE_OPTS=\"-j6 NO_OPENSSL=Y NO_WILDMATCH=YesPlease\" ./run v1.9.5 v2.12.0 p0100-globbing.sh\n Test v1.9.5 v2.12.0\n -------------------------------------------------------------------------------------------------\n 0100.2: refglob((a*)^nb) against tag (a^100).t; n = 1 0.00(0.00+0.00) 0.00(0.00+0.00) =\n 0100.3: refglob((a*)^nb) against tag (a^100).t; n = 2 0.00(0.00+0.00) 0.00(0.00+0.00) =\n 0100.4: refglob((a*)^nb) against tag (a^100).t; n = 3 0.00(0.00+0.00) 0.00(0.00+0.00) =\n 0100.5: refglob((a*)^nb) against tag (a^100).t; n = 4 0.00(0.00+0.00) 0.01(0.00+0.00) +inf\n 0100.6: refglob((a*)^nb) against tag (a^100).t; n = 5 0.00(0.00+0.00) 0.16(0.15+0.00) +inf\n 0100.7: refglob((a*)^nb) against tag (a^100).t; n = 6 0.00(0.00+0.00) 2.73(2.71+0.00) +inf\n 0100.8: fileglob((a*)^nb) against file (a^100).t; n = 1 0.00(0.00+0.00) 0.00(0.00+0.00) =\n 0100.9: fileglob((a*)^nb) against file (a^100).t; n = 2 0.00(0.00+0.00) 0.00(0.00+0.00) =\n 0100.10: fileglob((a*)^nb) against file (a^100).t; n = 3 0.00(0.00+0.00) 0.00(0.00+0.00) =\n 0100.11: fileglob((a*)^nb) against file (a^100).t; n = 4 0.00(0.00+0.00) 0.01(0.00+0.00) +inf\n 0100.12: fileglob((a*)^nb) against file (a^100).t; n = 5 0.00(0.00+0.00) 0.16(0.15+0.00) +inf\n 0100.13: fileglob((a*)^nb) against file (a^100).t; n = 6 0.00(0.00+0.00) 2.75(2.73+0.00) +inf\n\nÆvar Arnfjörð Bjarmason (2):\n perf: add function to setup a fresh test repo\n perf: add test showing exponential growth in path globbing\n\n t/perf/README | 1 +\n t/perf/p0100-globbing.sh | 43 +++++++++++++++++++++++++++++++++++++++++++\n t/perf/perf-lib.sh | 17 +++++++++++++----\n 3 files changed, 57 insertions(+), 4 deletions(-)\n create mode 100755 t/perf/p0100-globbing.sh\n\n-- \n2.11.0\n\n"},{"id":"319375","messageId":"20170511094108.8756-1-avarab@gmail.com","threadId":"45928","inReplyTo":"20170511074354.11336-1-avarab@gmail.com","subject":"[PATCH v4 0/2] perf: show that wildmatch() regressed for pathological cases in v2.0","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2017-05-11T09:41:06Z","receivedAt":"2017-05-11T09:41:26Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Fixes a brainfart in v3. I thought I'd fixed that subshell cd issue,\nbut turns out I didn't, actually fix that.\n\nÆvar Arnfjörð Bjarmason (2):\n perf: add function to setup a fresh test repo\n perf: add test showing exponential growth in path globbing\n\n t/perf/README | 1 +\n t/perf/p0100-globbing.sh | 43 +++++++++++++++++++++++++++++++++++++++++++\n t/perf/perf-lib.sh | 19 +++++++++++++++----\n 3 files changed, 59 insertions(+), 4 deletions(-)\n create mode 100755 t/perf/p0100-globbing.sh\n\n-- \n2.11.0\n\n"},{"id":"319376","messageId":"20170511094108.8756-2-avarab@gmail.com","threadId":"45928","inReplyTo":"20170511094108.8756-1-avarab@gmail.com","subject":"[PATCH v4 1/2] perf: add function to setup a fresh test repo","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2017-05-11T09:41:07Z","receivedAt":"2017-05-11T09:41:30Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Add a function to setup a fresh test repo via 'git init' to compliment\nthe existing functions to copy over a normal & large repo.\n\nSome performance tests don't need any existing repository data at all\nto be significant, e.g. tests which stress glob matches against single\npathological revisions or files, which I'm about to add in a\nsubsequent commit.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n t/perf/README | 1 +\n t/perf/perf-lib.sh | 19 +++++++++++++++----\n 2 files changed, 16 insertions(+), 4 deletions(-)\n\ndiff --git a/t/perf/README b/t/perf/README\nindex 49ea4349be..de2fe15696 100644\n--- a/t/perf/README\n+++ b/t/perf/README\n@@ -106,6 +106,7 @@ sources perf-lib.sh:\n \n After that you will want to use some of the following:\n \n+\ttest_perf_fresh_repo # sets up an empty repository\n \ttest_perf_default_repo # sets up a \"normal\" repository\n \ttest_perf_large_repo # sets up a \"large\" repository\n \ndiff --git a/t/perf/perf-lib.sh b/t/perf/perf-lib.sh\nindex ab4b8b06ae..b6fc880395 100644\n--- a/t/perf/perf-lib.sh\n+++ b/t/perf/perf-lib.sh\n@@ -78,6 +78,10 @@ if test -z \"$GIT_PERF_LARGE_REPO\"; then\n \tGIT_PERF_LARGE_REPO=$TEST_DIRECTORY/..\n fi\n \n+test_perf_do_repo_symlink_config_ () {\n+\ttest_have_prereq SYMLINKS || git config core.symlinks false\n+}\n+\n test_perf_create_repo_from () {\n \ttest \"$#\" = 2 ||\n \terror \"bug in the test script: not 2 parameters to test-create-repo\"\n@@ -102,15 +106,22 @@ test_perf_create_repo_from () {\n \t) &&\n \t(\n \t\tcd \"$repo\" &&\n-\t\t\"$MODERN_GIT\" init -q && {\n-\t\t\ttest_have_prereq SYMLINKS ||\n-\t\t\tgit config core.symlinks false\n-\t\t} &&\n+\t\t\"$MODERN_GIT\" init -q &&\n+\t\ttest_perf_do_repo_symlink_config_ &&\n \t\tmv .git/hooks .git/hooks-disabled 2>/dev/null\n \t) || error \"failed to copy repository '$source' to '$repo'\"\n }\n \n # call at least one of these to establish an appropriately-sized repository\n+test_perf_fresh_repo () {\n+\trepo=\"${1:-$TRASH_DIRECTORY}\"\n+\t\"$MODERN_GIT\" init -q \"$repo\" &&\n+\t(\n+\t\tcd \"$repo\" &&\n+\t\ttest_perf_do_repo_symlink_config_\n+\t)\n+}\n+\n test_perf_default_repo () {\n \ttest_perf_create_repo_from \"${1:-$TRASH_DIRECTORY}\" \"$GIT_PERF_REPO\"\n }\n-- \n2.11.0\n\n"},{"id":"319377","messageId":"20170511094108.8756-3-avarab@gmail.com","threadId":"45928","inReplyTo":"20170511094108.8756-1-avarab@gmail.com","subject":"[PATCH v4 2/2] perf: add test showing exponential growth in path globbing","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2017-05-11T09:41:08Z","receivedAt":"2017-05-11T09:41:33Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Add a test showing that runtimes of the wildmatch() function used for\nglobbing in git grow exponentially in the face of some pathological\nglobs.\n\nThis issue affects both globs matching filenames via e.g. ls-files,\nand globs matching refnames via e.g. for-each-ref.\n\nAs noted in the test description this is a test to see whether Git\nsuffers from the issue noted in an article Russ Cox posted today about\ncommon bugs in various glob implementations:\nhttps://research.swtch.com/glob\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n t/perf/p0100-globbing.sh | 43 +++++++++++++++++++++++++++++++++++++++++++\n 1 file changed, 43 insertions(+)\n create mode 100755 t/perf/p0100-globbing.sh\n\ndiff --git a/t/perf/p0100-globbing.sh b/t/perf/p0100-globbing.sh\nnew file mode 100755\nindex 0000000000..dd18a9ce2b\n--- /dev/null\n+++ b/t/perf/p0100-globbing.sh\n@@ -0,0 +1,43 @@\n+#!/bin/sh\n+\n+test_description=\"Tests pathological globbing performance\n+\n+Shows how Git's globbing performance performs when given the sort of\n+pathological patterns described in at https://research.swtch.com/glob\n+\"\n+\n+. ./perf-lib.sh\n+\n+test_globs_big='10 25 50 75 100'\n+test_globs_small='1 2 3 4 5 6'\n+\n+test_perf_fresh_repo\n+\n+test_expect_success 'setup' '\n+\tfor i in $(test_seq 1 100)\n+\tdo\n+\t\tprintf \"a\" >>refname &&\n+\t\tfor j in $(test_seq 1 $i)\n+\t\tdo\n+\t\t\tprintf \"a*\" >>refglob.$i\n+\t\tdone &&\n+\t\techo b >>refglob.$i\n+\tdone &&\n+\ttest_commit test $(cat refname).t \"\" $(cat refname).t\n+'\n+\n+for i in $test_globs_small\n+do\n+\ttest_perf \"refglob((a*)^nb) against tag (a^100).t; n = $i\" '\n+\t\tgit for-each-ref \"refs/tags/$(cat refglob.'$i')b\"\n+\t'\n+done\n+\n+for i in $test_globs_small\n+do\n+\ttest_perf \"fileglob((a*)^nb) against file (a^100).t; n = $i\" '\n+\t\tgit ls-files \"$(cat refglob.'$i')b\"\n+\t'\n+done\n+\n+test_done\n-- \n2.11.0\n\n"},{"id":"319442","messageId":"20170511191719.GC12516@aiede.svl.corp.google.com","threadId":"45928","inReplyTo":"20170511094108.8756-1-avarab@gmail.com","subject":"Re: [PATCH v4 0/2] perf: show that wildmatch() regressed for pathological cases in v2.0","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2017-05-11T19:17:19Z","receivedAt":"2017-05-11T19:18:59Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Ævar Arnfjörð Bjarmason wrote:\n\n> Ævar Arnfjörð Bjarmason (2):\n> perf: add function to setup a fresh test repo\n> perf: add test showing exponential growth in path globbing\n> \n> t/perf/README | 1 +\n> t/perf/p0100-globbing.sh | 43 +++++++++++++++++++++++++++++++++++++++++++\n> t/perf/perf-lib.sh | 19 +++++++++++++++----\n> 3 files changed, 59 insertions(+), 4 deletions(-)\n> create mode 100755 t/perf/p0100-globbing.sh\n\nReviewed-by: Jonathan Nieder \n\nThanks.\n"}]}