{"thread":{"id":"51665","subject":"Missing file in 2.23 (p5302-pack-index.subtests)?","startedAt":"2019-08-18T16:03:23Z","lastAt":"2019-08-27T03:35:06Z","messageCount":7,"participants":["Theodore Y. Ts'o","Jeff King"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"380622","messageId":"20190818160317.GA13772@mit.edu","threadId":"51665","inReplyTo":null,"subject":"Missing file in 2.23 (p5302-pack-index.subtests)?","fromName":"Theodore Y. Ts'o","fromEmail":"tytso@mit.edu","sentAt":"2019-08-18T16:03:17Z","receivedAt":"2019-08-18T16:03:23Z","isPatch":false,"sender":{"key":"tytso@mit.edu","avatar":"https://avatars.githubusercontent.com/u/51416?v=4"},"body":"I was trying to run \"make profile\" on the master branch (commit\n5fa0f5238b: \"Git 2.23\") and it died in the\n\n\t$(MAKE) PROFILE=GEN perf\n\ndies with:\n\n\tcannot open test-results/p5302-pack-index.subtests: No such file or directory at ./aggregate.perl line 153.\n\nI presume that's becuase the file t/perf/test-results/p5302-pack-index.subtests is missing?\n\n\t\t\t\t\t\t- Ted\n"},{"id":"381254","messageId":"20190826205012.GB27762@sigill.intra.peff.net","threadId":"51665","inReplyTo":"20190818160317.GA13772@mit.edu","subject":"Re: Missing file in 2.23 (p5302-pack-index.subtests)?","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2019-08-26T20:50:13Z","receivedAt":"2019-08-26T20:50:15Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Sun, Aug 18, 2019 at 12:03:17PM -0400, Theodore Y. Ts'o wrote:\n\n> I was trying to run \"make profile\" on the master branch (commit\n> 5fa0f5238b: \"Git 2.23\") and it died in the\n> \n> \t$(MAKE) PROFILE=GEN perf\n> \n> dies with:\n> \n> \tcannot open test-results/p5302-pack-index.subtests: No such file or directory at ./aggregate.perl line 153.\n> \n> I presume that's becuase the file t/perf/test-results/p5302-pack-index.subtests is missing?\n\nThat file should be created by running p5302. If you do:\n\n  cd t/perf\n  rm -rf test-results\n  ./run p5302-pack-index.sh\n\nis it generated?\n\nIf so, then perhaps something removed it while the perf tests were\nrunning. Did you run perhaps run \"make clean\" simultaneously?\n\n-Peff\n"},{"id":"381255","messageId":"20190826205142.GC27762@sigill.intra.peff.net","threadId":"51665","inReplyTo":"20190826205012.GB27762@sigill.intra.peff.net","subject":"Re: Missing file in 2.23 (p5302-pack-index.subtests)?","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2019-08-26T20:51:42Z","receivedAt":"2019-08-26T20:51:45Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Aug 26, 2019 at 04:50:13PM -0400, Jeff King wrote:\n\n> On Sun, Aug 18, 2019 at 12:03:17PM -0400, Theodore Y. Ts'o wrote:\n> \n> > I was trying to run \"make profile\" on the master branch (commit\n> > 5fa0f5238b: \"Git 2.23\") and it died in the\n> > \n> > \t$(MAKE) PROFILE=GEN perf\n> > \n> > dies with:\n> > \n> > \tcannot open test-results/p5302-pack-index.subtests: No such file or directory at ./aggregate.perl line 153.\n> > \n> > I presume that's becuase the file t/perf/test-results/p5302-pack-index.subtests is missing?\n> \n> That file should be created by running p5302. If you do:\n\nBy the way, I tried \"make perf\" as you specified above, and it worked\nfine for me. And I don't think there are any changes in v2.23 that\nshould touch this area. But certainly it would be interesting to know if\nit fails consistently for you, and if so, whether older versions work\n(so we can bisect).\n\n-Peff\n"},{"id":"381278","messageId":"20190827012942.GD28066@mit.edu","threadId":"51665","inReplyTo":"20190826205012.GB27762@sigill.intra.peff.net","subject":"Re: Missing file in 2.23 (p5302-pack-index.subtests)?","fromName":"Theodore Y. Ts'o","fromEmail":"tytso@mit.edu","sentAt":"2019-08-27T01:29:42Z","receivedAt":"2019-08-27T01:29:49Z","isPatch":false,"sender":{"key":"tytso@mit.edu","avatar":"https://avatars.githubusercontent.com/u/51416?v=4"},"body":"On Mon, Aug 26, 2019 at 04:50:13PM -0400, Jeff King wrote:\n> On Sun, Aug 18, 2019 at 12:03:17PM -0400, Theodore Y. Ts'o wrote:\n> \n> > I was trying to run \"make profile\" on the master branch (commit\n> > 5fa0f5238b: \"Git 2.23\") and it died in the\n> > \n> > \t$(MAKE) PROFILE=GEN perf\n> > \n> > dies with:\n> > \n> > \tcannot open test-results/p5302-pack-index.subtests: No such file or directory at ./aggregate.perl line 153.\n> > \n> > I presume that's becuase the file t/perf/test-results/p5302-pack-index.subtests is missing?\n> \n> That file should be created by running p5302. If you do:\n> \n>   cd t/perf\n>   rm -rf test-results\n>   ./run p5302-pack-index.sh\n> \n> is it generated?\n\nNo, it wasn't.  See below....\n\n\t\t\t\t\t- Ted\n\n<tytso@lambda> {/usr/projects/git/git-core/t/perf}   (master)\n1140% git show\ncommit 5fa0f5238b0cd46cfe7f6fa76c3f526ea98148d9 (HEAD -> master, tag: v2.23.0, origin/master, origin/HEAD)\nAuthor: Junio C Hamano <gitster@pobox.com>\nDate:   Fri Aug 16 10:28:23 2019 -0700\n\n    Git 2.23\n    \n    Signed-off-by: Junio C Hamano <gitster@pobox.com>\n\ndiff --git a/GIT-VERSION-GEN b/GIT-VERSION-GEN\nindex 551e607e73..a1539a7ce6 100755\n--- a/GIT-VERSION-GEN\n+++ b/GIT-VERSION-GEN\n@@ -1,7 +1,7 @@\n #!/bin/sh\n \n GVF=GIT-VERSION-FILE\n-DEF_VER=v2.23.0-rc2\n+DEF_VER=v2.23.0\n \n LF='\n '\n<tytso@lambda> {/usr/projects/git/git-core/t/perf}   (master)\n1141% rm -rf test-results ; ./run p5302-pack-index.sh\n=== Running 1 tests in this tree ===\nwarning: $GIT_PERF_LARGE_REPO is $GIT_BUILD_DIR.\nwarning: This will work, but may not be a sufficiently large repo\nwarning: for representative measurements.\nnot ok 1 - repack\n#\t\n#\t\tgit repack -ad &&\n#\t\tPACK=$(ls .git/objects/pack/*.pack | head -n1) &&\n#\t\ttest -f \"$PACK\" &&\n#\t\texport PACK\n#\t\ncannot open test-results/p5302-pack-index.subtests: No such file or directory at ./aggregate.perl line 153.\n<tytso@lambda> {/usr/projects/git/git-core/t/perf}   (master)\n1142% git status\nOn branch master\nYour branch is up to date with 'origin/master'.\n\nUntracked files:\n  (use \"git add <file>...\" to include in what will be committed)\n\t../../MAKELOG\n\t../../MAKELOG.1\n\t../../common-cmds.h\n\t../../git-lost-found\n\t../../git-peek-remote\n\t../../git-rebase--am\n\t../../git-rebase--common\n\t../../git-rebase--helper\n\t../../git-rebase--interactive\n\t../../git-rebase--merge\n\t../../git-relink\n\t../../git-remote-testgit\n\t../../git-repo-config\n\t../../git-tar-tree\n\t../../git_remote_helpers/\n\t../../perl/MYMETA.json\n\t../../perl/MYMETA.yml\n\t../../perl/PM.stamp\n\t../../perl/blib/\n\t../../perl/perl.mak\n\t../../perl/pm_to_blib\n\t../../test-chmtime\n\t../../test-config\n\t../../test-credential\n\t../../test-ctype\n\t../../test-date\n\t../../test-delta\n\t../../test-dump-cache-tree\n\t../../test-dump-split-index\n\t../../test-dump-untracked-cache\n\t../../test-fake-ssh\n\t../../test-genrandom\n\t../../test-hashmap\n\t../../test-index-version\n\t../../test-line-buffer\n\t../../test-match-trees\n\t../../test-mergesort\n\t../../test-mktemp\n\t../../test-obj-pool\n\t../../test-parse-options\n\t../../test-path-utils\n\t../../test-prio-queue\n\t../../test-read-cache\n\t../../test-regex\n\t../../test-revision-walking\n\t../../test-run-command\n\t../../test-scrap-cache-tree\n\t../../test-sha1\n\t../../test-sha1-array\n\t../../test-sigchain\n\t../../test-string-list\n\t../../test-string-pool\n\t../../test-submodule-config\n\t../../test-subprocess\n\t../../test-svn-fe\n\t../../test-treap\n\t../../test-urlmatch-normalization\n\t../../test-wildmatch\n\nnothing added to commit but untracked files present (use \"git add\" to track)\n"},{"id":"381282","messageId":"20190827022700.GA31125@sigill.intra.peff.net","threadId":"51665","inReplyTo":"20190827012942.GD28066@mit.edu","subject":"Re: Missing file in 2.23 (p5302-pack-index.subtests)?","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2019-08-27T02:27:00Z","receivedAt":"2019-08-27T02:27:02Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Aug 26, 2019 at 09:29:42PM -0400, Theodore Y. Ts'o wrote:\n\n> > > I presume that's becuase the file t/perf/test-results/p5302-pack-index.subtests is missing?\n> > \n> > That file should be created by running p5302. If you do:\n> > \n> >   cd t/perf\n> >   rm -rf test-results\n> >   ./run p5302-pack-index.sh\n> > \n> > is it generated?\n> \n> No, it wasn't.  See below....\n\nWeird. The output here:\n\n> 1141% rm -rf test-results ; ./run p5302-pack-index.sh\n> === Running 1 tests in this tree ===\n> warning: $GIT_PERF_LARGE_REPO is $GIT_BUILD_DIR.\n> warning: This will work, but may not be a sufficiently large repo\n> warning: for representative measurements.\n> not ok 1 - repack\n> #\t\n> #\t\tgit repack -ad &&\n> #\t\tPACK=$(ls .git/objects/pack/*.pack | head -n1) &&\n> #\t\ttest -f \"$PACK\" &&\n> #\t\texport PACK\n> #\t\n> cannot open test-results/p5302-pack-index.subtests: No such file or directory at ./aggregate.perl line 153.\n\nImplies that we're trying to _write_ to it, and that the problem is that\ntest-results doesn't exist. That should be set up by this part of\nperf-lib:\n\n  perf_results_dir=$TEST_OUTPUT_DIRECTORY/test-results\n  test -n \"$GIT_PERF_SUBSECTION\" && perf_results_dir=\"$perf_results_dir/$GIT_PERF_SUBSECTION\"\n  mkdir -p \"$perf_results_dir\"\n  rm -f \"$perf_results_dir\"/$(basename \"$0\" .sh).subtests\n\nI don't know why that would be failing for you (and not producing any\nkind of message!). You might need to poke at:\n\n  ./p5302-pack-index.sh -x\n\nto see what is going on.\n\nIt's also weird that the repack fails for you. And that (presumably?)\nthis is the only script that fails, since they would all want to do\nthis.\n\nThere are some bits of test-lib.sh that re-exec the script, but I think\nwe ironed out the weirdness there (and I use \"--verbose-log\" myself,\nwhich is one such option). But just in case: do you set GIT_TEST_OPTS in\nyour config.mak?\n\n-Peff\n"},{"id":"381283","messageId":"20190827025811.GJ28066@mit.edu","threadId":"51665","inReplyTo":"20190827022700.GA31125@sigill.intra.peff.net","subject":"Re: Missing file in 2.23 (p5302-pack-index.subtests)?","fromName":"Theodore Y. Ts'o","fromEmail":"tytso@mit.edu","sentAt":"2019-08-27T02:58:11Z","receivedAt":"2019-08-27T02:58:21Z","isPatch":false,"sender":{"key":"tytso@mit.edu","avatar":"https://avatars.githubusercontent.com/u/51416?v=4"},"body":"On Mon, Aug 26, 2019 at 10:27:00PM -0400, Jeff King wrote:\n> > cannot open test-results/p5302-pack-index.subtests: No such file or directory at ./aggregate.perl line 153.\n> \n> Implies that we're trying to _write_ to it, and that the problem is that\n> test-results doesn't exist. That should be set up by this part of\n> perf-lib:\n\nHmm....   test-results does exist after the failure:\n\n<tytso@lambda> {/usr/projects/git/git-core/t/perf}   (master)\n1174% rm -rf test-results ; ./run p5302-pack-index.sh\n=== Running 1 tests in this tree ===\nwarning: $GIT_PERF_LARGE_REPO is $GIT_BUILD_DIR.\nwarning: This will work, but may not be a sufficiently large repo\nwarning: for representative measurements.\nnot ok 1 - repack\n#\t\n#\t\tgit repack -ad &&\n#\t\tPACK=$(ls .git/objects/pack/*.pack | head -n1) &&\n#\t\ttest -f \"$PACK\" &&\n#\t\texport PACK\n#\t\ncannot open test-results/p5302-pack-index.subtests: No such file or directory at ./aggregate.perl line 153.\n<tytso@lambda> {/usr/projects/git/git-core/t/perf}   (master)\n1175% ls test-results\ntotal 0\n0 run_subsections.names\n\nMaybe we're not in the right cwd for some reason?!?\n\n\n> There are some bits of test-lib.sh that re-exec the script, but I think\n> we ironed out the weirdness there (and I use \"--verbose-log\" myself,\n> which is one such option). But just in case: do you set GIT_TEST_OPTS in\n> your config.mak?\n\nYou mean in my top-level directory?  I don't have a config.mak file in\nthere:\n\n<tytso@lambda> {/usr/projects/git/git-core}   (master)\n1151% grep GIT_TEST_OPTS config.mak.*\n<tytso@lambda> {/usr/projects/git/git-core}   (master)\n1181% ls config.mak*\n 4 config.mak.autogen   4 config.mak.dev   4 config.mak.in  24 config.mak.uname\n<tytso@lambda> {/usr/projects/git/git-core}   (master)\n1182% grep GIT_TEST_OPTS config.mak.*\n\n\t\t\t\t\t- Ted\n"},{"id":"381285","messageId":"20190827033503.GA571@sigill.intra.peff.net","threadId":"51665","inReplyTo":"20190827025811.GJ28066@mit.edu","subject":"Re: Missing file in 2.23 (p5302-pack-index.subtests)?","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2019-08-27T03:35:04Z","receivedAt":"2019-08-27T03:35:06Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Aug 26, 2019 at 10:58:11PM -0400, Theodore Y. Ts'o wrote:\n\n> On Mon, Aug 26, 2019 at 10:27:00PM -0400, Jeff King wrote:\n> > > cannot open test-results/p5302-pack-index.subtests: No such file or directory at ./aggregate.perl line 153.\n> > \n> > Implies that we're trying to _write_ to it, and that the problem is that\n> > test-results doesn't exist. That should be set up by this part of\n> > perf-lib:\n> \n> Hmm....   test-results does exist after the failure:\n\nAh, never mind. I was thinking it was coming from this line in perf-lib:\n\n   echo \"$test_count\" >>\"$perf_results_dir\"/$base.subtests\n\nBut it it is pretty clear from this line of your output:\n\n> cannot open test-results/p5302-pack-index.subtests: No such file or directory at ./aggregate.perl line 153.\n\n...that this is the aggregate script that is run afterwards complaining.\nSo for whatever reason, the actual p5302 script is exiting early, before\nit writes anything into the subtests file (from the line above).\n\nThat subtests write is done in test_wrapper_(), which is only triggered\nfor actual timing tests, not for setup tests (like the repack step that\nstarts this script). So the plausible sequence of events here is:\n\n  1. The first \"test_expect_success repack\" test fails for some reason.\n     Try running \"./p5302-pack-index -v -i -x\" to see more output.\n\n  2. After the initial failure, the script exits totally rather than\n     continue. That's due to this line in perf-lib, which acts as if\n     \"-i\" is always set:\n\n       # Performance tests should never fail.  If they do, stop immediately\n       immediate=t\n\n     That makes sense, since any timings we do after a setup step fails\n     would invalidate the result. And if it's not the _first_ such step\n     that fails, we'll just have a truncated subtests file, and some\n     timings will be missing. But when the first one fails, there's no\n     file at all.\n\n     We could probably leave a more consistent state in that case. E.g.,\n     something like this:\n\ndiff --git a/t/perf/perf-lib.sh b/t/perf/perf-lib.sh\nindex b58a43ea43..cfac2c3cbe 100644\n--- a/t/perf/perf-lib.sh\n+++ b/t/perf/perf-lib.sh\n@@ -48,7 +48,7 @@ export MODERN_GIT\n perf_results_dir=$TEST_OUTPUT_DIRECTORY/test-results\n test -n \"$GIT_PERF_SUBSECTION\" && perf_results_dir=\"$perf_results_dir/$GIT_PERF_SUBSECTION\"\n mkdir -p \"$perf_results_dir\"\n-rm -f \"$perf_results_dir\"/$(basename \"$0\" .sh).subtests\n+>\"$perf_results_dir\"/$(basename \"$0\" .sh).subtests\n \n die_if_build_dir_not_repo () {\n \tif ! ( cd \"$TEST_DIRECTORY/..\" &&\n\n\n     which would just quietly omit the rest of p5302. It is unfortunate\n     that you wouldn't get some note in the output saying \"hey, we\n     didn't run some tests!\".\n\n     A lot of this has to do with the hackish way that the list of tests\n     is generated. If you do something like:\n\n       ./run HEAD^ HEAD p5302-pack-index.sh\n\n     and the tests fail in HEAD^ but not HEAD, you'll get a nice table\n     listing all of the tests, with a nil field for each one that didn't\n     run in HEAD^.\n\n     But that's because the second run, on HEAD, actually generated a\n     correct p5302-pack-index.subtests file, that actually mentioned the\n     tests! If the order were reversed, you'd get an empty list.\n\n     So I think in the long run, it would be nice to have some way of\n     generating the list that's more robust to failures. But I suspect\n     doing that is going to be hard; a lot of this is rooted in the fact\n     that there's no data structure with the set of tests, but literally\n     an executable shell script that decides what to run.\n\nSo short answer: use \"-v\" to figure out why the repack is failing, and\nthe rest is just Git's perf suite being a bit hacky.\n\n-Peff\n"}]}