{"thread":{"id":"51251","subject":"Re: Git self test failure on Solaris 11.3","startedAt":"2019-06-06T17:18:15Z","lastAt":"2019-06-07T01:04:04Z","messageCount":5,"participants":["Eric Sunshine","Jeff King","Derrick Stolee"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"376773","messageId":"CAPig+cS98DeSaF6pNqKCn6bAy=jjiTuc3AZpPB2cXm6hQ6vcqQ@mail.gmail.com","threadId":"51251","inReplyTo":"CAH8yC8knkXtBii9KhS2iUo908zTx_NyPUgXZhqdwmwEF7Gz8Jg@mail.gmail.com","subject":"Re: Git self test failure on Solaris 11.3","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2019-06-06T17:18:01Z","receivedAt":"2019-06-06T17:18:15Z","isPatch":false,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"[forwarding to the Git list]\n\nOn Sun, Jun 2, 2019 at 6:23 AM Jeffrey Walton <noloader@gmail.com> wrote:\n> On Sun, Jun 2, 2019 at 5:09 AM Jeffrey Walton <noloader@gmail.com> wrote:\n> > I'm catching a self test failure on Solaris 11.3. Git 2.21 from sources.\n> >\n> > ok 8 - check normal git operations: two packs\n> > ok 9 - add more packs\n> > ok 10 - check normal git operations: mixed mode (two packs + extra)\n> > ok 11 - write midx with twelve packs\n> > not ok 12 - check normal git operations: twelve packs\n> > #\n> > #                       midx_git_two_modes \"rev-list --objects --all\" &&\n> > #                       midx_git_two_modes \"log --raw\" &&\n> > #                       midx_git_two_modes \"count-objects --verbose\" &&\n> > #                       midx_git_two_modes \"cat-file\n> > --batch-all-objects --buffer --batch-check\" &&\n> > #                       midx_git_two_modes \"cat-file\n> > --batch-all-objects --buffer --batch-check --unsorted\" sorted\n> > #\n> > ok 13 - verify multi-pack-index success\n> > ok 14 - verify bad signature\n> > ok 15 - verify bad version\n> > ...\n>\n> Ick, this is bad. Intermittent failure. The second time around it passed.\n\nJeff Walton reported this to me privately. I'm not familiar with this\ncode and don't have time presently to investigate it, so I'm\nforwarding it to the list in the hope that someone who knows the code\nintimately (Stolee) or someone who has touched this code (Peff) might\nhave some idea about it. The failure is intermittent, so perhaps the\noutput of some command is not stable sort-wise(?).\n"},{"id":"376774","messageId":"20190606173516.GA25089@sigill.intra.peff.net","threadId":"51251","inReplyTo":"CAPig+cS98DeSaF6pNqKCn6bAy=jjiTuc3AZpPB2cXm6hQ6vcqQ@mail.gmail.com","subject":"Re: Git self test failure on Solaris 11.3","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2019-06-06T17:35:17Z","receivedAt":"2019-06-06T17:35:20Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Jun 06, 2019 at 01:18:01PM -0400, Eric Sunshine wrote:\n\n> > > not ok 12 - check normal git operations: twelve packs\n> > > #\n> > > #                       midx_git_two_modes \"rev-list --objects --all\" &&\n> > > #                       midx_git_two_modes \"log --raw\" &&\n> > > #                       midx_git_two_modes \"count-objects --verbose\" &&\n> > > #                       midx_git_two_modes \"cat-file\n> > > --batch-all-objects --buffer --batch-check\" &&\n> > > #                       midx_git_two_modes \"cat-file\n> > > --batch-all-objects --buffer --batch-check --unsorted\" sorted\n> > > #\n> [...]\n> \n> Jeff Walton reported this to me privately. I'm not familiar with this\n> code and don't have time presently to investigate it, so I'm\n> forwarding it to the list in the hope that someone who knows the code\n> intimately (Stolee) or someone who has touched this code (Peff) might\n> have some idea about it. The failure is intermittent, so perhaps the\n> output of some command is not stable sort-wise(?).\n\nI can't seem to get it to fail for me, even under high load. Do you know\nwhich of those commands is failing (i.e., what does \"-v -x\" say)?\n\nIf it's the rev-list or log output, it could be due to traversal order.\nI notice that the commit creation doesn't increment the timestamps. This\nmight help:\n\ndiff --git a/t/t5319-multi-pack-index.sh b/t/t5319-multi-pack-index.sh\nindex 70926b5bc0..79aa97efce 100755\n--- a/t/t5319-multi-pack-index.sh\n+++ b/t/t5319-multi-pack-index.sh\n@@ -60,6 +60,7 @@ commit_and_list_objects () {\n \t} >file_101 &&\n \tgit update-index --add file_101 &&\n \ttree=$(git write-tree) &&\n+\ttest_tick &&\n \tcommit=$(git commit-tree $tree -p HEAD</dev/null) &&\n \t{\n \t\techo $tree &&\n\nThough I think in each case that the results should be deterministic for\na given set of commits (and the tests here are just running over the\nsame set of commits with and without midx; so we should see the same\noutput in both cases).\n\nThe normal \"cat-file\" output should be sorted by sha1, so should be\nstable. The unsorted output will be in whatever order we happen to hit\nthe packfiles, which _would_ be subject to timestamp variations.\nHowever, Jeffrey's email mentions Git 2.21, which predates my b4a14394af\n(t5319: fix bogus cat-file argument, 2019-04-05). So before that,\n\"cat-file --unsorted\" should always be producing no output. :)\n\nAfter the switch to correctly use --unordered, I could believe there\nmight be a timestamp issue, but the \"sorted\" flag there means that we\nshould be sorting the output before comparing.\n\nI can't reproduce the intermittent failure either on 2.21.0, or with\nv2.22.0-rc3.\n\n-Peff\n"},{"id":"376777","messageId":"CAPig+cQi6V=B7GtdN3+-1nPnLXiVjz-O+=WJt5f94o7wHy1VHg@mail.gmail.com","threadId":"51251","inReplyTo":"20190606173516.GA25089@sigill.intra.peff.net","subject":"Re: Git self test failure on Solaris 11.3","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2019-06-06T19:00:00Z","receivedAt":"2019-06-06T19:00:12Z","isPatch":false,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Thu, Jun 6, 2019 at 1:35 PM Jeff King <peff@peff.net> wrote:\n> On Thu, Jun 06, 2019 at 01:18:01PM -0400, Eric Sunshine wrote:\n> > > > not ok 12 - check normal git operations: twelve packs\n> >\n> > Jeff Walton reported this to me privately. I'm not familiar with this\n> > code and don't have time presently to investigate it, so I'm\n> > forwarding it to the list in the hope that someone who knows the code\n> > intimately (Stolee) or someone who has touched this code (Peff) might\n> > have some idea about it. The failure is intermittent, so perhaps the\n> > output of some command is not stable sort-wise(?).\n>\n> I can't seem to get it to fail for me, even under high load. Do you know\n> which of those commands is failing (i.e., what does \"-v -x\" say)?\n\nI don't know.\n\n> I can't reproduce the intermittent failure either on 2.21.0, or with\n> v2.22.0-rc3.\n\nI can't reproduce it either on Jeff's Solaris box. Perhaps Jeff can\nadd \"-v -x\" to his automated build/test script in order to help\ndiagnose the problem if it occurs again.\n\nThanks for looking into it.\n"},{"id":"376779","messageId":"20190606190503.GA3265@sigill.intra.peff.net","threadId":"51251","inReplyTo":"CAPig+cQi6V=B7GtdN3+-1nPnLXiVjz-O+=WJt5f94o7wHy1VHg@mail.gmail.com","subject":"Re: Git self test failure on Solaris 11.3","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2019-06-06T19:05:04Z","receivedAt":"2019-06-06T19:05:07Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Jun 06, 2019 at 03:00:00PM -0400, Eric Sunshine wrote:\n\n> > I can't reproduce the intermittent failure either on 2.21.0, or with\n> > v2.22.0-rc3.\n> \n> I can't reproduce it either on Jeff's Solaris box. Perhaps Jeff can\n> add \"-v -x\" to his automated build/test script in order to help\n> diagnose the problem if it occurs again.\n\nJeff: try \"--verbose-log -x\" if you're doing this as part of an automated\nrun. That will leave you nice logs in t/test-results without dumping it\nall (and in particular, if you run the tests in parallel, it will avoid\ninterleaving output from various tests).\n\n-Peff\n"},{"id":"376791","messageId":"0e75e5ed-3964-5027-1af3-23237674554c@gmail.com","threadId":"51251","inReplyTo":"20190606190503.GA3265@sigill.intra.peff.net","subject":"Re: Git self test failure on Solaris 11.3","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2019-06-07T01:03:58Z","receivedAt":"2019-06-07T01:04:04Z","isPatch":false,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 6/6/2019 3:05 PM, Jeff King wrote:\n> On Thu, Jun 06, 2019 at 03:00:00PM -0400, Eric Sunshine wrote:\n> \n>>> I can't reproduce the intermittent failure either on 2.21.0, or with\n>>> v2.22.0-rc3.\n>>\n>> I can't reproduce it either on Jeff's Solaris box. Perhaps Jeff can\n>> add \"-v -x\" to his automated build/test script in order to help\n>> diagnose the problem if it occurs again.\n> \n> Jeff: try \"--verbose-log -x\" if you're doing this as part of an automated\n> run. That will leave you nice logs in t/test-results without dumping it\n> all (and in particular, if you run the tests in parallel, it will avoid\n> interleaving output from various tests).\n\nPlease keep me posted with any updates with exactly which command failed.\nThese comparison tests are intended to catch surprising interactions, but\nare not very narrow. We'll need to know the exact failure spot, and the\nfailure could take a few forms:\n\n* The git command failed with an error.\n\n* The midx-enabled and midx-disabled commands differ in output.\n\nBetween these two options and the different commands that could have failed,\nit's hard to dig in without more to go on.\n\nThanks,\n-Stolee\n"}]}