{"thread":{"id":"59956","subject":"t2400 on freebsd12","startedAt":"2023-07-06T17:37:41Z","lastAt":"2023-07-16T02:52:02Z","messageCount":11,"participants":["D. Ben Knoble","Junio C Hamano","Eric Sunshine","Jacob Abel"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"479259","messageId":"CALnO6CDryTsguLshcQxx97ZxyY42Twu2hC2y1bLOsS-9zbqXMA@mail.gmail.com","threadId":"59956","inReplyTo":null,"subject":"t2400 on freebsd12","fromName":"D. Ben Knoble","fromEmail":"ben.knoble@gmail.com","sentAt":"2023-07-06T17:37:22Z","receivedAt":"2023-07-06T17:37:41Z","isPatch":false,"sender":{"key":"ben.knoble@gmail.com","avatar":"https://avatars.githubusercontent.com/u/22802209?v=4"},"body":"CI for an i18n PR [1] is failing on freebsd12, but I cannot reproduce\nit. Is this known (a search of archives and the master branch doesn't\nreveal anything)?\n\nSummary output:\n\nTest Summary Report\n-------------------\nt2400-worktree-add.sh (Wstat: 256 Tests:\n227 Failed: 27)\n Failed tests: 50-52, 91-93, 107-109, 123-125, 139-141\n 159-161, 175-177, 191-193, 207-209\n Non-zero exit status: 1\n\nProximate log entry:\n\n[16:19:43] t2400-worktree-add.sh ..............................\nDubious, test returned 1 (wstat 256, 0x100)\nFailed 27/227 subtests\n\nI can see no mention of git-bundle in these tests, so I don't think my\nPR is responsible for the breakage. I don't have a system on which to\nbisect, however.\n\n-- \nD. Ben Knoble\n\n[1]: https://github.com/gitgitgadget/git/pull/1550/\n"},{"id":"479467","messageId":"CALnO6CCc-J+fe9qKaoyYUMM3xMEUnV5w7NKWSbn6xTgEjbac5w@mail.gmail.com","threadId":"59956","inReplyTo":"CALnO6CDryTsguLshcQxx97ZxyY42Twu2hC2y1bLOsS-9zbqXMA@mail.gmail.com","subject":"Re: t2400 on freebsd12","fromName":"D. Ben Knoble","fromEmail":"ben.knoble@gmail.com","sentAt":"2023-07-13T19:17:09Z","receivedAt":"2023-07-13T19:17:27Z","isPatch":false,"sender":{"key":"ben.knoble@gmail.com","avatar":"https://avatars.githubusercontent.com/u/22802209?v=4"},"body":"Bump: this bug seems to affect several GitGitGadget PRs in CI, which\nalso renders GGG unusable for sending mail, IIUC.\n\n\n-- \nD. Ben Knoble\n"},{"id":"479476","messageId":"xmqqfs5ro8v7.fsf@gitster.g","threadId":"59956","inReplyTo":"CALnO6CCc-J+fe9qKaoyYUMM3xMEUnV5w7NKWSbn6xTgEjbac5w@mail.gmail.com","subject":"Re: t2400 on freebsd12","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2023-07-13T20:27:08Z","receivedAt":"2023-07-13T20:27:14Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"D. Ben Knoble\" writes:\n\n> \"D. Ben Knoble\" wrote:\n>> CI for an i18n PR [1] is failing on freebsd12, but I cannot reproduce\n>> it. Is this known (a search of archives and the master branch doesn't\n>> reveal anything)?\n>> \n>> Summary output:\n>> \n>> Test Summary Report\n>> -------------------\n>> t2400-worktree-add.sh (Wstat: 256 Tests:\n>> 227 Failed: 27)\n>> Failed tests: 50-52, 91-93, 107-109, 123-125, 139-141\n>> 159-161, 175-177, 191-193, 207-209\n>> Non-zero exit status: 1\n>> \n>> Proximate log entry:\n>> \n>> [16:19:43] t2400-worktree-add.sh ..............................\n>> Dubious, test returned 1 (wstat 256, 0x100)\n>> Failed 27/227 subtests\n\nThose listed in https://github.com/gitgitgadget/git/actions do not\nseem to even run Cirrus CI. Perhaps GitGitGadget folks wanted to\nmake sure that changes that may break FreeBSD will not escape to the\npublic list, and added it as an extra for pull requests? The runs\nshown at the end of PR in https://github.com/gitgitgadget/git/pulls\nhowever do run Cirrus CI and some, but not all, of them do fail.\n\nGiven that the tests seem to randomly fail, I can believe if this is\ndue to a flakey test that needs to be fixed, but from what we can\nsee in the webpage of Cirrus CI, I cannot even guess what the\nproblem is.\n\nI do not offhand know how well the FreeBSD port has been maintained,\nor those who have (or had once in the past) stake in it are keeping\nan eye on it. Anybody?\n\n> Bump: this bug seems to affect several GitGitGadget PRs in CI, which\n> also renders GGG unusable for sending mail, IIUC.\n\nBy the way, is this really \"blocking\" use of GGG in any way? I do\nrecall seeing messages regarding gitk from Jens Lidestrom that are\nshown in https://github.com/gitgitgadget/git/pull/1551 but the CI\nrun report at the end of that page does have a failing CirrusCI.\n"},{"id":"479479","messageId":"CAPig+cQvr-HuJBQsjt0jW0G95SmyzTc74R6JiznNeKbHmfu9PQ@mail.gmail.com","threadId":"59956","inReplyTo":"xmqqfs5ro8v7.fsf@gitster.g","subject":"Re: t2400 on freebsd12","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2023-07-13T20:43:37Z","receivedAt":"2023-07-13T20:44:05Z","isPatch":false,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Thu, Jul 13, 2023 at 4:31 PM Junio C Hamano wrote:\n> \"D. Ben Knoble\" writes:\n> > Bump: this bug seems to affect several GitGitGadget PRs in CI, which\n> > also renders GGG unusable for sending mail, IIUC.\n>\n> By the way, is this really \"blocking\" use of GGG in any way? I do\n> recall seeing messages regarding gitk from Jens Lidestrom that are\n> shown in https://github.com/gitgitgadget/git/pull/1551 but the CI\n> run report at the end of that page does have a failing CirrusCI.\n\nFailed CI doesn't block GGG (at least not last time I used it). You\ncan \"/submit\" the pull request to the Git mailing list even if CI\nfailed or is not yet finished running.\n"},{"id":"479489","messageId":"CALnO6CAFC23b2NYCsfcuuR3cHnLy2vm_vZbjDEEBP+nDFurXSw@mail.gmail.com","threadId":"59956","inReplyTo":"CAPig+cQvr-HuJBQsjt0jW0G95SmyzTc74R6JiznNeKbHmfu9PQ@mail.gmail.com","subject":"Re: t2400 on freebsd12","fromName":"D. Ben Knoble","fromEmail":"ben.knoble@gmail.com","sentAt":"2023-07-13T21:44:18Z","receivedAt":"2023-07-13T21:44:33Z","isPatch":false,"sender":{"key":"ben.knoble@gmail.com","avatar":"https://avatars.githubusercontent.com/u/22802209?v=4"},"body":"On Thu, Jul 13, 2023 at 4:43 PM Eric Sunshine wrote:\n> > By the way, is this really \"blocking\" use of GGG in any way? I do\n> > recall seeing messages regarding gitk from Jens Lidestrom that are\n> > shown in https://github.com/gitgitgadget/git/pull/1551 but the CI\n> > run report at the end of that page does have a failing CirrusCI.\n>\n> Failed CI doesn't block GGG (at least not last time I used it). You\n> can \"/submit\" the pull request to the Git mailing list even if CI\n> failed or is not yet finished running.\n\nHm, I thought I recalled that it did, but I will try again. Thanks for\nthe attention.\n\n-- \nD. Ben Knoble\n"},{"id":"479508","messageId":"356tacvizwbtwigdkz4byrrzsyjuktcb7cxaibf6wjocffgycp@iwhmszuuvzpl","threadId":"59956","inReplyTo":"xmqqfs5ro8v7.fsf@gitster.g","subject":"Re: t2400 on freebsd12","fromName":"Jacob Abel","fromEmail":"jacobabel@nullpo.dev","sentAt":"2023-07-14T06:22:13Z","receivedAt":"2023-07-14T06:22:34Z","isPatch":false,"sender":{"key":"jacobabel@nullpo.dev","avatar":"https://avatars.githubusercontent.com/u/9424043?v=4"},"body":"On 23/07/13 01:27PM, Junio C Hamano wrote:\n> \"D. Ben Knoble\" writes:\n> \n> > \"D. Ben Knoble\" wrote:\n> > [...]\n> >>\n> >> Summary output:\n> >>\n> >> Test Summary Report\n> >> -------------------\n> >> t2400-worktree-add.sh (Wstat: 256 Tests:\n> >> 227 Failed: 27)\n> >> Failed tests: 50-52, 91-93, 107-109, 123-125, 139-141\n> >> 159-161, 175-177, 191-193, 207-209\n> >> Non-zero exit status: 1\n> >>\n> >> Proximate log entry:\n> >>\n> >> [16:19:43] t2400-worktree-add.sh ..............................\n> >> Dubious, test returned 1 (wstat 256, 0x100)\n> >> Failed 27/227 subtests\n> \n> [...]\n> \n> Given that the tests seem to randomly fail, I can believe if this is\n> due to a flakey test that needs to be fixed, but from what we can\n> see in the webpage of Cirrus CI, I cannot even guess what the\n> problem is.\n> \n> I do not offhand know how well the FreeBSD port has been maintained,\n> or those who have (or had once in the past) stake in it are keeping\n> an eye on it. Anybody?\n\nI wrote these tests[1]. All the tests that are failing are:\n\n- running `git worktree add` without `--orphan` or `--quiet`.\n- running in a repo with 1 local branch with a valid commit.\n- running in a worktree with an invalid/unborn HEAD.\n\nThe resulting code path should be git failing with:\n\n- a warning containing the path to the HEAD file for the worktree \n that is in scope and the contents of that HEAD.\n- a hint to try using `--orphan`.\n- an error `fatal: invalid reference`.\n\nThere are other minor variations between all the tests but these are\nthe commonalities between the tests. No other tests stress that\nparticular codepath.\n\nMy guess is something in `can_use_local_ref()` (`builtin/worktree.c`) \nis triggering the crash.\n\n> \n> [...]\n\n1. https://lore.kernel.org/git/20230517214711.12467-1-jacobabel@nullpo.dev/\n\n"},{"id":"479520","messageId":"CAPig+cRMXJkrEgyVtC0u2QK=5QNnJOQnXBU_rE+JiGufEYH9sg@mail.gmail.com","threadId":"59956","inReplyTo":"356tacvizwbtwigdkz4byrrzsyjuktcb7cxaibf6wjocffgycp@iwhmszuuvzpl","subject":"Re: t2400 on freebsd12","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2023-07-14T16:19:41Z","receivedAt":"2023-07-14T16:19:56Z","isPatch":false,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Fri, Jul 14, 2023 at 2:30 AM Jacob Abel wrote:\n> On 23/07/13 01:27PM, Junio C Hamano wrote:\n> > \"D. Ben Knoble\" writes:\n> > >> t2400-worktree-add.sh (Wstat: 256 Tests:\n> > >> 227 Failed: 27)\n> > >> Failed tests: 50-52, 91-93, 107-109, 123-125, 139-141\n> > >> 159-161, 175-177, 191-193, 207-209\n>>\n> > I do not offhand know how well the FreeBSD port has been maintained,\n> > or those who have (or had once in the past) stake in it are keeping\n> > an eye on it. Anybody?\n>\n> I wrote these tests[1]. All the tests that are failing are:\n>\n> - running `git worktree add` without `--orphan` or `--quiet`.\n> - running in a repo with 1 local branch with a valid commit.\n> - running in a worktree with an invalid/unborn HEAD.\n>\n> 1. https://lore.kernel.org/git/20230517214711.12467-1-jacobabel@nullpo.dev/\n\nI haven't been following this thread closely, but I wonder if the\n`grep` introduced by patch [3/8] of the cited patch series is\nproblematic:\n\n grep -E \"fatal:( options)? .* cannot be used together\" actual\n\nsince BSD lineage regexp (including macOS) historically did not\nsupport the \"?\" repetition operator. Perhaps an easy fix would be to\nsimplify this to:\n\n grep \"cannot be used together\" actual\n"},{"id":"479533","messageId":"zznttv6pkjwtaoboiiz7z77of5pq7vtwk4gm7hkurkbbcnouq3@3c2yhx2tr5yw","threadId":"59956","inReplyTo":"CAPig+cRMXJkrEgyVtC0u2QK=5QNnJOQnXBU_rE+JiGufEYH9sg@mail.gmail.com","subject":"Re: t2400 on freebsd12","fromName":"Jacob Abel","fromEmail":"jacobabel@nullpo.dev","sentAt":"2023-07-14T19:45:55Z","receivedAt":"2023-07-14T19:46:25Z","isPatch":false,"sender":{"key":"jacobabel@nullpo.dev","avatar":"https://avatars.githubusercontent.com/u/9424043?v=4"},"body":"On 23/07/14 12:19PM, Eric Sunshine wrote:\n> On Fri, Jul 14, 2023 at 2:30 AM Jacob Abel wrote:\n> > On 23/07/13 01:27PM, Junio C Hamano wrote:\n> > > \"D. Ben Knoble\" writes:\n> > > >> t2400-worktree-add.sh (Wstat: 256 Tests:\n> > > >> 227 Failed: 27)\n> > > >> Failed tests: 50-52, 91-93, 107-109, 123-125, 139-141\n> > > >> 159-161, 175-177, 191-193, 207-209\n> >>\n> > > I do not offhand know how well the FreeBSD port has been maintained,\n> > > or those who have (or had once in the past) stake in it are keeping\n> > > an eye on it. Anybody?\n> >\n> > I wrote these tests[1]. All the tests that are failing are:\n> >\n> > - running `git worktree add` without `--orphan` or `--quiet`.\n> > - running in a repo with 1 local branch with a valid commit.\n> > - running in a worktree with an invalid/unborn HEAD.\n> >\n> > 1. https://lore.kernel.org/git/20230517214711.12467-1-jacobabel@nullpo.dev/\n> \n> I haven't been following this thread closely, but I wonder if the\n> `grep` introduced by patch [3/8] of the cited patch series is\n> problematic:\n> \n> grep -E \"fatal:( options)? .* cannot be used together\" actual\n> \n> since BSD lineage regexp (including macOS) historically did not\n> support the \"?\" repetition operator. Perhaps an easy fix would be to\n> simplify this to:\n> \n> grep \"cannot be used together\" actual\n\nThe only tests that use the `?` operator are tests 32-38 which use \n`test_wt_add_excl()`. Those tests all seem to be consistently passing.\n\nI probably should have mentioned in my original post the exact lines\nthat were causing the error. Line numbers mentioned below are from the\ncurrent head of the master branch (830b4a04c4, 'the tenth batch')\n\nTests 50-52 are the `test_wt_add_orphan_hint()` tests on lines 428-430\nof t2400 (the fn is defined right above them). These were introduced in \npatch 6/8 [1].\n\nThe rest of the tests correspond to the `test_dwim_orphan('warn_bad_head', ...)` \ntests on lines 1039-1041 and likewise that function is defined\ndirectly above those lines. These were introduced in patch 8/8 [2] \nhowever the bulk of the test code was introduced in the previous \npatch (7/8) [3].\n\nOf particular note out of the details I gave in my previous post is \nthat these tests all cause the command to emit the bad HEAD warning. \nI bring this up because in that warning code (patch 8/8 [2]) there \nis path string manipulation and a file read (both of which could be\nstepping on some platform dependent behavior).\n\n1. https://lore.kernel.org/git/20230517214711.12467-7-jacobabel@nullpo.dev/\n2. https://lore.kernel.org/git/20230517214711.12467-9-jacobabel@nullpo.dev/\n3. https://lore.kernel.org/git/20230517214711.12467-8-jacobabel@nullpo.dev/\n\n"},{"id":"479535","messageId":"xmqq8rbinqyg.fsf@gitster.g","threadId":"59956","inReplyTo":"CAPig+cRMXJkrEgyVtC0u2QK=5QNnJOQnXBU_rE+JiGufEYH9sg@mail.gmail.com","subject":"Re: t2400 on freebsd12","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2023-07-14T21:06:15Z","receivedAt":"2023-07-14T21:06:27Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Eric Sunshine writes:\n\n> I haven't been following this thread closely, but I wonder if the\n> `grep` introduced by patch [3/8] of the cited patch series is\n> problematic:\n>\n> grep -E \"fatal:( options)? .* cannot be used together\" actual\n>\n> since BSD lineage regexp (including macOS) historically did not\n> support the \"?\" repetition operator. Perhaps an easy fix would be to\n> simplify this to:\n>\n> grep \"cannot be used together\" actual\n\nWe do not seem to get the same breakage on macOS CI runs (otherwise\nthis would have been caught much earlier). We do have many \"grep\n-E\" invocations to ask for ERE but not many of them uses zero-or-one\n'?' in our test suite. But I would be somewhat surprised if\ntest_dir_is_empty is broken on FreeBSD and nobody has noticed it for\nthis long.\n\nI got an impression from the discussion so far that this breakage is\nflaky and not always reproducible. I wonder if \"stress\" thing helps\nthe chance to reproduce for those with FreeBSD boxes?\n\n $ cd t && sh ./t2400-* --stress\n\nThanks.\n"},{"id":"479540","messageId":"dk2fndv5aqshkgvfq55mrr5chdouenszedugyjoezcuatrsmn6@vqyih7nbqwbu","threadId":"59956","inReplyTo":"CAPig+cRMXJkrEgyVtC0u2QK=5QNnJOQnXBU_rE+JiGufEYH9sg@mail.gmail.com","subject":"Re: t2400 on freebsd12","fromName":"Jacob Abel","fromEmail":"jacobabel@nullpo.dev","sentAt":"2023-07-15T03:02:41Z","receivedAt":"2023-07-15T03:02:53Z","isPatch":false,"sender":{"key":"jacobabel@nullpo.dev","avatar":"https://avatars.githubusercontent.com/u/9424043?v=4"},"body":"On 23/07/14 12:19PM, Eric Sunshine wrote:\n>\n> [...]\n> \n> I haven't been following this thread closely, but I wonder if the\n> `grep` introduced by patch [3/8] of the cited patch series is\n> problematic:\n> \n> grep -E \"fatal:( options)? .* cannot be used together\" actual\n> \n> since BSD lineage regexp (including macOS) historically did not\n> support the \"?\" repetition operator. Perhaps an easy fix would be to\n> simplify this to:\n> \n> grep \"cannot be used together\" actual\n\nThank you for this insight. It didn't end up being exactly this issue\nbut it seems to be a grep issue nonetheless. I've submitted a \npatch [1] which should resolve the issue (I tested it on a freebsd12\nVM locally).\n\nThe TLDR of the issue is that grep 2.5 (GNU or BSD) doesn't seem to\nrecognise `\\s` (or its inverted counterpart) as ERE but newer GNU (and\npotentially other) grep versions do.\n\n1. https://lore.kernel.org/git/20230715025512.7574-1-jacobabel@nullpo.dev/\n\n"},{"id":"479554","messageId":"CAPig+cRUOMSUzb95nnDPRnPN0TyDY93E4EdMp9zxTmnOfc8vKg@mail.gmail.com","threadId":"59956","inReplyTo":"dk2fndv5aqshkgvfq55mrr5chdouenszedugyjoezcuatrsmn6@vqyih7nbqwbu","subject":"Re: t2400 on freebsd12","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2023-07-16T02:51:45Z","receivedAt":"2023-07-16T02:52:02Z","isPatch":false,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Fri, Jul 14, 2023 at 11:02 PM Jacob Abel wrote:\n> On 23/07/14 12:19PM, Eric Sunshine wrote:\n> > I haven't been following this thread closely, but I wonder if the\n> > `grep` introduced by patch [3/8] of the cited patch series is\n> > problematic:\n> >\n> > grep -E \"fatal:( options)? .* cannot be used together\" actual\n>\n> Thank you for this insight. It didn't end up being exactly this issue\n> but it seems to be a grep issue nonetheless. I've submitted a\n> patch [1] which should resolve the issue (I tested it on a freebsd12\n> VM locally).\n>\n> The TLDR of the issue is that grep 2.5 (GNU or BSD) doesn't seem to\n> recognise `\\s` (or its inverted counterpart) as ERE but newer GNU (and\n> potentially other) grep versions do.\n\nThat's great to hear. Thanks for digging into this and submitting a fix.\n\n(The FreeBSD 12 VM I created with the idea of investigating this is\nnow unnecessary.)\n"}]}