{"thread":{"id":"58748","subject":"[PATCH] t7610: fix flaky timeout issue, don't clone from example.com","startedAt":"2022-11-05T11:54:33Z","lastAt":"2022-11-17T21:50:36Z","messageCount":9,"participants":["Ævar Arnfjörð Bjarmason","Taylor Blau"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"466575","messageId":"patch-1.1-83eca7b5a7c-20221105T115420Z-avarab@gmail.com","threadId":"58748","inReplyTo":null,"subject":"[PATCH] t7610: fix flaky timeout issue, don't clone from example.com","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-11-05T11:54:21Z","receivedAt":"2022-11-05T11:54:33Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"When t7610-mergetool.sh runs without failures the git://example.com\nsubmodule URLs will never be used. That's because we \"git submodule\nadd\" it, but then manually populate them so that subsequent \"git\nsubmodule update -N\" won't attempt to clone it, only update it without\nfetching.\n\nBut if we fail in an earlier test it'll have the knock-on effect of\nhaving later tests hang on that \"git submodule update -N\" as we\nattempt to clone this repository from example.com.\n\nThis can be reproduced on \"master\" by running the test with\nSANITIZE=leak without \"--immediate\". With\n\"GIT_TEST_PASSING_SANITIZE_LEAK=true\" (which the linux-leaks job uses)\nwe'll skip the test entirely. So we'll only run into this when running\nit manually, or with the \"GIT_TEST_PASSING_SANITIZE_LEAK=check\" mode.\n\nThat's not because the failure has anything to do with leak detection\nper-se. It just so happens that we have a leak that'll fail before\nwe've managed to fully set these up, and therefore \"git submodule\nupdate -N\" ends up spawning \"git clone\".\n\nLet's instead continue lying about the origin of this submodule by\nproviding a URL for it that doesn't work, but now one that *really*\ndoesn't work: /dev/null. If the test is passing we won't ever use\nthis, and if we have knock-on failures we'll fail early, instead of\nwaiting for a timeout.\n\nThe behavior of \"-N\" here might be surprising to some, since it's\nexplained as \"[if you use -N we] don’t fetch new objects from the\nremote site\". But (perhaps counter-intuitively) it's only talking\nabout if it needs to do so via \"git fetch\". In this case we'll end up\nspawning a \"git clone\", as we have no submodule set up.\n\nSee ff7f089ed10 (mergetool: Teach about submodules, 2011-04-13) for\nthe commit that implemented these \"example.com\" tests.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>\n---\n t/t7610-mergetool.sh | 4 ++--\n 1 file changed, 2 insertions(+), 2 deletions(-)\n\ndiff --git a/t/t7610-mergetool.sh b/t/t7610-mergetool.sh\nindex 8cc64729adb..b1ba0d9a088 100755\n--- a/t/t7610-mergetool.sh\n+++ b/t/t7610-mergetool.sh\n@@ -33,7 +33,7 @@ test_expect_success 'setup' '\n \t\tgit add foo &&\n \t\tgit commit -m \"Add foo\"\n \t) &&\n-\tgit submodule add git://example.com/submod submod &&\n+\tgit submodule add /dev/null submod &&\n \tgit add file1 \"spaced name\" file1[1-4] subdir/file3 .gitmodules submod &&\n \tgit commit -m \"add initial versions\" &&\n \n@@ -614,7 +614,7 @@ test_expect_success 'submodule in subdirectory' '\n \t\t)\n \t) &&\n \ttest_when_finished \"rm -rf subdir/subdir_module\" &&\n-\tgit submodule add git://example.com/subsubmodule subdir/subdir_module &&\n+\tgit submodule add /dev/null subdir/subdir_module &&\n \tgit add subdir/subdir_module &&\n \tgit commit -m \"add submodule in subdirectory\" &&\n \n-- \n2.38.0.1452.g710f45c7951\n\n"},{"id":"466998","messageId":"Y2wuEDGMRg99Ifef@nand.local","threadId":"58748","inReplyTo":"patch-1.1-83eca7b5a7c-20221105T115420Z-avarab@gmail.com","subject":"Re: [PATCH] t7610: fix flaky timeout issue, don't clone from example.com","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2022-11-09T22:47:44Z","receivedAt":"2022-11-09T22:47:48Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Sat, Nov 05, 2022 at 12:54:21PM +0100, Ævar Arnfjörð Bjarmason wrote:\n> The behavior of \"-N\" here might be surprising to some, since it's\n> explained as \"[if you use -N we] don’t fetch new objects from the\n> remote site\". But (perhaps counter-intuitively) it's only talking\n> about if it needs to do so via \"git fetch\". In this case we'll end up\n> spawning a \"git clone\", as we have no submodule set up.\n\nMakes sense, though I'm not sure I agree this is worth patching as I\ncurrently understand it.\n\nIf I run t7610 today with '--run=2-' (IOW, skipping the setup test), I\nam definitely going to get failures. And I don't think we should have\nevery subsequent test depend on having run anything containing \"setup\"\nbefore it. That is, it is not surprising that we will see some test\nfailures when carving up and running portions of the test instead of the\nwhole file.\n\nI don't think this is substantively any different than that. Whether we\ndon't complete the \"setup\" test because we had some leak (and ran the\ntest suite with the appropriate LSan options), or because we neglected\nto run it at all, I don't think there is a significant difference.\n\nEither way, the end-state of the test repository isn't guaranteed to\nmatch the intent of the \"setup\" test.\n\nIf this is the only such problem in-tree, sure, I think it's fine to\npatch. But I would wager that there are *many* more than just this one\nlurking, and patching all of them would be less straightforward than\nthis one.\n\nSo... I don't know. I'm certainly leaning negative on this patch, but if\nyou have some compelling reason that I'm missing, I'm all-ears.\n\nThanks,\nTaylor\n"},{"id":"467010","messageId":"221110.864jv7ptbq.gmgdl@evledraar.gmail.com","threadId":"58748","inReplyTo":"Y2wuEDGMRg99Ifef@nand.local","subject":"Re: [PATCH] t7610: fix flaky timeout issue, don't clone from example.com","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-11-09T23:55:30Z","receivedAt":"2022-11-10T00:05:35Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Wed, Nov 09 2022, Taylor Blau wrote:\n\n> On Sat, Nov 05, 2022 at 12:54:21PM +0100, Ævar Arnfjörð Bjarmason wrote:\n>> The behavior of \"-N\" here might be surprising to some, since it's\n>> explained as \"[if you use -N we] don’t fetch new objects from the\n>> remote site\". But (perhaps counter-intuitively) it's only talking\n>> about if it needs to do so via \"git fetch\". In this case we'll end up\n>> spawning a \"git clone\", as we have no submodule set up.\n>\n> Makes sense, though I'm not sure I agree this is worth patching as I\n> currently understand it.\n>\n> If I run t7610 today with '--run=2-' (IOW, skipping the setup test), I\n> am definitely going to get failures. And I don't think we should have\n> every subsequent test depend on having run anything containing \"setup\"\n> before it. That is, it is not surprising that we will see some test\n> failures when carving up and running portions of the test instead of the\n> whole file.\n>\n> I don't think this is substantively any different than that. Whether we\n> don't complete the \"setup\" test because we had some leak (and ran the\n> test suite with the appropriate LSan options), or because we neglected\n> to run it at all, I don't think there is a significant difference.\n>\n> Either way, the end-state of the test repository isn't guaranteed to\n> match the intent of the \"setup\" test.\n>\n> If this is the only such problem in-tree, sure, I think it's fine to\n> patch. But I would wager that there are *many* more than just this one\n> lurking, and patching all of them would be less straightforward than\n> this one.\n>\n> So... I don't know. I'm certainly leaning negative on this patch, but if\n> you have some compelling reason that I'm missing, I'm all-ears.\n\nI agree with that in general, but the expected failure case for all\nthose other things that are broken with the missing setup is just that\nwe'll predictably error out immediately.\n\nWhereas in this case we'll end up hanging on a connect() to some box\nthat IANA's maintaining for the example.com landing page, and after we\nreach the connect() timeout we move onto the next failing test, which\nwill do that again. The test takes a long time to finally report an\noverall failure.\n\nI think that's worth fixing in general, aside from the leak\ndetection. I.e. yes in general we aren't good about running tests 3..9\nsuccessfully if the 2nd test fails.\n\nBut we generally just fail some or all of 3..9 pretty fast, and don't\nstart taking 20 minutes to run the test, when it took 10s before (or\nwhatever).\n\n\n\n"},{"id":"467020","messageId":"Y2xfNcCfkij6k4Uk@nand.local","threadId":"58748","inReplyTo":"221110.864jv7ptbq.gmgdl@evledraar.gmail.com","subject":"Re: [PATCH] t7610: fix flaky timeout issue, don't clone from example.com","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2022-11-10T02:17:25Z","receivedAt":"2022-11-10T02:17:31Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Thu, Nov 10, 2022 at 12:55:30AM +0100, Ævar Arnfjörð Bjarmason wrote:\n> But we generally just fail some or all of 3..9 pretty fast, and don't\n> start taking 20 minutes to run the test, when it took 10s before (or\n> whatever).\n\nOK. I still think that in principle this is indistinguishable from not\nrunning a setup test to completion.\n\n*But*: I'm willing to treat them differently since instead of\nmanifesting in an immediate failure later on in the suite, we hang for a\nsubstantial period of time.\n\nSo I'm content to merge this down, but I don't think it's worth\nsearching out for more of these in the future.\n\nThanks,\nTaylor\n"},{"id":"467336","messageId":"Y3PrjMLcEGfoHifZ@nand.local","threadId":"58748","inReplyTo":"Y2xfNcCfkij6k4Uk@nand.local","subject":"Re: [PATCH] t7610: fix flaky timeout issue, don't clone from example.com","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2022-11-15T19:42:04Z","receivedAt":"2022-11-15T19:42:12Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Wed, Nov 09, 2022 at 09:17:25PM -0500, Taylor Blau wrote:\n> On Thu, Nov 10, 2022 at 12:55:30AM +0100, Ævar Arnfjörð Bjarmason wrote:\n> > But we generally just fail some or all of 3..9 pretty fast, and don't\n> > start taking 20 minutes to run the test, when it took 10s before (or\n> > whatever).\n>\n> OK. I still think that in principle this is indistinguishable from not\n> running a setup test to completion.\n>\n> *But*: I'm willing to treat them differently since instead of\n> manifesting in an immediate failure later on in the suite, we hang for a\n> substantial period of time.\n>\n> So I'm content to merge this down, but I don't think it's worth\n> searching out for more of these in the future.\n\nHaving merged this down to 'next', it looks like there is some CI\nfallout on the Windows tests, see:\n\n  https://github.com/ttaylorr/git/actions/runs/3473324797/jobs/5805324776\n\nI am not sure how I might have caught this earlier not having a Windows\nmachine myself. Regardless, let's make sure that it is fixed up before\nthis graduates.\n\nThanks,\nTaylor\n"},{"id":"467347","messageId":"patch-1.1-21a9cb49299-20221115T233516Z-avarab@gmail.com","threadId":"58748","inReplyTo":"Y3PrjMLcEGfoHifZ@nand.local","subject":"[PATCH] t7610: use \"file:///dev/null\", not \"/dev/null\", fixes MinGW","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-11-15T23:40:14Z","receivedAt":"2022-11-15T23:40:43Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"On MinGW the \"/dev/null\" is translated to \"nul\" on command-lines, even\nthough as in this case it'll never end up referring to an actual file.\n\nSo on Windows the fix for the previous \"example.com\" timeout issue in\n8354cf752ec (t7610: fix flaky timeout issue, don't clone from\nexample.com, 2022-11-05) would yield:\n\n  fatal: repo URL: 'nul' must be absolute or begin with ./|../\n\nLet's evade this yet again by prefixing this with \"file://\", which\nmakes this pass in the Windows CI.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>\n---\n\nOn Tue, Nov 15 2022, Taylor Blau wrote:\n\n> On Wed, Nov 09, 2022 at 09:17:25PM -0500, Taylor Blau wrote:\n>> On Thu, Nov 10, 2022 at 12:55:30AM +0100, Ævar Arnfjörð Bjarmason wrote:\n>> > But we generally just fail some or all of 3..9 pretty fast, and don't\n>> > start taking 20 minutes to run the test, when it took 10s before (or\n>> > whatever).\n>>\n>> OK. I still think that in principle this is indistinguishable from not\n>> running a setup test to completion.\n>>\n>> *But*: I'm willing to treat them differently since instead of\n>> manifesting in an immediate failure later on in the suite, we hang for a\n>> substantial period of time.\n>>\n>> So I'm content to merge this down, but I don't think it's worth\n>> searching out for more of these in the future.\n>\n> Having merged this down to 'next', it looks like there is some CI\n> fallout on the Windows tests, see:\n>\n>   https://github.com/ttaylorr/git/actions/runs/3473324797/jobs/5805324776\n>\n> I am not sure how I might have caught this earlier not having a Windows\n> machine myself. Regardless, let's make sure that it is fixed up before\n> this graduates.\n\nSorry again :( I think my CI was quite queued up at the time, and I\nfigured surely *this* won't have any porability issues, but forgot\nabout MinGW's sneaky /dev/null string-replacement behavior.\n\nWindows CI passes with this:\nhttps://github.com/avar/git/actions/runs/3473817195\n\n t/t7610-mergetool.sh | 4 ++--\n 1 file changed, 2 insertions(+), 2 deletions(-)\n\ndiff --git a/t/t7610-mergetool.sh b/t/t7610-mergetool.sh\nindex b1ba0d9a088..7b957022f1a 100755\n--- a/t/t7610-mergetool.sh\n+++ b/t/t7610-mergetool.sh\n@@ -33,7 +33,7 @@ test_expect_success 'setup' '\n \t\tgit add foo &&\n \t\tgit commit -m \"Add foo\"\n \t) &&\n-\tgit submodule add /dev/null submod &&\n+\tgit submodule add file:///dev/null submod &&\n \tgit add file1 \"spaced name\" file1[1-4] subdir/file3 .gitmodules submod &&\n \tgit commit -m \"add initial versions\" &&\n \n@@ -614,7 +614,7 @@ test_expect_success 'submodule in subdirectory' '\n \t\t)\n \t) &&\n \ttest_when_finished \"rm -rf subdir/subdir_module\" &&\n-\tgit submodule add /dev/null subdir/subdir_module &&\n+\tgit submodule add file:///dev/null subdir/subdir_module &&\n \tgit add subdir/subdir_module &&\n \tgit commit -m \"add submodule in subdirectory\" &&\n \n-- \n2.38.0.1473.g172bcc0511c\n\n"},{"id":"467350","messageId":"Y3Q3xQeaXGPemcWv@nand.local","threadId":"58748","inReplyTo":"patch-1.1-21a9cb49299-20221115T233516Z-avarab@gmail.com","subject":"Re: [PATCH] t7610: use \"file:///dev/null\", not \"/dev/null\", fixes MinGW","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2022-11-16T01:07:17Z","receivedAt":"2022-11-16T01:07:25Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Wed, Nov 16, 2022 at 12:40:14AM +0100, Ævar Arnfjörð Bjarmason wrote:\n> On MinGW the \"/dev/null\" is translated to \"nul\" on command-lines, even\n> though as in this case it'll never end up referring to an actual file.\n>\n> So on Windows the fix for the previous \"example.com\" timeout issue in\n> 8354cf752ec (t7610: fix flaky timeout issue, don't clone from\n> example.com, 2022-11-05) would yield:\n>\n>   fatal: repo URL: 'nul' must be absolute or begin with ./|../\n>\n> Let's evade this yet again by prefixing this with \"file://\", which\n> makes this pass in the Windows CI.\n\nThanks very much. I just picked this up on top of my 'ab/t7610-timeout'\nbranch and pushed it out to ttaylorr/git to double-check your results\nagainst CI. (Not that I don't trust you, of course, but I'm feeling like\nI should be extra-cautious ;-)).\n\nAssuming that all looks good, then I'll push out a new version of 'next'\nand dependent branches to the usual spots so that we can get 'next' back\nto green.\n\nThanks for working on this so quickly.\n\nThanks,\nTaylor\n"},{"id":"467427","messageId":"221117.86o7t5j3jd.gmgdl@evledraar.gmail.com","threadId":"58748","inReplyTo":"Y3Q3xQeaXGPemcWv@nand.local","subject":"Re: [PATCH] t7610: use \"file:///dev/null\", not \"/dev/null\", fixes MinGW","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-11-17T10:08:08Z","receivedAt":"2022-11-17T10:09:45Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Tue, Nov 15 2022, Taylor Blau wrote:\n\n> On Wed, Nov 16, 2022 at 12:40:14AM +0100, Ævar Arnfjörð Bjarmason wrote:\n>> On MinGW the \"/dev/null\" is translated to \"nul\" on command-lines, even\n>> though as in this case it'll never end up referring to an actual file.\n>>\n>> So on Windows the fix for the previous \"example.com\" timeout issue in\n>> 8354cf752ec (t7610: fix flaky timeout issue, don't clone from\n>> example.com, 2022-11-05) would yield:\n>>\n>>   fatal: repo URL: 'nul' must be absolute or begin with ./|../\n>>\n>> Let's evade this yet again by prefixing this with \"file://\", which\n>> makes this pass in the Windows CI.\n>\n> Thanks very much. I just picked this up on top of my 'ab/t7610-timeout'\n> branch and pushed it out to ttaylorr/git to double-check your results\n> against CI. (Not that I don't trust you, of course, but I'm feeling like\n> I should be extra-cautious ;-)).\n>\n> Assuming that all looks good, then I'll push out a new version of 'next'\n> and dependent branches to the usual spots so that we can get 'next' back\n> to green.\n>\n> Thanks for working on this so quickly.\n\nAnd thank for picking it up! Looks like \"next\" is passing CI now:\nhttps://github.com/git/git/actions/runs/3475751137/jobs/5810308794\n\n"},{"id":"467464","messageId":"Y3asprytS9MY+TMH@nand.local","threadId":"58748","inReplyTo":"221117.86o7t5j3jd.gmgdl@evledraar.gmail.com","subject":"Re: [PATCH] t7610: use \"file:///dev/null\", not \"/dev/null\", fixes MinGW","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2022-11-17T21:50:30Z","receivedAt":"2022-11-17T21:50:36Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Thu, Nov 17, 2022 at 11:08:08AM +0100, Ævar Arnfjörð Bjarmason wrote:\n> And thank for picking it up! Looks like \"next\" is passing CI now:\n> https://github.com/git/git/actions/runs/3475751137/jobs/5810308794\n\nNo problem. I dislike pushing out 'next' without a corresponding WC\nreport, but I dislike 'next' failing CI even more. So I felt like here\nit was OK.\n\nThanks,\nTaylor\n"}]}