{"thread":{"id":"50410","subject":"t0025 flakey?","startedAt":"2019-02-06T10:25:42Z","lastAt":"2019-02-08T10:22:06Z","messageCount":14,"participants":["Johannes Schindelin","SZEDER Gábor","Torsten Bögershausen","Randall S. Becker","Jeff King","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"368573","messageId":"nycvar.QRO.7.76.6.1902061123410.41@tvgsbejvaqbjf.bet","threadId":"50410","inReplyTo":null,"subject":"t0025 flakey?","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2019-02-06T10:25:38Z","receivedAt":"2019-02-06T10:25:42Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Torsten,\n\nat first I thought that those intermittent test failures were limited to\nWindows, but they are not: I can see it now in a build on 32-bit Linux.\nFull logs here:\n\nhttps://dev.azure.com/gitgitgadget/git/_build/results?buildId=1032&_a=summary&view=ms.vss-test-web.build-test-results-tab\n\nExcerpt from the failing test case:\n\n-- snip --\nnot ok 2 - renormalize CRLF in repo\n expecting success: \n\techo \"*.txt text=auto\" >.gitattributes &&\n\tgit add --renormalize \"*.txt\" &&\n\tcat >expect <<-\\EOF &&\n\ti/lf w/crlf attr/text=auto CRLF.txt\n\ti/lf w/lf attr/text=auto LF.txt\n\ti/lf w/mixed attr/text=auto CRLF_mix_LF.txt\n\tEOF\n\tgit ls-files --eol |\n\tsed -e \"s/\t/ /g\" -e \"s/ */ /g\" |\n\tsort >actual &&\n\ttest_cmp expect actual\n\n+ echo *.txt text=auto\n+ git add --renormalize *.txt\n+ cat\n+ sort\n+ sed -e s/\t/ /g -e s/ */ /g\n+ git ls-files --eol\n+ test_cmp expect actual\n+ diff -u expect actual\n--- expect\t2019-02-06 09:39:42.080733629 +0000\n+++ actual\t2019-02-06 09:39:42.088733629 +0000\n@@ -1,3 +1,3 @@\n-i/lf w/crlf attr/text=auto CRLF.txt\n+i/crlf w/crlf attr/text=auto CRLF.txt\n i/lf w/lf attr/text=auto LF.txt\n-i/lf w/mixed attr/text=auto CRLF_mix_LF.txt\n+i/mixed w/mixed attr/text=auto CRLF_mix_LF.txt\nerror: last command exited with $?=1\n-- snap --\n\nAny ideas?\nJohannes\n"},{"id":"368575","messageId":"20190206104243.GJ10587@szeder.dev","threadId":"50410","inReplyTo":"nycvar.QRO.7.76.6.1902061123410.41@tvgsbejvaqbjf.bet","subject":"Re: t0025 flakey?","fromName":"SZEDER Gábor","fromEmail":"szeder.dev@gmail.com","sentAt":"2019-02-06T10:42:43Z","receivedAt":"2019-02-06T10:42:49Z","isPatch":false,"sender":{"key":"szeder.dev@gmail.com","avatar":"https://avatars.githubusercontent.com/u/116324?v=4"},"body":"On Wed, Feb 06, 2019 at 11:25:38AM +0100, Johannes Schindelin wrote:\n> Hi Torsten,\n> \n> at first I thought that those intermittent test failures were limited to\n> Windows, but they are not: I can see it now in a build on 32-bit Linux.\n> Full logs here:\n> \n> https://dev.azure.com/gitgitgadget/git/_build/results?buildId=1032&_a=summary&view=ms.vss-test-web.build-test-results-tab\n> \n> Excerpt from the failing test case:\n> \n> -- snip --\n> not ok 2 - renormalize CRLF in repo\n> expecting success: \n> \techo \"*.txt text=auto\" >.gitattributes &&\n> \tgit add --renormalize \"*.txt\" &&\n> \tcat >expect <<-\\EOF &&\n> \ti/lf w/crlf attr/text=auto CRLF.txt\n> \ti/lf w/lf attr/text=auto LF.txt\n> \ti/lf w/mixed attr/text=auto CRLF_mix_LF.txt\n> \tEOF\n> \tgit ls-files --eol |\n> \tsed -e \"s/\t/ /g\" -e \"s/ */ /g\" |\n> \tsort >actual &&\n> \ttest_cmp expect actual\n> \n> + echo *.txt text=auto\n> + git add --renormalize *.txt\n> + cat\n> + sort\n> + sed -e s/\t/ /g -e s/ */ /g\n> + git ls-files --eol\n> + test_cmp expect actual\n> + diff -u expect actual\n> --- expect\t2019-02-06 09:39:42.080733629 +0000\n> +++ actual\t2019-02-06 09:39:42.088733629 +0000\n> @@ -1,3 +1,3 @@\n> -i/lf w/crlf attr/text=auto CRLF.txt\n> +i/crlf w/crlf attr/text=auto CRLF.txt\n> i/lf w/lf attr/text=auto LF.txt\n> -i/lf w/mixed attr/text=auto CRLF_mix_LF.txt\n> +i/mixed w/mixed attr/text=auto CRLF_mix_LF.txt\n> error: last command exited with $?=1\n> -- snap --\n> \n> Any ideas?\n\nI reported this and Peff looked into it on the way to Git Merge, but\nnot working solution yet.\n\nhttps://public-inbox.org/git/20190129225121.GD1895@sigill.intra.peff.net/T/#u\n\n"},{"id":"368588","messageId":"nycvar.QRO.7.76.6.1902061450280.41@tvgsbejvaqbjf.bet","threadId":"50410","inReplyTo":"20190206104243.GJ10587@szeder.dev","subject":"Re: t0025 flakey?","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2019-02-06T13:52:53Z","receivedAt":"2019-02-06T13:53:04Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Gábor,\n\nOn Wed, 6 Feb 2019, SZEDER Gábor wrote:\n\n> On Wed, Feb 06, 2019 at 11:25:38AM +0100, Johannes Schindelin wrote:\n> \n> > at first I thought that those intermittent test failures were limited\n> > to Windows, but they are not: I can see it now in a build on 32-bit\n> > Linux.\n> > Full logs here:\n> > \n> > https://dev.azure.com/gitgitgadget/git/_build/results?buildId=1032&_a=summary&view=ms.vss-test-web.build-test-results-tab\n> > \n> > Excerpt from the failing test case:\n> > \n> > -- snip --\n> > not ok 2 - renormalize CRLF in repo\n> > expecting success: \n> > \techo \"*.txt text=auto\" >.gitattributes &&\n> > \tgit add --renormalize \"*.txt\" &&\n> > \tcat >expect <<-\\EOF &&\n> > \ti/lf w/crlf attr/text=auto CRLF.txt\n> > \ti/lf w/lf attr/text=auto LF.txt\n> > \ti/lf w/mixed attr/text=auto CRLF_mix_LF.txt\n> > \tEOF\n> > \tgit ls-files --eol |\n> > \tsed -e \"s/\t/ /g\" -e \"s/ */ /g\" |\n> > \tsort >actual &&\n> > \ttest_cmp expect actual\n> > \n> > + echo *.txt text=auto\n> > + git add --renormalize *.txt\n> > + cat\n> > + sort\n> > + sed -e s/\t/ /g -e s/ */ /g\n> > + git ls-files --eol\n> > + test_cmp expect actual\n> > + diff -u expect actual\n> > --- expect\t2019-02-06 09:39:42.080733629 +0000\n> > +++ actual\t2019-02-06 09:39:42.088733629 +0000\n> > @@ -1,3 +1,3 @@\n> > -i/lf w/crlf attr/text=auto CRLF.txt\n> > +i/crlf w/crlf attr/text=auto CRLF.txt\n> > i/lf w/lf attr/text=auto LF.txt\n> > -i/lf w/mixed attr/text=auto CRLF_mix_LF.txt\n> > +i/mixed w/mixed attr/text=auto CRLF_mix_LF.txt\n> > error: last command exited with $?=1\n> > -- snap --\n> > \n> > Any ideas?\n> \n> I reported this and Peff looked into it on the way to Git Merge, but\n> not working solution yet.\n> \n> https://public-inbox.org/git/20190129225121.GD1895@sigill.intra.peff.net/T/#u\n\nThank you!\nDscho"},{"id":"368595","messageId":"20190206171517.s5lskawpdodc74ui@tb-raspi4","threadId":"50410","inReplyTo":"nycvar.QRO.7.76.6.1902061450280.41@tvgsbejvaqbjf.bet","subject":"Re: t0025 flakey?","fromName":"Torsten Bögershausen","fromEmail":"tboegi@web.de","sentAt":"2019-02-06T17:15:17Z","receivedAt":"2019-02-06T17:15:29Z","isPatch":false,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"On Wed, Feb 06, 2019 at 02:52:53PM +0100, Johannes Schindelin wrote:\n> Hi Gábor,\n>\n> On Wed, 6 Feb 2019, SZEDER Gábor wrote:\n>\n> > On Wed, Feb 06, 2019 at 11:25:38AM +0100, Johannes Schindelin wrote:\n> >\n> > > at first I thought that those intermittent test failures were limited\n> > > to Windows, but they are not: I can see it now in a build on 32-bit\n> > > Linux.\n> > > Full logs here:\n> > >\n> > > https://dev.azure.com/gitgitgadget/git/_build/results?buildId=1032&_a=summary&view=ms.vss-test-web.build-test-results-tab\n> > >\n> > > Excerpt from the failing test case:\n> > >\n> > > -- snip --\n> > > not ok 2 - renormalize CRLF in repo\n> > > expecting success:\n> > > \techo \"*.txt text=auto\" >.gitattributes &&\n> > > \tgit add --renormalize \"*.txt\" &&\n> > > \tcat >expect <<-\\EOF &&\n> > > \ti/lf w/crlf attr/text=auto CRLF.txt\n> > > \ti/lf w/lf attr/text=auto LF.txt\n> > > \ti/lf w/mixed attr/text=auto CRLF_mix_LF.txt\n> > > \tEOF\n> > > \tgit ls-files --eol |\n> > > \tsed -e \"s/\t/ /g\" -e \"s/ */ /g\" |\n> > > \tsort >actual &&\n> > > \ttest_cmp expect actual\n> > >\n> > > + echo *.txt text=auto\n> > > + git add --renormalize *.txt\n> > > + cat\n> > > + sort\n> > > + sed -e s/\t/ /g -e s/ */ /g\n> > > + git ls-files --eol\n> > > + test_cmp expect actual\n> > > + diff -u expect actual\n> > > --- expect\t2019-02-06 09:39:42.080733629 +0000\n> > > +++ actual\t2019-02-06 09:39:42.088733629 +0000\n> > > @@ -1,3 +1,3 @@\n> > > -i/lf w/crlf attr/text=auto CRLF.txt\n> > > +i/crlf w/crlf attr/text=auto CRLF.txt\n> > > i/lf w/lf attr/text=auto LF.txt\n> > > -i/lf w/mixed attr/text=auto CRLF_mix_LF.txt\n> > > +i/mixed w/mixed attr/text=auto CRLF_mix_LF.txt\n> > > error: last command exited with $?=1\n> > > -- snap --\n> > >\n> > > Any ideas?\n> >\n> > I reported this and Peff looked into it on the way to Git Merge, but\n> > not working solution yet.\n> >\n> > https://public-inbox.org/git/20190129225121.GD1895@sigill.intra.peff.net/T/#u\n>\n> Thank you!\n> Dscho\n\nI shortly looked into the pointers here -\nIs t0025 flaky after the fix from Peff:\n\n[PATCH] add: use separate ADD_CACHE_RENORMALIZE flag\n\nOr has it always been shaky ?\nDoes anybody know ?\n"},{"id":"368614","messageId":"20190206173932.GM10587@szeder.dev","threadId":"50410","inReplyTo":"20190206171517.s5lskawpdodc74ui@tb-raspi4","subject":"Re: t0025 flakey?","fromName":"SZEDER Gábor","fromEmail":"szeder.dev@gmail.com","sentAt":"2019-02-06T17:39:32Z","receivedAt":"2019-02-06T17:39:39Z","isPatch":false,"sender":{"key":"szeder.dev@gmail.com","avatar":"https://avatars.githubusercontent.com/u/116324?v=4"},"body":"On Wed, Feb 06, 2019 at 05:15:17PM +0000, Torsten Bögershausen wrote:\n> On Wed, Feb 06, 2019 at 02:52:53PM +0100, Johannes Schindelin wrote:\n> > Hi Gábor,\n> >\n> > On Wed, 6 Feb 2019, SZEDER Gábor wrote:\n> >\n> > > On Wed, Feb 06, 2019 at 11:25:38AM +0100, Johannes Schindelin wrote:\n> > >\n> > > > at first I thought that those intermittent test failures were limited\n> > > > to Windows, but they are not: I can see it now in a build on 32-bit\n> > > > Linux.\n> > > > Full logs here:\n> > > >\n> > > > https://dev.azure.com/gitgitgadget/git/_build/results?buildId=1032&_a=summary&view=ms.vss-test-web.build-test-results-tab\n> > > >\n> > > > Excerpt from the failing test case:\n> > > >\n> > > > -- snip --\n> > > > not ok 2 - renormalize CRLF in repo\n> > > > expecting success:\n> > > > \techo \"*.txt text=auto\" >.gitattributes &&\n> > > > \tgit add --renormalize \"*.txt\" &&\n> > > > \tcat >expect <<-\\EOF &&\n> > > > \ti/lf w/crlf attr/text=auto CRLF.txt\n> > > > \ti/lf w/lf attr/text=auto LF.txt\n> > > > \ti/lf w/mixed attr/text=auto CRLF_mix_LF.txt\n> > > > \tEOF\n> > > > \tgit ls-files --eol |\n> > > > \tsed -e \"s/\t/ /g\" -e \"s/ */ /g\" |\n> > > > \tsort >actual &&\n> > > > \ttest_cmp expect actual\n> > > >\n> > > > + echo *.txt text=auto\n> > > > + git add --renormalize *.txt\n> > > > + cat\n> > > > + sort\n> > > > + sed -e s/\t/ /g -e s/ */ /g\n> > > > + git ls-files --eol\n> > > > + test_cmp expect actual\n> > > > + diff -u expect actual\n> > > > --- expect\t2019-02-06 09:39:42.080733629 +0000\n> > > > +++ actual\t2019-02-06 09:39:42.088733629 +0000\n> > > > @@ -1,3 +1,3 @@\n> > > > -i/lf w/crlf attr/text=auto CRLF.txt\n> > > > +i/crlf w/crlf attr/text=auto CRLF.txt\n> > > > i/lf w/lf attr/text=auto LF.txt\n> > > > -i/lf w/mixed attr/text=auto CRLF_mix_LF.txt\n> > > > +i/mixed w/mixed attr/text=auto CRLF_mix_LF.txt\n> > > > error: last command exited with $?=1\n> > > > -- snap --\n> > > >\n> > > > Any ideas?\n> > >\n> > > I reported this and Peff looked into it on the way to Git Merge, but\n> > > not working solution yet.\n> > >\n> > > https://public-inbox.org/git/20190129225121.GD1895@sigill.intra.peff.net/T/#u\n> >\n> > Thank you!\n> > Dscho\n> \n> I shortly looked into the pointers here -\n> Is t0025 flaky after the fix from Peff:\n> \n> [PATCH] add: use separate ADD_CACHE_RENORMALIZE flag\n> \n> Or has it always been shaky ?\n> Does anybody know ?\n\nI sort-of bisected it, and it pointed to Peff's fix.\n\nRunning 't0025 --stress' in the merge of 'sg/stress-test' and\n'jk/add-ignore-errors-bit-assignment-fix' results in a failure within\nseconds or <20 repetitions. In the merge with\n'jk/add-ignore-errors-bit-assignment-fix^' it runs successfully for\nthousands of repetitions.\n\n\n"},{"id":"368617","messageId":"005501d4be45$e4121c40$ac3654c0$@nexbridge.com","threadId":"50410","inReplyTo":"20190206171517.s5lskawpdodc74ui@tb-raspi4","subject":"RE: t0025 flakey?","fromName":"Randall S. Becker","fromEmail":"rsbecker@nexbridge.com","sentAt":"2019-02-06T18:00:43Z","receivedAt":"2019-02-06T18:01:03Z","isPatch":false,"sender":{"key":"randall.becker@nexbridge.ca","avatar":"https://avatars.githubusercontent.com/u/28956764?v=4"},"body":"On February 6, 2019 12:15, Torsten Bögershausen wrote:\n> To: Johannes Schindelin \n> Cc: SZEDER Gábor ; Jeff King ;\n> git@vger.kernel.org\n> Subject: Re: t0025 flakey?\n> \n> On Wed, Feb 06, 2019 at 02:52:53PM +0100, Johannes Schindelin wrote:\n> > Hi Gábor,\n> >\n> > On Wed, 6 Feb 2019, SZEDER Gábor wrote:\n> >\n> > > On Wed, Feb 06, 2019 at 11:25:38AM +0100, Johannes Schindelin wrote:\n> > >\n> > > > at first I thought that those intermittent test failures were\n> > > > limited to Windows, but they are not: I can see it now in a build\n> > > > on 32-bit Linux.\n> > > > Full logs here:\n> > > >\n> > > > https://dev.azure.com/gitgitgadget/git/_build/results?buildId=1032\n> > > > &_a=summary&view=ms.vss-test-web.build-test-results-tab\n> > > >\n> > > > Excerpt from the failing test case:\n> > > >\n> > > > -- snip --\n> > > > not ok 2 - renormalize CRLF in repo expecting success:\n> > > > \techo \"*.txt text=auto\" >.gitattributes &&\n> > > > \tgit add --renormalize \"*.txt\" &&\n> > > > \tcat >expect <<-\\EOF &&\n> > > > \ti/lf w/crlf attr/text=auto CRLF.txt\n> > > > \ti/lf w/lf attr/text=auto LF.txt\n> > > > \ti/lf w/mixed attr/text=auto CRLF_mix_LF.txt\n> > > > \tEOF\n> > > > \tgit ls-files --eol |\n> > > > \tsed -e \"s/\t/ /g\" -e \"s/ */ /g\" |\n> > > > \tsort >actual &&\n> > > > \ttest_cmp expect actual\n> > > >\n> > > > + echo *.txt text=auto\n> > > > + git add --renormalize *.txt\n> > > > + cat\n> > > > + sort\n> > > > + sed -e s/\t/ /g -e s/ */ /g\n> > > > + git ls-files --eol\n> > > > + test_cmp expect actual\n> > > > + diff -u expect actual\n> > > > --- expect\t2019-02-06 09:39:42.080733629 +0000\n> > > > +++ actual\t2019-02-06 09:39:42.088733629 +0000\n> > > > @@ -1,3 +1,3 @@\n> > > > -i/lf w/crlf attr/text=auto CRLF.txt\n> > > > +i/crlf w/crlf attr/text=auto CRLF.txt\n> > > > i/lf w/lf attr/text=auto LF.txt\n> > > > -i/lf w/mixed attr/text=auto CRLF_mix_LF.txt\n> > > > +i/mixed w/mixed attr/text=auto CRLF_mix_LF.txt\n> > > > error: last command exited with $?=1\n> > > > -- snap --\n> > > >\n> > > > Any ideas?\n> > >\n> > > I reported this and Peff looked into it on the way to Git Merge, but\n> > > not working solution yet.\n> > >\n> > > https://public-inbox.org/git/20190129225121.GD1895@sigill.intra.peff\n> > > .net/T/#u\n> >\n> > Thank you!\n> > Dscho\n> \n> I shortly looked into the pointers here - Is t0025 flaky after the fix\nfrom Peff:\n> \n> [PATCH] add: use separate ADD_CACHE_RENORMALIZE flag\n> \n> Or has it always been shaky ?\n> Does anybody know ?\n\nThe NonStop port has traditionally had issues with t0025, which we tended to\nignore because things did work. We wrote those off as bash issues in t0025\nsince they seemed to be corrected when we picked up a new bash version about\na year ago. I will keep monitoring this, particularly when 2.21 comes out.\n\nCheers,\nRandall\n\n-- Brief whoami:\n NonStop developer since approximately 211288444200000000\n UNIX developer since approximately 421664400\n-- In my real life, I talk too much.\n\n\n\n"},{"id":"368657","messageId":"20190207020022.GA29974@sigill.intra.peff.net","threadId":"50410","inReplyTo":"20190206104243.GJ10587@szeder.dev","subject":"[PATCH] add_to_index(): convert forgotten HASH_RENORMALIZE check","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2019-02-07T02:00:22Z","receivedAt":"2019-02-07T02:00:27Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Feb 06, 2019 at 11:42:43AM +0100, SZEDER Gábor wrote:\n\n> I reported this and Peff looked into it on the way to Git Merge, but\n> not working solution yet.\n> \n> https://public-inbox.org/git/20190129225121.GD1895@sigill.intra.peff.net/T/#u\n\nOof. Well, now I know why my attempts to fix the test failed. It was not\nmy new test that was failing at all, but rather the existing test. Which\nimplies that I severely bungled the actual code change.\n\nArmed with that knowledge, it was pretty easy to find said bungling. The\nfix is below.\n\nJunio, this should go on top of jk/add-ignore-errors-bit-assignment-fix\nas soon as possible, as the regression is already in master. And I'll go\nfind a brown paper bag. ;)\n\n-- >8 --\nSubject: [PATCH] add_to_index(): convert forgotten HASH_RENORMALIZE check\n\nCommit 9e5da3d055 (add: use separate ADD_CACHE_RENORMALIZE flag,\n2019-01-17) switched out using HASH_RENORMALIZE in our flags field for a\nnew ADD_CACHE_RENORMALIZE flag. However, it forgot to convert one of the\nchecks for HASH_RENORMALIZE into the new flag, which totally broke \"git\nadd --renormalize\".\n\nTo make matters even more confusing, the resulting code would racily\npass the tests! The forgotten check was responsible for defeating the\nup-to-date check of the index entry. That meant that \"git add\n--renormalize\" would refuse to renormalize things that appeared\nstat-clean. But most of the time the test commands run fast enough that\nthe file mtime is the same as the index mtime. And thus we err on the\nconservative side and re-hash the file, which is what \"--renormalize\"\nwould have wanted.\n\nBut if you're unlucky and cross that one-second boundary between writing\nthe file and writing the index (which is more likely to happen on a slow\nor heavily-loaded system), then the file appears stat-clean. And\n\"--renormalize\" would effectively do nothing.\n\nThe fix is straightforward: convert this check to use the right flag.\n\nNoticed-by: SZEDER Gábor \nSigned-off-by: Jeff King \n---\n read-cache.c | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/read-cache.c b/read-cache.c\nindex 9783c493a3..accc059951 100644\n--- a/read-cache.c\n+++ b/read-cache.c\n@@ -746,7 +746,7 @@ int add_to_index(struct index_state *istate, const char *path, struct stat *st,\n \tif (ignore_case) {\n \t\tadjust_dirname_case(istate, ce->name);\n \t}\n-\tif (!(flags & HASH_RENORMALIZE)) {\n+\tif (!(flags & ADD_CACHE_RENORMALIZE)) {\n \t\talias = index_file_exists(istate, ce->name,\n \t\t\t\t\t ce_namelen(ce), ignore_case);\n \t\tif (alias &&\n-- \n2.20.1.1122.g2972e48916\n\n"},{"id":"368662","messageId":"20190207041802.ai25fc64sqfg2h66@tb-raspi4","threadId":"50410","inReplyTo":"20190207020022.GA29974@sigill.intra.peff.net","subject":"Re: [PATCH] add_to_index(): convert forgotten HASH_RENORMALIZE check","fromName":"Torsten Bögershausen","fromEmail":"tboegi@web.de","sentAt":"2019-02-07T04:18:02Z","receivedAt":"2019-02-07T04:18:18Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"On Wed, Feb 06, 2019 at 09:00:22PM -0500, Jeff King wrote:\n> On Wed, Feb 06, 2019 at 11:42:43AM +0100, SZEDER Gábor wrote:\n>\n> > I reported this and Peff looked into it on the way to Git Merge, but\n> > not working solution yet.\n> >\n> > https://public-inbox.org/git/20190129225121.GD1895@sigill.intra.peff.net/T/#u\n>\n> Oof. Well, now I know why my attempts to fix the test failed. It was not\n> my new test that was failing at all, but rather the existing test. Which\n> implies that I severely bungled the actual code change.\n>\n> Armed with that knowledge, it was pretty easy to find said bungling. The\n> fix is below.\n>\n> Junio, this should go on top of jk/add-ignore-errors-bit-assignment-fix\n> as soon as possible, as the regression is already in master. And I'll go\n> find a brown paper bag. ;)\n>\n> -- >8 --\n> Subject: [PATCH] add_to_index(): convert forgotten HASH_RENORMALIZE check\n>\n> Commit 9e5da3d055 (add: use separate ADD_CACHE_RENORMALIZE flag,\n> 2019-01-17) switched out using HASH_RENORMALIZE in our flags field for a\n> new ADD_CACHE_RENORMALIZE flag. However, it forgot to convert one of the\n> checks for HASH_RENORMALIZE into the new flag, which totally broke \"git\n> add --renormalize\".\n>\n> To make matters even more confusing, the resulting code would racily\n> pass the tests! The forgotten check was responsible for defeating the\n> up-to-date check of the index entry. That meant that \"git add\n> --renormalize\" would refuse to renormalize things that appeared\n> stat-clean. But most of the time the test commands run fast enough that\n> the file mtime is the same as the index mtime. And thus we err on the\n> conservative side and re-hash the file, which is what \"--renormalize\"\n> would have wanted.\n>\n> But if you're unlucky and cross that one-second boundary between writing\n> the file and writing the index (which is more likely to happen on a slow\n> or heavily-loaded system), then the file appears stat-clean. And\n> \"--renormalize\" would effectively do nothing.\n>\n> The fix is straightforward: convert this check to use the right flag.\n>\n> Noticed-by: SZEDER Gábor \n> Signed-off-by: Jeff King \n> ---\n> read-cache.c | 2 +-\n> 1 file changed, 1 insertion(+), 1 deletion(-)\n>\n> diff --git a/read-cache.c b/read-cache.c\n> index 9783c493a3..accc059951 100644\n> --- a/read-cache.c\n> +++ b/read-cache.c\n> @@ -746,7 +746,7 @@ int add_to_index(struct index_state *istate, const char *path, struct stat *st,\n> \tif (ignore_case) {\n> \t\tadjust_dirname_case(istate, ce->name);\n> \t}\n> -\tif (!(flags & HASH_RENORMALIZE)) {\n> +\tif (!(flags & ADD_CACHE_RENORMALIZE)) {\n> \t\talias = index_file_exists(istate, ce->name,\n> \t\t\t\t\t ce_namelen(ce), ignore_case);\n> \t\tif (alias &&\n> --\n> 2.20.1.1122.g2972e48916\n>\n\nAck, of course.\n\nAnd trying to answer an older question:\n\n>>>The reason appears to be wrong bit mask usage\n>>>#define ADD_CACHE_IGNORE_ERRORS 4\n>>>and\n>>>#define HASH_RENORMALIZE 4\n\nWhat if we had renamed \"flags\" like this ?\n\ndiff --git a/read-cache.c b/read-cache.c\nindex 5096e395ee..f93d291613 100644\n--- a/read-cache.c\n+++ b/read-cache.c\n@@ -696,20 +696,20 @@ void set_object_name_for_intent_to_add_entry(struct cache_entry *ce)\n \toidcpy(&ce->oid, &oid);\n }\n\n-int add_to_index(struct index_state *istate, const char *path, struct stat *st, int flags)\n+int add_to_index(struct index_state *istate, const char *path, struct stat *st, int add_cache_flags)\n {\n \tint namelen, was_same;\n \tmode_t st_mode = st->st_mode;\n \tstruct cache_entry *ce, *alias = NULL;\n \tunsigned ce_option = CE_MATCH_IGNORE_VALID|CE_MATCH_IGNORE_SKIP_WORKTREE|CE_MATCH_RACY_IS_DIRTY;\n-\tint verbose = flags & (ADD_CACHE_VERBOSE | ADD_CACHE_PRETEND);\n-\tint pretend = flags & ADD_CACHE_PRETEND;\n-\tint intent_only = flags & ADD_CACHE_INTENT;\n+\tint verbose = add_cache_flags & (ADD_CACHE_VERBOSE | ADD_CACHE_PRETEND);\n+\tint pretend = add_cache_flags & ADD_CACHE_PRETEND;\n+\tint intent_only = add_cache_flags & ADD_CACHE_INTENT;\n \tint add_option = (ADD_CACHE_OK_TO_ADD|ADD_CACHE_OK_TO_REPLACE|\n \t\t\t (intent_only ? ADD_CACHE_NEW_ONLY : 0));\n \tint hash_flags = HASH_WRITE_OBJECT;\n\n-\tif (flags & ADD_CACHE_RENORMALIZE)\n+\tif (add_cache_flags & ADD_CACHE_RENORMALIZE)\n \t\thash_flags |= HASH_RENORMALIZE;\n\n \tif (!S_ISREG(st_mode) && !S_ISLNK(st_mode) && !S_ISDIR(st_mode))\n@@ -750,7 +750,7 @@ int add_to_index(struct index_state *istate, const char *path, struct stat *st,\n \tif (ignore_case) {\n \t\tadjust_dirname_case(istate, ce->name);\n \t}\n-\tif (!(flags & HASH_RENORMALIZE)) {\n+\tif (!(add_cache_flags & HASH_RENORMALIZE)) {\n \t\talias = index_file_exists(istate, ce->name,\n \t\t\t\t\t ce_namelen(ce), ignore_case);\n \t\tif (alias &&\n"},{"id":"368707","messageId":"nycvar.QRO.7.76.6.1902071416410.41@tvgsbejvaqbjf.bet","threadId":"50410","inReplyTo":"20190207020022.GA29974@sigill.intra.peff.net","subject":"Re: [PATCH] add_to_index(): convert forgotten HASH_RENORMALIZE check","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2019-02-07T13:17:03Z","receivedAt":"2019-02-07T13:17:33Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Peff,\n\nOn Wed, 6 Feb 2019, Jeff King wrote:\n\n> On Wed, Feb 06, 2019 at 11:42:43AM +0100, SZEDER Gábor wrote:\n> \n> > I reported this and Peff looked into it on the way to Git Merge, but\n> > not working solution yet.\n> > \n> > https://public-inbox.org/git/20190129225121.GD1895@sigill.intra.peff.net/T/#u\n> \n> Oof. Well, now I know why my attempts to fix the test failed. It was not\n> my new test that was failing at all, but rather the existing test. Which\n> implies that I severely bungled the actual code change.\n> \n> Armed with that knowledge, it was pretty easy to find said bungling. The\n> fix is below.\n> \n> Junio, this should go on top of jk/add-ignore-errors-bit-assignment-fix\n> as soon as possible, as the regression is already in master. And I'll go\n> find a brown paper bag. ;)\n\nThank you *so* much for the quick fix!\nDscho\n\n> -- >8 --\n> Subject: [PATCH] add_to_index(): convert forgotten HASH_RENORMALIZE check\n> \n> Commit 9e5da3d055 (add: use separate ADD_CACHE_RENORMALIZE flag,\n> 2019-01-17) switched out using HASH_RENORMALIZE in our flags field for a\n> new ADD_CACHE_RENORMALIZE flag. However, it forgot to convert one of the\n> checks for HASH_RENORMALIZE into the new flag, which totally broke \"git\n> add --renormalize\".\n> \n> To make matters even more confusing, the resulting code would racily\n> pass the tests! The forgotten check was responsible for defeating the\n> up-to-date check of the index entry. That meant that \"git add\n> --renormalize\" would refuse to renormalize things that appeared\n> stat-clean. But most of the time the test commands run fast enough that\n> the file mtime is the same as the index mtime. And thus we err on the\n> conservative side and re-hash the file, which is what \"--renormalize\"\n> would have wanted.\n> \n> But if you're unlucky and cross that one-second boundary between writing\n> the file and writing the index (which is more likely to happen on a slow\n> or heavily-loaded system), then the file appears stat-clean. And\n> \"--renormalize\" would effectively do nothing.\n> \n> The fix is straightforward: convert this check to use the right flag.\n> \n> Noticed-by: SZEDER Gábor \n> Signed-off-by: Jeff King \n> ---\n> read-cache.c | 2 +-\n> 1 file changed, 1 insertion(+), 1 deletion(-)\n> \n> diff --git a/read-cache.c b/read-cache.c\n> index 9783c493a3..accc059951 100644\n> --- a/read-cache.c\n> +++ b/read-cache.c\n> @@ -746,7 +746,7 @@ int add_to_index(struct index_state *istate, const char *path, struct stat *st,\n> \tif (ignore_case) {\n> \t\tadjust_dirname_case(istate, ce->name);\n> \t}\n> -\tif (!(flags & HASH_RENORMALIZE)) {\n> +\tif (!(flags & ADD_CACHE_RENORMALIZE)) {\n> \t\talias = index_file_exists(istate, ce->name,\n> \t\t\t\t\t ce_namelen(ce), ignore_case);\n> \t\tif (alias &&\n> -- \n> 2.20.1.1122.g2972e48916\n> \n> "},{"id":"368710","messageId":"001501d4bf06$506b8640$f14292c0$@nexbridge.com","threadId":"50410","inReplyTo":"20190206171517.s5lskawpdodc74ui@tb-raspi4","subject":"RE: t0025 flakey?","fromName":"Randall S. Becker","fromEmail":"rsbecker@nexbridge.com","sentAt":"2019-02-07T16:58:08Z","receivedAt":"2019-02-07T16:58:22Z","isPatch":false,"sender":{"key":"randall.becker@nexbridge.ca","avatar":"https://avatars.githubusercontent.com/u/28956764?v=4"},"body":"On February 6, 2019 13:01, I wrote:\n> On February 6, 2019 12:15, Torsten Bögershausen wrote:\n> > To: Johannes Schindelin \n> > Cc: SZEDER Gábor ; Jeff King ;\n> > git@vger.kernel.org\n> > Subject: Re: t0025 flakey?\n> >\n> > On Wed, Feb 06, 2019 at 02:52:53PM +0100, Johannes Schindelin wrote:\n> > > Hi Gábor,\n> > >\n> > > On Wed, 6 Feb 2019, SZEDER Gábor wrote:\n> > >\n> > > > On Wed, Feb 06, 2019 at 11:25:38AM +0100, Johannes Schindelin\n> wrote:\n> > > >\n> > > > > at first I thought that those intermittent test failures were\n> > > > > limited to Windows, but they are not: I can see it now in a\n> > > > > build on 32-bit Linux.\n> > > > > Full logs here:\n> > > > >\n> > > > > https://dev.azure.com/gitgitgadget/git/_build/results?buildId=10\n> > > > > 32 &_a=summary&view=ms.vss-test-web.build-test-results-tab\n> > > > >\n> > > > > Excerpt from the failing test case:\n> > > > >\n> > > > > -- snip --\n> > > > > not ok 2 - renormalize CRLF in repo expecting success:\n> > > > > \techo \"*.txt text=auto\" >.gitattributes &&\n> > > > > \tgit add --renormalize \"*.txt\" &&\n> > > > > \tcat >expect <<-\\EOF &&\n> > > > > \ti/lf w/crlf attr/text=auto CRLF.txt\n> > > > > \ti/lf w/lf attr/text=auto LF.txt\n> > > > > \ti/lf w/mixed attr/text=auto CRLF_mix_LF.txt\n> > > > > \tEOF\n> > > > > \tgit ls-files --eol |\n> > > > > \tsed -e \"s/\t/ /g\" -e \"s/ */ /g\" |\n> > > > > \tsort >actual &&\n> > > > > \ttest_cmp expect actual\n> > > > >\n> > > > > + echo *.txt text=auto\n> > > > > + git add --renormalize *.txt\n> > > > > + cat\n> > > > > + sort\n> > > > > + sed -e s/\t/ /g -e s/ */ /g\n> > > > > + git ls-files --eol\n> > > > > + test_cmp expect actual\n> > > > > + diff -u expect actual\n> > > > > --- expect\t2019-02-06 09:39:42.080733629 +0000\n> > > > > +++ actual\t2019-02-06 09:39:42.088733629 +0000\n> > > > > @@ -1,3 +1,3 @@\n> > > > > -i/lf w/crlf attr/text=auto CRLF.txt\n> > > > > +i/crlf w/crlf attr/text=auto CRLF.txt\n> > > > > i/lf w/lf attr/text=auto LF.txt -i/lf w/mixed attr/text=auto\n> > > > > CRLF_mix_LF.txt\n> > > > > +i/mixed w/mixed attr/text=auto CRLF_mix_LF.txt\n> > > > > error: last command exited with $?=1\n> > > > > -- snap --\n> > > > >\n> > > > > Any ideas?\n> > > >\n> > > > I reported this and Peff looked into it on the way to Git Merge,\n> > > > but not working solution yet.\n> > > >\n> > > > https://public-inbox.org/git/20190129225121.GD1895@sigill.intra.pe\n> > > > ff\n> > > > .net/T/#u\n> > >\n> > > Thank you!\n> > > Dscho\n> >\n> > I shortly looked into the pointers here - Is t0025 flaky after the fix\nfrom\n> Peff:\n> >\n> > [PATCH] add: use separate ADD_CACHE_RENORMALIZE flag\n> >\n> > Or has it always been shaky ?\n> > Does anybody know ?\n> \n> The NonStop port has traditionally had issues with t0025, which we tended\n> to ignore because things did work. We wrote those off as bash issues in\n> t0025 since they seemed to be corrected when we picked up a new bash\n> version about a year ago. I will keep monitoring this, particularly when\n2.21\n> comes out.\n\nFYI: t0020-t0027 all passed on the NonStop port for 2.21.0-rc0 - so no\nissues for us on this one.\n\nCheers,\nRandall\n\n-- Brief whoami:\n NonStop developer since approximately 211288444200000000\n UNIX developer since approximately 421664400\n-- In my real life, I talk too much.\n\n\n\n"},{"id":"368713","messageId":"xmqqtvhfpkep.fsf@gitster-ct.c.googlers.com","threadId":"50410","inReplyTo":"001501d4bf06$506b8640$f14292c0$@nexbridge.com","subject":"Re: t0025 flakey?","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2019-02-07T17:39:58Z","receivedAt":"2019-02-07T17:40:03Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Randall S. Becker\" writes:\n\n> On February 6, 2019 13:01, I wrote:\n>> On February 6, 2019 12:15, Torsten Bögershausen wrote:\n>> > To: Johannes Schindelin \n>> > ...\n>> > [PATCH] add: use separate ADD_CACHE_RENORMALIZE flag\n>> >\n>> > Or has it always been shaky ?\n>> > Does anybody know ?\n>> \n>> The NonStop port has traditionally had issues with t0025, which we tended\n>> to ignore because things did work. We wrote those off as bash issues in\n>> t0025 since they seemed to be corrected when we picked up a new bash\n>> version about a year ago. I will keep monitoring this, particularly when\n> 2.21\n>> comes out.\n>\n> FYI: t0020-t0027 all passed on the NonStop port for 2.21.0-rc0 - so no\n> issues for us on this one.\n\nYup, the preview has both the renormalize fix and another hotfix for\nit Peff made. Thanks all for quick fixes and confirmations.\n"},{"id":"368738","messageId":"20190207210009.GA30825@sigill.intra.peff.net","threadId":"50410","inReplyTo":"20190207041802.ai25fc64sqfg2h66@tb-raspi4","subject":"Re: [PATCH] add_to_index(): convert forgotten HASH_RENORMALIZE check","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2019-02-07T21:00:09Z","receivedAt":"2019-02-07T21:00:13Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Feb 07, 2019 at 04:18:02AM +0000, Torsten Bögershausen wrote:\n\n> And trying to answer an older question:\n> \n> >>>The reason appears to be wrong bit mask usage\n> >>>#define ADD_CACHE_IGNORE_ERRORS 4\n> >>>and\n> >>>#define HASH_RENORMALIZE 4\n> \n> What if we had renamed \"flags\" like this ?\n> [...]\n> -int add_to_index(struct index_state *istate, const char *path, struct stat *st, int flags)\n> +int add_to_index(struct index_state *istate, const char *path, struct stat *st, int add_cache_flags)\n\nYes, changing the name of the variable in the original patch would have\ncaught this case. I don't know if it is worth doing now or not (the code\nas it is now seems pretty clear to me, but of course I've looked at it a\nlot lately).\n\n-Peff\n"},{"id":"368758","messageId":"20190207235726.GR10587@szeder.dev","threadId":"50410","inReplyTo":"001501d4bf06$506b8640$f14292c0$@nexbridge.com","subject":"Re: t0025 flakey?","fromName":"SZEDER Gábor","fromEmail":"szeder.dev@gmail.com","sentAt":"2019-02-07T23:57:26Z","receivedAt":"2019-02-07T23:57:32Z","isPatch":false,"sender":{"key":"szeder.dev@gmail.com","avatar":"https://avatars.githubusercontent.com/u/116324?v=4"},"body":"On Thu, Feb 07, 2019 at 11:58:08AM -0500, Randall S. Becker wrote:\n> > The NonStop port has traditionally had issues with t0025, which we tended\n> > to ignore because things did work. We wrote those off as bash issues in\n> > t0025 since they seemed to be corrected when we picked up a new bash\n> > version about a year ago. I will keep monitoring this, particularly when\n> 2.21\n> > comes out.\n> \n> FYI: t0020-t0027 all passed on the NonStop port for 2.21.0-rc0 - so no\n> issues for us on this one.\n\nNote that t0021 is very likely flaky on NonStop, too:\n\n https://public-inbox.org/git/20190111140408.GC840@szeder.dev/T/#u\n\n"},{"id":"368815","messageId":"000901d4bf98$1ab08fe0$5011afa0$@nexbridge.com","threadId":"50410","inReplyTo":"20190207235726.GR10587@szeder.dev","subject":"RE: t0025 flakey?","fromName":"Randall S. Becker","fromEmail":"rsbecker@nexbridge.com","sentAt":"2019-02-08T10:21:44Z","receivedAt":"2019-02-08T10:22:06Z","isPatch":false,"sender":{"key":"randall.becker@nexbridge.ca","avatar":"https://avatars.githubusercontent.com/u/28956764?v=4"},"body":"On February 7, 2019 18:57, SZEDER Gábor wrote:\n> On Thu, Feb 07, 2019 at 11:58:08AM -0500, Randall S. Becker wrote:\n> > > The NonStop port has traditionally had issues with t0025, which we\n> > > tended to ignore because things did work. We wrote those off as bash\n> > > issues in\n> > > t0025 since they seemed to be corrected when we picked up a new bash\n> > > version about a year ago. I will keep monitoring this, particularly\n> > > when\n> > 2.21\n> > > comes out.\n> >\n> > FYI: t0020-t0027 all passed on the NonStop port for 2.21.0-rc0 - so no\n> > issues for us on this one.\n> \n> Note that t0021 is very likely flaky on NonStop, too:\n> \n> https://public-inbox.org/git/20190111140408.GC840@szeder.dev/T/#u\n\nWe will keep a watch on it, thanks. t0021 has been stable on this platform for at least a year and passes for 2.21.0-rc0 as well as 2.20.0.\n\nCheers,\nRandall\n\n"}]}