{"thread":{"id":"63001","subject":"win+Meson test failures due to ps/build-meson-fixes-0130 topic?","startedAt":"2025-02-25T19:45:50Z","lastAt":"2025-02-26T22:07:20Z","messageCount":10,"participants":["Junio C Hamano","Patrick Steinhardt","Jeff King"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"513036","messageId":"xmqqo6ypiz9w.fsf@gitster.g","threadId":"63001","inReplyTo":null,"subject":"win+Meson test failures due to ps/build-meson-fixes-0130 topic?","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-02-25T19:45:47Z","receivedAt":"2025-02-25T19:45:50Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"As of yesterday, 'next' started failing all the \"win+Meson test\"\njobs.\n\nhttps://github.com/git/git/actions/runs/13506034920\n\nI had to do one of my least favourite activities working on Git:\nbisecting CI breakage on a platform I have no idea how well it\nworks.\n\nHere is the full \"stack\" of changes, it contains all the topics\nmerged between 'master' and 'next', plus a single change that\nbutchers .github/workflows/main.yml to leave only three jobs\n(ci-config, windows-meson-build, and windows-meson-test).\n\n    3a7e83b527 Merge branch 'ps/build-meson-fixes-0130'\n    e59193e984 Merge branch 'ek/mingw-rename-symlink'\n    a209cf1427 Merge branch 'po/meson-perl-fix'\n    9f1e73677c Merge branch 'ua/os-version-capability'\n    82f01cb375 Merge branch 'ms/rename-match-name-with-pattern'\n    8b04381049 Merge branch 'ad/set-default-target-in-makefiles'\n    2cab1bd7d0 Merge branch 'pw/merge-tree-stdin-deadlock-fix'\n    92b43de9df Merge branch 'mh/doc-commit-title-not-subject'\n    685c150623 Merge branch 'bc/diff-reject-empty-arg-to-pickaxe'\n    c5093fc2d8 Merge branch 'tb/new-make-fix'\n    9091607e95 Merge branch 'da/xdiff-w-sign-compare-workaround'\n    01e86410fe GitHub CI: trim and slim for win+Meson test breakage hunting\n    2d2a71ce85 The thirteenth batch\n\nAt 01e86410fe and e59193e984, CI seems to pass.\n\nhttps://github.com/git/git/actions/runs/13528755524\n\nWith ps/build-meson-fixes-0130 at 3a7e83b527, all the \"win+Meson\ntest\" jobs fail.  I suspect that the series does not break\nindividual tests but the breakage is in somewhere more\nframework-ish?  I didn't look into individual test job failures.\n\nhttps://github.com/git/git/actions/runs/13529323827\n\nI'll also note that in one of my earlier attempts (which had\nek/mingw-rename-symlink merged before po/meson-perl-fix), one of the\n10 \"win+Meson test\" jobs failed (\"win+Meson test (7)\" to be exact).\nRerunning the failed test succeeds, so there must be something flaky\nin t0610-reftable-basics test that is exposed by this change,\nperhaps?\n\n  https://github.com/git/git/actions/runs/13528071468/attempts/1\n  https://github.com/git/git/actions/runs/13528071468/attempts/2\n\nAs the ps/build-meson-fixes-0130 topic has 13 changes, and each\niteration in the above exercise took somewhere around 20 minutes, I\nwon't dive into the topic myself to identify which one of these\nindividual topics causes the breakage.  This however gave me enough\nto go on to today's 'master' integration (mainly, by mark the topic\nto be held in 'next' until this gets resolved, and deal with the\nother topics).\n\nThanks.\n"},{"id":"513049","messageId":"Z76qYgV7B2eUJHiP@pks.im","threadId":"63001","inReplyTo":"xmqqo6ypiz9w.fsf@gitster.g","subject":"Re: win+Meson test failures due to ps/build-meson-fixes-0130 topic?","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-02-26T05:45:06Z","receivedAt":"2025-02-26T05:45:10Z","isPatch":false,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Tue, Feb 25, 2025 at 11:45:47AM -0800, Junio C Hamano wrote:\n> As of yesterday, 'next' started failing all the \"win+Meson test\"\n> jobs.\n> \n> https://github.com/git/git/actions/runs/13506034920\n> \n> I had to do one of my least favourite activities working on Git:\n> bisecting CI breakage on a platform I have no idea how well it\n> works.\n> \n> Here is the full \"stack\" of changes, it contains all the topics\n> merged between 'master' and 'next', plus a single change that\n> butchers .github/workflows/main.yml to leave only three jobs\n> (ci-config, windows-meson-build, and windows-meson-test).\n> \n>     3a7e83b527 Merge branch 'ps/build-meson-fixes-0130'\n>     e59193e984 Merge branch 'ek/mingw-rename-symlink'\n>     a209cf1427 Merge branch 'po/meson-perl-fix'\n>     9f1e73677c Merge branch 'ua/os-version-capability'\n>     82f01cb375 Merge branch 'ms/rename-match-name-with-pattern'\n>     8b04381049 Merge branch 'ad/set-default-target-in-makefiles'\n>     2cab1bd7d0 Merge branch 'pw/merge-tree-stdin-deadlock-fix'\n>     92b43de9df Merge branch 'mh/doc-commit-title-not-subject'\n>     685c150623 Merge branch 'bc/diff-reject-empty-arg-to-pickaxe'\n>     c5093fc2d8 Merge branch 'tb/new-make-fix'\n>     9091607e95 Merge branch 'da/xdiff-w-sign-compare-workaround'\n>     01e86410fe GitHub CI: trim and slim for win+Meson test breakage hunting\n>     2d2a71ce85 The thirteenth batch\n> \n> At 01e86410fe and e59193e984, CI seems to pass.\n> \n> https://github.com/git/git/actions/runs/13528755524\n> \n> With ps/build-meson-fixes-0130 at 3a7e83b527, all the \"win+Meson\n> test\" jobs fail.  I suspect that the series does not break\n> individual tests but the breakage is in somewhere more\n> framework-ish?  I didn't look into individual test job failures.\n> \n> https://github.com/git/git/actions/runs/13529323827\n> \n> I'll also note that in one of my earlier attempts (which had\n> ek/mingw-rename-symlink merged before po/meson-perl-fix), one of the\n> 10 \"win+Meson test\" jobs failed (\"win+Meson test (7)\" to be exact).\n> Rerunning the failed test succeeds, so there must be something flaky\n> in t0610-reftable-basics test that is exposed by this change,\n> perhaps?\n> \n>   https://github.com/git/git/actions/runs/13528071468/attempts/1\n>   https://github.com/git/git/actions/runs/13528071468/attempts/2\n> \n> As the ps/build-meson-fixes-0130 topic has 13 changes, and each\n> iteration in the above exercise took somewhere around 20 minutes, I\n> won't dive into the topic myself to identify which one of these\n> individual topics causes the breakage.  This however gave me enough\n> to go on to today's 'master' integration (mainly, by mark the topic\n> to be held in 'next' until this gets resolved, and deal with the\n> other topics).\n\nHm, interesting. I didn't notice those issues in our CI. Thanks for\ndiving into it, I'll figure out what has happened here.\n\nPatrick\n"},{"id":"513050","messageId":"20250226072731.GA20372@coredump.intra.peff.net","threadId":"63001","inReplyTo":"Z76qYgV7B2eUJHiP@pks.im","subject":"Re: win+Meson test failures due to ps/build-meson-fixes-0130 topic?","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2025-02-26T07:27:31Z","receivedAt":"2025-02-26T07:27:39Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Feb 26, 2025 at 06:45:06AM +0100, Patrick Steinhardt wrote:\n\n> > I'll also note that in one of my earlier attempts (which had\n> > ek/mingw-rename-symlink merged before po/meson-perl-fix), one of the\n> > 10 \"win+Meson test\" jobs failed (\"win+Meson test (7)\" to be exact).\n> > Rerunning the failed test succeeds, so there must be something flaky\n> > in t0610-reftable-basics test that is exposed by this change,\n> > perhaps?\n> > \n> >   https://github.com/git/git/actions/runs/13528071468/attempts/1\n> >   https://github.com/git/git/actions/runs/13528071468/attempts/2\n> > \n> > As the ps/build-meson-fixes-0130 topic has 13 changes, and each\n> > iteration in the above exercise took somewhere around 20 minutes, I\n> > won't dive into the topic myself to identify which one of these\n> > individual topics causes the breakage.  This however gave me enough\n> > to go on to today's 'master' integration (mainly, by mark the topic\n> > to be held in 'next' until this gets resolved, and deal with the\n> > other topics).\n> \n> Hm, interesting. I didn't notice those issues in our CI. Thanks for\n> diving into it, I'll figure out what has happened here.\n\nI spent a little time poking at this yesterday. Here's what I found in\ncase it helps.\n\nI arbitrarily picked t0210 to poke at because it's short and it was\nfailing. Interestingly, the failing command was clone, with the client\ncomplaining that it did not understand the server's protocol version\nresponse.\n\nRunning with GIT_TRACE_PACKET shows that we got a line:\n\n  version 2\\r\\n\n\nAnd the extra \"\\r\" confuses the parser. So perhaps something in the\nbuild changed that is affecting how CRLF is output by standard\nfunctions? That would explain why we see failures all over the test\nsuite. I didn't dig into any other cases, though.\n\n-Peff\n"},{"id":"513051","messageId":"Z77EJRJwPDGUglLk@pks.im","threadId":"63001","inReplyTo":"Z76qYgV7B2eUJHiP@pks.im","subject":"Re: win+Meson test failures due to ps/build-meson-fixes-0130 topic?","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-02-26T07:35:01Z","receivedAt":"2025-02-26T07:35:12Z","isPatch":false,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Wed, Feb 26, 2025 at 06:45:06AM +0100, Patrick Steinhardt wrote:\n> On Tue, Feb 25, 2025 at 11:45:47AM -0800, Junio C Hamano wrote:\n> > As of yesterday, 'next' started failing all the \"win+Meson test\"\n> > jobs.\n> > \n> > https://github.com/git/git/actions/runs/13506034920\n> > \n> > I had to do one of my least favourite activities working on Git:\n> > bisecting CI breakage on a platform I have no idea how well it\n> > works.\n> > \n> > Here is the full \"stack\" of changes, it contains all the topics\n> > merged between 'master' and 'next', plus a single change that\n> > butchers .github/workflows/main.yml to leave only three jobs\n> > (ci-config, windows-meson-build, and windows-meson-test).\n> > \n> >     3a7e83b527 Merge branch 'ps/build-meson-fixes-0130'\n> >     e59193e984 Merge branch 'ek/mingw-rename-symlink'\n> >     a209cf1427 Merge branch 'po/meson-perl-fix'\n> >     9f1e73677c Merge branch 'ua/os-version-capability'\n> >     82f01cb375 Merge branch 'ms/rename-match-name-with-pattern'\n> >     8b04381049 Merge branch 'ad/set-default-target-in-makefiles'\n> >     2cab1bd7d0 Merge branch 'pw/merge-tree-stdin-deadlock-fix'\n> >     92b43de9df Merge branch 'mh/doc-commit-title-not-subject'\n> >     685c150623 Merge branch 'bc/diff-reject-empty-arg-to-pickaxe'\n> >     c5093fc2d8 Merge branch 'tb/new-make-fix'\n> >     9091607e95 Merge branch 'da/xdiff-w-sign-compare-workaround'\n> >     01e86410fe GitHub CI: trim and slim for win+Meson test breakage hunting\n> >     2d2a71ce85 The thirteenth batch\n> > \n> > At 01e86410fe and e59193e984, CI seems to pass.\n> > \n> > https://github.com/git/git/actions/runs/13528755524\n> > \n> > With ps/build-meson-fixes-0130 at 3a7e83b527, all the \"win+Meson\n> > test\" jobs fail.  I suspect that the series does not break\n> > individual tests but the breakage is in somewhere more\n> > framework-ish?  I didn't look into individual test job failures.\n> > \n> > https://github.com/git/git/actions/runs/13529323827\n> > \n> > I'll also note that in one of my earlier attempts (which had\n> > ek/mingw-rename-symlink merged before po/meson-perl-fix), one of the\n> > 10 \"win+Meson test\" jobs failed (\"win+Meson test (7)\" to be exact).\n> > Rerunning the failed test succeeds, so there must be something flaky\n> > in t0610-reftable-basics test that is exposed by this change,\n> > perhaps?\n> > \n> >   https://github.com/git/git/actions/runs/13528071468/attempts/1\n> >   https://github.com/git/git/actions/runs/13528071468/attempts/2\n> > \n> > As the ps/build-meson-fixes-0130 topic has 13 changes, and each\n> > iteration in the above exercise took somewhere around 20 minutes, I\n> > won't dive into the topic myself to identify which one of these\n> > individual topics causes the breakage.  This however gave me enough\n> > to go on to today's 'master' integration (mainly, by mark the topic\n> > to be held in 'next' until this gets resolved, and deal with the\n> > other topics).\n> \n> Hm, interesting. I didn't notice those issues in our CI. Thanks for\n> diving into it, I'll figure out what has happened here.\n\nOkay, the issue was in 163a3e97ac1 (meson: simplify use of the\ncommon-main library, 2025-01-30). In that commit we stopped using the\ncommon-main dependency for a couple of binaries, including\ngit-remote-https and others. What I didn't realize though is that this\nalso made us not use the linker arguments anymore, but those are\nrequired on Windows when linking an executable.\n\nI've kicked off CI jobs on GitHub to verify that the revised version of\nthis patch series works alright. Once those tests pass I'll send another\nversion of this patch series.\n\nPatrick\n"},{"id":"513053","messageId":"20250226074653.GA27455@coredump.intra.peff.net","threadId":"63001","inReplyTo":"Z77EJRJwPDGUglLk@pks.im","subject":"Re: win+Meson test failures due to ps/build-meson-fixes-0130 topic?","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2025-02-26T07:46:53Z","receivedAt":"2025-02-26T07:46:54Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Feb 26, 2025 at 08:35:01AM +0100, Patrick Steinhardt wrote:\n\n> > Hm, interesting. I didn't notice those issues in our CI. Thanks for\n> > diving into it, I'll figure out what has happened here.\n> \n> Okay, the issue was in 163a3e97ac1 (meson: simplify use of the\n> common-main library, 2025-01-30). In that commit we stopped using the\n> common-main dependency for a couple of binaries, including\n> git-remote-https and others. What I didn't realize though is that this\n> also made us not use the linker arguments anymore, but those are\n> required on Windows when linking an executable.\n\nAh, thanks, I was just closing in on that commit, too, by reverting\nbackwards from the tip of the series\n\nThere's one other oddity I found that you might be interested in. After\neach revert doing \"meson test ...\" mostly worked. But when I reverted\nbf4a16b84a (meson: drop separate version library, 2025-01-30), the build\nthen fails with:\n\n  $ GIT_TRACE_PACKET=1 meson test -C build t0210-* --interactive --test-args=-vi\n  ...\n  [305/571] Linking static target libgit.a\n  FAILED: libgit.a\n  \"gcc-ar\" \"csrDT\" libgit.a @libgit.a.rsp\n  C:/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/bin/ar.exe: libgit.a: No such file or directory\n\nI'd imagine that if I did a complete \"rm -rf build && meson setup build\"\nit would work. But it's pretty lousy to have to do that preemptively\nduring a bisection.\n\nI didn't dig into the actual source of the problem in that commit, so I\ndon't know if there's a way to write it that is happy working with both\nthe before and after states.\n\n-Peff\n"},{"id":"513056","messageId":"20250226082002.GA30633@coredump.intra.peff.net","threadId":"63001","inReplyTo":"20250226074653.GA27455@coredump.intra.peff.net","subject":"Re: win+Meson test failures due to ps/build-meson-fixes-0130 topic?","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2025-02-26T08:20:02Z","receivedAt":"2025-02-26T08:20:04Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Feb 26, 2025 at 02:46:53AM -0500, Jeff King wrote:\n\n> I'd imagine that if I did a complete \"rm -rf build && meson setup build\"\n> it would work. But it's pretty lousy to have to do that preemptively\n> during a bisection.\n\nLooks like this runs even deeper. On my Linux machine:\n\n  $ git checkout origin/next\n  $ meson setup build\n  [...ok...]\n  $ meson compile -C build\n  [...ok...]\n\n  $ git checkout origin/master\n  $ meson compile -C build\n  [...]\n  ../meson.build:215:35: ERROR: The `!=` operator of array does not accept objects of type str ()\n  FAILED: build.ninja\n  /usr/bin/meson --internal regenerate /home/peff/compile/git .\n  ninja: error: rebuilding 'build.ninja': subcommand failed\n\n  $ rm -rf build\n  $ meson setup build\n  $ meson compile -C build\n  [...ok again...]\n\nI'm hoping you can tell me I'm holding it wrong, and there's some way to\ndo an incremental build when crossing these sorts of boundaries (without\nblowing away all of the build products).\n\nI do at least use ccache which makes the \"rm -rf\" case a little less\npainful.\n\n-Peff\n"},{"id":"513071","messageId":"Z77bxuad7pFsz7ow@pks.im","threadId":"63001","inReplyTo":"20250226082002.GA30633@coredump.intra.peff.net","subject":"Re: win+Meson test failures due to ps/build-meson-fixes-0130 topic?","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-02-26T09:15:50Z","receivedAt":"2025-02-26T09:15:55Z","isPatch":false,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Wed, Feb 26, 2025 at 03:20:02AM -0500, Jeff King wrote:\n> On Wed, Feb 26, 2025 at 02:46:53AM -0500, Jeff King wrote:\n> \n> > I'd imagine that if I did a complete \"rm -rf build && meson setup build\"\n> > it would work. But it's pretty lousy to have to do that preemptively\n> > during a bisection.\n> \n> Looks like this runs even deeper. On my Linux machine:\n> \n>   $ git checkout origin/next\n>   $ meson setup build\n>   [...ok...]\n>   $ meson compile -C build\n>   [...ok...]\n> \n>   $ git checkout origin/master\n>   $ meson compile -C build\n>   [...]\n>   ../meson.build:215:35: ERROR: The `!=` operator of array does not accept objects of type str ()\n>   FAILED: build.ninja\n>   /usr/bin/meson --internal regenerate /home/peff/compile/git .\n>   ninja: error: rebuilding 'build.ninja': subcommand failed\n> \n>   $ rm -rf build\n>   $ meson setup build\n>   $ meson compile -C build\n>   [...ok again...]\n> \n> I'm hoping you can tell me I'm holding it wrong, and there's some way to\n> do an incremental build when crossing these sorts of boundaries (without\n> blowing away all of the build products).\n> \n> I do at least use ccache which makes the \"rm -rf\" case a little less\n> painful.\n\nYou're not holding it wrong. The problem is that this patch series\nintroduces a backwards-incompatible change to the \"sane_tools_path\"\noption as it changes it from a string to an array, which makes Meson\nhiccup. Once Meson is not considered to be experimental anymore we\nshould refrain from changing option types going forward, but until then\nI consider it to be okayish to break things like this.\n\nFrom my point of view we can treat Meson as stable as soon as Git v2.49\nis out. So from thereon, I will make sure to not include backwards\nincompatible changes anymore.\n\nPatrick\n"},{"id":"513104","messageId":"xmqqh64gg0pu.fsf@gitster.g","threadId":"63001","inReplyTo":"Z77EJRJwPDGUglLk@pks.im","subject":"Re: win+Meson test failures due to ps/build-meson-fixes-0130 topic?","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-02-26T15:55:09Z","receivedAt":"2025-02-26T15:55:12Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Patrick Steinhardt <ps@pks.im> writes:\n\n> Okay, the issue was in 163a3e97ac1 (meson: simplify use of the\n> common-main library, 2025-01-30). In that commit we stopped using the\n> common-main dependency for a couple of binaries, including\n> git-remote-https and others. What I didn't realize though is that this\n> also made us not use the linker arguments anymore, but those are\n> required on Windows when linking an executable.\n>\n> I've kicked off CI jobs on GitHub to verify that the revised version of\n> this patch series works alright. Once those tests pass I'll send another\n> version of this patch series.\n\nThanks.  What is a bit more worrying is that you earlier said your\nother CI didn't notice anything funny.  Is the reason for the\nnon-breakage also known (like: the two CI jobs build things\ndifferently)?\n\nThanks.\n\n\n"},{"id":"513108","messageId":"Z79BVcFMdCpgbKqM@pks.im","threadId":"63001","inReplyTo":"xmqqh64gg0pu.fsf@gitster.g","subject":"Re: win+Meson test failures due to ps/build-meson-fixes-0130 topic?","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-02-26T16:29:09Z","receivedAt":"2025-02-26T16:29:14Z","isPatch":false,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Wed, Feb 26, 2025 at 07:55:09AM -0800, Junio C Hamano wrote:\n> Patrick Steinhardt <ps@pks.im> writes:\n> \n> > Okay, the issue was in 163a3e97ac1 (meson: simplify use of the\n> > common-main library, 2025-01-30). In that commit we stopped using the\n> > common-main dependency for a couple of binaries, including\n> > git-remote-https and others. What I didn't realize though is that this\n> > also made us not use the linker arguments anymore, but those are\n> > required on Windows when linking an executable.\n> >\n> > I've kicked off CI jobs on GitHub to verify that the revised version of\n> > this patch series works alright. Once those tests pass I'll send another\n> > version of this patch series.\n> \n> Thanks.  What is a bit more worrying is that you earlier said your\n> other CI didn't notice anything funny.  Is the reason for the\n> non-breakage also known (like: the two CI jobs build things\n> differently)?\n\nOh, yeah, I forgot to mention that detail: the jobs failed, but the\nfailure wasn't reported (see e.g. [1]). It's a bug in the CI setup\nitself. I'll send a patch series tomorrow to fix that.\n\nPatrick\n\n[1]: https://gitlab.com/gitlab-org/git/-/jobs/8996075448\n"},{"id":"513121","messageId":"20250226220719.GB600528@coredump.intra.peff.net","threadId":"63001","inReplyTo":"Z77bxuad7pFsz7ow@pks.im","subject":"Re: win+Meson test failures due to ps/build-meson-fixes-0130 topic?","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2025-02-26T22:07:19Z","receivedAt":"2025-02-26T22:07:20Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Feb 26, 2025 at 10:15:50AM +0100, Patrick Steinhardt wrote:\n\n> > I'm hoping you can tell me I'm holding it wrong, and there's some way to\n> > do an incremental build when crossing these sorts of boundaries (without\n> > blowing away all of the build products).\n> > \n> > I do at least use ccache which makes the \"rm -rf\" case a little less\n> > painful.\n> \n> You're not holding it wrong. The problem is that this patch series\n> introduces a backwards-incompatible change to the \"sane_tools_path\"\n> option as it changes it from a string to an array, which makes Meson\n> hiccup. Once Meson is not considered to be experimental anymore we\n> should refrain from changing option types going forward, but until then\n> I consider it to be okayish to break things like this.\n> \n> From my point of view we can treat Meson as stable as soon as Git v2.49\n> is out. So from thereon, I will make sure to not include backwards\n> incompatible changes anymore.\n\nThanks. It's comforting to hear that this is something that's under our\ncontrol.\n\n-Peff\n"}]}