{"thread":{"id":"62122","subject":"fatal from submodule status --recursive when used with grep -q","startedAt":"2024-09-16T15:09:10Z","lastAt":"2024-09-20T13:12:34Z","messageCount":11,"participants":["Matt Liberty","Phillip Wood","Junio C Hamano","phillip.wood123@gmail.com","Eric Sunshine"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"502893","messageId":"CAKDm0rNaHbzoiPg=DeuCoxzooNAsxw2BJfc0wg7fC_-=o9uJ7w@mail.gmail.com","threadId":"62122","inReplyTo":null,"subject":"fatal from submodule status --recursive when used with grep -q","fromName":"Matt Liberty","fromEmail":"mliberty@precisioninno.com","sentAt":"2024-09-16T15:08:59Z","receivedAt":"2024-09-16T15:09:10Z","isPatch":false,"sender":{"key":"mliberty@precisioninno.com","avatar":null},"body":"Hi\n\nI have a repo with submodule, like:\n\n% git submodule status --recursive\n+834b7a0fb6de03c8e584256af2ecb7889b568343 tools/OpenROAD (v2.0-15587-g834b7a0fb)\n aafee90f8a21bf7867cef2e159929440cf45b2e5 tools/OpenROAD/src/sta\n(v2.2.0-1106-gaafee90)\n ef5389d31526003c2ebd7e6d6d6fe3848a20f0a2\ntools/OpenROAD/third-party/abc (remotes/origin/HEAD)\n 80ba43d26264738c93900129dc0aab7fab36c53f tools/yosys (yosys-0.44)\n 28d955ca97a1c4be3aed4062aec0241a734fac5d tools/yosys/abc (yosys-0.44)\n\nIf I try to grep the output I get a fatal error:\n\n% git submodule status --recursive | grep -q \"^+\"\nfatal: failed to recurse into submodule 'tools/OpenROAD'\n\nI didn't expect any output but did want the return status (0). I'm\nguessing git is unhappy that grep -q exits on the first occurrence of\nthe pattern. I don't feel fatal is appropriate here.\n\nI can work around this by writing to a tmpfile but would like to see\nthis addressed.\n\nThanks!\nMatt\n"},{"id":"502994","messageId":"7b4ce2fc-3763-4881-8459-e0c1f2cbc69d@gmail.com","threadId":"62122","inReplyTo":"CAKDm0rNaHbzoiPg=DeuCoxzooNAsxw2BJfc0wg7fC_-=o9uJ7w@mail.gmail.com","subject":"Re: fatal from submodule status --recursive when used with grep -q","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2024-09-18T10:05:08Z","receivedAt":"2024-09-18T10:05:11Z","isPatch":false,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"Hi Matt\n\nOn 16/09/2024 16:08, Matt Liberty wrote:\n> \n> If I try to grep the output I get a fatal error:\n> \n> % git submodule status --recursive | grep -q \"^+\"\n> fatal: failed to recurse into submodule 'tools/OpenROAD'\n> \n> I didn't expect any output but did want the return status (0). I'm\n> guessing git is unhappy that grep -q exits on the first occurrence of\n> the pattern. I don't feel fatal is appropriate here.\n\nI assume git is dying with SIGPIPE. As the only purpose of \"git \nsubmodule status\" is to write the status information to stdout that \nsounds reasonable. If you want to collect the exit status you need to \nconsume the whole output so that the command runs to completion.\n\nBest Wishes\n\nPhillip\n\n> I can work around this by writing to a tmpfile but would like to see\n> this addressed.\n> \n> Thanks!\n> Matt\n> \n"},{"id":"503003","messageId":"CAKDm0rNjAYMu8=h8ApvEHw71fgH2VW0iCjTMqvysQ7YBUnJZyw@mail.gmail.com","threadId":"62122","inReplyTo":"7b4ce2fc-3763-4881-8459-e0c1f2cbc69d@gmail.com","subject":"Re: fatal from submodule status --recursive when used with grep -q","fromName":"Matt Liberty","fromEmail":"mliberty@precisioninno.com","sentAt":"2024-09-18T14:45:33Z","receivedAt":"2024-09-18T14:45:45Z","isPatch":false,"sender":{"key":"mliberty@precisioninno.com","avatar":null},"body":"I'm not trying to capture the exit status of git but of grep. Compare\nit to \"yes|grep -q y\" which doesn't generate any error even though yes\ngenerates endless output.\n\nThanks\n\nMatt\n\n\nOn Wed, Sep 18, 2024 at 3:05 AM Phillip Wood wrote:\n>\n> Hi Matt\n>\n> On 16/09/2024 16:08, Matt Liberty wrote:\n> >\n> > If I try to grep the output I get a fatal error:\n> >\n> > % git submodule status --recursive | grep -q \"^+\"\n> > fatal: failed to recurse into submodule 'tools/OpenROAD'\n> >\n> > I didn't expect any output but did want the return status (0). I'm\n> > guessing git is unhappy that grep -q exits on the first occurrence of\n> > the pattern. I don't feel fatal is appropriate here.\n>\n> I assume git is dying with SIGPIPE. As the only purpose of \"git\n> submodule status\" is to write the status information to stdout that\n> sounds reasonable. If you want to collect the exit status you need to\n> consume the whole output so that the command runs to completion.\n>\n> Best Wishes\n>\n> Phillip\n>\n> > I can work around this by writing to a tmpfile but would like to see\n> > this addressed.\n> >\n> > Thanks!\n> > Matt\n> >\n"},{"id":"503045","messageId":"xmqq1q1ga1dp.fsf@gitster.g","threadId":"62122","inReplyTo":"7b4ce2fc-3763-4881-8459-e0c1f2cbc69d@gmail.com","subject":"Re: fatal from submodule status --recursive when used with grep -q","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-09-19T01:16:50Z","receivedAt":"2024-09-19T01:16:53Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Phillip Wood writes:\n\n> On 16/09/2024 16:08, Matt Liberty wrote:\n>> If I try to grep the output I get a fatal error:\n>> % git submodule status --recursive | grep -q \"^+\"\n>> fatal: failed to recurse into submodule 'tools/OpenROAD'\n>> I didn't expect any output but did want the return status (0). I'm\n>> guessing git is unhappy that grep -q exits on the first occurrence of\n>> the pattern. I don't feel fatal is appropriate here.\n>\n> I assume git is dying with SIGPIPE. As the only purpose of \"git\n> submodule status\" is to write the status information to stdout that\n> sounds reasonable. If you want to collect the exit status you need to\n> consume the whole output so that the command runs to completion.\n\nYes, but isn't the main point of that complaint that \"failed to\nrecurse into\" is not a good way to say \"because you closed the\nreading end of the pipe, we are aborting the operation and not\ndecending into 'tools/OpenROAD' submodule\", is it?\n\n"},{"id":"503060","messageId":"CAKDm0rN-jvtmVhwODQTcQJu5LYTJ7GwTKUtfA9kzifHR6WWf9w@mail.gmail.com","threadId":"62122","inReplyTo":"xmqq1q1ga1dp.fsf@gitster.g","subject":"Re: fatal from submodule status --recursive when used with grep -q","fromName":"Matt Liberty","fromEmail":"mliberty@precisioninno.com","sentAt":"2024-09-19T13:23:20Z","receivedAt":"2024-09-19T13:23:32Z","isPatch":false,"sender":{"key":"mliberty@precisioninno.com","avatar":null},"body":"I'm saying most unix tools do not error when the pipe is closed and I\ngave 'yes' as a simple example but there are many others. grep found\nwhat I was looking for and I don't care that git didn't recurse\nfurther (in fact I'm happy that it didn't waste time doing so). There\nis no problem except that git insists on complaining about the closed\npipe.\n\nMatt\n\n\nOn Wed, Sep 18, 2024 at 6:16 PM Junio C Hamano wrote:\n>\n> Phillip Wood writes:\n>\n> > On 16/09/2024 16:08, Matt Liberty wrote:\n> >> If I try to grep the output I get a fatal error:\n> >> % git submodule status --recursive | grep -q \"^+\"\n> >> fatal: failed to recurse into submodule 'tools/OpenROAD'\n> >> I didn't expect any output but did want the return status (0). I'm\n> >> guessing git is unhappy that grep -q exits on the first occurrence of\n> >> the pattern. I don't feel fatal is appropriate here.\n> >\n> > I assume git is dying with SIGPIPE. As the only purpose of \"git\n> > submodule status\" is to write the status information to stdout that\n> > sounds reasonable. If you want to collect the exit status you need to\n> > consume the whole output so that the command runs to completion.\n>\n> Yes, but isn't the main point of that complaint that \"failed to\n> recurse into\" is not a good way to say \"because you closed the\n> reading end of the pipe, we are aborting the operation and not\n> decending into 'tools/OpenROAD' submodule\", is it?\n>\n"},{"id":"503061","messageId":"467f6fb4-5f83-416b-bc09-572a82f210ea@gmail.com","threadId":"62122","inReplyTo":"CAKDm0rMsXUpFSWRWkW0J0FUpEbot6wQqdzYTT5CLZ=3TGbV-OA@mail.gmail.com","subject":"Re: fatal from submodule status --recursive when used with grep -q","fromName":"","fromEmail":"phillip.wood123@gmail.com","sentAt":"2024-09-19T14:22:26Z","receivedAt":"2024-09-19T14:22:28Z","isPatch":false,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"On 18/09/2024 15:28, Matt Liberty wrote:\n> I'm not trying to capture the exit status of git but of grep.  Compare \n> it to \"yes|grep -q y\" which doesn't generate any error even though yes \n> generates endless output.\n\nSorry, I'd misunderstood your original message. So the problem is that \nyou want \"git submodule status\" to be quiet (i.e. not print an error \nmessage) when it dies of SIGPIPE?\n\nBest Wishes\n\nPhillip\n\n> Matt\n> \n> On Wed, Sep 18, 2024 at 3:05 AM Phillip Wood > wrote:\n> \n> Hi Matt\n> \n> On 16/09/2024 16:08, Matt Liberty wrote:\n> >\n> > If I try to grep the output I get a fatal error:\n> >\n> > % git submodule status --recursive | grep -q \"^+\"\n> > fatal: failed to recurse into submodule 'tools/OpenROAD'\n> >\n> > I didn't expect any output but did want the return status (0).  I'm\n> > guessing git is unhappy that grep -q exits on the first occurrence of\n> > the pattern.  I don't feel fatal is appropriate here.\n> \n> I assume git is dying with SIGPIPE. As the only purpose of \"git\n> submodule status\" is to write the status information to stdout that\n> sounds reasonable. If you want to collect the exit status you need to\n> consume the whole output so that the command runs to completion.\n> \n> Best Wishes\n> \n> PhillipSo\n> \n> > I can work around this by writing to a tmpfile but would like to see\n> > this addressed.\n> >\n> > Thanks!\n> > Matt\n> >\n> \n"},{"id":"503062","messageId":"CAKDm0rO6H4RTa5EXMrNvSy2swVegHveasY1koZXF5V7HvZzK0Q@mail.gmail.com","threadId":"62122","inReplyTo":"467f6fb4-5f83-416b-bc09-572a82f210ea@gmail.com","subject":"Re: fatal from submodule status --recursive when used with grep -q","fromName":"Matt Liberty","fromEmail":"mliberty@precisioninno.com","sentAt":"2024-09-19T15:17:17Z","receivedAt":"2024-09-19T15:17:28Z","isPatch":false,"sender":{"key":"mliberty@precisioninno.com","avatar":null},"body":"Yes and also to have a 0 exit code. No error occurred from my perspective.\n\nMatt\n\n\nOn Thu, Sep 19, 2024 at 7:22 AM wrote:\n>\n> On 18/09/2024 15:28, Matt Liberty wrote:\n> > I'm not trying to capture the exit status of git but of grep. Compare\n> > it to \"yes|grep -q y\" which doesn't generate any error even though yes\n> > generates endless output.\n>\n> Sorry, I'd misunderstood your original message. So the problem is that\n> you want \"git submodule status\" to be quiet (i.e. not print an error\n> message) when it dies of SIGPIPE?\n>\n> Best Wishes\n>\n> Phillip\n>\n> > Matt\n> >\n> > On Wed, Sep 18, 2024 at 3:05 AM Phillip Wood > > wrote:\n> >\n> > Hi Matt\n> >\n> > On 16/09/2024 16:08, Matt Liberty wrote:\n> > >\n> > > If I try to grep the output I get a fatal error:\n> > >\n> > > % git submodule status --recursive | grep -q \"^+\"\n> > > fatal: failed to recurse into submodule 'tools/OpenROAD'\n> > >\n> > > I didn't expect any output but did want the return status (0). I'm\n> > > guessing git is unhappy that grep -q exits on the first occurrence of\n> > > the pattern. I don't feel fatal is appropriate here.\n> >\n> > I assume git is dying with SIGPIPE. As the only purpose of \"git\n> > submodule status\" is to write the status information to stdout that\n> > sounds reasonable. If you want to collect the exit status you need to\n> > consume the whole output so that the command runs to completion.\n> >\n> > Best Wishes\n> >\n> > PhillipSo\n> >\n> > > I can work around this by writing to a tmpfile but would like to see\n> > > this addressed.\n> > >\n> > > Thanks!\n> > > Matt\n> > >\n> >\n"},{"id":"503087","messageId":"xmqqsetv795s.fsf@gitster.g","threadId":"62122","inReplyTo":"CAKDm0rO6H4RTa5EXMrNvSy2swVegHveasY1koZXF5V7HvZzK0Q@mail.gmail.com","subject":"Re: fatal from submodule status --recursive when used with grep -q","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-09-19T19:09:19Z","receivedAt":"2024-09-19T19:09:21Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Matt Liberty writes:\n\n[administrivia: you are not supposed to top-post on this list]\n\n> On Thu, Sep 19, 2024 at 7:22 AM wrote:\n>>\n>> On 18/09/2024 15:28, Matt Liberty wrote:\n>> > I'm not trying to capture the exit status of git but of grep. Compare\n>> > it to \"yes|grep -q y\" which doesn't generate any error even though yes\n>> > generates endless output.\n>>\n>> Sorry, I'd misunderstood your original message. So the problem is that\n>> you want \"git submodule status\" to be quiet (i.e. not print an error\n>> message) when it dies of SIGPIPE?\n>\n> Yes and also to have a 0 exit code. No error occurred from my perspective.\n\nI am sympathetic to the line of reasoning, but I wouldn't go that\nfar.\n\nRunning \"yes | less\" and killing it with 'q' saying \"ok I've seen\nenough\", and running \"yes\" and killing it with '^C' saying \"ok I've\nseen enough\" are pretty much the same from the end-user perspective.\nYou told the command to go away, and the command complied. But I\nhaven't seen anybody complaining if they see a \"Killed\" message or\n\"echo $?\" immediately after it says 130 for the latter.\n\n"},{"id":"503091","messageId":"CAPig+cTDFcJiig7=Sf5vxsgzc8_ax5t7Vqy+-_x8OK_U5U6Qnw@mail.gmail.com","threadId":"62122","inReplyTo":"xmqqsetv795s.fsf@gitster.g","subject":"Re: fatal from submodule status --recursive when used with grep -q","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2024-09-19T19:40:42Z","receivedAt":"2024-09-19T19:40:53Z","isPatch":false,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Thu, Sep 19, 2024 at 3:10 PM Junio C Hamano wrote:\n> Matt Liberty writes:\n> > Yes and also to have a 0 exit code. No error occurred from my perspective.\n>\n> I am sympathetic to the line of reasoning, but I wouldn't go that\n> far.\n>\n> Running \"yes | less\" and killing it with 'q' saying \"ok I've seen\n> enough\", and running \"yes\" and killing it with '^C' saying \"ok I've\n> seen enough\" are pretty much the same from the end-user perspective.\n> You told the command to go away, and the command complied. But I\n> haven't seen anybody complaining if they see a \"Killed\" message or\n> \"echo $?\" immediately after it says 130 for the latter.\n\nRegarding the exit code, it's not even clear how that factors into\nthis discussion considering that, in the presented example,\ngit-submodule is upstream of a pipe, thus its exit code is lost\nanyhow.\n"},{"id":"503092","messageId":"CAKDm0rPNLtv48P4JOKNicMcrz=ZZwr9PwE207D_foj1r-pn5UQ@mail.gmail.com","threadId":"62122","inReplyTo":"CAPig+cTDFcJiig7=Sf5vxsgzc8_ax5t7Vqy+-_x8OK_U5U6Qnw@mail.gmail.com","subject":"Re: fatal from submodule status --recursive when used with grep -q","fromName":"Matt Liberty","fromEmail":"mliberty@precisioninno.com","sentAt":"2024-09-19T20:10:51Z","receivedAt":"2024-09-19T20:11:03Z","isPatch":false,"sender":{"key":"mliberty@precisioninno.com","avatar":null},"body":"The exit code matters if you do \"set -euo pipefail\".\n\nMatt\n\n\nOn Thu, Sep 19, 2024 at 12:40 PM Eric Sunshine wrote:\n>\n> On Thu, Sep 19, 2024 at 3:10 PM Junio C Hamano wrote:\n> > Matt Liberty writes:\n> > > Yes and also to have a 0 exit code. No error occurred from my perspective.\n> >\n> > I am sympathetic to the line of reasoning, but I wouldn't go that\n> > far.\n> >\n> > Running \"yes | less\" and killing it with 'q' saying \"ok I've seen\n> > enough\", and running \"yes\" and killing it with '^C' saying \"ok I've\n> > seen enough\" are pretty much the same from the end-user perspective.\n> > You told the command to go away, and the command complied. But I\n> > haven't seen anybody complaining if they see a \"Killed\" message or\n> > \"echo $?\" immediately after it says 130 for the latter.\n>\n> Regarding the exit code, it's not even clear how that factors into\n> this discussion considering that, in the presented example,\n> git-submodule is upstream of a pipe, thus its exit code is lost\n> anyhow.\n"},{"id":"503124","messageId":"c2ce33c1-b9d2-4798-b91e-4e0a55e97052@gmail.com","threadId":"62122","inReplyTo":"CAKDm0rPNLtv48P4JOKNicMcrz=ZZwr9PwE207D_foj1r-pn5UQ@mail.gmail.com","subject":"Re: fatal from submodule status --recursive when used with grep -q","fromName":"","fromEmail":"phillip.wood123@gmail.com","sentAt":"2024-09-20T13:12:30Z","receivedAt":"2024-09-20T13:12:34Z","isPatch":false,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"Hi Matt\n\n[please reply in-line to avoid breaking the flow of the conversation]\n\nOn 19/09/2024 21:10, Matt Liberty wrote:\n> On Thu, Sep 19, 2024 at 12:40 PM Eric Sunshine wrote:\n>>\n>> Regarding the exit code, it's not even clear how that factors into\n>> this discussion considering that, in the presented example,\n>> git-submodule is upstream of a pipe, thus its exit code is lost\n>> anyhow.\n >\n > The exit code matters if you do \"set -euo pipefail\".\n\nWhen you use \"grep -q\" you know that if it is successful the upstream \ncommands in the pipeline are likely to receive SIGPIPE. Therefore if you \nwant to use \"-o pipefail\" to check that they do not fail unexpectedly \nyou need to remap the exit code associated with SIGPIPE to zero. \nSomething like [1]\n\n\thide_sigpipe() {\n\t\tstatus=$?\n\t\ttest $status -eq 141 && exit 0\n\t\texit $status\n\t}\n\t(yes; hide_sigpipe) | grep -q y\n\n\nI've posted a patch[2] that fixes the error message you were seeing.\n\nBest Wishes\n\nPhillip\n\n[1] \nhttps://unix.stackexchange.com/questions/582844/how-to-suppress-sigpipe-in-bash\n[2] \nhttps://lore.kernel.org/git/pull.1799.git.1726837642511.gitgitgadget@gmail.com\n"}]}