# fatal from submodule status --recursive when used with grep -q 11 messages from 2024-09-16 to 2024-09-20. Participants: Matt Liberty, Phillip Wood, Junio C Hamano, phillip.wood123@gmail.com, Eric Sunshine. Thread: https://gitlist.dev/t/62122 ## Matt Liberty, 2024-09-16 15:08 Subject: fatal from submodule status --recursive when used with grep -q Message-ID: URL: https://gitlist.dev/e/CAKDm0rNaHbzoiPg%3DDeuCoxzooNAsxw2BJfc0wg7fC_-%3Do9uJ7w%40mail.gmail.com ``` Hi I have a repo with submodule, like: % git submodule status --recursive +834b7a0fb6de03c8e584256af2ecb7889b568343 tools/OpenROAD (v2.0-15587-g834b7a0fb) aafee90f8a21bf7867cef2e159929440cf45b2e5 tools/OpenROAD/src/sta (v2.2.0-1106-gaafee90) ef5389d31526003c2ebd7e6d6d6fe3848a20f0a2 tools/OpenROAD/third-party/abc (remotes/origin/HEAD) 80ba43d26264738c93900129dc0aab7fab36c53f tools/yosys (yosys-0.44) 28d955ca97a1c4be3aed4062aec0241a734fac5d tools/yosys/abc (yosys-0.44) If I try to grep the output I get a fatal error: % git submodule status --recursive | grep -q "^+" fatal: failed to recurse into submodule 'tools/OpenROAD' I didn't expect any output but did want the return status (0). I'm guessing git is unhappy that grep -q exits on the first occurrence of the pattern. I don't feel fatal is appropriate here. I can work around this by writing to a tmpfile but would like to see this addressed. Thanks! Matt ``` ## Phillip Wood, 2024-09-18 10:05 Subject: Re: fatal from submodule status --recursive when used with grep -q Message-ID: <7b4ce2fc-3763-4881-8459-e0c1f2cbc69d@gmail.com> URL: https://gitlist.dev/e/7b4ce2fc-3763-4881-8459-e0c1f2cbc69d%40gmail.com In-Reply-To: ``` Hi Matt On 16/09/2024 16:08, Matt Liberty wrote: > > If I try to grep the output I get a fatal error: > > % git submodule status --recursive | grep -q "^+" > fatal: failed to recurse into submodule 'tools/OpenROAD' > > I didn't expect any output but did want the return status (0). I'm > guessing git is unhappy that grep -q exits on the first occurrence of > the pattern. I don't feel fatal is appropriate here. I assume git is dying with SIGPIPE. As the only purpose of "git submodule status" is to write the status information to stdout that sounds reasonable. If you want to collect the exit status you need to consume the whole output so that the command runs to completion. Best Wishes Phillip > I can work around this by writing to a tmpfile but would like to see > this addressed. > > Thanks! > Matt > ``` ## Matt Liberty, 2024-09-18 14:45 Subject: Re: fatal from submodule status --recursive when used with grep -q Message-ID: URL: https://gitlist.dev/e/CAKDm0rNjAYMu8%3Dh8ApvEHw71fgH2VW0iCjTMqvysQ7YBUnJZyw%40mail.gmail.com In-Reply-To: <7b4ce2fc-3763-4881-8459-e0c1f2cbc69d@gmail.com> ``` I'm not trying to capture the exit status of git but of grep. Compare it to "yes|grep -q y" which doesn't generate any error even though yes generates endless output. Thanks Matt On Wed, Sep 18, 2024 at 3:05 AM Phillip Wood wrote: > > Hi Matt > > On 16/09/2024 16:08, Matt Liberty wrote: > > > > If I try to grep the output I get a fatal error: > > > > % git submodule status --recursive | grep -q "^+" > > fatal: failed to recurse into submodule 'tools/OpenROAD' > > > > I didn't expect any output but did want the return status (0). I'm > > guessing git is unhappy that grep -q exits on the first occurrence of > > the pattern. I don't feel fatal is appropriate here. > > I assume git is dying with SIGPIPE. As the only purpose of "git > submodule status" is to write the status information to stdout that > sounds reasonable. If you want to collect the exit status you need to > consume the whole output so that the command runs to completion. > > Best Wishes > > Phillip > > > I can work around this by writing to a tmpfile but would like to see > > this addressed. > > > > Thanks! > > Matt > > ``` ## Junio C Hamano, 2024-09-19 01:16 Subject: Re: fatal from submodule status --recursive when used with grep -q Message-ID: URL: https://gitlist.dev/e/xmqq1q1ga1dp.fsf%40gitster.g In-Reply-To: <7b4ce2fc-3763-4881-8459-e0c1f2cbc69d@gmail.com> ``` Phillip Wood writes: > On 16/09/2024 16:08, Matt Liberty wrote: >> If I try to grep the output I get a fatal error: >> % git submodule status --recursive | grep -q "^+" >> fatal: failed to recurse into submodule 'tools/OpenROAD' >> I didn't expect any output but did want the return status (0). I'm >> guessing git is unhappy that grep -q exits on the first occurrence of >> the pattern. I don't feel fatal is appropriate here. > > I assume git is dying with SIGPIPE. As the only purpose of "git > submodule status" is to write the status information to stdout that > sounds reasonable. If you want to collect the exit status you need to > consume the whole output so that the command runs to completion. Yes, but isn't the main point of that complaint that "failed to recurse into" is not a good way to say "because you closed the reading end of the pipe, we are aborting the operation and not decending into 'tools/OpenROAD' submodule", is it? ``` ## Matt Liberty, 2024-09-19 13:23 Subject: Re: fatal from submodule status --recursive when used with grep -q Message-ID: URL: https://gitlist.dev/e/CAKDm0rN-jvtmVhwODQTcQJu5LYTJ7GwTKUtfA9kzifHR6WWf9w%40mail.gmail.com In-Reply-To: ``` I'm saying most unix tools do not error when the pipe is closed and I gave 'yes' as a simple example but there are many others. grep found what I was looking for and I don't care that git didn't recurse further (in fact I'm happy that it didn't waste time doing so). There is no problem except that git insists on complaining about the closed pipe. Matt On Wed, Sep 18, 2024 at 6:16 PM Junio C Hamano wrote: > > Phillip Wood writes: > > > On 16/09/2024 16:08, Matt Liberty wrote: > >> If I try to grep the output I get a fatal error: > >> % git submodule status --recursive | grep -q "^+" > >> fatal: failed to recurse into submodule 'tools/OpenROAD' > >> I didn't expect any output but did want the return status (0). I'm > >> guessing git is unhappy that grep -q exits on the first occurrence of > >> the pattern. I don't feel fatal is appropriate here. > > > > I assume git is dying with SIGPIPE. As the only purpose of "git > > submodule status" is to write the status information to stdout that > > sounds reasonable. If you want to collect the exit status you need to > > consume the whole output so that the command runs to completion. > > Yes, but isn't the main point of that complaint that "failed to > recurse into" is not a good way to say "because you closed the > reading end of the pipe, we are aborting the operation and not > decending into 'tools/OpenROAD' submodule", is it? > ``` ## phillip.wood123@gmail.com, 2024-09-19 14:22 Subject: Re: fatal from submodule status --recursive when used with grep -q Message-ID: <467f6fb4-5f83-416b-bc09-572a82f210ea@gmail.com> URL: https://gitlist.dev/e/467f6fb4-5f83-416b-bc09-572a82f210ea%40gmail.com In-Reply-To: ``` On 18/09/2024 15:28, Matt Liberty wrote: > I'm not trying to capture the exit status of git but of grep.  Compare > it to "yes|grep -q y" which doesn't generate any error even though yes > generates endless output. Sorry, I'd misunderstood your original message. So the problem is that you want "git submodule status" to be quiet (i.e. not print an error message) when it dies of SIGPIPE? Best Wishes Phillip > Matt > > On Wed, Sep 18, 2024 at 3:05 AM Phillip Wood > wrote: > > Hi Matt > > On 16/09/2024 16:08, Matt Liberty wrote: > > > > If I try to grep the output I get a fatal error: > > > > % git submodule status --recursive | grep -q "^+" > > fatal: failed to recurse into submodule 'tools/OpenROAD' > > > > I didn't expect any output but did want the return status (0).  I'm > > guessing git is unhappy that grep -q exits on the first occurrence of > > the pattern.  I don't feel fatal is appropriate here. > > I assume git is dying with SIGPIPE. As the only purpose of "git > submodule status" is to write the status information to stdout that > sounds reasonable. If you want to collect the exit status you need to > consume the whole output so that the command runs to completion. > > Best Wishes > > PhillipSo > > > I can work around this by writing to a tmpfile but would like to see > > this addressed. > > > > Thanks! > > Matt > > > ``` ## Matt Liberty, 2024-09-19 15:17 Subject: Re: fatal from submodule status --recursive when used with grep -q Message-ID: URL: https://gitlist.dev/e/CAKDm0rO6H4RTa5EXMrNvSy2swVegHveasY1koZXF5V7HvZzK0Q%40mail.gmail.com In-Reply-To: <467f6fb4-5f83-416b-bc09-572a82f210ea@gmail.com> ``` Yes and also to have a 0 exit code. No error occurred from my perspective. Matt On Thu, Sep 19, 2024 at 7:22 AM wrote: > > On 18/09/2024 15:28, Matt Liberty wrote: > > I'm not trying to capture the exit status of git but of grep. Compare > > it to "yes|grep -q y" which doesn't generate any error even though yes > > generates endless output. > > Sorry, I'd misunderstood your original message. So the problem is that > you want "git submodule status" to be quiet (i.e. not print an error > message) when it dies of SIGPIPE? > > Best Wishes > > Phillip > > > Matt > > > > On Wed, Sep 18, 2024 at 3:05 AM Phillip Wood > > wrote: > > > > Hi Matt > > > > On 16/09/2024 16:08, Matt Liberty wrote: > > > > > > If I try to grep the output I get a fatal error: > > > > > > % git submodule status --recursive | grep -q "^+" > > > fatal: failed to recurse into submodule 'tools/OpenROAD' > > > > > > I didn't expect any output but did want the return status (0). I'm > > > guessing git is unhappy that grep -q exits on the first occurrence of > > > the pattern. I don't feel fatal is appropriate here. > > > > I assume git is dying with SIGPIPE. As the only purpose of "git > > submodule status" is to write the status information to stdout that > > sounds reasonable. If you want to collect the exit status you need to > > consume the whole output so that the command runs to completion. > > > > Best Wishes > > > > PhillipSo > > > > > I can work around this by writing to a tmpfile but would like to see > > > this addressed. > > > > > > Thanks! > > > Matt > > > > > ``` ## Junio C Hamano, 2024-09-19 19:09 Subject: Re: fatal from submodule status --recursive when used with grep -q Message-ID: URL: https://gitlist.dev/e/xmqqsetv795s.fsf%40gitster.g In-Reply-To: ``` Matt Liberty writes: [administrivia: you are not supposed to top-post on this list] > On Thu, Sep 19, 2024 at 7:22 AM wrote: >> >> On 18/09/2024 15:28, Matt Liberty wrote: >> > I'm not trying to capture the exit status of git but of grep. Compare >> > it to "yes|grep -q y" which doesn't generate any error even though yes >> > generates endless output. >> >> Sorry, I'd misunderstood your original message. So the problem is that >> you want "git submodule status" to be quiet (i.e. not print an error >> message) when it dies of SIGPIPE? > > Yes and also to have a 0 exit code. No error occurred from my perspective. I am sympathetic to the line of reasoning, but I wouldn't go that far. Running "yes | less" and killing it with 'q' saying "ok I've seen enough", and running "yes" and killing it with '^C' saying "ok I've seen enough" are pretty much the same from the end-user perspective. You told the command to go away, and the command complied. But I haven't seen anybody complaining if they see a "Killed" message or "echo $?" immediately after it says 130 for the latter. ``` ## Eric Sunshine, 2024-09-19 19:40 Subject: Re: fatal from submodule status --recursive when used with grep -q Message-ID: URL: https://gitlist.dev/e/CAPig%2BcTDFcJiig7%3DSf5vxsgzc8_ax5t7Vqy%2B-_x8OK_U5U6Qnw%40mail.gmail.com In-Reply-To: ``` On Thu, Sep 19, 2024 at 3:10 PM Junio C Hamano wrote: > Matt Liberty writes: > > Yes and also to have a 0 exit code. No error occurred from my perspective. > > I am sympathetic to the line of reasoning, but I wouldn't go that > far. > > Running "yes | less" and killing it with 'q' saying "ok I've seen > enough", and running "yes" and killing it with '^C' saying "ok I've > seen enough" are pretty much the same from the end-user perspective. > You told the command to go away, and the command complied. But I > haven't seen anybody complaining if they see a "Killed" message or > "echo $?" immediately after it says 130 for the latter. Regarding the exit code, it's not even clear how that factors into this discussion considering that, in the presented example, git-submodule is upstream of a pipe, thus its exit code is lost anyhow. ``` ## Matt Liberty, 2024-09-19 20:10 Subject: Re: fatal from submodule status --recursive when used with grep -q Message-ID: URL: https://gitlist.dev/e/CAKDm0rPNLtv48P4JOKNicMcrz%3DZZwr9PwE207D_foj1r-pn5UQ%40mail.gmail.com In-Reply-To: ``` The exit code matters if you do "set -euo pipefail". Matt On Thu, Sep 19, 2024 at 12:40 PM Eric Sunshine wrote: > > On Thu, Sep 19, 2024 at 3:10 PM Junio C Hamano wrote: > > Matt Liberty writes: > > > Yes and also to have a 0 exit code. No error occurred from my perspective. > > > > I am sympathetic to the line of reasoning, but I wouldn't go that > > far. > > > > Running "yes | less" and killing it with 'q' saying "ok I've seen > > enough", and running "yes" and killing it with '^C' saying "ok I've > > seen enough" are pretty much the same from the end-user perspective. > > You told the command to go away, and the command complied. But I > > haven't seen anybody complaining if they see a "Killed" message or > > "echo $?" immediately after it says 130 for the latter. > > Regarding the exit code, it's not even clear how that factors into > this discussion considering that, in the presented example, > git-submodule is upstream of a pipe, thus its exit code is lost > anyhow. ``` ## phillip.wood123@gmail.com, 2024-09-20 13:12 Subject: Re: fatal from submodule status --recursive when used with grep -q Message-ID: URL: https://gitlist.dev/e/c2ce33c1-b9d2-4798-b91e-4e0a55e97052%40gmail.com In-Reply-To: ``` Hi Matt [please reply in-line to avoid breaking the flow of the conversation] On 19/09/2024 21:10, Matt Liberty wrote: > On Thu, Sep 19, 2024 at 12:40 PM Eric Sunshine wrote: >> >> Regarding the exit code, it's not even clear how that factors into >> this discussion considering that, in the presented example, >> git-submodule is upstream of a pipe, thus its exit code is lost >> anyhow. > > The exit code matters if you do "set -euo pipefail". When you use "grep -q" you know that if it is successful the upstream commands in the pipeline are likely to receive SIGPIPE. Therefore if you want to use "-o pipefail" to check that they do not fail unexpectedly you need to remap the exit code associated with SIGPIPE to zero. Something like [1] hide_sigpipe() { status=$? test $status -eq 141 && exit 0 exit $status } (yes; hide_sigpipe) | grep -q y I've posted a patch[2] that fixes the error message you were seeing. Best Wishes Phillip [1] https://unix.stackexchange.com/questions/582844/how-to-suppress-sigpipe-in-bash [2] https://lore.kernel.org/git/pull.1799.git.1726837642511.gitgitgadget@gmail.com ```