Add new check for error ignoring on command substitutions

[felipecrs]

For new checks and feature suggestions

• https://www.shellcheck.net/ (i.e. the latest commit) currently gives no useful warnings about this
• I searched through https://github.com/koalaman/shellcheck/issues and didn't find anything related

Suggestion

Given the following script:

#!/bin/bash

set -e

basename "$(curl -fsSL -w '%{url_effective}' -o /dev/null https://github.com/docker/compose/releases/latest)"

It would normally work, but curl could actually fail (such as with command not found). So, for the sake of the example, I will force it to fail by using curl2.

Examples:

$ bash <<'EOF'; echo $?
#!/bin/bash

set -e

basename "$(curl -fsSL -w '%{url_effective}' -o /dev/null https://github.com/docker/compose/releases/latest)"
EOF
v2.0.0
0

With curl2:

$ bash <<'EOF'; echo $?
#!/bin/bash

set -e

basename "$(curl2 -fsSL -w '%{url_effective}' -o /dev/null https://github.com/docker/compose/releases/latest)"
EOF
bash: line 5: curl2: command not found

0

The status code of the latter example is 0 even if the command failed. This is misleading. ShellCheck could advise to be written like so instead:

$ bash <<'EOF'; echo $?
#!/bin/bash

set -e

curl_output="$(curl2 -fsSL -w '%{url_effective}' -o /dev/null https://github.com/docker/compose/releases/latest)"
basename "$curl_output"
EOF
bash: line 5: curl2: command not found
127

References twpayne/chezmoi#1466 (comment)

[kurahaupo]

Yet another strike against set -e for creating expectations that directly conflict with how the shell actually works and how correct, idiomatic shell code is normally written.

If such a warning were given, I would like it to be gated on the presence of set -e, and for the wording of the warning to point out that set -e makes promises that cannot be met.

[felipecrs]

I couldn't agree more with @kurahaupo.

[koalaman]

Thanks to @DoxasticFox you can now enable an optional check for this:

#!/bin/bash
# shellcheck enable=check-extra-masked-returns
set -e
basename "$(curl -fsSL -w '%{url_effective}' -o /dev/null https://github.com/docker/compose/releases/latest)"

results in

basename "$(curl -fsSL -w '%{url_effective}' -o /dev/null https://github.com/docker/compose/releases/latest)"
            ^-- SC2312 (info): Consider invoking this command separately to avoid masking its return value (or use '|| true' to ignore).

[felipecrs]

This is awesome!

[felipecrs]

For reference, this is @DoxasticFox's PR: #2320

[felipecrs]

Using this feature, I realized there are:

[[ $(command -v curl) ]]

and

my_function "$(cat <<EOF
my
multiline
parameter
EOF
)"

Which are two cases very known, and return code on them is very likely not to mean anything. Should this rule have exceptions like these above?

[DoxasticFox]

@felipecrs I think this:

command -v curl

and

my_function "my
multiline
parameter
"

should have similar effects. Why not these?

[felipecrs]

For the first, you are right. Btw the drop-in replacement would be command -v curl &>/dev/null, as my first does not print to stdout too.

For the second, there are multiple benefits of using here-docs, such as 1. better readability in the code, 2. allow to be indented with <<-EOF and 3. no need to escape quotes inside of it.

[DoxasticFox]

Ah, I didn't understand the heredoc was the important part. Well, #2320 assumes that certain commands' exit codes can be safely masked. We could add cat to that list of commands. But that seems overly permissive to me, because cat file/that/might/not/exist is a common use of cat and it can fail. Also, based on @koalaman's comment #2320 (comment), he might need some convincing to accept a PR dealing with special cases, like where cat takes a heredoc. It might be worth raising an issue to discuss it.

In the meanwhile, another alternative is this:

arg="$(cat <<EOF
my
multiline
parameter
EOF
)"
my_function "$arg"

It's a bit of shame to have to introduce a variable to deal with this case though, I'll admit. But it wouldn't be the only case where using set -e would force you to do that.

[felipecrs]

That's right. Thanks for the rationale.

Perhaps an alternative solution would be to allow # shellcheck disable=check-extra-masked-returns instead of having to use || true, as it would be just a comment rather than being an actual command in the code which bash will have to test whether the command failed or not at runtime.

[DoxasticFox]

You can use # shellcheck disable=SC2312.

[felipecrs]

Btw I created the issue for discussing: #2359