Variable Interpolation in Bash Heredoc: Mechanisms and Advanced Applications

In Bash scripting, heredoc (here document) is a common method for inputting multiline strings, allowing users to embed large text blocks directly without concatenating multiple echo commands. However, when interpolating variables within heredoc, developers often encounter issues where variables are not expanded correctly. This paper starts from basic mechanisms and delves into the causes, providing multiple solutions.

Basic Syntax and Variable Interpolation Mechanism

The Bash heredoc syntax is: <<[-]word ... delimiter. According to the Bash manual, if word is unquoted (i.e., without single quotes, double quotes, or backslashes), all lines in the heredoc undergo parameter expansion, command substitution, and arithmetic expansion. This means variables like $var are automatically replaced with their values. Conversely, if word is quoted, the heredoc content is treated as literal text, and variables are not expanded.

For example, in the following code, because the delimiter EOF is enclosed in double quotes, the variable $var is not processed:

var=$1
sudo tee "/path/to/outfile" > /dev/null << "EOF"
Some text that contains my $var
EOF

To fix this, simply remove the quotes from the delimiter:

var=$1
sudo tee "/path/to/outfile" > /dev/null <<EOF
Some text that contains my $var
EOF

This ensures $var is correctly expanded within the heredoc. The core mechanism lies in Bash's handling of delimiters: quoting a delimiter disables all expansions, including variable interpolation.

Challenges in Sudo Environments

In scenarios requiring sudo privileges, heredoc usage becomes more complex. For instance, attempting to use sudo cat with redirection may fail because the redirection operation > /path/to/outfile is executed in the current shell, which lacks sudo permissions. The following code does not work:

sudo cat > /path/to/outfile <<EOT
my text...
EOT

This is because sudo only applies to the cat command, while redirection is performed by the current user. To resolve this, wrap the entire command in a subshell to ensure redirection occurs in the sudo context:

sudo sh -c "cat > /path/to/outfile" <<EOT
my text...
EOT

Alternatively, use the tee command (as in the initial example), which allows writing files via sudo while variables in heredoc expand normally (provided the delimiter is unquoted). This approach combines sudo's privilege escalation with heredoc flexibility, a common technique for privileged file writes.

Advanced Techniques: Mixed Interpolation and Literal Text Control

In practical applications, it may be necessary to mix variable expansions with literal text in heredoc. For example, if the text contains dollar signs or backticks that should not be interpreted, use backslashes for escaping. The following example demonstrates control over interpolation:

Name='Rich Ba$tard'
dough='$$$dollars$$$'
cat <<____HERE
$Name, you can win a lot of $dough this week!
Notice that \`backticks\` need escaping if you want
literal text, not `pwd`, just like in variables like
\$HOME (current value: $HOME)
____HERE

Here, $Name and $dough are expanded, while \`backticks\` and \$HOME remain literal via backslashes. This provides fine-grained control, enabling dynamic content embedding while avoiding unwanted expansions.

Cross-Shell and Remote Execution Scenarios

Heredoc is particularly useful for cross-shell or remote execution tasks. For instance, when executing commands on a remote host via SSH, local variables may need to be passed while preventing certain parts from expanding locally. The following code illustrates this:

local=$(uname)
ssh -t remote <<:
    echo "$local is the value from the host which ran the ssh command"
    # Prevent here doc from expanding locally; remote won't see backslash
    remote=\$(uname)
    # Same here
    echo "\$remote is the value from the host we ssh:ed to"
:

In this case, $local is expanded locally and passed to the remote, while \$(uname) and \$remote are escaped to ensure interpretation only in the remote shell. This leverages heredoc flexibility for complex automation scripts, especially in DevOps and system administration tasks.

Conclusion and Best Practices

Understanding variable interpolation in Bash heredoc hinges on the quoting state of delimiters: unquoted enables expansion, quoted disables it. In sudo environments, use subshells or tee to ensure redirection occurs with correct permissions. For mixed content, escaping mechanisms offer necessary control. In practice, always test heredoc behavior, especially with privileged operations or cross-environment tasks, to avoid security vulnerabilities or functional errors. By mastering these techniques, developers can efficiently utilize heredoc for multiline text and variable interpolation, enhancing script reliability and maintainability.