In-depth Analysis and Practice of Executing Multiple Bash Commands with Python Subprocess Module

Problem Background and Core Challenges

In Python development, using the subprocess module to call external commands is a common requirement. However, when multiple Bash commands need to be executed, developers often encounter issues where command separators are incorrectly parsed. The original code example illustrates this typical scenario:

import subprocess, shlex

def subprocess_cmd(command):
    process = subprocess.Popen(shlex.split(command), stdout=subprocess.PIPE)
    proc_stdout = process.communicate()[0].strip()
    print(proc_stdout)

subprocess_cmd("echo a; echo b")

This code expects to output two lines: a and b, but the actual output is a; echo b. The root cause is that shlex.split() treats the entire string as a single command argument, and the semicolon ; as a Bash command separator is not properly recognized.

Core Solution: The shell=True Parameter

To resolve this issue, the Bash shell itself must parse the command string. This can be achieved by setting the shell=True parameter:

import subprocess

command = "echo a; echo b"
ret = subprocess.run(command, capture_output=True, shell=True)
print(ret.stdout.decode())

This improved code correctly outputs:

a
b

Technical Principle Analysis

When shell=True, Python executes the command string through the system's shell program (typically /bin/bash on Unix-like systems). This allows Bash to recognize and process special characters such as command separators ;, pipes |, and redirections >.

In contrast, when shell=False (the default), the command string is directly passed as an argument to the executable, and special characters are not interpreted by the shell, causing command separators to fail.

Alternative Approach: Direct Interaction with Bash Process

For more complex multi-line command scenarios, another effective method is to directly create a Bash process and pass commands through standard input:

commands = '''
echo "a"
echo "b"
echo "c"
echo "d"
'''

process = subprocess.Popen('/bin/bash', stdin=subprocess.PIPE, stdout=subprocess.PIPE, text=True)
out, err = process.communicate(commands)
print(out)

This method is particularly suitable for:

• Scenarios requiring execution of numerous consecutive commands
• Commands containing complex logical structures (e.g., loops, conditional statements)
• Situations where maintaining continuous Bash environment state is necessary

Security Considerations and Best Practices

Although shell=True offers convenience, it also introduces security risks. If the command string includes user input, it may be vulnerable to shell injection attacks. Recommendations include:

1. Prefer using shell=False and pass commands and arguments in list form
2. If shell=True is necessary, strictly validate and escape user input
3. Consider using subprocess.run() instead of the older Popen interface for better error handling and resource management

Extended Application: Interactive Bash Subshells

Referencing related technical discussions, in certain advanced scenarios, developers may need to launch an interactive Bash subshell and pre-execute initialization commands. While this is not the primary focus of this article, such requirements highlight the complexity of Bash process control.

By combining Popen, standard input/output redirection, and appropriate Bash parameters, various complex process interaction patterns can be implemented, providing powerful foundational capabilities for automation scripts and system management tools.

Conclusion

The key to correctly executing multiple Bash commands lies in understanding the mechanism of the shell parameter. shell=True enables Bash to properly parse special characters in the command string, while direct interaction with the Bash process offers a more flexible solution for complex scenarios. Developers should choose the appropriate method based on specific requirements and security considerations in practical applications.