How to Execute Linux Commands in Python

In this guide, we will learn how to run Linux commands using Python. Python is a powerful programming language that can help automate many tasks, including running commands on a Linux system. By the end of this guide, you’ll be able to use Python to execute Linux commands easily and efficiently. Whether you’re a beginner or an experienced programmer, this tutorial will provide simple steps and examples to get you started.

Using the os Module

The os module is part of the Python Standard Library, which provides a way of using operating system-dependent functionality.

One of the simplest ways to run a Linux command is by using the os.system function:

import os

# Running a simple command
os.system("ls -l")

While os.system is easy to use, it’s not recommended for more complex tasks because it is less flexible and does not provide a way to capture command output directly.

Using the subprocess Module

The subprocess module is a more powerful alternative for running system commands in Python. It allows you to spawn new processes, connect to their input/output/error pipes, and obtain their return codes.

Python 3.5 introduced the run function, which is the recommended approach as it is more versatile.

import subprocess

# Running a command and capturing its output
completed_process = subprocess.run(["ls", "-l"], capture_output=True, text=True)

# Accessing the output
print(completed_process.stdout)

# Checking the return code
if completed_process.returncode != 0:
    print("The command failed with return code", completed_process.returncode)

The Popen Class

For more complex interactions with subprocesses, Popen class is used.

import subprocess

# Starting a process
process = subprocess.Popen(["ls", "-l"], stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)

# Communicating with the process
stdout, stderr = process.communicate()

# Printing the output
print(stdout)

# Checking if the process had an error
if process.returncode != 0:
    print("Error:", stderr)

Capturing Command Output

As seen in the examples above, capturing the output of a command is often essential. The subprocess.run and subprocess.Popen functions provide the stdout and stderr parameters to capture the standard output and standard error of the command, respectively.

Handling Shell Pipelines

Sometimes you may want to execute a shell pipeline with multiple commands. This can be done by passing shell=True to the subprocess.run or subprocess.Popen functions.

import subprocess

# Running a shell pipeline
completed_process = subprocess.run(
    "grep 'some_text' some_file.txt | wc -l",
    shell=True,
    capture_output=True,
    text=True
)

# Printing the output
print(completed_process.stdout.strip())

Be cautious with `shell=True`, especially when using user-generated input, as it is more vulnerable to shell injection attacks.

Handling Errors

When executing Linux commands from Python, it’s important to handle potential errors, such as non-zero exit codes or exceptions.

import subprocess

try:
    completed_process = subprocess.run(["ls", "-l", "/non/existent/directory"], check=True, text=True)
except subprocess.CalledProcessError as e:
    print(f"The command '{e.cmd}' failed with return code {e.returncode}")

Conclusion

Running Linux commands in Python is easy using the os and subprocess modules, with subprocess offering more features. It’s important to execute commands securely, especially with user input, and to properly manage the output and errors for reliable scripts and applications.