Subprocess Module

Indices and tables

Introduction

Python provides the subprocess module to allow your programs to spawn new processes in an OS-agnostic way. Using the subprocess module, you can connect to the input, output and error streams of the new process and retrieve its return code.

Tip

The os module also has functions for spawning processes using the exec() functions. However, these functions cause the current program to be replaced (i.e. terminated). While this can be useful, be aware of this difference.

Spawning Processes

subprocess.run(args, *, input=None, stdin=None, stdout=None, stderr=None, shell=False, check=False)

Run the command described by args. Wait for command to complete, then return a subprocess.CompletedProcess instance.

  • args - A sequence (preferred) or a string that contains the program to run and the arguments to it.
  • input - A byte sequence or string of data to send the program. Can’t be used if stdin is not None.
  • stdin - The standard input file handle to the program. Can’t be used if input is not None.
  • stdout - The standard output file handle to the program. By default, this is a bytes object.
  • stderr - The standard error file handle to the program. By default, this is a bytes object.
  • shell - When True, a shell is spawned first and then the command is executed. If False, the command is executed directly.
  • check - When true and the process exists with a non-zero return code, a CalledProcessError exception will be raised.

The stdin, stdout and stderr arguments can take on the following values:

  • None - Inherit file handles of the parent.
  • subprocess.PIPE - Create a communications pipe with the child process. Can send input data and/or retrieve output data.
  • subprocess.DEVNULL - Connect the stream to the null device.
  • Existing file descriptor or file object - Use that object for IO
  • subprocess.STDOUT - Only valid for stderr; redirects stderr to stdout.

For example:

>>> #
>>> # Spawn a process to get a directory listing
>>> # NOTE: There are better ways to get a directory
>>> #       listing in Python than this!
>>> #
>>> import subprocess
>>> retval = subprocess.run(["ls", "-l"], stdout=subprocess.PIPE)

>>> #
>>> # Examine the type of the returned object
>>> #
>>> type(retval)
subprocess.CompletedProcess

>>> #
>>> # See what args were sent into the call to subprocess.run
>>> #
>>> retval.args
['ls', '-l']

>>> #
>>> # See what exit code the child process returned.
>>> #
>>> retval.returncode
0

>>> #
>>> # See the output returned via stdout from the child process.
>>> # NOTE: By default, a bytes object is returned, so, must
>>> #       decode it back to a string and print it.
>>> #
>>> print(retval.stdout.decode())
total 77576
drwxr-xr-x 1 vagrant users          0 Nov 28 18:07 bin
-rwxr--r-- 1 vagrant users        717 Mar 22 02:24 conky_fn.sh
drwxr-xr-x 1 vagrant users        386 Mar 22 02:30 Desktop
drwxr-xr-x 1 vagrant vagrant      644 Mar 22 02:22 Python-3.5.5
-rw-r--r-- 1 root    root    79431680 Feb  5 00:53 Python-3.5.5.tar

Using the Shell

If the shell argument to the subprocess.run() command is True, an intermediate shell will be spawned which causes variables, glob patterns and other special shell features that may be present in args, to be processed.

Tip

When you set shell to True, you have to pass args as a string, not a sequence.

The $HOME component in the following example will be expanded out by the shell:

>>> import subprocess
>>> retval = subprocess.run(["ls -l $HOME/Desktop"], shell=True, stdout=subprocess.PIPE)
>>> print(retval.stdout.decode())
total 32
-rwxr-xr-x 1 vagrant users  528 Mar 22 03:29 firefox.desktop
-rwxr-xr-x 1 vagrant users  164 Mar 22 03:29 GitEye.desktop
-rwxr-xr-x 1 vagrant users  273 Mar 22 03:29 jedit.desktop
-rwxr-xr-x 1 vagrant users  290 Mar 22 03:29 jetbrains-pycharm-ce.desktop
-rwxr-xr-x 1 vagrant users  233 Mar 22 03:29 Jupyter-Notebook.desktop
drwxr-xr-x 1 vagrant users    0 Mar 22 03:23 notebooks
-rwxr-xr-x 1 vagrant users 3318 Mar 22 03:29 org.kde.konsole.desktop
drwxr-xr-x 1 vagrant users   72 Mar 22 03:29 pythonedu
-rwxr-xr-x 1 vagrant users  255 Mar 22 03:29 PythonEDU-course.desktop
-rwxr-xr-x 1 vagrant users  207 Mar 22 03:29 Update-PythonEDU-Materials.desktop

Process IO

The earlier example demonstrated how we can capture the output from a child process. But you can feed the child process a bytes object when it is spawned using the input argument. For example:

>>> #
>>> # Call the child process to do some work.
>>> #
>>> import subprocess
>>> retval = subprocess.run(["grep", "Py"], input="0-Zed\n1-PyB\n2-Lam\n3-Pyk".encode(), stdout=subprocess.PIPE)

>>> #
>>> # Get it's output.
>>> #
>>>print(retval.stdout.decode())
1-PyB
3-Pyk

The subprocess.run() command is a convenience function that can handle the most common use cases. For more interactive communication with the spawned process, you must use the underlying subprocess.Popen() function.