Subprocess Module
=================

.. toctree::
   :maxdepth: 1


Indices and tables
------------------

* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`


Introduction
------------
Python provides the :py:mod:`subprocess` module to allow your programs to spawn new processes in an OS-agnostic way. Using the :py:mod:`subprocess` module, you can connect to the input, output and error streams of the new process and retrieve its return code.


.. tip:: The :py:mod:`os` module also has functions for spawning processes using the :py:func:`~os.exec` functions. However, these functions cause the current program to be **replaced** (i.e. terminated). While this can be useful, be aware of this difference.


.. _section_heading-Spawning_Processes:

Spawning Processes
------------------

.. py:function:: subprocess.run(args, *, input=None, stdin=None, stdout=None, stderr=None, shell=False, check=False)

   Run the command described by ``args``. Wait for command to complete, then return a :py:class:`subprocess.CompletedProcess` instance.


   * ``args`` - A sequence (preferred) or a string that contains the program to run and the arguments to it.

   * ``input`` - A byte sequence or string of data to send the program. Can't be used if ``stdin`` is not :py:data:`None`.

   * ``stdin`` - The standard input file handle to the program. Can't be used if ``input`` is not :py:data:`None`.

   * ``stdout`` - The standard output file handle to the program. By default, this is a :py:class:`bytes` object.

   * ``stderr`` - The standard error file handle to the program. By default, this is a :py:class:`bytes` object.

   * ``shell`` - When :py:data:`True`, a shell is spawned first and then the command is executed. If :py:data:`False`, the command is executed directly.


   * ``check`` - When true and the process exists with a non-zero return code, a :py:exc:`CalledProcessError` exception will be raised.


The ``stdin``, ``stdout`` and ``stderr`` arguments can take on the following values:

   * :py:data:`None` - Inherit file handles of the parent.

   * :py:data:`subprocess.PIPE` - Create a communications pipe with the child process. Can send input data and/or retrieve output data.

   * :py:data:`subprocess.DEVNULL` - Connect the stream to the null device.

   * Existing file descriptor or file object - Use that object for IO

   * :py:data:`subprocess.STDOUT` - Only valid for ``stderr``; redirects ``stderr`` to ``stdout``.


For example:

>>> #
>>> # Spawn a process to get a directory listing
>>> # NOTE: There are better ways to get a directory
>>> #       listing in Python than this!
>>> #
>>> import subprocess
>>> retval = subprocess.run(["ls", "-l"], stdout=subprocess.PIPE)

>>> #
>>> # Examine the type of the returned object
>>> #
>>> type(retval)
subprocess.CompletedProcess

>>> #
>>> # See what args were sent into the call to subprocess.run
>>> #
>>> retval.args
['ls', '-l']

>>> #
>>> # See what exit code the child process returned.
>>> #
>>> retval.returncode
0

>>> #
>>> # See the output returned via stdout from the child process.
>>> # NOTE: By default, a bytes object is returned, so, must
>>> #       decode it back to a string and print it.
>>> #
>>> print(retval.stdout.decode())
total 77576
drwxr-xr-x 1 vagrant users          0 Nov 28 18:07 bin
-rwxr--r-- 1 vagrant users        717 Mar 22 02:24 conky_fn.sh
drwxr-xr-x 1 vagrant users        386 Mar 22 02:30 Desktop
drwxr-xr-x 1 vagrant vagrant      644 Mar 22 02:22 Python-3.5.5
-rw-r--r-- 1 root    root    79431680 Feb  5 00:53 Python-3.5.5.tar



.. _section_heading-Using_The_Shell:

Using the Shell
---------------

If the ``shell`` argument to the :py:func:`subprocess.run` command is :py:data:`True`, an intermediate shell will be spawned which causes variables, glob patterns and other special shell features that may be present in ``args``, to be processed.

.. tip:: When you set ``shell`` to :py:data:`True`, you have to pass ``args`` as a string, not a sequence.

The ``$HOME`` component in the following example will be expanded out by the shell:

>>> import subprocess
>>> retval = subprocess.run(["ls -l $HOME/Desktop"], shell=True, stdout=subprocess.PIPE)
>>> print(retval.stdout.decode())
total 32
-rwxr-xr-x 1 vagrant users  528 Mar 22 03:29 firefox.desktop
-rwxr-xr-x 1 vagrant users  164 Mar 22 03:29 GitEye.desktop
-rwxr-xr-x 1 vagrant users  273 Mar 22 03:29 jedit.desktop
-rwxr-xr-x 1 vagrant users  290 Mar 22 03:29 jetbrains-pycharm-ce.desktop
-rwxr-xr-x 1 vagrant users  233 Mar 22 03:29 Jupyter-Notebook.desktop
drwxr-xr-x 1 vagrant users    0 Mar 22 03:23 notebooks
-rwxr-xr-x 1 vagrant users 3318 Mar 22 03:29 org.kde.konsole.desktop
drwxr-xr-x 1 vagrant users   72 Mar 22 03:29 pythonedu
-rwxr-xr-x 1 vagrant users  255 Mar 22 03:29 PythonEDU-course.desktop
-rwxr-xr-x 1 vagrant users  207 Mar 22 03:29 Update-PythonEDU-Materials.desktop


.. _section_heading-Process_IO:

Process IO
----------

The earlier example demonstrated how we can capture the output from a child process. But you can feed the child process a bytes object when it is spawned using the ``input`` argument. For example:

>>> #
>>> # Call the child process to do some work.
>>> #
>>> import subprocess
>>> retval = subprocess.run(["grep", "Py"], input="0-Zed\n1-PyB\n2-Lam\n3-Pyk".encode(), stdout=subprocess.PIPE)

>>> #
>>> # Get it's output.
>>> #
>>>print(retval.stdout.decode())
1-PyB
3-Pyk


The :py:func:`subprocess.run` command is a convenience function that can handle the most common use cases. For more interactive communication with the spawned process, you must use the underlying :py:func:`subprocess.Popen` function.