Chapter 9. Using Python

9.1. Introduction to Python

Python is a high-level programming language that supports multiple programming paradigms, such as object-oriented, imperative, functional, and procedural. Python has dynamic semantics and can be used for general-purpose programming.

With Red Hat Enterprise Linux, many packages that are installed on the system, such as packages providing system tools, tools for data analysis or web applications are written in Python. To be able to use these packages, you need to have the python packages installed.

9.1.1. Python versions

Two incompatible versions of Python are widely used, Python 2.x and Python 3.x.

RHEL 8 provides the following versions of Python.

VersionPackage to installCommand examplesAvailable sinceLife cycle

Python 3.6

python3

python3, pip3

RHEL 8.0

full RHEL 8

Python 2.7

python2

python2, pip2

RHEL 8.0

shorter

Python 3.8

python38

python3.8, pip3.8

RHEL 8.2

shorter

See Red Hat Enterprise Linux Life Cycle and Red Hat Enterprise Linux 8 Application Streams Life Cycle for details about the length of support.

Each of the Python versions is distributed in a separate module, and by design, you can install multiple modules in parallel on the same system.

The python38 module does not include the same bindings to system tools (RPM, DNF, SELinux, and others) that are provided for the python36 module.

Important

Always specify the version of Python when installing it, invoking it, or otherwise interacting with it. For example, use python3 instead of python in package and command names. All Python-related commands should also include the version, for example, pip3, pip2, or pip3.8.

The unversioned python command (/usr/bin/python) is not available by default in RHEL 8. You can configure it using the alternatives command; for instructions, see Configuring the unversioned Python. Any manual changes to /usr/bin/python, except changes made using the alternatives command, may be overwritten upon an update.

As a system administrator, you are recommended to use preferably Python 3 for the following reasons:

  • Python 3 represents the main development direction of the Python project.
  • Support for Python 2 in the upstream community ends in 2020.
  • Popular Python libraries are dropping Python 2 support in upstream.
  • Python 2 in Red Hat Enterprise Linux 8 will have a shorter life cycle and its aim is to facilitate smoother transition to Python 3 for customers.

For developers, Python 3 has the following advantages over Python 2:

  • Python 3 allows writing expressive, maintainable, and correct code more easily.
  • Code written in Python 3 will have greater longevity.
  • Python 3 has new features, including asyncio, f-strings, advanced unpacking, keyword only arguments, chained exceptions and more.

However, existing software tends to require /usr/bin/python to be Python 2. For this reason, no default python package is distributed with Red Hat Enterprise Linux 8, and you can choose between using Python 2 and 3 as /usr/bin/python, as described in Section 9.2.5, “Configuring the unversioned Python”.

9.1.2. The internal platform-python package

System tools in Red Hat Enterprise Linux 8 use a Python version 3.6 provided by the internal platform-python package. Red Hat advises customers to use the python36 package instead.

9.2. Installing and using Python

Warning

Using the unversioned python command to install or run Python does not work by default due to ambiguity. Always specify the version of Python, or configure the system default version by using the alternatives command.

9.2.1. Installing Python 3

In Red Hat Enterprise Linux 8, Python 3 is distributed in versions 3.6 and 3.8, provided by the python36 and python38 modules in the AppStream repository.

Procedure

  • To install Python 3.6 from the python36 module, execute the following command:

    # yum install python3

    The python36:3.6 module stream is enabled automatically.

  • To install Python 3.8 from the python38 module, use:

    # yum install python38

    The python38:3.8 module stream is enabled automatically.

For details regarding modules in RHEL 8, see Installing, managing, and removing user-space components.

Note

By design, RHEL 8 modules can be installed in parallel, including the python27, python36, and python38 modules. Note that parallel installation is not supported for multiple streams within a single module.

Python 3.8 and packages built for it can be installed in parallel with Python 3.6 on the same system, with the exception of the mod_wsgi module. Due to a limitation of the Apache HTTP Server, only one of the python3-mod_wsgi and python38-mod_wsgi packages can be installed on a system.

Packages with add-on modules for Python 3.6 generally use the python3- prefix; packages for Python 3.8 include the python38- prefix. Always include the prefix when installing additional Python packages, as shown in the examples below.

Procedure

  • To install the Requests module for Python 3.6, execute this command:

    # yum install python3-requests
  • To install the Cython extension to Python 3.8, use:

    # yum install python38-Cython

9.2.1.1. Installing additional Python 3 packages for developers

Additional Python 3.8 packages for developers are distributed through the CodeReady Linux Builder repository in the python38-devel module. This module contains the python38-pytest package and its dependencies: the pyparsing, atomicwrites, attrs, packaging, py, more-itertools, pluggy, and wcwidth packages.

Important

The CodeReady Linux Builder repository and its content is unsupported by Red Hat.

To install packages from the python38-devel module, follow the procedure below.

Procedure

  • Enable the unsupported CodeReady Linux Builder repository:

    # subscription-manager repos --enable codeready-builder-for-rhel-8-x86_64-rpms
  • Enable the python38-devel module:

    # yum module enable python38-devel
  • Install the python38-pytest package:

    # yum install python38-pytest

For more information about the CodeReady Linux Builder repository, see How to enable and make use of content within CodeReady Linux Builder.

9.2.2. Installing Python 2

Some software has not yet been fully ported to Python 3, and needs Python 2 to operate. Red Hat Enterprise Linux 8 allows parallel installation of Python 3 and Python 2. If you need the Python 2 functionality, install the python27 module, which is available in the AppStream repository.

Warning

Note that Python 3 is the main development direction of the Python project. The support for Python 2 is being phased out. The python27 module has a shorter support period than Red Hat Enterprise Linux 8.

Procedure

  • To install Python 2.7 from the python27 module, execute this command:

    # yum install python2

    The python27:2.7 module stream is enabled automatically.

Note

By design, RHEL 8 modules can be installed in parallel, including the python27, python36, and python38 modules.

For details regarding modules, see Installing, managing, and removing user-space components.

Packages with add-on modules for Python 2 generally use the python2- prefix. Always include the prefix when installing additional Python packages, as shown in the examples below.

Procedure

  • To install the Requests module for Python 2, execute this command:

    # yum install python2-requests
  • To install the Cython extension to Python 2, use:

    # yum install python2-Cython

9.2.3. Using Python 3

When running the Python interpreter or Python-related commands, always specify the version.

Procedure

  • To run the Python 3.6 interpreter or related commands, use, for example:

    $ python3
    $ python3 -m cython --help
    $ pip3 install <package>
  • To run the Python 3.8 interpreter or related commands, use, for example:

    $ python3.8
    $ python3.8 -m cython --help
    $ pip3.8 install <package>

9.2.4. Using Python 2

When running the Python 2 interpreter or Python2-related commands, always specify the version.

Procedure

  • To run the Python 2 interpreter or related commands, use, for example:

    $ python2
    $ python2 -m cython --help
    $ pip2 install <package>

9.2.5. Configuring the unversioned Python

System administrators can configure the unversioned python command, located at /usr/bin/python, using the alternatives command. Note that the required package, python3, python38, or python2, needs to be installed before configuring the unversioned command to the respective version.

Important

The /usr/bin/python executable is controlled by the alternatives system. Any manual changes may be overwritten upon an update.

Additional Python-related commands, such as pip3, do not have configurable unversioned variants.

9.2.5.1. Configuring the unversioned python command directly

To configure the unversioned python command directly to a selected version of Python, use this procedure.

Procedure

  • To configure the unversioned python command to Python 3.6, execute this command:

    # alternatives --set python /usr/bin/python3
  • To configure the unversioned python command to Python 3.8, use the following command:

    # alternatives --set python /usr/bin/python3.8
  • To configure the unversioned python command to Python 2, use:

    # alternatives --set python /usr/bin/python2

9.2.5.2. Configuring the unversioned python command to the required Python version interactively

You can also configure the unversioned python command to the required Python version interactively.

To configure the unversioned python command interactively, use this procedure.

Procedure

  1. Execute the following command:

    # alternatives --config python
  2. Select the required version from the provided list.
  3. To reset this configuration and remove the unversioned python command, run:

    # alternatives --auto python

9.3. Migration from Python 2 to Python 3

As a developer, you may want to migrate your former code that is written in Python 2 to Python 3. For more information on how to migrate large code bases to Python 3, see The Conservative Python 3 Porting Guide.

Note that after this migration, the original Python 2 code becomes interpretable by the Python 3 interpreter and stays interpretable for the Python 2 interpreter as well.

9.4. Packaging of Python 3 RPMs

Most Python projects use Setuptools for packaging, and define package information in the setup.py file. For more information on Setuptools packaging, see Setuptools documentation.

You can also package your Python project into an RPM package, which provides the following advantages compared to Setuptools packaging:

  • Specification of dependencies of a package on other RPMs (even non-Python)
  • Cryptographic signing

    With cryptographic signing, content of RPM packages can be verified, integrated, and tested with the rest of the operating system.

9.4.1. SPEC file description for a Python package

A SPEC file contains instructions that the rpmbuild utility uses to build an RPM. The instructions are included in a series of sections. A SPEC file has two main parts in which the sections are defined:

  • Preamble (contains a series of metadata items that are used in the Body)
  • Body (contains the main part of the instructions)

For further information about SPEC files, see Packaging and distributing software.

An RPM SPEC file for Python projects has some specifics compared to non-Python RPM SPEC files. Most notably, a name of any RPM package of a Python library must always include the prefix determining the version, for example, python3 for Python 3.6 or python38 for Python 3.8.

Other specifics are shown in the following SPEC file example for the python3-detox package. For description of such specifics, see the notes below the example.

%global modname detox                                                           1

Name:           python3-detox                                                   2
Version:        0.12
Release:        4%{?dist}
Summary:        Distributing activities of the tox tool
License:        MIT
URL:            https://pypi.io/project/detox
Source0:        https://pypi.io/packages/source/d/%{modname}/%{modname}-%{version}.tar.gz

BuildArch:      noarch

BuildRequires:  python36-devel                                                  3
BuildRequires:  python3-setuptools
BuildRequires:  python36-rpm-macros
BuildRequires:  python3-six
BuildRequires:  python3-tox
BuildRequires:  python3-py
BuildRequires:  python3-eventlet

%?python_enable_dependency_generator                                            4

%description

Detox is the distributed version of the tox python testing tool. It makes efficient use of multiple CPUs by running all possible activities in parallel.
Detox has the same options and configuration that tox has, so after installation you can run it in the same way and with the same options that you use for tox.

    $ detox

%prep
%autosetup -n %{modname}-%{version}

%build
%py3_build                                                                      5

%install
%py3_install

%check
%{__python3} setup.py test                                                      6

%files -n python3-%{modname}
%doc CHANGELOG
%license LICENSE
%{_bindir}/detox
%{python3_sitelib}/%{modname}/
%{python3_sitelib}/%{modname}-%{version}*

%changelog
...
1
The modname macro contains the name of the Python project. In this example it is detox.
2
When packaging a Python project into RPM, the python3 prefix always needs to be added to the original name of the project. The original name here is detox and the name of the RPM is python3-detox.
3
BuildRequires specifies what packages are required to build and test this package. In BuildRequires, always include items providing tools necessary for building Python packages: python36-devel and python3-setuptools. The python36-rpm-macros package is required so that files with /usr/bin/python3 shebangs are automatically changed to /usr/bin/python3.6. For more information, see Section 9.4.4, “Handling hashbangs in Python scripts”.
4
Every Python package requires some other packages to work correctly. Such packages need to be specified in the SPEC file as well. To specify the dependencies, you can use the %python_enable_dependency_generator macro to automatically use dependencies defined in the setup.py file. If a package has dependencies that are not specified using Setuptools, specify them within additional Requires directives.
5
The %py3_build and %py3_install macros run the setup.py build and setup.py install commands, respectively, with additional arguments to specify installation locations, the interpreter to use, and other details.
6
The check section provides a macro that runs the correct version of Python. The %{__python3} macro contains a path for the Python 3 interpreter, for example /usr/bin/python3. We recommend to always use the macro rather than a literal path.

9.4.2. Common macros for Python 3 RPMs

In a SPEC file, always use the macros below rather than hardcoding their values.

In macro names, always use python3 or python2 instead of unversioned python. Configure the particular Python 3 version in the BuildRequires of the SPEC file to either python36-rpm-macros or python38-rpm-macros.

MacroNormal DefinitionDescription

%{__python3}

/usr/bin/python3

Python 3 interpreter

%{python3_version}

3.6

The full version of the Python 3 interpreter.

%{python3_sitelib}

/usr/lib/python3.6/site-packages

Where pure-Python modules are installed.

%{python3_sitearch}

/usr/lib64/python3.6/site-packages

Where modules containing architecture-specific extensions are installed.

%py3_build

 

Runs the setup.py build command with arguments suitable for a system package.

%py3_install

 

Runs the setup.py install command with arguments suitable for a system package.

9.4.3. Automatic provides for Python RPMs

When packaging a Python project, make sure that, if present, the following directories are included in the resulting RPM:

  • .dist-info
  • .egg-info
  • .egg-link

From these directories, the RPM build process automatically generates virtual pythonX.Ydist provides, for example, python3.6dist(detox). These virtual provides are used by packages that are specified by the %python_enable_dependency_generator macro.

9.4.4. Handling hashbangs in Python scripts

In Red Hat Enterprise Linux 8, executable Python scripts are expected to use hashbangs (shebangs) specifying explicitly at least the major Python version.

The /usr/lib/rpm/redhat/brp-mangle-shebangs buildroot policy (BRP) script is run automatically when building any RPM package, and attempts to correct hashbangs in all executable files.

Note

The BRP script generates errors when encountering a Python script with an ambiguous hashbang, such as:

#! /usr/bin/python

or

#! /usr/bin/env python

9.4.4.1. Modifying hashbangs in Python scripts

To modify hashbangs in the Python scripts that cause the build errors at RPM build time, use this procedure.

Procedure

  • Apply the pathfix.py script from the platform-python-devel package:

    # pathfix.py -pn -i %{__python3} PATH …​

    Note that multiple PATHs can be specified. If a PATH is a directory, pathfix.py recursively scans for any Python scripts matching the pattern ^[a-zA-Z0-9_]+\.py$, not only those with an ambiguous hashbang. Add this command to the %prep section or at the end of the %install section.

Alternatively, modify the packaged Python scripts so that they conform to the expected format. For this purpose, pathfix.py can be used outside the RPM build process, too. When running pathfix.py outside a RPM build, replace __python3 from the example above with a path for the hashbang, such as /usr/bin/python3.

If the packaged Python scripts require other version than Python 3.6, adjust the commands above to include the respective version.

9.4.4.2. Changing /usr/bin/python3 hashbangs in their custom packages

Additionally, hashbangs in the form /usr/bin/python3 are by default replaced with hashbangs pointing to Python from the platform-python package used for system tools with Red Hat Enterprise Linux.

To change the /usr/bin/python3 hashbangs in their custom packages to point to a version of Python installed from Application Stream, in the form /usr/bin/python3.6, use the following procedure.

Procedure

  • Add the python36-rpm-macros package into the BuildRequires section of the SPEC file by including the following line:

    BuildRequires:  python36-rpm-macros
Note

To prevent hashbang check and modification by the BRP script, use the following RPM directive:

%undefine %brp_mangle_shebangs

If you are using other version than Python 3.6, adjust the commands above to include the respective version.

9.4.5. Additional resources