Chapter 7. Using Python in Red Hat Enterprise Linux 8

7.1. Introduction to Python

Python is a high-level programming language that supports multiple programming paradigms, such as object-oriented, imperative, functional, and procedural. Python has dynamic semantics and can be used for general-purpose programming.

With Red Hat Enterprise Linux, many packages that are installed on the system, such as packages providing system tools, tools for data analysis or web applications are written in Python. To be able to use these packages, you need to have the python packages installed.

7.1.1. Python versions

Two incompatible versions of Python are widely used, Python 2.x and Python 3.x.

Red Hat Enterprise Linux 8 uses Python 3.6 by default. However, Python 2.7 is also provided to support existing software.

Warning

Neither the default python package nor the unversioned /usr/bin/python executable is distributed with Red Hat Enterprise Linux 8.

Important

Always specify the major version of Python when installing it, invoking it, or otherwise interacting with it. For example, use python3, instead of python, in package and command names. All Python-related commands should also include the version, for example, pip3 or pip2.

Alternatively, configure the system default version by using the alternatives command as described in Configuring the unversioned Python.

As a system administrator, you are recommended to use preferably Python 3 for the following reasons:

Python 3 represents the main development direction of the Python project.
Support for Python 2 in the upstream community ends in 2020.
Popular Python libraries are dropping Python 2 support in upstream.
Python 2 in Red Hat Enterprise Linux 8 will have a shorter life cycle and its aim is to facilitate smoother transition to Python 3 for customers.

For developers, Python 3 has the following advantages over Python 2:

Python 3 allows writing expressive, maintainable, and correct code more easily.
Code written in Python 3 will have greater longevity.
Python 3 has new features, including asyncio, f-strings, advanced unpacking, keyword only arguments, chained exceptions and more.

However, existing software tends to require /usr/bin/python to be Python 2. For this reason, no default python package is distributed with Red Hat Enterprise Linux 8, and you can choose between using Python 2 and 3 as /usr/bin/python, as described in Section 7.2.5, “Configuring the unversioned Python”.

7.1.2. The internal platform-python package

System tools in Red Hat Enterprise Linux 8 use a Python version 3.6 provided by the internal platform-python package. Red Hat advises customers to use the python36 package instead.

7.2. Installing and using Python

Warning

Using the unversioned python command to install or run Python does not work by default due to ambiguity. Always specify the major version of Python, or configure the system default version by using the alternatives command.

7.2.1. Installing Python 3

In Red Hat Enterprise Linux 8, Python 3 is distributed as the python36 module in the AppStream repository.

To install Python 3.6 from the python36 module in AppStream, use the following procedure.

Procedure

To install Python 3, execute the following command:
```
# yum install python3
```

For details regarding modules, see Installing, managing, and removing user space components.

7.2.2. Installing Python 2

Some software has not yet been fully ported to Python 3, and needs Python 2 to operate. Red Hat Enterprise Linux 8 allows parallel installation of Python 3 and Python 2. If you need the Python 2 functionality, install the python27 module, which is available in the AppStream repository.

Warning

Note that Python 3 is the main development direction of the Python project. The support for Python 2 is being phased out. The python27 module has a shorter support period than Red Hat Enterprise Linux 8.

To install Python 2.7 from the python27 module in AppStream, use the following procedure.

Procedure

To install Python 2, execute the following command:
```
# yum install python2
```

For details regarding modules, see Installing, managing, and removing user space components.

7.2.3. Using Python 3

To run Python 3, always use the python3 command. Use the version also in all other related commands, such as pip3.

Packages with add-on modules for Python 3 generally use the python3- prefix.

For example, to install the Requests module that is used for writing HTTP clients, use this procedure.

Procedure

Execute the following command:
```
# yum install python3-requests
```

7.2.4. Using Python 2

To run Python 2, always use the python2 command. Use the version also in all other related commands, such as pip2.

Packages with add-on modules for Python 2 generally use the python2- prefix.

For example, to install the Requests module that is used for writing HTTP clients, use the following procedure.

Procedure

Execute the following command:
```
yum install python2-requests
```

7.2.5. Configuring the unversioned Python

System administrators can configure the unversioned python command on the system using the alternatives command. Note that the required package, either python3 or python2, needs to be installed before configuring the unversioned command to the respective version.

7.2.5.1. Configuring the unversioned python command to Python 3 directly

To configure the unversioned python command to Python 3 directly, use this procedure.

Procedure

Execute the following command:

# alternatives --set python /usr/bin/python3

7.2.5.2. Configuring the unversioned python command to Python 2 directly

To configure the unversioned python command to Python 2 directly, use this procedure.

Procedure

Execute the following command:

# alternatives --set python /usr/bin/python2

7.2.5.3. Configuring the unversioned python command to the required Python version interactively

You can also configure the unversioned python command to the required Python version interactively.

To configure the unversioned python command interactively, use this procedure.

Procedure

Execute the following command:
```
# alternatives --config python
```
Select the required version from the provided list.
To reset this configuration and remove the unversioned python command, run:
```
# alternatives --auto python
```

Warning

Additional Python-related commands, such as pip3, do not have configurable unversioned variants.

7.3. Migration from Python 2 to Python 3

As a developer, you may want to migrate your former code that is written in Python 2 to Python 3. For more information on how to migrate large code bases to Python 3, see The Conservative Python 3 Porting Guide.

Note that after this migration, the original Python 2 code becomes interpretable by the Python 3 interpreter and stays interpretable for the Python 2 interpreter as well.

7.4. Packaging of Python 3 RPMs

Most Python projects use Setuptools for packaging, and define package information in the setup.py file. For more information on Setuptools packaging, see Setuptools documentation.

You can also package your Python project into an RPM package, which provides the following advantages compared to Setuptools packaging:

Specification of dependencies of a package on other RPMs (even non-Python)
Cryptographic signing
With cryptographic signing, content of RPM packages can be verified, integrated, and tested with the rest of the operating system.

7.4.1. SPEC file description for a Python package

A SPEC file contains instructions that the rpmbuild utility uses to build an RPM. The instructions are included in a series of sections. A SPEC file has two main parts in which the sections are defined:

Preamble (contains a series of metadata items that are used in the Body)
Body (contains the main part of the instructions)

For further information about SPEC files, see Packaging and distributing software.

An RPM SPEC file for Python projects has some specifics compared to non-Python RPM SPEC files. Most notably, a name of any RPM package of a Python library must always include the python3 prefix.

Other specifics are shown in the following SPEC file example for the python3-detox package. For description of such specifics, see the notes below the example.

%global modname detox                                                           1

Name:           python3-detox                                                   2
Version:        0.12
Release:        4%{?dist}
Summary:        Distributing activities of the tox tool
License:        MIT
URL:            https://pypi.io/project/detox
Source0:        https://pypi.io/packages/source/d/%{modname}/%{modname}-%{version}.tar.gz

BuildArch:      noarch

BuildRequires:  python36-devel                                                  3
BuildRequires:  python3-setuptools
BuildRequires:  python36-rpm-macros
BuildRequires:  python3-six
BuildRequires:  python3-tox
BuildRequires:  python3-py
BuildRequires:  python3-eventlet

%?python_enable_dependency_generator                                            4

%description

Detox is the distributed version of the tox python testing tool. It makes efficient use of multiple CPUs by running all possible activities in parallel.
Detox has the same options and configuration that tox has, so after installation you can run it in the same way and with the same options that you use for tox.

    $ detox

%prep
%autosetup -n %{modname}-%{version}

%build
%py3_build                                                                      5

%install
%py3_install

%check
%{__python3} setup.py test                                                      6

%files -n python3-%{modname}
%doc CHANGELOG
%license LICENSE
%{_bindir}/detox
%{python3_sitelib}/%{modname}/
%{python3_sitelib}/%{modname}-%{version}*

%changelog
...

1: The modname macro contains the name of the Python project. In this example it is detox.
2: When packaging a Python project into RPM, the python3 prefix always needs to be added to the original name of the project. The original name here is detox and the name of the RPM is python3-detox.
3: BuildRequires specifies what packages are required to build and test this package. In BuildRequires, always include items providing tools necessary for building Python packages: python36-devel and python3-setuptools. The python36-rpm-macros package is required so that files with /usr/bin/python3 shebangs are automatically changed to /usr/bin/python3.6. For more information, see Section 7.4.4, “Handling hashbangs in Python scripts”.
4: Every Python package requires some other packages to work correctly. Such packages need to be specified in the SPEC file as well. To specify the dependencies, you can use the %python_enable_dependency_generator macro to automatically use dependencies defined in the setup.py file. If a package has dependencies that are not specified using Setuptools, specify them within additional Requires directives.
5: The %py3_build and %py3_install macros run the setup.py build and setup.py install commands, respectively, with additional arguments to specify installation locations, the interpreter to use, and other details.
6: The check section provides a macro that runs the correct version of Python. The %{__python3} macro contains a path for the Python 3 interpreter, for example /usr/bin/python3. We recommend to always use the macro rather than a literal path.

7.4.2. Common macros for Python 3 RPMs

In a SPEC file, always use the macros below rather than hardcoding their values.

In macro names, always use python3 or python2 instead of unversioned python.

Macro	Normal Definition	Description
%{__python3}	/usr/bin/python3	Python 3 interpreter
%{python3_version}	3.6	The full version of the Python 3 interpreter.
%{python3_sitelib}	/usr/lib/python3.6/site-packages	Where pure-Python modules are installed.
%{python3_sitearch}	/usr/lib64/python3.6/site-packages	Where modules containing architecture-specific extensions are installed.
%py3_build		Runs the `setup.py build` command with arguments suitable for a system package.
%py3_install		Runs the `setup.py install` command with arguments suitable for a system package.

7.4.3. Automatic provides for Python RPMs

When packaging a Python project, make sure that, if present, the following directories are included in the resulting RPM:

.dist-info
.egg-info
.egg-link

From these directories, the RPM build process automatically generates virtual pythonX.Ydist provides, for example python3.6dist(detox). These virtual provides are used by packages that are specified by the %python_enable_dependency_generator macro.

7.4.4. Handling hashbangs in Python scripts

In Red Hat Enterprise Linux 8, executable Python scripts are expected to use hashbangs (shebangs) specifying explicitly at least the major Python version.

The /usr/lib/rpm/redhat/brp-mangle-shebangs buildroot policy (BRP) script is run automatically when building any RPM package, and attempts to correct hashbangs in all executable files.

Note

The BRP script generates errors when encountering a Python script with an ambiguous hashbang, such as:

#! /usr/bin/python

#! /usr/bin/env python

7.4.4.1. Modifying hashbangs in Python scripts

To modify hashbangs in the Python scripts that cause the build errors at RPM build time, use this procedure.

Procedure

Apply the pathfix.py script from the platform-python-devel package:
```
# pathfix.py -pn -i %{__python3} PATH …
```
Note that multiple PATHs can be specified. If a PATH is a directory, pathfix.py recursively scans for any Python scripts matching the pattern ^[a-zA-Z0-9_]+\.py$, not only those with an ambiguous hashbang. Add this command to the %prep section or at the end of the %install section.

Alternatively, modify the packaged Python scripts so that they conform to the expected format. For this purpose, pathfix.py can be used outside the RPM build process, too. When running pathfix.py outside a RPM build, replace __python3 from the example above with a path for the hashbang, such as /usr/bin/python3.

If the packaged Python scripts require Python version 2, replace the number 3 with 2 in the commands above.

7.4.4.2. Changing /usr/bin/python3 hashbangs in their custom packages

Additionally, hashbangs in the form /usr/bin/python3 are by default replaced with hashbangs pointing to Python from the platform-python package used for system tools with Red Hat Enterprise Linux.

To change the /usr/bin/python3 hashbangs in their custom packages to point to a version of Python installed from Application Stream, in the form /usr/bin/python3.6, ue the following procedure.

Procedure

Add the python36-rpm-macros package into the BuildRequires section of the SPEC file by including the following line:
```
BuildRequires:  python36-rpm-macros
```

Note

To prevent hashbang check and modification by the BRP script, use the following RPM directive:

%undefine %brp_mangle_shebangs

7.4.5. Additional resources

For more information on RPM packaging, see Packaging and distributing software.

Where did the comment section go?

Red Hat's documentation publication system recently went through an upgrade to enable speedier, more mobile-friendly content. We decided to re-evaluate our commenting platform to ensure that it meets your expectations and serves as an optimal feedback mechanism. During this redesign, we invite your input on providing feedback on Red Hat documentation via the discussion platform.