Chapter 15. Using Python
15.1. Introduction to Python
Python is a high-level programming language that supports multiple programming paradigms, such as object-oriented, imperative, functional, and procedural. Python has dynamic semantics and can be used for general-purpose programming.
With Red Hat Enterprise Linux, many packages that are installed on the system, such as packages providing system tools, tools for data analysis or web applications are written in Python. To be able to use these packages, you need to have the python
packages installed.
15.1.1. Python versions
Two incompatible versions of Python are widely used, Python 2.x and Python 3.x.
RHEL 8 provides the following versions of Python.
Version | Package to install | Command examples | Available since | Life cycle |
---|---|---|---|---|
Python 3.6 |
|
| RHEL 8.0 | full RHEL 8 |
Python 2.7 |
|
| RHEL 8.0 | shorter |
Python 3.8 |
|
| RHEL 8.2 | shorter |
See Red Hat Enterprise Linux Life Cycle and Red Hat Enterprise Linux 8 Application Streams Life Cycle for details about the length of support.
Each of the Python versions is distributed in a separate module, and by design, you can install multiple modules in parallel on the same system.
The python38
module does not include the same bindings to system tools (RPM, DNF, SELinux, and others) that are provided for the python36
module.
Always specify the version of Python when installing it, invoking it, or otherwise interacting with it. For example, use python3
instead of python
in package and command names. All Python-related commands should also include the version, for example, pip3
, pip2
, or pip3.8
.
The unversioned python
command (/usr/bin/python
) is not available by default in RHEL 8. You can configure it using the alternatives
command; for instructions, see Configuring the unversioned Python. Any manual changes to /usr/bin/python
, except changes made using the alternatives
command, may be overwritten upon an update.
As a system administrator, you are recommended to use preferably Python 3 for the following reasons:
- Python 3 represents the main development direction of the Python project.
- Support for Python 2 in the upstream community ends in 2020.
- Popular Python libraries are dropping Python 2 support in upstream.
-
Python 2 in Red Hat Enterprise Linux 8 will have a shorter life cycle and its aim is to facilitate smoother transition to
Python 3
for customers.
For developers, Python 3 has the following advantages over Python 2:
- Python 3 allows writing expressive, maintainable, and correct code more easily.
- Code written in Python 3 will have greater longevity.
- Python 3 has new features, including asyncio, f-strings, advanced unpacking, keyword only arguments, chained exceptions and more.
However, existing software tends to require /usr/bin/python
to be Python 2. For this reason, no default python
package is distributed with Red Hat Enterprise Linux 8, and you can choose between using Python 2 and 3 as /usr/bin/python
, as described in Section 15.2.5, “Configuring the unversioned Python”.
15.1.2. The internal platform-python package
System tools in Red Hat Enterprise Linux 8 use a Python version 3.6 provided by the internal platform-python
package. Red Hat advises customers to use the python36
package instead.
15.2. Installing and using Python
Using the unversioned python
command to install or run Python does not work by default due to ambiguity. Always specify the version of Python, or configure the system default version by using the alternatives
command.
15.2.1. Installing Python 3
In Red Hat Enterprise Linux 8, Python 3 is distributed in versions 3.6 and 3.8, provided by the python36
and python38
modules in the AppStream repository.
Procedure
To install Python 3.6 from the
python36
module, execute the following command:# yum install python3
The python36:3.6 module stream is enabled automatically.
To install Python 3.8 from the
python38
module, use:# yum install python38
The python38:3.8 module stream is enabled automatically.
For details regarding modules in RHEL 8, see Installing, managing, and removing user-space components.
By design, RHEL 8 modules can be installed in parallel, including the python27
, python36
, and python38
modules. Note that parallel installation is not supported for multiple streams within a single module.
Python 3.8 and packages built for it can be installed in parallel with Python 3.6 on the same system, with the exception of the mod_wsgi
module. Due to a limitation of the Apache HTTP Server, only one of the python3-mod_wsgi
and python38-mod_wsgi
packages can be installed on a system.
Packages with add-on modules for Python 3.6 generally use the python3-
prefix; packages for Python 3.8 include the python38-
prefix. Always include the prefix when installing additional Python packages, as shown in the examples below.
Procedure
To install the
Requests
module for Python 3.6, execute this command:# yum install python3-requests
To install the
Cython
extension to Python 3.8, use:# yum install python38-Cython
15.2.1.1. Installing additional Python 3 packages for developers
Additional Python 3.8 packages for developers are distributed through the CodeReady Linux Builder repository in the python38-devel
module. This module contains the python38-pytest
package and its dependencies: the pyparsing
, atomicwrites
, attrs
, packaging
, py
, more-itertools
, pluggy
, and wcwidth
packages.
The CodeReady Linux Builder repository and its content is unsupported by Red Hat.
To install packages from the python38-devel
module, follow the procedure below.
Procedure
Enable the unsupported CodeReady Linux Builder repository:
# subscription-manager repos --enable codeready-builder-for-rhel-8-x86_64-rpms
Enable the
python38-devel
module:# yum module enable python38-devel
Install the
python38-pytest
package:# yum install python38-pytest
For more information about the CodeReady Linux Builder repository, see How to enable and make use of content within CodeReady Linux Builder.
15.2.2. Installing Python 2
Some software has not yet been fully ported to Python 3, and needs Python 2 to operate. Red Hat Enterprise Linux 8 allows parallel installation of Python 3 and Python 2. If you need the Python 2 functionality, install the python27
module, which is available in the AppStream repository.
Note that Python 3 is the main development direction of the Python project. The support for Python 2 is being phased out. The python27
module has a shorter support period than Red Hat Enterprise Linux 8.
Procedure
To install Python 2.7 from the
python27
module, execute this command:# yum install python2
The python27:2.7 module stream is enabled automatically.
By design, RHEL 8 modules can be installed in parallel, including the python27
, python36
, and python38
modules.
For details regarding modules, see Installing, managing, and removing user-space components.
Packages with add-on modules for Python 2 generally use the python2-
prefix. Always include the prefix when installing additional Python packages, as shown in the examples below.
Procedure
To install the
Requests
module for Python 2, execute this command:# yum install python2-requests
To install the
Cython
extension to Python 2, use:# yum install python2-Cython
15.2.3. Using Python 3
When running the Python interpreter or Python-related commands, always specify the version.
Procedure
To run the Python 3.6 interpreter or related commands, use, for example:
$ python3 $ python3 -m cython --help $ pip3 install <package>
To run the Python 3.8 interpreter or related commands, use, for example:
$ python3.8 $ python3.8 -m cython --help $ pip3.8 install <package>
15.2.4. Using Python 2
When running the Python 2 interpreter or Python2-related commands, always specify the version.
Procedure
To run the Python 2 interpreter or related commands, use, for example:
$ python2 $ python2 -m cython --help $ pip2 install <package>
15.2.5. Configuring the unversioned Python
System administrators can configure the unversioned python
command, located at /usr/bin/python
, using the alternatives
command. Note that the required package, python3
, python38
, or python2
, needs to be installed before configuring the unversioned command to the respective version.
The /usr/bin/python
executable is controlled by the alternatives
system. Any manual changes may be overwritten upon an update.
Additional Python-related commands, such as pip3
, do not have configurable unversioned variants.
15.2.5.1. Configuring the unversioned python command directly
To configure the unversioned python
command directly to a selected version of Python, use this procedure.
Procedure
To configure the unversioned
python
command to Python 3.6, execute this command:# alternatives --set python /usr/bin/python3
To configure the unversioned
python
command to Python 3.8, use the following command:# alternatives --set python /usr/bin/python3.8
To configure the unversioned
python
command to Python 2, use:# alternatives --set python /usr/bin/python2
15.2.5.2. Configuring the unversioned python command to the required Python version interactively
You can also configure the unversioned python
command to the required Python version interactively.
To configure the unversioned python
command interactively, use this procedure.
Procedure
Execute the following command:
# alternatives --config python
- Select the required version from the provided list.
To reset this configuration and remove the unversioned
python
command, run:# alternatives --auto python
15.3. Migration from Python 2 to Python 3
As a developer, you may want to migrate your former code that is written in Python 2 to Python 3. For more information on how to migrate large code bases to Python 3, see The Conservative Python 3 Porting Guide.
Note that after this migration, the original Python 2 code becomes interpretable by the Python 3 interpreter and stays interpretable for the Python 2 interpreter as well.
15.4. Packaging of Python 3 RPMs
Most Python projects use Setuptools for packaging, and define package information in the setup.py
file. For more information on Setuptools packaging, see Setuptools documentation.
You can also package your Python project into an RPM package, which provides the following advantages compared to Setuptools packaging:
- Specification of dependencies of a package on other RPMs (even non-Python)
Cryptographic signing
With cryptographic signing, content of RPM packages can be verified, integrated, and tested with the rest of the operating system.
15.4.1. SPEC file description for a Python package
A SPEC file contains instructions that the rpmbuild
utility uses to build an RPM. The instructions are included in a series of sections. A SPEC file has two main parts in which the sections are defined:
- Preamble (contains a series of metadata items that are used in the Body)
- Body (contains the main part of the instructions)
For further information about SPEC files, see Packaging and distributing software.
An RPM SPEC file for Python projects has some specifics compared to non-Python RPM SPEC files. Most notably, a name of any RPM package of a Python library must always include the prefix determining the version, for example, python3
for Python 3.6 or python38
for Python 3.8.
Other specifics are shown in the following SPEC file example for the python3-detox
package. For description of such specifics, see the notes below the example.
%global modname detox 1 Name: python3-detox 2 Version: 0.12 Release: 4%{?dist} Summary: Distributing activities of the tox tool License: MIT URL: https://pypi.io/project/detox Source0: https://pypi.io/packages/source/d/%{modname}/%{modname}-%{version}.tar.gz BuildArch: noarch BuildRequires: python36-devel 3 BuildRequires: python3-setuptools BuildRequires: python36-rpm-macros BuildRequires: python3-six BuildRequires: python3-tox BuildRequires: python3-py BuildRequires: python3-eventlet %?python_enable_dependency_generator 4 %description Detox is the distributed version of the tox python testing tool. It makes efficient use of multiple CPUs by running all possible activities in parallel. Detox has the same options and configuration that tox has, so after installation you can run it in the same way and with the same options that you use for tox. $ detox %prep %autosetup -n %{modname}-%{version} %build %py3_build 5 %install %py3_install %check %{__python3} setup.py test 6 %files -n python3-%{modname} %doc CHANGELOG %license LICENSE %{_bindir}/detox %{python3_sitelib}/%{modname}/ %{python3_sitelib}/%{modname}-%{version}* %changelog ...
- 1
- The modname macro contains the name of the Python project. In this example it is
detox
. - 2
- When packaging a Python project into RPM, the
python3
prefix always needs to be added to the original name of the project. The original name here isdetox
and the name of the RPM ispython3-detox
. - 3
- BuildRequires specifies what packages are required to build and test this package. In BuildRequires, always include items providing tools necessary for building Python packages:
python36-devel
andpython3-setuptools
. Thepython36-rpm-macros
package is required so that files with/usr/bin/python3
shebangs are automatically changed to/usr/bin/python3.6
. For more information, see Section 15.4.4, “Handling hashbangs in Python scripts”. - 4
- Every Python package requires some other packages to work correctly. Such packages need to be specified in the SPEC file as well. To specify the dependencies, you can use the %python_enable_dependency_generator macro to automatically use dependencies defined in the
setup.py
file. If a package has dependencies that are not specified using Setuptools, specify them within additionalRequires
directives. - 5
- The %py3_build and %py3_install macros run the
setup.py build
andsetup.py install
commands, respectively, with additional arguments to specify installation locations, the interpreter to use, and other details. - 6
- The check section provides a macro that runs the correct version of Python. The %{__python3} macro contains a path for the Python 3 interpreter, for example
/usr/bin/python3
. We recommend to always use the macro rather than a literal path.
15.4.2. Common macros for Python 3 RPMs
In a SPEC file, always use the macros below rather than hardcoding their values.
In macro names, always use python3
or python2
instead of unversioned python
. Configure the particular Python 3 version in the BuildRequires
of the SPEC file to either python36-rpm-macros
or python38-rpm-macros
.
Macro | Normal Definition | Description |
---|---|---|
%{__python3} | /usr/bin/python3 | Python 3 interpreter |
%{python3_version} | 3.6 | The full version of the Python 3 interpreter. |
%{python3_sitelib} | /usr/lib/python3.6/site-packages | Where pure-Python modules are installed. |
%{python3_sitearch} | /usr/lib64/python3.6/site-packages | Where modules containing architecture-specific extensions are installed. |
%py3_build |
Runs the | |
%py3_install |
Runs the |
15.4.3. Automatic provides for Python RPMs
When packaging a Python project, make sure that, if present, the following directories are included in the resulting RPM:
-
.dist-info
-
.egg-info
-
.egg-link
From these directories, the RPM build process automatically generates virtual pythonX.Ydist
provides, for example, python3.6dist(detox)
. These virtual provides are used by packages that are specified by the %python_enable_dependency_generator macro.
15.4.4. Handling hashbangs in Python scripts
In Red Hat Enterprise Linux 8, executable Python scripts are expected to use hashbangs (shebangs) specifying explicitly at least the major Python version.
The /usr/lib/rpm/redhat/brp-mangle-shebangs
buildroot policy (BRP) script is run automatically when building any RPM package, and attempts to correct hashbangs in all executable files.
The BRP script generates errors when encountering a Python script with an ambiguous hashbang, such as:
#! /usr/bin/python
or
#! /usr/bin/env python
15.4.4.1. Modifying hashbangs in Python scripts
To modify hashbangs in the Python scripts that cause the build errors at RPM build time, use this procedure.
Procedure
Apply the
pathfix.py
script from theplatform-python-devel
package:# pathfix.py -pn -i %{__python3} PATH …
Note that multiple
PATHs
can be specified. If aPATH
is a directory,pathfix.py
recursively scans for any Python scripts matching the pattern^[a-zA-Z0-9_]+\.py$
, not only those with an ambiguous hashbang. Add this command to the%prep
section or at the end of the%install
section.
Alternatively, modify the packaged Python scripts so that they conform to the expected format. For this purpose, pathfix.py
can be used outside the RPM build process, too. When running pathfix.py
outside a RPM build, replace __python3
from the example above with a path for the hashbang, such as /usr/bin/python3
.
If the packaged Python scripts require other version than Python 3.6, adjust the commands above to include the respective version.
15.4.4.2. Changing /usr/bin/python3 hashbangs in their custom packages
Additionally, hashbangs in the form /usr/bin/python3
are by default replaced with hashbangs pointing to Python from the platform-python
package used for system tools with Red Hat Enterprise Linux.
To change the /usr/bin/python3
hashbangs in their custom packages to point to a version of Python installed from Application Stream, in the form /usr/bin/python3.6
, use the following procedure.
Procedure
Add the
python36-rpm-macros
package into the BuildRequires section of the SPEC file by including the following line:BuildRequires: python36-rpm-macros
To prevent hashbang check and modification by the BRP script, use the following RPM directive:
%undefine %brp_mangle_shebangs
If you are using other version than Python 3.6, adjust the commands above to include the respective version.
15.4.5. Additional resources
- For more information on RPM packaging, see Packaging and distributing software.