FIPS: custom python 3.9 application loading pymssql package crashes with "OpenSSL internal error, assertion failed: FATAL FIPS SELFTEST FAILURE"

Solution Verified - Updated -

Environment

  • Red Hat Enterprise Linux 8
    • Python 3.9
    • 3rd party modules

Issue

  • When executing my custom Python 3.9 application using pymssql module (3rd party), the following error is displayed on the standard error and Python dumps a core

    fips.c(145): OpenSSL internal error, assertion failed: FATAL FIPS SELFTEST FAILURE
Aborted (core dumped)

  • My Python 3.9 application using 3rd party Python modules installed through pip crashes similarly

Resolution

Follow the procedure described in the Diagnostic Steps section.
If this is a match, proceed further, otherwise open a case on the Customer Portal referencing this solution.

For the failing modules only (e.g. pymssql in the solution below), instead of relying on pre-built binaries, install the module from the source, as shown in the procedure below.

  1. Install pre-requisites

    $ sudo yum -y install gcc python3-devel

  2. Check on the 3rd party project website what are the other pre-requisites to compile the project

    In the case of pymssql, the additional packages below are required (freetds-devel is found in EPEL):

    $ sudo yum -y install freetds-devel openssl-devel krb5-devel

    Important Note: the project being 3rd party, we may not be able to always help, you may have to find the information by yourself.

  3. Execute the pip command to build from the sources

    $ pip-3.9 install --ignore-installed --no-binary :all: pymssql
[...]
Installing collected packages: pymssql
Successfully installed pymssql-2.2.8

    If the command fails, this probably means the project has some missing dependencies, you then have to find the information from the pip error being displayed.

  4. Verify the module can now load properly

    $ python3.9
[...]
>>> import pymssql
>>>

    Since there was no crash, we know the module is now usable.

Root Cause

pymssql module delivered through pip uses a deprecated openssl library internally, which crashes on recent RHEL8 systems due to not satisfying to FIPS self test specifications.

Diagnostic Steps

  1. Execute python3.9 in the terminal

    $ python3.9
[...]
>>>

  2. At the interactive prompt, try loading the help for all installed modules

    >>> help("modules")
fips.c(145): OpenSSL internal error, assertion failed: FATAL FIPS SELFTEST FAILURE
Aborted (core dumped)

    If module loading crashes as shown above, this means some 3rd party module is not usable with RHEL8+FIPS.

  3. List all 3rd party modules installed on the system

    $ pip-3.9 list
Package    Version
---------- -------
cffi          1.15.1
pip           20.2.4
pycparser     2.21
pymssql       2.2.8
python-augeas 1.1.0
setuptools    50.3.2

    In the example above we have a few modules, including cffi, pycparser, pymssql and python-augeas.

  4. For each 3rd party module installed on the system, try importing the module one by one to find the culprit

    $ python3.9
[...]
>>> import cffi
>>> import pycparser
>>> import pymssql
fips.c(145): OpenSSL internal error, assertion failed: FATAL FIPS SELFTEST FAILURE
Aborted (core dumped)

    In the example above, 2 modules (cffi and pycparser) could be imported successfully, then pymssql made Python crash, the latter is hence the culprit.

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments