EL9 issues with mariadb connectors
This has been reproduced on:
- RockyLinux 9
- AlmaLinux 9
- RHEL 9
Initially I found the issue when installing the frappe framework on a RL9 machine. From there I have been able to easily reproduce the issue in several environments.
Frappe uses PyMySQL lib, and on all the tests I did, the only lib that actually worked was MySQL. I'll get to that ahead.
So I started with a minimum install base system (always), then added dnf groupinstall 'Development Tools'
I initially compiled Python3.11 from sources.
Then used Python3.11 supplied by @appstream
I reproduce these results either from sources or from repo.
I did an additional test using system supplied Python3.9 and the results are also reproducible. So it is not related to Python being 3.9 or 3.11.
So I added python3.11, python3.11-pip, python3.11-devel
After I installed the MariaDB-Connector-C and -devel from the repos.
Running pip3.11 install mariadb fails with:
# pip3.11 install mariadb
Collecting mariadb
Using cached mariadb-1.1.10.tar.gz (84 kB)
Installing build dependencies ... done
Getting requirements to build wheel ... error
error: subprocess-exited-with-error
× Getting requirements to build wheel did not run successfully.
│ exit code: 2
╰─> [1 lines of output]
MariaDB Connector/Python requires MariaDB Connector/C >= 3.3.1, found version 3.2.6
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error
× Getting requirements to build wheel did not run successfully.
│ exit code: 2
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.
So, since the only connector provided is 3.2.6, I tried to install a lower version of MariaDB on Python:
# pip3.11 install mariadb==1.0.11
Collecting mariadb==1.0.11
Using cached mariadb-1.0.11.zip (85 kB)
Preparing metadata (setup.py) ... done
Installing collected packages: mariadb
DEPRECATION: mariadb is being installed using the legacy 'setup.py install' method, because it does not have a 'pyproject.toml' and the 'wheel' package is not installed. pip 23.1 will enforce this behaviour change. A possible replacement is to enable the '--use-pep517' option. Discussion can be found at https://github.com/pypa/pip/issues/8559
Running setup.py install for mariadb ... error
error: subprocess-exited-with-error
× Running setup.py install for mariadb did not run successfully.
│ exit code: 1
╰─> [40 lines of output]
running install
/usr/lib/python3.11/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
warnings.warn(
running build
running build_py
creating build
creating build/lib.linux-x86_64-cpython-311
creating build/lib.linux-x86_64-cpython-311/mariadb
copying mariadb/__init__.py -> build/lib.linux-x86_64-cpython-311/mariadb
creating build/lib.linux-x86_64-cpython-311/mariadb/constants
copying mariadb/constants/__init__.py -> build/lib.linux-x86_64-cpython-311/mariadb/constants
copying mariadb/constants/CLIENT.py -> build/lib.linux-x86_64-cpython-311/mariadb/constants
copying mariadb/constants/INDICATOR.py -> build/lib.linux-x86_64-cpython-311/mariadb/constants
copying mariadb/constants/CURSOR.py -> build/lib.linux-x86_64-cpython-311/mariadb/constants
copying mariadb/constants/FIELD_TYPE.py -> build/lib.linux-x86_64-cpython-311/mariadb/constants
running build_ext
building 'mariadb._mariadb' extension
creating build/temp.linux-x86_64-cpython-311
creating build/temp.linux-x86_64-cpython-311/mariadb
gcc -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -march=x86-64-v2 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -march=x86-64-v2 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -march=x86-64-v2 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -fPIC -DPY_MARIADB_MAJOR_VERSION=1 -DPY_MARIADB_MINOR_VERSION=0 -DPY_MARIADB_PATCH_VERSION=11 -I/usr/include/mysql -I/usr/include/mysql/mysql -I./include -I/usr/include/python3.11 -c mariadb/mariadb.c -o build/temp.linux-x86_64-cpython-311/mariadb/mariadb.o -DDEFAULT_PLUGINS_SUBDIR=\"/usr/lib64/mariadb/plugin\"
mariadb/mariadb.c: In function ‘PyInit__mariadb’:
mariadb/mariadb.c:155:35: error: lvalue required as left operand of assignment
155 | Py_TYPE(&MrdbConnection_Type) = &PyType_Type;
| ^
mariadb/mariadb.c:168:31: error: lvalue required as left operand of assignment
168 | Py_TYPE(&MrdbCursor_Type) = &PyType_Type;
| ^
mariadb/mariadb.c:174:29: error: lvalue required as left operand of assignment
174 | Py_TYPE(&MrdbPool_Type) = &PyType_Type;
| ^
mariadb/mariadb.c:180:34: error: lvalue required as left operand of assignment
180 | Py_TYPE(&MrdbIndicator_Type) = &PyType_Type;
| ^
mariadb/mariadb.c:186:38: error: lvalue required as left operand of assignment
186 | Py_TYPE(&Mariadb_Fieldinfo_Type) = &PyType_Type;
| ^
mariadb/mariadb.c:192:38: error: lvalue required as left operand of assignment
192 | Py_TYPE(&Mariadb_DBAPIType_Type) = &PyType_Type;
| ^
error: command '/usr/bin/gcc' failed with exit code 1
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure
× Encountered error while trying to install package.
╰─> mariadb
note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.
I tried all versions down to mariadb==1.0.0
The result is exactly the same, no matter what EL9 distro I'm using.
So since mariadb-connector-c-3.2.6 doesn't make it, I went to mariadb-connector-c-3.3.10 from MariaDB website, which only provides a tar.gz with stuff pre-compiled? weird.
Anyway got the package from GitHub:
MariaDB Corporation - MariaDB Connector C - Archive - Refs - Tags - v3.3.10
to build:
cd mariadb-connector-c
cmake -S . -B build
cd build && make
make install
And adding MariaDB to python:
# pip3.11 install mariadb
Collecting mariadb
Using cached mariadb-1.1.10.tar.gz (84 kB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Collecting packaging
Using cached packaging-24.1-py3-none-any.whl (53 kB)
Building wheels for collected packages: mariadb
Building wheel for mariadb (pyproject.toml) ... done
Created wheel for mariadb: filename=mariadb-1.1.10-cp311-cp311-linux_x86_64.whl size=193885 sha256=d5873661130b26edc8f5a596d8a6e4f33a692cf10141e8246f1480ace82ac088
Stored in directory: /root/.cache/pip/wheels/ad/96/e6/aa53808d09799f9d1c4f53eb127936c98d8a6bf971af53fe86
Successfully built mariadb
Installing collected packages: packaging, mariadb
Successfully installed mariadb-1.1.10 packaging-24.1
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
All good until now. So now let's test our script:
# python3.11 mariadb-db-test.py
Traceback (most recent call last):
File "/root/mariadb-db-test.py", line 2, in <module>
import mariadb
File "/usr/local/lib64/python3.11/site-packages/mariadb/__init__.py", line 7, in <module>
from ._mariadb import (
ImportError: libmariadb.so.3: cannot open shared object file: No such file or directory
To correct this, add a file to/etc/ld.so.conf.d like mariadb-connector-c-libs.conf with content:
/usr/local/lib/mariadb
And running the Python Script:
# python3.11 mariadb-db-test.py
Traceback (most recent call last):
File "/root/mariadb-db-test.py", line 29, in <module>
display_user()
File "/root/mariadb-db-test.py", line 26, in display_user
result = execute_query(query)
^^^^^^^^^^^^^^^^^^^^
File "/root/mariadb-db-test.py", line 10, in execute_query
conn = mariadb.connect(
^^^^^^^^^^^^^^^^
File "/usr/local/lib64/python3.11/site-packages/mariadb/__init__.py", line 146, in connect
connection = connectionclass(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib64/python3.11/site-packages/mariadb/connections.py", line 85, in __init__
super().__init__(*args, **kwargs)
mariadb.OperationalError: Access denied for user 'rh-test'@'10.0.0.4' (using password: YES)
So I install the package mysql using dnf which installs mysql client and shared libs.
Then:
# pip3.11 install mysql-connector-python
Collecting mysql-connector-python
Downloading mysql_connector_python-9.0.0-cp311-cp311-manylinux_2_17_x86_64.whl (19.3 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 19.3/19.3 MB 30.4 MB/s eta 0:00:00
Installing collected packages: mysql-connector-python
Successfully installed mysql-connector-python-9.0.0
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
[root@rh-test ~]# python3.11 mysql-db-test.py
[('rh-user@10.0.0.4',)]
So the exact same script instead of using mariadb I'm using mysql.connector. Result:
[root@rh-test ~]# python3.11 mysql-db-test.py
[('rh-user@10.0.0.4',)]
Works.
mariadb-db-test.py uses mariadb
mysql-db-test.py uses mysql.connector
Using PyMySQL fails the same as if using mariadb. The error is exactly the same.
I can reproduce this in any of the EL9 environments tested. Rocky, Alma and RHEL.
