OCS / ODF Database Workloads Must Not Use CephFS PVs/PVCs (RDBMSs, NoSQL, PostgreSQL, Mongo DBs, etc.)
Issue
CephFS is intended to be a distributed filesystem to certain workloads. There were many issues noted with customers when running database applications using PVs based in CephFS.
In some scenarios, administrators were also taking snapshots of CephFS PV running databases. This is a corner case that is difficult to hit, but when hit, it can cause severe impact (complete CephFS service outage).
The impact includes damage to the metadata of files used by the database application (only file metadata is damaged, not the data itself). This can cause both Ceph MDS pods to crash when the database application is deleting a file with damaged metadata. In this scenario, Ceph can no longer serve any IO for CephFS volumes, causing loss of data access to CephFSs (at least one MDS (Meta Data Server) pod must be up and running to serve CephFS IO).
Related Articles
Environment
- Red Hat OpenShift Data Foundations (RHODF) Version 4.x
- Red Hat OpenShift Container Storage (RHOCS) Version 4.x
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.
