Ceph/ODF: MDS service crashes with "MDLog.cc: 651: FAILED ceph_assert(segments.size() >= pre_segments_size)"
Issue
Ceph/ODF: MDS service crashes with "MDLog.cc: 651: FAILED ceph_assert(segments.size() >= pre_segments_size)"
Example from ceph crash info:
sh-5.1$ ceph crash info 2024-05-05T05:23:33.483384Z_38907b28-03d5-4805-9080-69c982f4c381
{
"assert_condition": "segments.size() >= pre_segments_size",
"assert_file": "/builddir/build/BUILD/ceph-17.2.6/src/mds/MDLog.cc",
"assert_func": "void MDLog::trim(int)",
"assert_line": 651,
"assert_msg": "/builddir/build/BUILD/ceph-17.2.6/src/mds/MDLog.cc: In function 'void MDLog::trim(int)' thread 7f9d3c208640 time 2024-05-05T05:23:33.482451+0000\n/builddir/build/BUILD/ceph-17.2.6/src/mds/MDLog.cc: 651: FAILED ceph_assert(segments.size() >= pre_segments_size)\n",
"assert_thread_name": "safe_timer",
"backtrace": [
"/lib64/libc.so.6(+0x54db0) [0x7f9d428a7db0]",
"/lib64/libc.so.6(+0xa154c) [0x7f9d428f454c]",
"raise()",
"abort()",
"(ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f9d42f03ae1]",
"/usr/lib64/ceph/libceph-common.so.2(+0x142c45) [0x7f9d42f03c45]",
"(MDLog::trim(int)+0xb06) [0x5611463c02a6]",
"(MDSRankDispatcher::tick()+0x365) [0x561146142c65]",
"ceph-mds(+0x11c71d) [0x56114611571d]",
"(CommonSafeTimer<ceph::fair_mutex>::timer_thread()+0x15e) [0x7f9d42fed4ae]",
"/usr/lib64/ceph/libceph-common.so.2(+0x22cda1) [0x7f9d42fedda1]",
"/lib64/libc.so.6(+0x9f802) [0x7f9d428f2802]",
"/lib64/libc.so.6(+0x3f450) [0x7f9d42892450]"
],
"ceph_version": "17.2.6-170.el9cp",
"crash_id": "2024-05-05T05:23:33.483384Z_38907b28-03d5-4805-9080-69c982f4c381",
"entity_name": "mds.ocs-storagecluster-cephfilesystem-b",
"os_id": "rhel",
"os_name": "Red Hat Enterprise Linux",
"os_version": "9.3 (Plow)",
"os_version_id": "9.3",
"process_name": "ceph-mds",
"stack_sig": "21cf82abf00a9a80ef194472005415a53e94d6965c4e910d756a9f711243f498",
"timestamp": "2024-05-05T05:23:33.483384Z",
"utsname_hostname": "rook-ceph-mds-ocs-storagecluster-cephfilesystem-b-xxxxxxyyyyyy",
"utsname_machine": "x86_64",
"utsname_release": "5.14.0-284.52.1.el9_2.x86_64",
"utsname_sysname": "Linux",
"utsname_version": "#1 SMP PREEMPT_DYNAMIC Tue Jan 30 08:35:38 EST 2024"
Environment
Red Hat OpenShift Container Platform (OCP) 4.x
Red Hat OpenShift Container Storage (OCS) 4.x
Red Hat OpenShift Data Foundation (ODF) 4.x
Red Hat Ceph Storage (RHCS) 5.x
Red Hat Ceph Storage (RHCS) 6.x
Red Hat Ceph Storage (RHCS) 7.x
Ceph File System (CephFS)
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.