Ceph/ODF: OSD service crashes "/lib64/libc.so.6(+0x3e6f0) [0x7f8efee496f0]", "thread_name:tp_osd_tp".

Solution Verified - Updated -

Issue

OSD service crashes "/lib64/libc.so.6(+0x3e6f0) [0x7f8efee496f0]", "thread_name:tp_osd_tp".

Example:

2025-02-26T02:59:26.502+0000 7fe85db68640 -1 *** Caught signal (Aborted) **
 in thread 7fe85db68640 thread_name:tp_osd_tp

 ceph version 18.2.1-229.el9cp (ef652b206f2487adfc86613646a4cac946f6b4e0) reef (stable)
 1: /lib64/libc.so.6(+0x3e6f0) [0x7fe88022f6f0]
 2: /lib64/libc.so.6(+0x8b94c) [0x7fe88027c94c]
 3: raise()
 4: abort()
 5: /lib64/libc.so.6(+0x2871b) [0x7fe88021971b]
 6: /lib64/libc.so.6(+0x37386) [0x7fe880228386]
 7: ceph-osd(+0x8fb758) [0x55e17f5c6758]
 8: (Message::encode(unsigned long, int, bool)+0x2e) [0x55e17f8f3e5e]
 9: (ProtocolV2::send_message(Message*)+0xc9) [0x55e17fa82c89]
 10: (AsyncConnection::send_message(Message*)+0x266) [0x55e17fa6f466]
 11: (OSDService::send_message_osd_cluster(int, Message*, unsigned int)+0x129) [0x55e17f298509]
 12: (ReplicatedBackend::issue_op(hobject_t const&, eversion_t const&, unsigned long, osd_reqid_t, eversion_t, eversion_t, hobject_t, hobject_t, std::vector<pg_log_entry_t, std::allocator<pg_log_entry_t> > const&, std::optional<pg_hit_set_history_t>&, ReplicatedBackend::InProgressOp*, ceph::os::Transaction&)+0x70d) [0x55e17f5dd97d]
 13: (ReplicatedBackend::submit_transaction(hobject_t const&, object_stat_sum_t const&, eversion_t const&, std::unique_ptr<PGTransaction, std::default_delete<PGTransaction> >&&, eversion_t const&, eversion_t const&, std::vector<pg_log_entry_t, std::allocator<pg_log_entry_t> >&&, std::optional<pg_hit_set_history_t>&, Context*, unsigned long, osd_reqid_t, boost::intrusive_ptr<OpRequest>)+0x652) [0x55e17f5d78c2]
 14: (PrimaryLogPG::issue_repop(PrimaryLogPG::RepGather*, PrimaryLogPG::OpContext*)+0x37f) [0x55e17f3e16ef]
 15: (PrimaryLogPG::simple_opc_submit(std::unique_ptr<PrimaryLogPG::OpContext, std::default_delete<PrimaryLogPG::OpContext> >)+0x57) [0x55e17f3e65b7]
 16: (PrimaryLogPG::AwaitAsyncWork::react(PrimaryLogPG::DoSnapWork const&)+0x519) [0x55e17f415c19]
 17: ceph-osd(+0x599e31) [0x55e17f264e31]
 18: ceph-osd(+0xf91c1b) [0x55e17fc5cc1b]
 19: (PrimaryLogPG::snap_trimmer(unsigned int)+0xc8) [0x55e17f3b6648]
 20: (ceph::osd::scheduler::PGSnapTrim::run(OSD*, OSDShard*, boost::intrusive_ptr<PG>&, ThreadPool::TPHandle&)+0x1f) [0x55e17f4ea6cf]
 21: (OSD::ShardedOpWQ::_process(unsigned int, ceph::heartbeat_handle_d*)+0xd67) [0x55e17f3044b7]
 22: (ShardedThreadPool::shardedthreadpool_worker(unsigned int)+0x25b) [0x55e17f7c36cb]
 23: ceph-osd(+0xaf8c34) [0x55e17f7c3c34]
 24: /lib64/libc.so.6(+0x89c02) [0x7fe88027ac02]
 25: /lib64/libc.so.6(+0x10ec40) [0x7fe8802ffc40]

Environment

Red Hat OpenShift Container Platform (OCP) 4.x
Red Hat OpenShift Data Foundation (ODF) 4.x
Red Hat Ceph Storage (RHCS) 6.x
Red Hat Ceph Storage (RHCS) 7.x
Red Hat Ceph Storage (RHCS) 8.x
Ceph Object Storage Daemon (OSD)

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content