Ceph - OSD failed with error "hit suicide timeout"
Issue
- OSD failed with error "hit suicide timeout"
2017-05-16 12:31:34.187444 7f894122a700 1 heartbeat_map is_healthy 'OSD::osd_op_tp thread 0x7f8929214700' had timed out after 15
2017-05-16 12:31:39.187602 7f894122a700 1 heartbeat_map is_healthy 'OSD::osd_op_tp thread 0x7f8929214700' had timed out after 15
2017-05-16 12:31:44.187753 7f894122a700 1 heartbeat_map is_healthy 'OSD::osd_op_tp thread 0x7f8929214700' had timed out after 15
2017-05-16 12:31:49.187927 7f894122a700 1 heartbeat_map is_healthy 'OSD::osd_op_tp thread 0x7f8929214700' had timed out after 15
2017-05-16 12:31:54.188060 7f894122a700 1 heartbeat_map is_healthy 'OSD::osd_op_tp thread 0x7f8929214700' had timed out after 15
2017-05-16 12:31:59.188214 7f894122a700 1 heartbeat_map is_healthy 'OSD::osd_op_tp thread 0x7f8929214700' had timed out after 15
2017-05-16 12:32:04.188372 7f894122a700 1 heartbeat_map is_healthy 'OSD::osd_op_tp thread 0x7f8929214700' had timed out after 15
2017-05-16 12:32:09.188535 7f894122a700 1 heartbeat_map is_healthy 'OSD::osd_op_tp thread 0x7f8929214700' had timed out after 15
2017-05-16 12:32:14.188688 7f894122a700 1 heartbeat_map is_healthy 'OSD::osd_op_tp thread 0x7f8929214700' had timed out after 15
2017-05-16 12:32:14.188700 7f894122a700 1 heartbeat_map is_healthy 'OSD::osd_op_tp thread 0x7f8929214700' had suicide timed out after 150
2017-05-16 12:32:14.592404 7f894122a700 -1 common/HeartbeatMap.cc: In function 'bool ceph::HeartbeatMap::_check(ceph::heartbeat_handle_d*, const char*, time_t)' thread 7f894122a700 time 2017-05-16 12:32:14.188711
common/HeartbeatMap.cc: 79: FAILED assert(0 == "hit suicide timeout")
ceph version 0.94.5-14.el7cp (ff6967ce0543fb0b60fe23f10da7b4a35bf046a0)
1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x85) [0xb08f35]
2: (ceph::HeartbeatMap::_check(ceph::heartbeat_handle_d*, char const*, long)+0x2d9) [0xa3ca49]
3: (ceph::HeartbeatMap::is_healthy()+0xde) [0xa3d33e]
4: (ceph::HeartbeatMap::check_touch_file()+0x2c) [0xa3da5c]
5: (CephContextServiceThread::entry()+0x15b) [0xb1934b]
6: (()+0x7dc5) [0x7f8944a34dc5]
7: (clone()+0x6d) [0x7f89435151cd]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
Environment
- Red Hat Ceph Storage 1.3
- Red Hat Ceph Storage 2.x
- Red Hat Ceph Storage 3.x
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.