Ceph: OSD_TOO_MANY_REPAIRS: Too many repaired reads on 1 OSDs along with "PG_DEGRADED" and "SLOW_OPS".

Solution Verified - Updated -

Issue

The output of ceph health detail shows "OSD_TOO_MANY_REPAIRS": Too many repaired reads on 1 OSDs along with "PG_DEGRADED" and "SLOW_OPS".

Example:

[root@edon-00 ~]# ceph health detail
HEALTH_WARN Too many repaired reads on 1 OSDs; Degraded data redundancy: 1/104988123404 objects degraded (0.000%), 1 pg degraded; 1 slow ops, oldest one blocked for 25234 sec, osd.33 has slow ops
[WRN] OSD_TOO_MANY_REPAIRS: Too many repaired reads on 1 OSDs
    osd.33 had 246285 reads repaired
[WRN] PG_DEGRADED: Degraded data redundancy: 1/104988123404 objects degraded (0.000%), 1 pg degraded
    pg 8.53a is active+recovering+degraded+repair, acting [33,50,14]
[WRN] SLOW_OPS: 1 slow ops, oldest one blocked for 25234 sec, osd.33 has slow ops

Also, this will be seen in the osd log files:

[root@edon-02 ~]# vim ceph-osd.33.log
2023-12-08T12:45:40.749+0000 log_channel(cluster) log [ERR] : 8.53a full-object read crc 0x6b9bf47a != expected 0xffffffff on 8:5ca91bd9:::300017b6c72.00000000:head
2023-12-08T12:45:40.749+0000 log_channel(cluster) log [ERR] : 8.53a missing primary copy of 8:5ca91bd9:::300017b6c72.00000000:head, will try copies on 14,50
2023-12-08T12:45:40.769+0000 osd.33 592819 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.213685203.0:1223450696 8.53a 8:5ca91bd9:::300017b6c72.00000000:head [read 0~16384 [1508@0] out=16384b] snapc 0=[] RETRY=82 ondisk+retry+read+known_if_redirected e592608)

Environment

Red Hat Ceph Storage (RHCS) 5.x
Red Hat Ceph Storage (RHCS) 6.x
Red Hat Ceph Storage (RHCS) 7.x

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content