After migrating Ceph from Ubuntu to Red Hat Enterprise Linux ceph-osd daemons crash on service start with 'Error initializing leveldb : Corruption: XX missing files'

Issue

• When starting ceph-osd daemons after migrating from Ubuntu to RHEL 7.3, the ceph-osd daemons crash with:

2017-02-28 10:42:23.615868 7f16719ef880  0 ceph version 0.87.2 (87a7cec9ab11c677de2ab23a7668a77d2f5b955e), process ceph-osd, pid 12992
2017-02-28 10:42:23.670375 7f16719ef880  0 filestore(/var/lib/ceph/osd/ceph-25) backend xfs (magic 0x58465342)
2017-02-28 10:42:23.670393 7f16719ef880  1 filestore(/var/lib/ceph/osd/ceph-25)  disabling 'filestore replica fadvise' due to known issues with fadvise(DONTNEED) on xfs
2017-02-28 10:42:23.770667 7f16719ef880  0 genericfilestorebackend(/var/lib/ceph/osd/ceph-25) detect_features: FIEMAP ioctl is supported and appears to work
2017-02-28 10:42:23.770677 7f16719ef880  0 genericfilestorebackend(/var/lib/ceph/osd/ceph-25) detect_features: FIEMAP ioctl is disabled via 'filestore fiemap' config option
2017-02-28 10:42:23.795473 7f16719ef880  0 genericfilestorebackend(/var/lib/ceph/osd/ceph-25) detect_features: syncfs(2) syscall fully supported (by glibc and kernel)
2017-02-28 10:42:23.795586 7f16719ef880  0 xfsfilestorebackend(/var/lib/ceph/osd/ceph-25) detect_feature: extsize is disabled by conf
2017-02-28 10:42:25.001055 7f16719ef880 -1 filestore(/var/lib/ceph/osd/ceph-25) Error initializing leveldb : Corruption: 49 missing files; e.g.: /var/lib/ceph/osd/ceph-25/current/omap/385569.sst
2017-02-28 10:42:25.001100 7f16719ef880 -1  ** ERROR: error converting store /var/lib/ceph/osd/ceph-25: (1) Operation not permitted