Ceph - protected snapshot deletion attempt after HEAD flattening can lead to stale child reference
Issue
After flattening of some images and deleting the parent snapshots, attempting to delete a protected snapshot led to a situation where we can't unprotect nor delete it:
# rbd snap unprotect virtual_machines/8adfa17c-e730-4f68-85fa-6f1b01c84331_disk_to_be_deleted_by_glance@9fb1737dce0241249f7bcfb83bd51502_to_be_deleted_by_glance
2016-11-29 16:17:20.653509 8f5b29b437c0 -1 librbd: snap_unprotect: can't unprotect; at least 1 child(ren) in pool virtual_machines
#
# rbd snap rm virtual_machines/8adfa17c-e730-4f68-85fa-6f1b01c84331_disk_to_be_deleted_by_glance@9fb1737dce0241249f7bcfb83bd51502_to_be_deleted_by_glance
rbd: snapshot '9fb1737dce0241249f7bcfb83bd51502_to_be_deleted_by_glance' is protected from removal.
2016-11-28 13:24:01.107502 8fdeba89e7c0 -1 librbd: removing snapshot from header failed: (16) Device or resource busy
Printing the children will refer to a non-existing parent snapshot:
# rbd children virtual_machines/8adfa17c-e730-4f68-85fa-6f1b01c84331_disk_to_be_deleted_by_glance@9fb1737dce0241249f7bcfb83bd51502_to_be_deleted_by_glance
2016-11-22 09:38:16.972395 8fbdedeaf7c0 -1 librbd: parent snapshot does not exist
2016-11-22 09:38:16.972591 8fbdedeaf7c0 -1 librbd: error opening parent snapshot: (2) No such file or directory
rbd: listing children failed: (2) No such file or directory
2016-11-22 09:38:16.999014 8fbdedeaf7c0 -1 librbd: Error looking up name for image id ebc7d244bd6e04 in pool virtual_machines
Listing all objects in the pool does just show a single header object for this id:
# rados -p rbd ls | grep ebc7d244bd6e04
rbd_header. ebc7d244bd6e04
#
The stale image is not shown when doing a rbd ls
on the pool.
Environment
- Red Hat Ceph Storage 1.3
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.