Ceph - protected snapshot deletion attempt after HEAD flattening can lead to stale child reference

Solution In Progress - Updated -

Issue

After flattening of some images and deleting the parent snapshots, attempting to delete a protected snapshot led to a situation where we can't unprotect nor delete it:

#  rbd snap unprotect virtual_machines/8adfa17c-e730-4f68-85fa-6f1b01c84331_disk_to_be_deleted_by_glance@9fb1737dce0241249f7bcfb83bd51502_to_be_deleted_by_glance
2016-11-29 16:17:20.653509 8f5b29b437c0 -1 librbd: snap_unprotect: can't unprotect; at least 1 child(ren) in pool virtual_machines
#
# rbd snap rm virtual_machines/8adfa17c-e730-4f68-85fa-6f1b01c84331_disk_to_be_deleted_by_glance@9fb1737dce0241249f7bcfb83bd51502_to_be_deleted_by_glance
rbd: snapshot '9fb1737dce0241249f7bcfb83bd51502_to_be_deleted_by_glance' is protected from removal.
2016-11-28 13:24:01.107502 8fdeba89e7c0 -1 librbd: removing snapshot from header failed: (16) Device or resource busy

Printing the children will refer to a non-existing parent snapshot:

# rbd children virtual_machines/8adfa17c-e730-4f68-85fa-6f1b01c84331_disk_to_be_deleted_by_glance@9fb1737dce0241249f7bcfb83bd51502_to_be_deleted_by_glance
2016-11-22 09:38:16.972395 8fbdedeaf7c0 -1 librbd: parent snapshot does not exist
2016-11-22 09:38:16.972591 8fbdedeaf7c0 -1 librbd: error opening parent snapshot: (2) No such file or directory
rbd: listing children failed: (2) No such file or directory
2016-11-22 09:38:16.999014 8fbdedeaf7c0 -1 librbd: Error looking up name for image id ebc7d244bd6e04 in pool virtual_machines

Listing all objects in the pool does just show a single header object for this id:

# rados -p rbd ls | grep ebc7d244bd6e04
rbd_header. ebc7d244bd6e04
#

The stale image is not shown when doing a rbd ls on the pool.

Environment

  • Red Hat Ceph Storage 1.3

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content