How to delete volume backups created before RHOSP upgrade

Solution Verified - Updated -

Environment

  • Red Hat OpenStack Platform 12.0
  • Red Hat OpenStack Platform 13.0

Issue

  • Getting error while deleting volume : Delete backup aborted, the backup service currently configured [cinder.backup.drivers.swift] is not the backup service that was used to create this backup [cinder.backup.drivers.swift.SwiftBackupDriver].
  • Volume backups which were created before the RHOSP13 upgrade cannot be deleted anymore.

Resolution

1. Workaround

  • Change cinder backup driver on all the controller nodes and set it as below.

    [root@controller-3 ~]# grep drivers.swift /etc/cinder/cinder.conf
    backup_driver = cinder.backup.drivers.swift
    
  • Restart service

    # systemctl restart openstack-cinder-backup.service
    

2. Workaround

  • Change the service backend from the database itself.

    MariaDB [cinder]> select service from backups where id='45d982c0-aabd-4d77-b499-07d1b3769f8b';
    +-----------------------------------------------+
    | service                                       |
    +-----------------------------------------------+
    | cinder.backup.drivers.swift.SwiftBackupDriver |
    +-----------------------------------------------+
    1 row in set (0.00 sec)
    
    MariaDB [cinder]> select * from backups where id='45d982c0-aabd-4d77-b499-07d1b3769f8b';
    
    MariaDB [cinder]> select service from backups where id='278bf65a-bf1c-40a5-8a77-d98b9331b45c';
    +-----------------------------+
    | service                     |
    +-----------------------------+
    | cinder.backup.drivers.swift |
    +-----------------------------+
    1 row in set (0.00 sec)
    
    MariaDB [cinder]> update backups set service = 'cinder.backup.drivers.swift' where id='45d982c0-aabd-4d77-b499-07d1b3769f8b';
    Query OK, 1 row affected (0.00 sec)
    Rows matched: 1  Changed: 1  Warnings: 0
    
    MariaDB [cinder]> select service from backups where id='45d982c0-aabd-4d77-b499-07d1b3769f8b';
    +-----------------------------+
    | service                     |
    +-----------------------------+
    | cinder.backup.drivers.swift |
    +-----------------------------+
    1 row in set (0.00 sec)
    

Root Cause

  • There seems to be changed in the code, Swift driver is now called in a different way, but for some reason, cinder backup is not able to work with this change and fails.
  • In RHSO12 the cinder backup_driver is configured as cinder.backup.drivers.swift whereas in RHSOP13 the cinder backup_driver is configured as cinder.backup.drivers.swift.SwiftBackupDriver .

Diagnostic Steps

  • Check for cinder backup service.

  • You will notice a difference in the backup service configured using RHSOP12 and RHSOP13

    MariaDB [(none)]> use cinder;
    
    MariaDB [cinder]> select service from backups where id='45d982c0-aabd-4d77-b499-07d1b3769f8b';
    +-----------------------------------------------+
    | service                                       |
    +-----------------------------------------------+
    | cinder.backup.drivers.swift.SwiftBackupDriver |
    +-----------------------------------------------+
    1 row in set (0.00 sec)
    
    MariaDB [cinder]> select service from backups where id='278bf65a-bf1c-40a5-8a77-d98b9331b45c';
    +-----------------------------+
    | service                     |
    +-----------------------------+
    | cinder.backup.drivers.swift |
    +-----------------------------+
    1 row in set (0.00 sec)
    
  • The below backup volume was created in RHOSP12, When we try deleting this volume it goes into error state.

    (myenvironment)[test@localhost LAB]$ openstack volume backup show 65dbb187-0770-4f8b-93dd-ca759683ed82
    +-----------------------+--------------------------------------+
    | Field                 | Value                                |
    +-----------------------+--------------------------------------+
    | availability_zone     | nova                                 |
    | container             | volumebackups                        |
    | created_at            | 2018-10-29T13:38:05.000000           |
    | data_timestamp        | 2018-10-29T13:38:05.000000           |
    | description           |                                      |
    | fail_reason           | None                                 |
    | has_dependent_backups | False                                |
    | id                    | 65dbb187-0770-4f8b-93dd-ca759683ed82 |
    | is_incremental        | False                                |
    | name                  | loreg_test_backup                    |
    | object_count          | 16                                   |
    | size                  | 15                                   |
    | snapshot_id           | None                                 |
    | status                | available                            |
    | updated_at            | 2019-01-29T10:54:19.000000           |
    | volume_id             | 53762af0-4c69-4b2d-a186-0693b402ee79 |
    +-----------------------+--------------------------------------+
    
    (myenvironment)[test@localhost LAB]$ openstack volume backup delete 65dbb187-0770-4f8b-93dd-ca759683ed82
    
    (myenvironment)[test@localhost LAB]$ openstack volume backup show 65dbb187-0770-4f8b-93dd-ca759683ed82
    +-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
    | Field                 | Value                                                                                                                                                                                                       |
    +-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
    | availability_zone     | nova                                                                                                                                                                                                        |
    | container             | volumebackups                                                                                                                                                                                               |
    | created_at            | 2018-10-29T13:38:05.000000                                                                                                                                                                                  |
    | data_timestamp        | 2018-10-29T13:38:05.000000                                                                                                                                                                                  |
    | description           |                                                                                                                                                                                                             |
    | fail_reason           | Delete backup aborted, the backup service currently configured [cinder.backup.drivers.swift] is not the backup service that was used to create this backup [cinder.backup.drivers.swift.SwiftBackupDriver]. |
    | has_dependent_backups | False                                                                                                                                                                                                       |
    | id                    | 65dbb187-0770-4f8b-93dd-ca759683ed82                                                                                                                                                                        |
    | is_incremental        | False                                                                                                                                                                                                       |
    | name                  | test_backup                                                                                                                                                                                           |
    | object_count          | 16                                                                                                                                                                                                          |
    | size                  | 15                                                                                                                                                                                                          |
    | snapshot_id           | None                                                                                                                                                                                                        |
    | status                | error                                                                                                                                                                                                       |
    | updated_at            | 2019-03-11T15:10:31.000000                                                                                                                                                                                  |
    | volume_id             | 53762af0-4c69-4b2d-a186-0693b402ee79                                                                                                                                                                        |
    +-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
    

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.