Chapter 6. Restoring a Core Backup

6.1. Overview

You can back up a Core by following the Backing up a Core procedure. Once you have created the backup, you can restore a Core to the state at the time of backup.

6.2. Requirements

  • A self-managed Core installation on an OpenShift platform
  • The oc binary has a logged in user with permission to edit deployment configurations and view persistent volumes.
  • A backup of the data for Mongodb, MySQL and all the components persistent volumes as described in Backing up Filesystem Data.

6.3. What Data is Restored

  • Mongodb Replica Set data
  • MySQL data
  • Nagios historical data
  • Core metrics files
  • Git files
  • Core SCM files

6.4. Preparing for Restoration

Before restoring the data:

  1. Scale the Mongodb and MySQL to run 1 replica.

  2. Scale all the other pods to 0, waiting for the scaling to take effect.

    You can scale the components down through the web interface or using the following commands: 

    oc scale --replicas=0 dc/fh-aaa
oc scale --replicas=0 dc/fh-appstore
oc scale --replicas=0 dc/fh-messaging
oc scale --replicas=0 dc/fh-metrics
oc scale --replicas=0 dc/fh-ngui
oc scale --replicas=0 dc/fh-scm
oc scale --replicas=0 dc/fh-supercore
oc scale --replicas=0 dc/gitlab-shell
oc scale --replicas=0 dc/memcached
oc scale --replicas=0 dc/millicore
oc scale --replicas=0 dc/nagios
oc scale --replicas=0 dc/redis
oc scale --replicas=0 dc/ups

  3. Verify all pods have been scaled down: 

    oc get pods

    Only mysql and mongo should be listed.

6.5. Restoring Mongodb Data

Note

If you did not create a gzipped archive dump as described in Backing up Mongodb Data, you might need to change the mongorestore arguments used in this procedure.

  1. Copy the backup file into the tmp directory of the mongodb pod using the oc rsync command: 

    oc rsync /path/to/backups/dir <mongo-pod-name>:/tmp
    Tip

    rsync copies everything in a directory. To save time put the Mongodb dump file into an empty directory before using rsync.

    You can ignore errors similar to the following: 

    rsync: failed to set permissions on "/tmp/.": Operation not permitted (1)
rsync error: some files/attrs were not transferred (see previous errors)
(code 23) at main.c(1052) [sender=3.0.9] error: exit status 23

  2. Connect to the remote shell of the Mongodb pod: 

    oc rsh <mongo-pod-name>

  3. Run mongorestore on the backup data: 

    mongorestore -u admin -p ${MONGODB_ADMIN_PASSWORD} --gzip --archive=mongodb-backup.gz

6.6. Restoring MySQL Data

  1. Copy the MySQL dump file into the tmp directory of the mysql pod using the oc rsync command: 

    oc rsync /path/to/backups/dir <mysql-pod-name>:/tmp
    Tip

    rsync copies everything in a directory. To save time put the MySQL dump file into an empty directory before using rsync.

    You can ignore errors similar to the following: 

    rsync: failed to set permissions on "/tmp/.": Operation not permitted (1)
rsync error: some files/attrs were not transferred (see previous errors)
(code 23) at main.c(1052) [sender=3.0.9] error: exit status 23

  2. Connect to the remote shell of the MySQL pod: 

    oc rsh <mysql-pod-name>

  3. Restore the backed up data: 

    mysql -h mysql -u ${MYSQL_USER} -p${MYSQL_PASSWORD} < mysql-backup.sql

6.7. Restoring Components Persistent Data

Tip

Every component other than MySQL and Mongodb is restored using this procedure and the components can be restored in any order

  1. Take note of which persistent volume each component is using, by entering the following command: 

    oc get pvc

  2. For each volume-name, other than MySQL and Mongodb: 

    oc describe pv <volume-name>

    The output contains a section similar to the following, use this to determine the server and directory the data is stored in.: 

    Source:
    Type:       NFS (an NFS mount that lasts the lifetime of a pod)
    Server:     1.2.3.4
    Path:       /data/nfs/pv
    ReadOnly:   false
Tip

Take note of the permissions and ownership of any files in the data directory, and make sure that the restored files match these permissions and ownership

To restore the data:

  1. Delete the contents of the pod’s current persistent volume.
  2. Extract the backed up data into that directory.

Once all the components data has been restored, you can scale the components up through the web interface or start the components using the following commands: 

oc scale --replicas=1 dc/fh-aaa
oc scale --replicas=1 dc/fh-appstore
oc scale --replicas=1 dc/fh-messaging
oc scale --replicas=1 dc/fh-metrics
oc scale --replicas=1 dc/fh-ngui
oc scale --replicas=1 dc/fh-scm
oc scale --replicas=1 dc/fh-supercore
oc scale --replicas=1 dc/gitlab-shell
oc scale --replicas=1 dc/memcached
oc scale --replicas=1 dc/millicore
oc scale --replicas=1 dc/nagios
oc scale --replicas=1 dc/redis
oc scale --replicas=1 dc/ups

After running this command, wait until all pods have been scaled up. Verify they are running using the following command: 

oc get pods

All the components should be listed, including mysql and mongo and the left and right values in the READY column should be equal.

6.8. Confirming Restoration

Log in to the Studio and ensure that all the projects, environments and MBaaS targets are correctly restored.