Chapter 5. Backup and Disaster Recovery

This chapter describes the minimum backup and restore procedures required to ensure continuity of your Red Hat Satellite deployment and associated data in the event of a disaster. If your deployment uses custom configurations you should take these into account when planning your backup and disaster recovery policy.

5.1. Backing up Satellite Server or Capsule Server

This section describes creating a backup of your Satellite Server or Capsule Server and all associated data using the katello-backup script. Backing up to a separate location is recommended. Backing up to a separate storage device on a separate system is highly recommended. As no Satellite services will be available during the backup, the backup can be scheduled for a quiet time (for example, using cron).
You can also use conventional backup methods such as that described in the System Backup and Recovery section of the Red Hat Enterprise Linux 7 System Administrator's Guide.

Note

  • When planning a scheduled backup, ensure that no other tasks are scheduled by other administrators for the same time. This is particularly important when administrators are working in different locations and time zones.
  • When creating a snapshot or conventional backup, stop all services (Do not do this if using the katello-backup script): 
    # katello-service stop
    Start the services after creating a snapshot or conventional backup: 
    # katello-service start
Red Hat Satellite 6.2 uses the katello-backup script to make and restore backups. To see the usage statement, enter a command as follows: 
# katello-backup --help
From Satellite 6.2.8, the katello-backup creates a time-stamped subdirectory in the backup directory you specify. The katello-backup script does not overwrite backups and the correct directory or subdirectory has to be selected when restoring from a backup or an incremental backup. The script will stop and restart services as required.

Known Issues

The katello-backup script has the following known issues:
  • In Satellite 6.2.7 and earlier:
    • The script does not support using the same directory multiple times for a full backup. Update to Satellite 6.2.8 or newer if possible. Alternatively, use a new directory every time or move the previous backup to a safe location until the new backup has completed successfully.
    • If an existing directory is used as the backup target directory, the script will change the group to the postgres group. This can have unexpected consequences if other processes are also using the directory. Update to Satellite 6.2.8 or newer if possible. Alternatively, use a new directory or subdirectory exclusively for backups.
  • In Satellite 6.2.8 and earlier, using the --online-backup option leaves Satellite services down. Update to Satellite 6.2.9 or newer if possible. Additionally, until Red Hat Bug 1432013 is resolved, use only the /tmp/ or /var/tmp/ directory to create online backups.
  • In Satellite 6.2.9 and earlier, using the --incremental option created full backups. This was fixed in Red Hat Bug 1445989 included in the asynchronous erratum RHBA-2017:1234 - Bug Fix Advisory. On Satellite 6.2.9, apply the erratum or upgrade to a newer version if possible.

Checking the Satellite or Capsule Version

To check the version of Satellite 6, use a command as follows: 
# yum info satellite
Red Hat Satellite 6 does not support updating individual packages. To update to the latest minor version, see Updating Satellite Server, Capsule Server, and Content Hosts in the Red Hat Satellite 6.2 Installation Guide. To upgrade to the next major version, see Upgrading Satellite Server and Capsule Server.

Procedure 5.1. To Perform a Full Backup of Satellite Server or Capsule Server:

This procedure performs a full off-line backup. No Satellite services will be available during the backup.
  1. Ensure your backup location has enough disk space to contain a copy of the following directories:
    • /etc/
    • /var/lib/pulp/
    • /var/lib/mongodb/
    • /var/lib/pgsql/
    In Satellite 6.2.7 and earlier, the backup target directory must be a new directory or subdirectory created exclusively for backups. You can use the du -sh directory_name command to check the space used by a directory.
  2. Request other users of Satellite Server or Capsule Server to save any changes and warn them that for the duration of the backup, no Satellite services will be available. Ensure that no other tasks are scheduled for the same time as the backup.
  3. Run the backup script: 
    # katello-backup backup_directory
    The katello-backup script stops all services which could impact the backup, performs the backup, then restarts the required services. If the target directory does not exist when trying to create a backup file the script will create it.

    Note

    From Satellite 6.2.13 the katello-backup script prompts for confirmation that the backup is to proceed. To run the backup without being prompted, add parameter --assumeyes.
    This process can take a long time to complete, due to the amount of data to copy.

Procedure 5.2. To Perform a Backup without Pulp Content:

This procedure performs an off-line backup but excludes the contents of the Pulp directory. This backup is useful for debugging purposes and is only intended to provide access to configuration files without spending time backing up the Pulp database. You cannot restore from a directory that does not contain Pulp content.
  1. Ensure your backup location has enough disk space to contain a copy of the following directories:
    • /etc/
    • /var/lib/mongodb/
    • /var/lib/pgsql/
    You can use the du -sh directory_name command to check the space used by a directory.
  2. Request other users of Satellite Server or Capsule Server to save any changes and warn them that for the duration of the backup, no Satellite services will be available. Ensure that no other tasks are scheduled for the same time as the backup.
  3. Run the backup script: 
    # katello-backup --skip-pulp-content backup_directory
    The katello-backup script stops all services which could impact the backup, performs the backup, then restarts the required services. If the target directory does not exist when trying to create a backup file the script will create it.

    Note

    From Satellite 6.2.13 the katello-backup script prompts for confirmation that the backup is to proceed. To run the backup without being prompted, add parameter --assumeyes.

Procedure 5.3. To Perform an Incremental Backup:

This procedure performs an off-line backup of any changes since a previous backup. Use a full backup as a reference to make the first incremental backup of a sequence. Keep at least the last known good full backup and a complete sequence of incremental backups to restore from.
  1. Ensure your backup location has enough disk space to contain a copy of all changes in the following directories:
    • /etc/
    • /var/lib/pulp/
    • /var/lib/mongodb/
    • /var/lib/pgsql/
    You can use the du -sh directory_name command to check the space used by a directory.
  2. Request other users of Satellite Server or Capsule Server to save any changes and warn them that for the duration of the backup, no Satellite services will be available. Ensure that no other tasks are scheduled for the same time as the backup.
  3. Run the backup script:
    With Pulp content: 
    # katello-backup backup_directory --incremental backup_directory/previous_time-stamped_subdirectory
    Without Pulp content: 
    # katello-backup backup_directory --skip-pulp-content --incremental backup_directory/previous_time-stamped_subdirectory
    The katello-backup script stops all services which could impact the backup, performs the backup, then restarts the required services. If the target directory does not exist when trying to create a backup file the script will create it. It is possible to make incremental backups using a backup older than the previous backup as a starting point, but with a corresponding increase in time to make the backup.

    Note

    From Satellite 6.2.13 the katello-backup script prompts for confirmation that the backup is to proceed. To run the backup without being prompted, add parameter --assumeyes.

Procedure 5.4. To Perform an Online Backup:

This procedure performs a full backup while Satellite Server or Capsule Server is running. If there are procedures affecting the Pulp database, the Pulp part of the backup procedure will repeat until it is no longer being altered. Since the backup of the Pulp database is the most time consuming part of backing up a Satellite, it is highly recommended to not alter the Pulp database during this time. It would prolong the procedure as the Pulp part of the backup will restart.

Note

Until Red Hat Bug 1432013 is resolved, use only the /tmp/ or /var/tmp/ directory to create online backups.

Important

Satellite 6 uses two database systems, Postgres and Mongo. There are records that exist in both Postgres and Mongo that need to remain synchronized.
The --online-backup option keeps all services running which means there is a possibility that data can be modified while the backup is being made. There is a basic check to see if the databases were modified during the backup. If this occurs, the script starts the database portion of the backup again. This check is rudimentary and cannot ensure with 100% certainty that there were no modifications to the databases while the backup script was running. This check can also result in repeated loops if there is continuous modification occurring to the databases.
If you still want to use the --online-backup method in production, ensure that no modifications occur during the backup.
  1. Ensure you have updated to Satellite 6.2.9 or later. To check the version of Satellite 6, use a command as follows: 
    # yum info satellite
  2. Ensure your backup location has enough disk space to contain a copy of the following directories:
    • /etc/
    • /var/lib/pulp/
    • /var/lib/mongodb/
    • /var/lib/pgsql/
    You can use the du -sh directory_name command to check the space used by a directory.
  3. Request other users of Satellite Server or Capsule Server to save any changes and ask them not to make changes to repositories and Content Views for the duration of the backup. Ensure that no other tasks, such as synchronizing of repositories, are scheduled for the same time as the backup.
  4. Run the backup script: 
    # katello-backup --online-backup /tmp/backup_directory

    Note

    From Satellite 6.2.13 the katello-backup script prompts for confirmation that the backup is to proceed. To run the backup without being prompted, add parameter --assumeyes.

Example 5.1. A Weekly Full Backup Followed by Daily Incremental Backups

An example script which makes a full backup on a Sunday and incremental backups on all other days of the week: 
#!/bin/bash -e
DESTINATION=/var/backup
if [[ $(date +%w) == 0 ]]; then
  katello-backup $DESTINATION --assumeyes
else
  LAST=$(ls -td -- $DESTINATION/*/ | head -n 1)
  katello-backup $DESTINATION --incremental "$LAST" --assumeyes
fi
exit 0
Add the --assumeyes parameter only if your Satellite version is 6.2.13 and higher.