Common issues with Satellite 6 upgrade or installation

Summary

Please note that many of the below issues as well as other bugs have been resolved in Red Hat Satellite 6 errata listed in these Articles: Red Hat Satellite Release Dates

Red Hat Satellite is on an "n-2" support schedule as documented in the Product Lifecycle Page. Therefore, when a release such as Satellite 6.6 is released customers on 6.2 should immediately upgrade to the latest version. Upgrades are always done from one minor version to the next, so customers would need to go from 6.2 to 6.3, 6.3 to 6.4, and then 6.4, 6.5 to 6.6. This document outlines the major known issues for migrating to a given version. This is outlined in this article.

Install or Upgrade

How to Install or Upgrade Red Hat Satellite 6 server to latest 6.7 version?

Please refer to the Documentation of Satellite 6.7 and follow the guide that correspond to your situation Red Hat Satellite 6.7 Installation and Upgrade Guides

How to Install or Upgrade Red Hat Satellite 6 server to latest 6.6 version?

Please refer to the Documentation of Satellite 6.6 and follow the guide that correspond to your situation Red Hat Satellite 6.6 Installation and Upgrade Guides

Interactive Red Hat Satellite Upgrade Helper

There is also an interactive Red Hat Satellite Upgrade Helper app designed to help you upgrade your Satellite 6 server to a newer version. All you need to provide is information about your upgrade path. We will not only present you the steps to upgrade Red Hat Satellite 6, but also give you some extra steps that will prevent known issues specific to your upgrade scenario.

Upgrade Issues Lists

Common Issues Applicable to Any Satellite6 Version

foreman-maintain fails with "Could not acquire change log lock"

https://access.redhat.com/solutions/3666621

Common Issues - Satellite 6.10

Red Hat Satellite upgrade to version 6.10 from version 6.9 fails with the following error message - ForemanTasks::TaskError: Task : Katello::Errors::Pulp3Error: Sort exceeded memory limit of 104857600 bytes

https://access.redhat.com/solutions/6716081

Satellite upgrade check command is printing "Please run 'foreman-maintain prep-6.10-upgrade' prior to upgrading."

https://access.redhat.com/solutions/6532821

[Satellite 6.10] Satellite upgrade fails due to orphaned RPMs: rubygem-sinatra, rubygem-rack-protection

https://access.redhat.com/solutions/6528671

Red Hat Satellite 6.10 upgrade fails with PG::NotNullViolation: ERROR: column "subscription_id" contains null values

https://access.redhat.com/solutions/6519901

[Satellite 6.10] Candlepin service failed after upgrading from Red Hat Satellite 6.9 to 6.10

https://access.redhat.com/solutions/6534951

The "hammer ping" command shows pulp and pulp_auth as failed even after upgrading to Red Hat Satellite 6.10

https://access.redhat.com/solutions/6601321

Upgrade to Red Hat Satellite 6.10 fails with error '/usr/sbin/foreman-rake db:migrate' returned 1 instead of one of [0]

https://access.redhat.com/solutions/6628631

Upgrade to Red Hat Satellite 6.10 fails at db:seed step with error "ActiveRecord::RecordInvalid: Validation failed"

https://access.redhat.com/solutions/6475691

Yum is failing with [Errno 14] HTTPS Error 403 - Forbidden error on the client systems after upgrading to Red Hat Satellite 6.10

https://access.redhat.com/solutions/6504131

How to use HTTPS (https://) proxy with Red Hat Satellite 6.10?

https://access.redhat.com/solutions/6414991

[Satellite 6.10] CV publish fails with "get() returned more than one UpdateRecord -- it returned X!"

https://access.redhat.com/solutions/6528211

Repository sync tasks are randomly failing with message Pulp task error after upgrading to Red Hat Satellite 6.10

https://access.redhat.com/solutions/6563341

Repository sync or Content-view publish task fails with error 'PulpRpmClient::ApiError: Error message: the server returns an error' in Red Hat Satellite 6.10

https://access.redhat.com/solutions/6667331

[Satellite 6.10] Content View publish fails with: 'No route matches {:action=>"show", :controller=>"foreman_tasks/tasks", :id=>nil}, missing required keys: [:id] (ActionController::UrlGenerationError)'

https://access.redhat.com/solutions/6567991

The hammer content-import command fails to complete if /tmp does not have enough free space in Red Hat Satellite 6.10

https://access.redhat.com/solutions/6595581

[Satellite6] removing or publishing a content view version fails on "PulpRpmClient::ApiError Error message: the server returns an error HTTP status code: 400 Response headers"

https://access.redhat.com/solutions/6607551

The 'satellite-maintain content remove-pulp2' command fails with error 'pulp-admin-client is needed by (installed) satellite-debug-tools-6.9.8-1.el7sat.noarch' after upgrading to Red Hat Satellite 6.10

https://access.redhat.com/solutions/6687131

[Satellite 6.10] Repo sync failed with "aiohttp.client_exceptions.ClientPayloadError: Response payload is not completed"

https://access.redhat.com/solutions/6596191

Repository sync or Content-view publish task fails with error 'PulpRpmClient::ApiError: Error message: the server returns an error' in Red Hat Satellite 6.10

https://access.redhat.com/solutions/6667331

The "Subscriptions" & "Red Hat Repositories" pages are showing up blank in Web UI after upgrading to Red Hat Satellite 6.10.4

https://access.redhat.com/solutions/6859391

How to find errata or package artifacts files under /var/lib/pulp on pulp-3 ?

https://access.redhat.com/solutions/6665871

Common Issues - Satellite 6.9

Upgrade to Satellite 6.9 fails at db:migrate stage with 'could not create unique index "indexlookup_values_on_lookup_key_id_andmatch"' error

https://access.redhat.com/solutions/6029911

[Satellite 6] Candlepin service is down after an upgrade to Satellite 6.9

https://access.redhat.com/solutions/6027421

Upgrade to Satellite 6.9 custom httpd configuration ScriptAlias non functional

https://access.redhat.com/solutions/6100891

Upgrade to Red Hat Satellite 6.9 failing with Could not find rdoc (>= 3.9.0) among 229 total gem(s) (Gem::LoadError).

https://access.redhat.com/solutions/5775131

Upgrade to Red Hat Satellite 6.9 fails in "Executing: katello-certs-check" step while custom SSL certificates are in use

https://access.redhat.com/solutions/6006081

The mongo engine upgrade to wiredTiger fails with error "`execute': wrong number of arguments (given 1, expected 3) (ArgumentError)" in Satellite 6.9

https://access.redhat.com/solutions/6165312

[Satellte6] Satellite upgrade fails with error message 'pulp-manage-db returned 69'

https://access.redhat.com/solutions/6252811

Red Hat Satellite upgrade to 6.9 fails with error 'Following repositories are not available on your system: rhel-7-server-satellite-6.9-rpms'

https://access.redhat.com/solutions/5993901

Upgrade from Red Hat Satellite 6.8 to 6.9 fails with error '/usr/sbin/foreman-rake db:migrate' returned 1 instead of one of [0]

https://access.redhat.com/solutions/6003311

Common Issues - Satellite 6.7

Red Hat Insights not working after upgrade to Red Hat Satellite 6.7, if there is HTTP Proxy used in the environment.

https://access.redhat.com/solutions/5010111

After upgrading to Red Hat Satellite 6.7, the "Tasks" page goes blank when loading.

https://access.redhat.com/solutions/5056971

Common Issues - Satellite 6.6

Candlepin gets stuck during startup forever, logging huge thread dump to error.log

https://access.redhat.com/solutions/3803901

# Legacy references
The following text is kept for legacy reasons, as it contains references to unsupported versions of Satellite. It can help in the only supported scenario of upgrading Satellite to a supported version.

Install or upgrade of older Sat6 versions

How to Install or Upgrade Red Hat Satellite 6 server to latest 6.5 version?

Please refer to the Documentation of Satellite 6.5 and follow the guide that correspond to your situation Red Hat Satellite 6.5 Installation and Upgrade Guides

How to Install or Upgrade Red Hat Satellite 6 server to latest 6.4 version?

Please refer to the Documentation of Satellite 6.4 and follow the guide that correspond to your situation Red Hat Satellite 6.4 Installation and Upgrade Guides

NOTE that Red Hat Satellite 6.3 and 6.4 do not support self-registered Satellites. You must migrate self-registered Satellite Servers to the Red Hat Content Delivery Network following steps in the Upgrade Guide

How to Install or Upgrade Red Hat Satellite 6 server to latest 6.3 version?

Follow Red Hat Satellite 6 Installation Guide which contains pre-requisites and installation steps for both Satellite and Capsule.

Follow Upgrading and Updating Red Hat Satellite which contains pre-requisites and upgrade steps for both Satellite and Capsule.

NOTE that Red Hat Satellite 6.3 and 6.4 do not support self-registered Satellites. You must migrate self-registered Satellite Servers to the Red Hat Content Delivery Network following steps in the Upgrade Guide

Common issues in older versions of Sat6

Common Issues - Satellite 6.5

Upgrade to Satellite 6.5 fails at 'migrate_foreman'

https://access.redhat.com/solutions/4147231

Common Issues - Satellite 6.4

Satellite upgrade to 6.4.3 failed with NoMethodError: undefined methodimport_product_content' for Katello::Glue::Candlepin::Product:Module`

https://access.redhat.com/solutions/4095781

Foreman-maintain utility does not list the version 6.4 for an upgrade.

https://access.redhat.com/solutions/3657831

Red Hat Satellite upgrade fails from 6.3.x to 6.4.x with an error undefined local variable or method `server_version' for satellite

https://access.redhat.com/solutions/4996411

Satellite upgrade from 6.3 to 6.4 fails at updating "rh-mongodb34-syspaths" package stage

https://access.redhat.com/solutions/3658241

Satellite 6.4 upgrade fails on upgrade step migrate_candlepin

https://access.redhat.com/solutions/3698131

Upgrade to Satellite 6.4 is failing because of invalid configuration in .hammer/defaults.yml

https://access.redhat.com/solutions/3659041

Unable to perform disconnected Satellite 6.4 installation or Upgrade using installation media.

https://access.redhat.com/solutions/3659241

Cannot enable Satellite Tools Repo on 6.4

https://access.redhat.com/solutions/2475941

Red Hat Satellite 6.4 upgrade fails with, Upstream username and password may only be set on custom repositories

https://access.redhat.com/solutions/3675571

Puppet 3 to Puppet 4 upgrade does not migrate autosign entries

https://access.redhat.com/solutions/3767071

Common Issues - Satellite 6.3

Installing or upgrading Satellite 6.1, 6.2 or 6.3 on Red Hat Enterprise Linux 7.5 dependency errors

https://access.redhat.com/solutions/3401241

Unable to enable Red Hat Satellite 6.3 repositories using subscription-manager

https://access.redhat.com/solutions/2472261

Running cpdb or foreman-maintain upgrade check failed on candlepin-validate-db

https://access.redhat.com/solutions/3362821

Red Hat Satellite upgrade from 6.2 to 6.3 fails at migrate_candlepin step with ERROR: duplicate key value violates unique constraint "cp_consumer_type_label_key"

https://access.redhat.com/solutions/3380851

Satellite 6 Can't set target version 6.3.z, 6.3 already in progress

https://access.redhat.com/solutions/3497291

Upgrade from 6.2.14 to 6.3 fails with Could not evaluate: Exception Connection refused - connect(2)

https://access.redhat.com/solutions/3425081

Upgrade Fails with error "rake aborted! NoMethodError: undefined method `first' for nil:NilClass"

https://access.redhat.com/solutions/3345191

Upgrade to Satellite 6.3 fails with certificate errors if the server has been renamed

https://access.redhat.com/solutions/3467491

Upgrade from 6.2.14 -> 6.3.1 failed at katello:clean_backend_objects

https://access.redhat.com/solutions/4004291

Upgrade to 6.3 failed with "Could not find dependency Class[Puppet::Server::Install] for File[/etc/pki/katello/puppet]"

https://access.redhat.com/solutions/4004371

Common Issues - Satellite 6.2

Red Hat Satellite upgrade from 6.1.11 to 6.2.x fails with error "Could not start Service[qpidd]: Execution of '/usr/share/katello-installer-base/modules/service_wait/bin/service-wait start qpidd' returned 6"

https://access.redhat.com/solutions/3031811

[Satellite6.2.11] qpidd fails to start with "Queue ..: recoverQueues() failed: jexception 0x0000 (Journal file .. has fid=0x1 which already exists for this journal." error

https://access.redhat.com/solutions/3157651

After upgrading to Satellite-6.2.10 manifest refresh is failing with "password component depends user component"

https://access.redhat.com/solutions/3093191

Upgrade to 6.2.11 fails on "mv: cannot move ‘/var/lib/qpidd/.qpidd/qls/jrnl/resource_manager/.’ .." or similar

https://access.redhat.com/solutions/3148641

Candlepin Failure when updating Red Hat Satellite 6.2 to RHEL 7.4

https://access.redhat.com/solutions/3136351

High memory usage and unresponded hosts search query after upgrade to 6.2.9

https://access.redhat.com/solutions/3023131

Satellite 6 : Content Hosts selection under Hosts empty "You currently don't have any Content Hosts" or keeps loading endlessly after Satellite upgrade.

https://access.redhat.com/solutions/1758323

Cannot enable Red Hat Satellite Tools Repo on Satellite 6.2

Selecting Red Hat Satellite Tools 6.2 inside Red Hat Repositories for:
- RHEL 5 (Workstation/Server)
- RHEL 6 (Workstation/Server)
- RHEL 7 (Workstation/Server)
- All result in CDN forbidden errors like "CDN loading error: access forbidden to https://cdn.redhat.com:443/content/dist/rhel/server/6/6Server/x86_64/sat-tools/6.2/os/repodata/repomd.xml"

https://access.redhat.com/solutions/2475941

Unable to enable Red Hat Satellite 6.2 repositories using subscription-manager, fails with "Error: rhel-7-server-satellite-6.2-rpms is not a valid repository ID. Use --list option to see valid repositories."

Users are unable to access the newly released Red Hat Satellite 6.2 repositories that are part of their subscription:
- rhel-6-server-satellite-6.2-rpms
- rhel-7-server-satellite-6.2-rpms

Attempts to enable these repositories are met with errors that the repository does not exist:

# subscription-manager repos --enable rhel-7-server-satellite-6.2-rpms
Error: rhel-7-server-satellite-6.2-rpms is not a valid repository ID. Use --list option to see valid repositories.

https://access.redhat.com/solutions/2472261

Red Hat Satellite 6.1.9 to 6.2 upgrade causes duplicate Hosts and Content Hosts

After Red Hat Satellite 6.1.9 to 6.2.0 upgrade, registering a system with short hostname configured on it causes duplicate profiles created in Hosts and Content Hosts
https://access.redhat.com/solutions/2482641

Red Hat Satellite 6.2 manifest import fails

https://access.redhat.com/solutions/2488481

Satellite 6.2 Virtual Datacenter (VDC) subscription does not show any product content, guest subscriptions show as null in activation key

https://access.redhat.com/solutions/2594971

Upgraded Satellite 6.2 Server Cannot Sync "Katello::Resources::Candlepin::Owner: 500 Internal Server Error"

https://access.redhat.com/solutions/2525481

Receiving PLP0034: distributor errors while attempting to publish new version of Content Views / sync repositories on Red Hat Satellite Capsule 6.2

https://access.redhat.com/solutions/2600231

Satellite 6.1.9 upgrade to 6.2 GA is taking significant amount of time

https://access.redhat.com/solutions/2484641

[Satellite 6.2] Error while trying to evaluate the upgrade process using '--noop' option

Getting below error in satellite.logs while trying to evaluate the upgrade process using --noop option:

Processing report from satellite.example.com with processor Puppet::Reports::Store
Puppet has finished, bye!
Executing hooks in group post
Hook /usr/share/katello-installer-base/hooks/post/10-post_install.rb returned nil
Upgrade failed during the installation phase. Fix the error and re-run the upgrade.
Hook /usr/share/katello-installer-base/hooks/post/30-upgrade.rb returned [<Logging::Logger:0x19f4150 name="main">, <Logging::Logger:0x19fa924 name="fatal">, <Logging::Logger:0x1b690a8 name="verbose">]
All hooks in group post finished
Exit with status code: 4 (signal was 4)

https://access.redhat.com/solutions/2477571

Upgrade from satellite 6.1 to 6.2 fails with Could not evaluate: Proxy cannot be registered (Connection refused - connect(2)): N/A"

Related to above "[Satellite 6.2] Error while trying to evaluate the upgrade process using '--noop' option"
https://access.redhat.com/solutions/2479161
Satellite 6.2 documentation has been added to clarify this and avoid error.

DHCP configuration in Capsule gets overridden during 6.2 upgrade

Manual edits to DNS or DHCP configuration files are overwritten during upgrade.

https://access.redhat.com/solutions/2482981

Satellite 6.2 upgrade fails with Upgrade step start_tomcat failed despite tomcat starting promptly

https://access.redhat.com/solutions/2476551

Upgrade to Satellite 6.2 fails with "tomcat fails to start" ; getting "401 Unauthorized" errors

https://access.redhat.com/solutions/2487971

Upgrade to Red Hat Satellite 6.2 Fails at Update_Subscription_Facet_Backend_Data Step

https://access.redhat.com/solutions/2485931

Various performance issues after upgrade from 6.1

Manual removal of some gutterball bits is required.
https://access.redhat.com/solutions/2608341

Error: "Access to was denied . You don't have authorization to view this page after upgrade. HTTP ERROR 403"

https://access.redhat.com/solutions/3200562