Satellite certificate or manifest activation error: "You might need to deallocate some entitlements from non-base organization(s)"

Solution Verified - Updated -

Environment

  • Red Hat Satellite 5 - 5.6, 5.7,5.8

Issue

  • Activation of a new Satellite certificate via rhn-satellite-activate fails with the following error(s) at the command line or in /var/log/rhn/rhn_server_satellite.log:

    Certificate specifies 70 of rhn-tools-rhel-workstation-6 non-flex entitlements.
    There are 82 non-flex entitlements allocated to non-base org(s) (5 used).
    You might need to deallocate some entitlements from non-base organization(s).
    You need to free 12 entitlements to match the new certificate.
    Activation failed, will now exit with no changes.
    

or:

    ERROR:: You do not have enough entitlements in the base org.

    Exception type satellite_tools.satCerts.NoFreeEntitlementsError

    Exception Handler Information
    Traceback (most recent call last):
      File "/usr/bin/satellite-sync", line 136, in main
        return satsync.Runner().main()
      File "/usr/share/rhn/satellite_tools/satsync.py", line 199, in main
        ret = method()
      File "/usr/share/rhn/satellite_tools/satsync.py", line 268, in _step_channel_families
        self.syncer.syncCert()
      File "/usr/share/rhn/satellite_tools/satsync.py", line 506, in syncCert
        return self._process_cert(cert)
      File "/usr/share/rhn/satellite_tools/satsync.py", line 541, in _process_cert
        satCerts.storeRhnCert(cert)
      File "/usr/share/rhn/satellite_tools/satCerts.py", line 289, in storeRhnCert
        set_slots_from_cert(sc)
      File "/usr/share/rhn/satellite_tools/satCerts.py", line 189, in set_slots_from_cert
        raise NoFreeEntitlementsError()
    NoFreeEntitlementsError 

or:

Exception type common.rhnException.rhnFault

Exception Handler Information
Traceback (most recent call last):
  File "/usr/bin/satellite-sync", line 136, in main
    return satsync.Runner().main()
  File "/usr/share/rhn/satellite_tools/satsync.py", line 199, in main
    ret = method()
  File "/usr/share/rhn/satellite_tools/satsync.py", line 268, in _step_channel_families
    self.syncer.syncCert()
  File "/usr/share/rhn/satellite_tools/satsync.py", line 506, in syncCert
    return self._process_cert(cert)
  File "/usr/share/rhn/satellite_tools/satsync.py", line 533, in _process_cert
    sync_handlers.populate_channel_family_permissions(sat_cert)
  File "/usr/share/rhn/satellite_tools/sync_handlers.py", line 643, in populate_channel_family_permissions
    importer.run()
  File "/usr/share/rhn/server/importlib/importLib.py", line 617, in run
    self.submit()
  File "/usr/share/rhn/server/importlib/channelImport.py", line 229, in submit
    self.backend.processChannelFamilyPermissions(self.batch)
  File "/usr/share/rhn/server/importlib/backend.py", line 1014, in processChannelFamilyPermissions
    (cfp['org_id'], cfp['channel_family'], cfp['max_members']), explain=0)
rhnFault: (23, 'ORA-20290: (not_enough_entitlements_in_base_org) - You do not have enough entitlements in the base org.: org_id [1] family [rhel-server-cluster] max [-87]', 'Could not update database entry.')

or

  <rhnFault class (code = 23, text = 'ORA-20290: (not_enough_entitlements_in_base_org) - You do not have enough entitlements in the base org.: org_id [1] family [rhel-server] max [-53]')>
(23, 'ORA-20290: (not_enough_entitlements_in_base_org) - You do not have enough entitlements in the base org.: org_id [1] family [rhel-server] max [-53]', 'Could not update database entry.'

or

RHN_PARENT: satellite.rhn.redhat.com
Certificate specifies `x` number of rhel-server non-flex entitlements.
    There are `y` number of non-flex systems in the base organization.
    You might need to unentitle some systems in the base organization.
    You need to free `z` number of entitlements to match the new certificate.
Activation failed, will now exit with no changes.

For Satellite 5.8:

# rhn-satellite-activate --manifest=/tmp/manifest.zip --verbose

15:28:33 HTTP_PROXY: None
15:28:33 HTTP_PROXY_USERNAME: None
15:28:33 HTTP_PROXY_PASSWORD: <password>
15:28:33 Checking cert XML sanity and GPG signature: '/usr/bin/validate-sat-cert.pl --keyring /etc/webapp-keyring.gpg /etc/sysconfig/rhn/rhsm-manifest.zip-cert-Lmzfsh'
15:28:53 Database connectioned initialized: refer to /etc/rhn/rhn.conf
15:28:53 Attempting local RHN Certificate push (and therefore activation)
Certificate specifies 501 of enterprise_entitled entitlements.
    There are 19 entitlements used by systems in the base (id 1) organization,
    plus 1000 entitlements allocated to non-base org(s) (0 used).
    You might need to unentitle some systems in the base organization,
    or deallocate some entitlements from non-base organization(s).
    You need to free 518 entitlements to match the new certificate.
    In the WebUI, the entitlement is named Management.
Certificate specifies 501 of provisioning_entitled entitlements.
    There are 5 entitlements used by systems in the base (id 1) organization,
    plus 1000 entitlements allocated to non-base org(s) (0 used).
    You might need to unentitle some systems in the base organization,
    or deallocate some entitlements from non-base organization(s).
    You need to free 504 entitlements to match the new certificate.
    In the WebUI, the entitlement is named Provisioning.
Activation failed, will now exit with no changes.
  • satellite-sync fails with:

    SYNC ERROR: unhandled exception occurred:
    
    (Check logs/email for potentially more detail)
    
    Error: You do not have enough unused provisioning_entitled 
    entitlements in the base org. You will need at least 1 free 
    entitlements, based on your current consumption. Please un-entitle the 
    remaining systems for the activation to proceed.
    
  • Red Hat Satellite upgrade fails with error: "You do not have enough entitlements in the base org" during Satellite certificate activation step, how to fix this?

Resolution

  • This error indicates that the new Satellite certificate or manifest contains fewer entitlements than is currently being allocated to organizations and/or consumed by systems on the Satellite. Refer Root Cause section for more details.

Satellite 5.8:

  • Compare the system and software channel entitlements on the Satellite with the numbers in the new Satellite Manifest (see Diagnostic Steps below). Either purchase additional entitlements if needed, or decrease the number of entitlements on the Satellite by un-entitling / un-subscribing systems, or decrease the total allocations from non-default Organizations, or attach additional available subscriptions under your account to the Satellite, leaving the slots value as 1. Follow this section in the Satellite 5 documentation.
  • For example, if there are multiple organizations and the base organization does not have enough free/unused entitlements, then log in as the Satellite Administrator and for each non-base organization go to Admin/Organizations/<organization>/Subscriptions. Under System Entitlements and Software Channel Entitlements, reduce the Proposed Total to the number consumed (Usage). This would transfer the unused but allocated entitlements from non-base (non-default) organizations back to the base (default) organization. Try to activate the new manifest again.
  • In addition to the Web UI, Red Hat Satellite API calls can also be used to adjust system and software channel entitlements - look for "Entitlements" related calls in the Satellite 5 API Guide , some example script can be found in How to remove allocated entitlements from organization after grace period with Satellite 5.8?.

Satellite 5.6-5.7:

Root Cause

  • Occasionally this issue can be caused by an incorrect organization name in the Satellite certificate. But most often, this error indicates that the new Satellite certificate or manifest contains fewer entitlements than is currently being allocated to organizations and/or consumed by systems on the Satellite. Some specific examples below:
    • The Satellite version is 5.4 or older and Virtualization/Virtualization Platform entitlements are being consumed, which had been replaced with Flex Guest Entitlements, as contained in the new certificate.
    • There are multiple organizations on the Satellite, and the total number of entitlements allocated (even though they may not be consumed by systems at the moment) to the organizations exceeds the number specified in the certificate. This scenario may be apparent for CAUDIT customers moving onto new SKUs, but it could also happen to non CAUDIT customers who have decreased entitlements since the old certificate was generated.
    • The customer renews their subscriptions, but purchases fewer entitlements than currently consumed/allocated on the Satellite server.
    • There is also an edge use case where the customer may have been taking advantage of the complimentary 20 provisioning and 20 monitoring slots as part of a demo program which had been discontinued. For example, the customer's satellite has monitoring-entitled systems, but they have not purchased any monitoring slots and since the new cert no longer includes the free monitoring slots, the certificate activation fails.
    • The customer tries to follow What is the workflow for generating a Satellite Certificate for use with the 2013 Packaging Model? article but does not attach the rest of the available subscriptions under his account to the newly created certificate.

Diagnostic Steps

  • Diff/compare the current working certificate with the new certificate or manifest that's failing to activate.
  • If available, run "entitlement-report" on the Satellite and add up the total numbers of each type of entitlement and compare to the numbers in the new certificate.
  • Log in to the Satellite web UI as the Satellite Administrator and compare how many system entitlements (management, provisioning, monitoring, virtualization, etc.) and software channel entitlements (rhel-server, rhn-tools. etc.) are being allocated/consumed on the Satellite compared to the number of slots available on the new Satellite certificate.

    • Go to the Admin tab and then click on each of the Organizations in the list, then go to the Subscriptions tab for that organization, and look at System Entitlements and Software Channel Entitlements subtabs. Add up the total allotted (maximum) entitlements for each organization on the Satellite, not just entitlements that are currently consumed by systems. 'Total' refers to the total available entitlements to the Red Hat Satellite as reflected in this Red Hat Satellite's certificate. 'Available' refers to the number entitlements that are available to be allocated to various organizations. 'Allotted' refers to the total number of entitlements allocated to any organization from initial pool as reflected in this Red Hat Satellite's certificate. 'Used' refers to entitlements that have been consumed by systems in various organizations.

      • For example, in Subscriptions/Software Channel Entitlements (https://satellite.example.com/rhn/admin/multiorg/SoftwareEntitlements.do)
Entitlement Name                Regular Entitlement Counts  Regular Usage       Flex Entitlement Counts     Flex Usage
                                (Available/Total)*          (Used/Allotted)**   (Available/Total)*          (Used/Allotted)**
Workstation technology beta     49 / 50                     0 of 1 (0%)         50 / 50                     None Allotted

In this example, looking at regular (physical) entitlements, there are 50 in the certificate, 1 allocated, and 0 used. For flex entitlements, there are 50 in the certificate, 0 allocated, and 0 used.

  • For Virtualization entitlements, check how many regular (AKA physical) and flex guest entitlements are consumed then see if the total consumed is less than the quantity present in the certificate. For example, if the certificate has name="channel-families" flex="93" quantity="169" family="rhel-server" then 93 flex guest systems can be registered to the Satellite and 169-93 = 76 regular systems can be registered.
  • Run the following command to get more verbose output during certificate activation for Satellite 5.6-5.7:
        # rhn-satellite-activate --rhn-cert=/path/to/satellite.cert  -vvv            

For Satellite 5.8:

       # rhn-satellite-activate --manifest=manifes.zip --verbose

Comments

Related Bugzillas - all addressed as of http://rhn.redhat.com/errata/RHEA-2009-1434.html:

Bug 479439 "Satellite cert activation needs to handle case where entitlements has been discontinued/removed: ideally warn user to cancel and purchase subscriptions, or proceed and force-unentitle"

Bug 439042 "Need better error for reassigning ents. to base org"

Bug 485183 "Add sat cert activation documentation specific for multi-org (as seen on whitepaper) to Satellite guide/manual"

Videos

You can also refer to our Video Presentations and Demos:
* What is a Red Hat Satellite Certificate
* How to activate a Red Hat Satellite Certificate
* Troubleshooting Red Hat Satellite Certificate activation issues

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments