Azure Redhat OpenShift - Deny Assignment in Resourcegroup

Solution Verified - Updated -

Environment

  • Azure Red Hat OpenShift (ARO)
    • 4.x

Issue

  • If Azure Redhat OpenShift (ARO) by default makes a Resource Group "Read Only", is there anyway to modify the "Deny Assignment" so that access to this Resource Group can be given to other Users?
  • Can I reboot virtual Machines Manually from the Azure Portal?
  • Can I modify the resources e.g. ( Disks,Virtual Machines,Network Interface,Load balancers, Public IP, Storage account and etc ) under the randomly generated resource group (aro-infra-xxxxxxx-clustername / aro-randomXXXXX) after cluster creation as Day-2 operations?

Resolution

Disclaimer: Links contained herein to external website(s) are provided for convenience only. Red Hat has not reviewed the links and is not responsible for the content or its availability. The inclusion of any link to an external website does not imply endorsement by Red Hat of the website or their entities, products or services. You agree that Red Hat is not responsible or liable for any loss or expenses that may result due to your use of (or reliance on) the external site or content.

  • By default it's not allowed to stop/start machines and modify the resources via the Azure Portal or Azure-cli for resources created under that randomly generated resource group for your Azure Redhat OpenShift cluster.
  • Currently, it is also not possible to add new users (with viewing roles) to an already existing Azure Redhat OpenShift resource group after creation due to the Deny Assignments placed on the Resource Group. Although it is understandable the Deny Assignment (that is automatically put in place) is a very effective protective measure, it can also be problematic if there is a need to later modify after the Resource Groups initial creation.
  • The deny assignment is documented and is the expected behavior. You may also check the ARO Support Policy on which actions are not supported that would violate the policy and void support from Microsoft and Red Hat here.
  • You may also refer to Microsoft Official documentation here for Azure Redhat Openshift FAQ.

Root Cause

  • For Production deployments the Azure ARO-RP configures a DenyAssignment attached to the auto generated (aro-infra-xxxxxxx-clustername / aro-randomXXXXX) Resource Group.
  • The Cluster Service Principal is the only one excluded from the DenyAssignment and the rest are denied.
  • More information can be found here.

Diagnostic Steps

You will see this from the Notifications and Activity logs on the Azure Portal.

  • Scenario 1: Adding an inbound rule in the Network Security Groups
    • User is trying to modify/add a rule on the Network Security Groups and the deny assignment prevented this action.
Failed to create security rule 'AllowAnyCustom8080Inbound'. Error: The client 'user@example.com' with object id 'XXXXXX-XXXX-4XX4-XXXX-bXXXXXXXXX8c' has permission to perform action 'Microsoft.Network/networkSecurityGroups/securityRules/write' on scope 'aro-infra-XXXXXX-XXXX/providers/Microsoft.Network/networkSecurityGroups/XXXX-XXXX-nsg/securityRules/AllowAnyCustom8080Inbound'>XXXX-95XX5-nsg/AllowAnyCustom8080Inbound'; however, the access is denied because of the deny assignment with name '0f1e3f53-d1a1-5e1d-87c2-9677f0992cd6' and Id '0f1e3f53d1a15e1d87c29677f0992cd6' at scope '/subscriptions/3cXXXXX-XXXX-XXXX-ba84-f84XXXXXXXX/resourcegroups/aro-infra-XXXXXXX-testaroclustername'.
  • Scenario 2: Changing the configuration of the Virtual Machine
    • The user is trying to modify the worker node virtual machine and the deny assignment prevented this action.
Failed to update virtual machine 'ocp-XXXXX-worker-australiaeast1-XXXX'. Error: The client 'user1@example.com' with object id '540e5759-b7b6-4d74-87f3-ce5e9cef150e' has permission to perform action 'Microsoft.Compute/virtualMachines/write' on scope 'aro-XXXXXXX/providers/Microsoft.Compute/virtualMachines/ocp-XXXXX-worker-australiaeast1-XXXX'>ocp-XXXXX-worker-australiaeast1-XXXX'; however, the access is denied because of the deny assignment with name '5901f0c2-9094-59b4-9a77-c83ca3b768ca' and Id '5901f0c2909459b49a77c83ca3b768ca' at scope '/subscriptions/6eXXXXX-XXXX-XXXX-aXXX-XXXXXXXXXXXXXb/resourcegroups/aro-XXXXX'.

These are just some of the few examples that are given here but there could be more depending on the resource type that is being modified under that resource group.

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments