How to setup Remote Execution using non-root user on RHEL systems connected to Red Hat Satellite 6?

Solution Verified - Updated -

Environment

  • Red Hat Satellite 6

Issue

  • How to setup Remote Execution using non-root user on RHEL system connected to Red Hat Satellite 6?

Resolution

  • On the client machine, create a user and add the account to sudoers file.

    [root@client ~]# useradd rexuser
    [root@client ~]# passwd rexuser
    [root@client ~]# echo "rexuser   ALL=NOPASSWD:   ALL" | tee -a /etc/sudoers.d/rexuser
    
  • In case you only need this rexuser to run yum related commands and nothing else, you may use:

    # echo "%rexuser ALL = NOPASSWD: /usr/bin/yum *,  /var/tmp/foreman-ssh-cmd*/script" | tee -a /etc/sudoers.d/rexuser
    
  • Check if rexuser can run the sudo commands without password.

    [root@client ~]# su - rexuser
    [rexuser@client ~]# sudo yum install tree
    
  • Copy over the foreman-proxy public key under rexuser account on client.example.com

    [root@satellite ~]# ssh-copy-id -i ~foreman-proxy/.ssh/id_rsa_foreman_proxy.pub rexuser@client.example.com
    
  • Now check if rexuser can execute the sudo commands without requiring any password interactions, using id_rsa_foreman_proxy private key from Satellite server

    [root@satellite ~]# ssh -i ~foreman-proxy/.ssh/id_rsa_foreman_proxy rexuser@client.example.com 'sudo yum repolist'
    
  • If foreman-proxy user can execute commands, then add the following parameter to the client host from the Satellite Server.
    Satellite webUI >> Hosts >> All Hosts >> Edit the client.example.com >> Parameters tab >> Add Parameter >> Specify Name as remote_execution_ssh_user and set its value to rexuser >> click Submit

  • The same can be done through hammer for individual clients:

    # hammer host list | grep client.example.com
    

    Note the id from the output and run:

    # hammer host set-parameter --host-id=XX --name='remote_execution_ssh_user' --parameter-type='string' --value='rexuser'
    

    Replace XX with the id from the previous output.

  • Please note that parameter remote_execution_ssh_user can also be set by Host Group, Operating System, Domain, Location, or Organization as well as globally.

  • Now Remote Execution jobs can be scheduled using a non-root user.

On Red Hat Satellite 6.4 and above:

  • Remote execution is possible without deploying the SSH keys, no requirement to set NOPASSWD in sudoers file and also if private key is guarded by a password, that too can be specified during the REX operation through Remote Job Advanced fields.

  • When executing scheduling a job, click on Display advanced fields >> specify the options Effective user, Password and Sudo password, which should allow REX job to be completed configuring SSH keys or NOPASSWD in sudoers file.

  • To set SSH user and Effective user globally, change the respective parameters from Administer >> Settings >> Remote Execution tab or using the following hammer commands

    [root@satellite ~]# hammer settings set --name remote_execution_ssh_user --value rexuser
    [root@satellite ~]# hammer settings set --name remote_execution_effective_user --value root
    
  • Note: When running Ansible roles on a client using non root user in this case if you set the SSH user and Effective user as the same user then Ansible will not work because Ansible allows you to ‘become’ another user, different from the user that logged into the machine (remote user). For more information refer this article.

  • Configure the satellite remote execution to use SSH key other than foreman-proxy SSH key

Video : Overview of Red Hat Satellite Remote Execution

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

21 Comments

Hello, I want to ask, if there is a way to do step 5 (Add Parameter) for more servers at once or I have to modify parameters on every server separately?

Can be automate with hammer :

hammer host set-parameter --host client.example.com --name remote_execution_ssh_user --value "testuser"

+50 for the hammer parameter. Thank you Pascal!

No need for the option NOPASSWD :

# useradd -m testuser

visudo

testuser ALL=NOPASSWD: ALL

Step 4 did not work for me. It is likely do to the pam configuratioins, but how are we to do REX on machines that are STIGed and use AD (SLDAP) authentication.

Need to add one more parameter in remote settings

Settings ---> Remote Execution ---> remote_execution_ssh_user ---> edit ---> testuser

Many customers have to comply with some security controls (STIG, others) that prohibit the use of allowing direct ssh to root, also disallow the use of "NOPASSWD" directive in /etc/sudoers. (among many other security mandates)

Apparently, from this blog discussion, https://access.redhat.com/blogs/1169563/posts/2482081, there is an RFE (#1437538 ) to make this work without these vulnerabilities (I suspect that RFE is for this, I might be wrong, I'm not sure).

So now to my question. A couple of people in that blog discussion (the person named Todd who submitted the RFE, and Red-Hatter James Mills who posted a bit after him), give some idea that maybe there will be some means to achieve Remote Execution that does not have to rely on either opening ssh directly to root, or having "NOPASSWD" directive in /etc/sudoers, both of which are not acceptable in STIG requirements (or my and other customers).

Is there any means currently, has anyone pioneered a method to achieve remote execution from the Satellite without suffering allowing ssh directly to root (ok, this solution addresses that, however....) or resorting to using "NOPASSWD" directive in /etc/sudoers for satellite version 6.2.current? (or perhaps it's coming soon?)

We along with many others who must suffer STIG or other controls (STIG is not constrained to just government), can not necessarily bypass specific security controls. we're hoping for a method to use Red Hat Satellite's remote execution and still have our force-fed pie of security controls..

Not everyone can use ALL=NOPASSWD if company policy does not allow it.

Added, we've been using Ansible (not Ansible Tower) to achieve what we could not with Satellite due to the constraint of not being able to use "NOPASSWD" sudoer directives.

Does Red Hat seriously expect this to be set at the client level? I have 2000 of them.

If you want to use non-root remote execution it is needed, however you can use LDAP/IPA for external users or Puppet or even root remote execution to set all stuff on client before switch to non-root remote execution. There is multiple ways to achieve this.

Probably it's worth noting that if someone wants to use a non root user as a REX user, he needs to modify the relevant templates and use "sudo " because the templates are build with the assumption that root will be used as the ssh user.

Wow! This article has me thoroughly confused! I'm trying to follow the process step by step, but the prompts in the article don't follow what the text is saying. Specifically:

Now check from the Satellite server if testuser can execute the sudo commands without requiring any password interactions.

Raw

[root@satellite ~]# su - foreman-proxy -s /bin/bash
(who am I here, root, testuser, or foreman-proxy? The prompt says I'm root on the satellite box. If I did an "su - testuser", wouldn't the prompt change? And why did I copy the foreman-proxy public key to my content host (client) if I'm issuing this command on the satellite server?? What does foreman-proxy have to do with any of this? ) Color me stupid, but I really do not understand this process at all.

On my Satellite server, foreman-proxy is not granted a shell, so how can I "su - foreman-proxy -s /bin/bash"?

Has anyone considered pam_ssh_agent_auth ?

How is this secure and passing anyone's security policy in 2019? This seems like a huge hole?

Could you please elaborate on how the rexuser might be misused? If I got it right, then:

  • only admin / root on a client system knows rexuser's password, so nobody else can sudo as a root
  • foreman-proxy SSH keys extend that to foreman-proxy user on Satellite/Capsule - but since foreman-proxy has /bin/false shell, nobody else than root on Caps/Sate can use that account

So until I miss something, just admins of Sat/Caps and of the Host(s) are granted root's permissions on the Host(s).

Multiply that across 1000's of servers and you have a single account that now has sudo access with no password... Its an attack vector that can be exploited. You get that one account, now you have access across your entire fleet of servers. You are now flagged on every audit system in 2019... Welcome to the world of security and audits...

I see. That's the (mandatory?) price for the convenience of automation of a remote execution accross managed systems. Three possible ways of limiting the scaled attack vector:

  • Use (more) Capsules and split managed systems among them. As the ssh from foreman-proxy account is initiated from the Capsule where the system is registered to. Then technically one foreman-proxy account (on a Caps) has access to as many systems as they are registered to the Caps. Though, still one Satellite user with sufficient permissions can invoke a remote execution to all systems on all Capsules - and once an intruder get to this account..

  • Use more Satellites with separate accounts. This resolves the above limitation.

  • Optionally, limit a user to some systems only (per Organization or Location). But still, an admin user can access all systems in all Orgs and Locs. As well as root user on the Satellite. Those will (always?) be the single point of authentication to managing all systems manageable via REX without passwords.

Anyway, the password-less REX is just an option or a feature that can but does not need to be used/set. Until configured, no such "one user can manage all the systems" exists.

Umm, yes. If you want to manage machines at scale, you need to have an identity with the required rights to manage said machines at scale. Regardless of operating system used or any other environment factors. Security will need to figure out a way to secure these identities and "live with it", unless they want all admins to leave the field, because managing hundreds or thousands of servers by hand is utter insanity.

Besides of that, there's not one line mentioning SELINUX MLS Hosts. ;-)

How can i tweak the command execution and add parameters to sudo ?

I have a bunch of high secure servers which cannot be updated by using sudo.

In fact, the call has to be:

sudo -r sysadm_r -i <command>

Reading the foreman docs, there's a parameter called: "remote_execution_effective_user_method" but this will be called by exec.

So there are no parameters applied to the command, and it will fail.

How to accomplish this, other than writing a wrapper script and try to use the parameter "remote_execution_effective_user_method" ?

Any help would be greatly appreciated !

I think, as the question has been unanswered for more than one moth i'll open a support case before upgrading to satellite 7. It seems as if SELINUX has been left out at the design stage. Also, when integrating OpenShift and RHVM, you have to deal with SELINUX considerations ;-) (After 20 years of development... Happy birthday SELINUX btw.)