Configuring Satellite 6.4 for Insights' remediation playbook execution

As explained in my previous post, in Satellite 6.4 Insights integration has gone deeper than ever. With this new release, now Insights remediation playbooks can be executed from within the Satellite Web interface.

In this post, we are going to cover how Satellite 6.4 has to be configured in order to do so.

Basically, we simply need to allow Foreman to execute commands remotely.

This plugin enables Foreman to run arbitrary commands on hosts using different providers. Initially only an SSH provider is supported but we plan to add more.

Communication goes through the smart proxy so Foreman does not have to have direct access to the target host and can scale to control many hosts. A command can be customized similarly to provisioning templates or partition tables. A number of templates are included for running basic commands, installing packages, etc.

This plugin is installed in Satellite 6.4 by default.

Satellite server configuration

Next you have to setup ssh keys. By default smart proxy loads the key from /usr/share/foreman-proxy/.ssh/id_rsa_foreman_proxy. To customize it you can edit the configuration in /etc/foreman-proxy/settings.d/remote_execution_ssh.yml. Without customization you need to create new ssh key and distribute it to target hosts. The key must not use a passphrase.

In this post, we are not going to customize it, and use the default location, but just be aware of the options you have.

[root@sat]# cat /etc/foreman-proxy/settings.d/remote_execution_ssh.yml
---
:enabled: https
:ssh_identity_key_file: /var/lib/foreman-proxy/ssh/id_rsa_foreman_proxy
:local_working_dir: /var/tmp
:remote_working_dir: /var/tmp
:kerberos_auth: false
# Whether to run remote execution jobs asynchronously
:async_ssh: false

Please note these are NOT root's ssh keys, but specific to foreman-proxy.

To generate a key, run the following commands on the host where Smart Proxy runs:

[root@sat ~]# mkdir ~foreman-proxy/.ssh
[root@sat ~]# chown foreman-proxy ~foreman-proxy/.ssh
[root@sat ~]# sudo -u foreman-proxy ssh-keygen -f ~foreman-proxy/.ssh/id_rsa_foreman_proxy -N ''

When using SELinux, make sure the directory and the files have correct labels of ssh_home_t. If not, restore the context:

[root@sat ~]# restorecon -RvF ~foreman-proxy/.ssh

Don’t forget to restart Foreman, Smart Proxy and Foreman tasks, so plugins are reloaded:

[root@sat ~]# service httpd restart
[root@sat ~]# service foreman-tasks restart
[root@sat ~]# service foreman-proxy restart

Finally, you have to refresh the Smart Proxy features in the Foreman.

[root@sat ~]# systemctl restart smart_proxy_dynflow_core

NOTE: There's a known bug affecting systems using directory information services, such as IdM. Basically, when executing ssh, it's executing /usr/bin/sss_ssh_knownhostsproxy and this does not work with users that do not have a TTY (like the case of foreman-proxy). The workaround to solve this its commenting the following lines in /etc/ssh/ssh_config :

#ProxyCommand /usr/bin/sss_ssh_knownhostsproxy -p %p %h
#       GSSAPIAuthentication yes
## If this option is set to yes then remote X11 clients will have full access
## to the original X11 display. As virtually no X11 client supports the untrusted
## mode correctly we set this to yes.
#       ForwardX11Trusted yes
## Send locale-related environment variables
#       SendEnv LANG LC_CTYPE LC_NUMERIC LC_TIME LC_COLLATE LC_MONETARY LC_MESSAGES
#       SendEnv LC_PAPER LC_NAME LC_ADDRESS LC_TELEPHONE LC_MEASUREMENT
#       SendEnv LC_IDENTIFICATION LC_ALL LANGUAGE
#       SendEnv XMODIFIER

Remote Hosts Configuration

The remote hosts need to be configured to accept the private key that the smart proxy is using. Root is used as the default user for accessing remote hosts via SSH. You may set the remote_execution_ssh_user global setting to change the default. If you would like to override at the host group, host, or other level, you may use parameters to do so. Set a parameter called remote_execution_ssh_user.

The ssh keys for the smart proxies are available as a host parameter (remote_execution_ssh_keys). This allows you to manage the authorized keys with your configuration management platform of choice, or through a provisioning template.

[root@sat ~]# ssh-copy-id -i /var/lib/foreman-proxy/ssh/id_rsa_foreman_proxy.pub root@icX.example.com

NOTE: This step has to be repeated for all the client machines you want Insights to operate.

And that’s all that you need to do, happy remediating!