Chapter 6. Discovering Bare-metal Hosts on Satellite

Red Hat Satellite 6.5-beta includes the Discovery plug-in. The Discovery plug-in enables automatic bare-metal discovery of unknown hosts on the provisioning network. These new hosts are registered to the Satellite Server and the Puppet agent on the client uploads system facts collected by Facter, such as serial ID, network interface, memory, and disk information. After registration, the hosts are displayed on the Discovered Hosts page in the Satellite web UI. You can then initiate provisioning either manually (using the web UI, CLI, or API) or automatically, using predefined discovery rules.

The Discovery plug-in communicates through the Satellite Capsule Server, which has direct access both to the provisioning network and the Satellite Server instance. It is possible to discover hosts directly from the Satellite Server, but Red Hat recommends the following scheme be used:

Satellite Server (Satellite Server Discovery plug-in) <--> Satellite Capsule (Satellite Capsule Discovery plug-in) <--> Discovered Host (Satellite Discovery image)

The Satellite Discovery plug-in consists of three different components:

The Satellite Server Discovery plug-in
This runs on the Satellite Server and provides API and UI functionality for working with discovered hosts. The tfm-rubygem-foreman_discovery package contains this plug-in.
The Satellite Capsule Server Discovery plug-in
This is a communication proxy between discovered hosts on a provisioning network and the Satellite Server. The rubygem-smart_proxy_discovery package contains this plug-in.
The Satellite Discovery image
This is the minimal operating system based on Red Hat Enterprise Linux that is PXE-booted on hosts to acquire initial hardware information and to check in to the Satellite Server. Discovered hosts keep running the Satellite Discovery image until they are rebooted into Anaconda, which then initiates the provisioning process. The foreman-discovery-image package contains this image. It must be installed on the Satellite Capsule Server that provides TFTP services.

6.1. Network Configuration for PXE-based Discovery

The discovery process is based on PXE: Systems must boot from the network using a single Ethernet connection to the LAN or VLAN. All other network interface configurations are not supported (bonding, teaming, bridging, DSL, Wi-Fi and others).

You must have a separate LAN or VLAN for discovery and PXE provisioning. You can configure systems to use VLAN trunks, but you must also configure the provisioning interface with the correct VLAN tag for the provisioning VLAN, and then change the tag to the production VLAN using a post-installation script.

Although using special network configurations is technically possible in PXE-less mode, where discovered systems use kexec to load a new kernel with Anaconda which avoids PXE booting completely, the discovery image currently does not allow such a configuration. While it is possible to use discovery extensions or a script to re-configure the network, Satellite 6 discovery plug-in cannot work with such a configuration.

Because the discovery process currently has limited possibilities for configuring network interfaces, and because the provisioning interface is also the primary interface, to simplify the configuration, have a separate primary interface from the interface used in production. Satellite 6 template features can be used to deploy post-installation scripts to configure interfaces if required.

6.2. Configuring the Satellite Discovery Plug-in

The following sections describe how to configure the Satellite Discovery plug-in and how to prepare the PXE-boot template on the Satellite Server.

6.2.1. Deploying the Satellite Discovery Image

Install the package containing the Satellite Discovery image on the Satellite Capsule Server that provides TFTP services (not on the Satellite Server itself):

# yum install foreman-discovery-image

This package contains the Linux kernel and initial RAM disk image as a bootable ISO file which is used for PXE-booting discovered hosts. You can run the following command to investigate the contents of the package. This produces output similar to the following:

$ rpm -ql foreman-discovery-image
/usr/share/foreman-discovery-image
/usr/share/foreman-discovery-image/fdi-image-rhel_7-2.1.0-20150212.1.iso

When you install this package, it extracts the kernel and image from the ISO file into the TFTP directory and creates symbolic links to the latest versions of the image and kernel. Use the symbolic links in the PXE-boot provisioning template to make sure that you do not need to change the version in the template every time the foreman-discovery-image package is upgraded. For example:

$ find /var/lib/tftpboot/boot
/var/lib/tftpboot/boot
/var/lib/tftpboot/boot/fdi-image-rhel_7-2.1.0-20150212.1-img
/var/lib/tftpboot/boot/fdi-image-rhel_7-2.1.0-20150212.1-vmlinuz
/var/lib/tftpboot/boot/fdi-image-rhel_7-img
/var/lib/tftpboot/boot/fdi-image-rhel_7-vmlinuz
Note

Currently, only Red Hat Enterprise Linux 7 Discovery images are provided, even for Satellite 6 installations on Red Hat Enterprise Linux 6. If there are discovered hosts running during the upgrade of the foreman-discovery-image package, reboot them all to load the updated version of the image as soon as possible. This can be done using the Satellite 6 web UI, CLI, or API.

6.2.2. Configuring PXE-booting

When an unknown host is booted on the provisioning network, Satellite Server provides a PXELinux boot menu with a single option: to boot from the local hard drive. You can use following procedure to build a default PXE template in Satellite to enable hardware discovery.

To Configure PXE-booting for host discovery:

  1. In the Satellite web UI, navigate to Hosts > Provisioning Templates.
  2. In the upper-right of the Provisioning Templates page, click Build PXE Default, and click OK.

The template becomes the default template on all TFTP servers. Every new unknown host that is in the provisioning subnet uses this configuration and uses the Foreman Discovery Image as the default.

6.2.3. Reviewing Global Discovery Settings

You can review global settings related to the Discovery plug-in in the Satellite web UI. Navigate to Administer > Settings and open the Discovered tab. Notable settings are:

Discovery organization, Discovery location
These variables specify where to place the discovered hosts. By default, the discovered hosts are automatically placed under the first organization and location created.
Interface fact
This variable specifies which incoming fact to use to determine the MAC address of the discovered host. By default, the PXELinux BOOTIF kernel command line option is used.
Hostname facts
This variable allows you to list facts to use for the host name. These are separated by commas, and the first fact in the list takes precedence.
Auto provisioning
This variable enables automatic provisioning according to specified rules. Set to false by default. Red Hat recommends that you test the configuration with manual provisioning before enabling Auto provisioning. See Section 6.4, “Provisioning Discovered Hosts” for more information.
Reboot
This variable enables automatic reboot of a host discovered by PXE, or the use of kexec for a host booted from local media, during provisioning. This is set to true by default.
Hostname prefix
This variable specifies the default prefix to use for the host name. Set to "mac" by default. The variable must start with a letter.
Fact columns
This variable allows you to add any fact reported by Facter as an additional column in discovered host lists.
Highlighted facts
This variable uses regular expressions to organize facts for the highlights section.
Storage facts
This variable uses regular expressions to organize facts for the storage section.
Hardware facts
This variable uses regular expressions to organize facts for the hardware section.
Network facts
This variable uses regular expressions to organize facts for the network section.
IPMI facts
This variable uses regular expressions to organize facts for the IPMI section.

6.3. Configuring the Satellite Capsule Server Discovery Plug-in

Ensure the foreman_url setting exists in the Satellite Capsule Server configuration file. The setting can appear as follows:

# grep foreman_url /etc/foreman-proxy/settings.yml
:foreman_url: https://satellite.example.com

The satellite-installer command configures this variable automatically, but Red Hat recommends that you check that the host responds correctly and there are no firewall rules blocking communication.

6.3.1. Configuring Discovery Subnets

You need to configure all subnets with discovered hosts to communicate to the Satellite Server or a Capsule Server for host discovery and provisioning. You must first enable a Capsule to provide a proxy service for provisioning templates for the subnet. The template Capsule for a subnet does not have to be the same Capsule that is set as the TFTP Capsule.

If you want to configure a new subnet, see Adding a Subnet to the Satellite Server in the Provisioning Guide.

To enable a Template Capsule:

You can enable the Satellite’s integrated Capsule and any external Capsule Servers to provide a proxy service for provisioning templates with the following command:

# satellite-installer --foreman-proxy-templates=1

To verify that a Capsule Server provides a template proxy service:

  1. Navigate to Infrastructure > Capsules.
  2. From the Actions menu, select Refresh to ensure that the list is up-to-date.
  3. In the Features column, search for the word "Templates".

To Configure a Subnet with a TFTP Capsule, Template Capsule, and Discovery Capsule:

  1. In the Satellite web UI, navigate to Infrastructure > Subnets.
  2. Select the subnet you want to configure.
  3. On the Capsules tab, select the TFTP Capsule, Template Capsule, and Discovery Capsule for this subnet.

6.3.2. Using Hammer with the Discovery Plug-in

To use the hammer command with the Discovery plug-in, you need to enable the Discovery plug-in in /etc/hammer/cli.modules.d/foreman_discovery.yml as follows:

:foreman_discovery:
  :enable_module: true

See hammer configuration directories for more information about the files and directories that hammer uses.

6.3.3. Reviewing User Permissions

When it first starts, the Satellite Capsule Server Discovery plug-in creates a role called Discovery. You can assign this role to non-administrative users to allow them to use the Discovery plug-in. Alternatively, assign the perform_discovery permission to an existing role. For more information on roles and permissions, see Creating and Managing Users in Administering Red Hat Satellite.

6.4. Provisioning Discovered Hosts

After you have correctly configured Discovery plug-ins on both Satellite Server and Capsule Server, you can automatically detect bare-metal hosts. To do so, boot a machine in any provisioning network that was configured with the PXE configuration template described in Section 6.2.2, “Configuring PXE-booting”. The machine is automatically registered with the Satellite Server and appears in the Hosts > Discovered Hosts list in the Satellite web UI.

You can either provision the discovered host manually, or you can configure automatic provisioning.

6.4.1. Manually Provisioning Hosts

The following procedure describes how to manually provision discovered hosts from the Satellite web UI.

To Manually Provision a Discovered Host:

  1. Navigate to Hosts > Discovered Hosts.
  2. Select the host you want to provision and click Provision.
  3. On the host’s Edit page, complete the necessary details, and then click Save.

When the host configuration is saved, Satellite modifies the host’s PXELinux file on the TFTP server and reboots the discovered host. It then boots into an installer for the chosen operating system, and finally into the installed operating system.

If you decide to re-provision an existing discovered host, delete the operating system from the machine and reboot it. The host then reappears on the Discovered Hosts page.

6.4.2. Decommissioning Discovered Hosts

If you no longer require Red Hat Satellite to manage a specific host, you need to decommission that host to prevent it from being discovered.

To Decommission a Discovered Host:

  1. Shut down the host.
  2. Navigate to Hosts > Discovered Hosts.
  3. In the Name column find the host you want to decommission and then select Delete from the corresponding Edit drop-down menu.

6.4.3. Automatically Provisioning Hosts

With Satellite 6.5-beta, it is possible to define provisioning rules that will assign a host group to provisioned hosts and trigger provisioning automatically.

To Create a Provisioning Rule:

  1. Navigate to Configure > Discovery rules.
  2. Click New Rule. Specify the following parameters of the provisioning rule:

    • Name is the name of the rule displayed in the list of rules. This name must not contain spaces or non-alphanumeric characters.
    • Search is the search statement used to match discovered hosts for the particular rule. You can use scoped search syntax to define it. See Section 6.4.4, “Scoped Search Syntax” for examples of using scoped search.
    • Host Group is the host group to be assigned to a matching host before starting the provisioning process. Make sure that the selected host group has all the required parameters set; required parameters are marked with an asterisk (*).
    • Hostname defines a pattern for assigning human-readable host names to the matching hosts. When left blank, the host name is assigned in the format "macMACADDRESS" by default. The same syntax used for provisioning templates is used in this instance. See Section 6.4.5, “Host Name Patterns” for more information and examples.
    • Hosts limit is the maximum number of provisioned hosts per rule. If the limit is reached, the rule will not take effect until one or more hosts are deleted. Typical use cases are rules per server rack or row when it is necessary to change provisioning parameters such as host name or host group per entry. You can set this value to zero (0) to specify no limit.
    • Priority specifies the order of execution of rules. The value must be greater than or equal to zero. A lower value indicates a higher priority. If two rules have the same priority, the first rule encountered is applied.
    • Enabled provides the option to temporarily enable or disable rules.
  3. Click Submit to save the rule.

By default, Satellite does not enable automatic discovery of hosts. The following procedure describes how to enable the Auto provisioning variable to provide automatic provisioning according to specified rules.

To Enable Automatic Provisioning:

  1. Navigate to Administer > Settings > Discovered in the Satellite web UI.
  2. Locate Auto provisioning in the Name column, and set its value to true.
  3. Click Save.

After you have defined some rules, Red Hat recommends that you discover a host and apply the rules using the Auto discover button on the host. This triggers auto-provisioning without the need to enable the global option.

6.4.4. Scoped Search Syntax

This section shows how to use scoped search syntax to filter the discovered hosts according to selected parameters. This is useful when creating a rule for automatic provisioning (see Section 6.4.3, “Automatically Provisioning Hosts”).

The search fields in the Satellite web UI support automatic completion to make building search strings easier. For example, you can test search patterns on the Hosts > Discovered Hosts page. The following are examples of typical search queries:

  • facts.architecture = x86_64
  • facts.bios_vendor ~ 'Dell*'
  • facts.macaddress = "aa:bb:cc:dd:ee:ff"
  • facts.macaddress_eth0 = "aa:bb:cc:dd:ee:ff"
  • facts.ipaddress_eth1 ~ "192.168.*"
  • facts.architecture ^ (x86_64,i386)
Note

The caret symbol (^) in scoped searches means "in" (the same usage as in SQL) and not "starts with" as it is used in regular expressions. You can review the full list of scoped search operators at https://github.com/wvanbergen/scoped_search/blob/master/lib/scoped_search/query_language/tokenizer.rb

In Satellite 6.5-beta, all facts are strings, so it is not possible to do numeric comparisons. However, three important facts are extracted and converted to numbers. These are described in Table 6.1, “Facts that Allow Numerical Comparison”.

Table 6.1. Facts that Allow Numerical Comparison

Search ParameterDescriptionExample Usage

cpu_count

The number of CPUs

cpu_count >= 8

disk_count

The number of disks attached

disk_count < 10

disks_size

The total amount of disk space (in MiB)

disks_size > 1000000

6.4.5. Host Name Patterns

This section lists the host name patterns that you can use when creating a rule for automatic provisioning (see Section 6.4.3, “Automatically Provisioning Hosts”).

The target host name template pattern has the same syntax as the provisioning templates (ERB). The domain is appended automatically. In addition to the @host attribute, the rand() function for random integers is available. For example:

  • application-server-<%= rand(99999) %>
  • load-balancer-<%= @host.facts['bios_vendor'] + '-' + rand(99999) %>
  • wwwsrv-<%= @host.hostgroup.name %>
  • minion-<%= @host.discovery_rule.name %>
  • db-server-<%= @host.ip.gsub('.','-') + '-' + @host.hostgroup.subnet.name %>
Important

When creating host name patterns, ensure the resulting host names are unique. Host names must not start with numbers. A good approach is to use unique information provided by Facter (for example, the MAC address, BIOS or serial ID) or to otherwise randomize the host name.

6.4.6. Using the Discovery Plug-in on the Command Line

You can use the hammer command to perform certain tasks related to discovery. Run the hammer -h command to verify your configuration:

$ hammer -h | grep discovery
discovery                     Manipulate discovered hosts.
discovery_rule                Manipulate discovered rules.

Use the hammer discovery -h command to view the available options. For example, you can use the following command to reboot a discovered host (assuming its ID is 130):

$ hammer discovery reboot -id 130
Host reboot started

6.5. Extending the Discovery Image

It is possible to extend the Satellite Discovery image with custom facts, software, or device drivers. You can also provide a compressed archive file containing extra code for the image to use.

First, create the following directory structure:

.
├── autostart.d
│   └── 01_zip.sh
├── bin
│   └── ntpdate
├── facts
│   └── test.rb
└── lib
    ├── libcrypto.so.1.0.0
    └── ruby
        └── test.rb

Where:

  • The autostart.d directory contains scripts that are executed in POSIX order by the image when it starts, but before the host is registered to Satellite.
  • The bin directory is added to the $PATH variable; you can place binary files here and use them in the autostart scripts.
  • The facts directory is added to the FACTERLIB variable so that custom facts can be configured and sent to Satellite.
  • The lib directory is added to the LD_LIBRARY_PATH variable and lib/ruby is added to the RUBYLIB variable, so that binary files in /bin can be executed correctly.

New directives and options are appended to the existing environment variables (PATH, LD_LIBRARY_PATH, RUBYLIB and FACTERLIB). If you need to specify the path to something explicitly in your scripts, the zip contents are extracted to the /opt/extension directory on the image.

After creating the above directory structure, package it into a zip archive with the following command:

zip -r my_extension.zip .

You can create multiple zip files but be aware they will be extracted to the same place on the Discovery image, so files in later zips will overwrite earlier ones if they have the same file name.

To inform the Discovery image of the extensions it should use, place your zip files on your TFTP server with the Discovery image, and then update the APPEND line of the PXELinux template with the fdi.zips option where the paths are relative to the TFTP root. For example, if you have two archives at $TFTP/zip1.zip and $TFTP/boot/zip2.zip, use the following syntax:

fdi.zips=zip1.zip,boot/zip2.zip

See Section 6.2.2, “Configuring PXE-booting” for more information on updating the PXE template.

6.6. Troubleshooting Satellite Discovery

If a machine is not listed in the Satellite web UI under Hosts > Discovered Hosts, inspect the following configuration areas to help isolate the error:

  • Navigate to Hosts > Provisioning Templates and redeploy the default PXELinux template using the Build PXE Default button.
  • Verify the pxelinux.cfg/default configuration file on the TFTP Capsule Server.
  • Ensure adequate network connectivity between hosts, Capsule Server, and Satellite Server.
  • Check the PXELinux template in use and determine the PXE discovery snippet it includes. Snippets are named as follows: pxelinux_discovery, pxegrub_discovery, or pxegrub2_discovery. Verify the proxy.url and proxy.type options in the PXE discovery snippet.
  • Ensure that DNS is working correctly for the discovered nodes, or use an IP address in the proxy.url option in the PXE discovery snippet included in the PXELinux template you are using.
  • Ensure that the DHCP server is delivering IP addresses to the booted image correctly.
  • Ensure the discovered host or virtual machine has at least 1200 MB of memory. Less memory can lead to various random kernel panic errors as the image needs to be extracted in-memory.

For gathering important system facts, use the discovery-debug command. It prints out system logs, network configuration, list of facts, and other information on the standard output. The typical use case is to redirect this output and copy it with the scp command for further investigation.

The first virtual console on the discovered host is reserved for systemd logs. Particularly useful system logs are tagged as follows:

  • discover-host — initial facts upload
  • foreman-discovery — facts refresh, reboot remote commands
  • nm-prepare — boot script which pre-configures NetworkManager
  • NetworkManager — networking information

Use TTY2 or higher to log in to a discovered host. The root account and SSH access are disabled by default, but you can enable SSH and set the root password using the following kernel command-line options in the Default PXELinux template on the APPEND line:

fdi.ssh=1 fdi.rootpw=redhat