Chapter 1. Using secure communications between two systems with OpenSSH

SSH (Secure Shell) is a protocol which provides secure communications between two systems using a client-server architecture and allows users to log in to server host systems remotely. Unlike other remote communication protocols, such as FTP or Telnet, SSH encrypts the login session, which prevents intruders to collect unencrypted passwords from the connection.

Red Hat Enterprise Linux includes the basic OpenSSH packages: the general openssh package, the openssh-server package and the openssh-clients package. Note that the OpenSSH packages require the OpenSSL package openssl-libs, which installs several important cryptographic libraries that enable OpenSSH to provide encrypted communications.

1.1. SSH and OpenSSH

SSH (Secure Shell) is a program for logging into a remote machine and executing commands on that machine. The SSH protocol provides secure encrypted communications between two untrusted hosts over an insecure network. You can also forward X11 connections and arbitrary TCP/IP ports over the secure channel.

The SSH protocol mitigates security threats, such as interception of communication between two systems and impersonation of a particular host, when you use it for remote shell login or file copying. This is because the SSH client and server use digital signatures to verify their identities. Additionally, all communication between the client and server systems is encrypted.

OpenSSH is an implementation of the SSH protocol supported by a number of Linux, UNIX, and similar operating systems. It includes the core files necessary for both the OpenSSH client and server. The OpenSSH suite consists of the following user-space tools:

  • ssh is a remote login program (SSH client)
  • sshd is an OpenSSH SSH daemon
  • scp is a secure remote file copy program
  • sftp is a secure file transfer program
  • ssh-agent is an authentication agent for caching private keys
  • ssh-add adds private key identities to ssh-agent
  • ssh-keygen generates, manages, and converts authentication keys for ssh
  • ssh-copy-id is a script that adds local public keys to the authorized_keys file on a remote SSH server
  • ssh-keyscan - gathers SSH public host keys

Two versions of SSH currently exist: version 1, and the newer version 2. The OpenSSH suite in Red Hat Enterprise Linux 8 supports only SSH version 2, which has an enhanced key-exchange algorithm not vulnerable to known exploits in version 1.

OpenSSH, as one of the RHEL core cryptographic subsystems uses system-wide crypto policies. This ensures that weak cipher suites and cryptographic algorithms are disabled in the default configuration. To adjust the policy, the administrator must either use the update-crypto-policies command to make settings stricter or looser or manually opt-out of the system-wide crypto policies.

The OpenSSH suite uses two different sets of configuration files: those for client programs (that is, ssh, scp, and sftp), and those for the server (the sshd daemon). System-wide SSH configuration information is stored in the /etc/ssh/ directory. User-specific SSH configuration information is stored in ~/.ssh/ in the user’s home directory. For a detailed list of OpenSSH configuration files, see the FILES section in the sshd(8) man page.

Additional resources

1.2. Configuring and starting an OpenSSH server

Use the following procedure for a basic configuration that might be required for your environment and for starting an OpenSSH server. Note that after the default RHEL installation, the sshd daemon is already started and server host keys are automatically created.

Prerequisites

  • The openssh-server package is installed.

Procedure

  1. Start the sshd daemon in the current session and set it to start automatically at boot time:

    # systemctl start sshd
    # systemctl enable sshd
  2. To specify different addresses than the default 0.0.0.0 (IPv4) or :: (IPv6) for the ListenAddress directive in the /etc/ssh/sshd_config configuration file and to use a slower dynamic network configuration, add the dependency on the network-online.target target unit to the sshd.service unit file. To achieve this, create the /etc/systemd/system/sshd.service.d/local.conf file with the following content:

    [Unit]
    Wants=network-online.target
    After=network-online.target
  3. Review if OpenSSH server settings in the /etc/ssh/sshd_config configuration file meet the requirements of your scenario.
  4. Optionally, change the welcome message that your OpenSSH server displays before a client authenticates by editing the /etc/issue file, for example:

    Welcome to ssh-server.example.com
    Warning: By accessing this server, you agree to the referenced terms and conditions.

    Note that to change the message displayed after a successful login you have to edit the /etc/motd file on the server. See the pam_motd man page for more information.

  5. Reload the systemd configuration to apply the changes:

    # systemctl daemon-reload

Verification steps

  1. Check that the sshd daemon is running:

    # systemctl status sshd
    ● sshd.service - OpenSSH server daemon
       Loaded: loaded (/usr/lib/systemd/system/sshd.service; enabled; vendor preset: enabled)
       Active: active (running) since Mon 2019-11-18 14:59:58 CET; 6min ago
         Docs: man:sshd(8)
               man:sshd_config(5)
     Main PID: 1149 (sshd)
        Tasks: 1 (limit: 11491)
       Memory: 1.9M
       CGroup: /system.slice/sshd.service
               └─1149 /usr/sbin/sshd -D -oCiphers=aes128-ctr,aes256-ctr,aes128-cbc,aes256-cbc -oMACs=hmac-sha2-256,>
    
    Nov 18 14:59:58 ssh-server-example.com systemd[1]: Starting OpenSSH server daemon...
    Nov 18 14:59:58 ssh-server-example.com sshd[1149]: Server listening on 0.0.0.0 port 22.
    Nov 18 14:59:58 ssh-server-example.com sshd[1149]: Server listening on :: port 22.
    Nov 18 14:59:58 ssh-server-example.com systemd[1]: Started OpenSSH server daemon.
  2. Connect to the SSH server with an SSH client.

    # ssh user@ssh-server-example.com
    ECDSA key fingerprint is SHA256:dXbaS0RG/UzlTTku8GtXSz0S1++lPegSy31v3L/FAEc.
    Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
    Warning: Permanently added 'ssh-server-example.com' (ECDSA) to the list of known hosts.
    
    user@ssh-server-example.com's password:

Additional resources

  • sshd(8) and sshd_config(5) man pages

1.3. Using key pairs instead of passwords for SSH authentication

To improve system security even further, generate SSH key pairs and then enforce key-based authentication by disabling password authentication.

1.3.1. Setting an OpenSSH server for key-based authentication

Follow these steps to configure your OpenSSH server for enforcing key-based authentication.

Prerequisites

  • The openssh-server package is installed.
  • The sshd daemon is running on the server.

Procedure

  1. Open the /etc/ssh/sshd_config configuration in a text editor, for example:

    # vi /etc/ssh/sshd_config
  2. Change the PasswordAuthentication option to no:

    PasswordAuthentication no

    On a system other than a new default installation, check that PubkeyAuthentication no has not been set and the ChallengeResponseAuthentication directive is set to no. If you are connected remotely, not using console or out-of-band access, test the key-based login process before disabling password authentication.

  3. To use key-based authentication with NFS-mounted home directories, enable the use_nfs_home_dirs SELinux boolean:

    # setsebool -P use_nfs_home_dirs 1
  4. Reload the sshd daemon to apply the changes:

    # systemctl reload sshd

Additional resources

  • sshd(8), sshd_config(5), and setsebool(8) man pages

1.3.2. Generating SSH key pairs

Use this procedure to generate an SSH key pair on a local system and to copy the generated public key to an OpenSSH server. If the server is configured accordingly, you can log in to the OpenSSH server without providing any password.

Important

If you complete the following steps as root, only root is able to use the keys.

Procedure

  1. To generate an ECDSA key pair for version 2 of the SSH protocol:

    $ ssh-keygen -t ecdsa
    Generating public/private ecdsa key pair.
    Enter file in which to save the key (/home/joesec/.ssh/id_ecdsa):
    Enter passphrase (empty for no passphrase):
    Enter same passphrase again:
    Your identification has been saved in /home/joesec/.ssh/id_ecdsa.
    Your public key has been saved in /home/joesec/.ssh/id_ecdsa.pub.
    The key fingerprint is:
    SHA256:Q/x+qms4j7PCQ0qFd09iZEFHA+SqwBKRNaU72oZfaCI joesec@localhost.example.com
    The key's randomart image is:
    +---[ECDSA 256]---+
    |.oo..o=++        |
    |.. o .oo .       |
    |. .. o. o        |
    |....o.+...       |
    |o.oo.o +S .      |
    |.=.+.   .o       |
    |E.*+.  .  . .    |
    |.=..+ +..  o     |
    |  .  oo*+o.      |
    +----[SHA256]-----+

    You can also generate an RSA key pair by using the -t rsa option with the ssh-keygen command or an Ed25519 key pair by entering the ssh-keygen -t ed25519 command.

  2. To copy the public key to a remote machine:

    $ ssh-copy-id joesec@ssh-server-example.com
    /usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
    ...
    Number of key(s) added: 1
    
    Now try logging into the machine, with: "ssh 'joesec@ssh-server-example.com'" and check to make sure that only the key(s) you wanted were added.

    If you do not use the ssh-agent program in your session, the previous command copies the most recently modified ~/.ssh/id*.pub public key if it is not yet installed. To specify another public-key file or to prioritize keys in files over keys cached in memory by ssh-agent, use the ssh-copy-id command with the -i option.

Note

If you reinstall your system and want to keep previously generated key pairs, back up the ~/.ssh/ directory. After reinstalling, copy it back to your home directory. You can do this for all users on your system, including root.

Verification steps

  1. Log in to the OpenSSH server without providing any password:

    $ ssh joesec@ssh-server-example.com
    Welcome message.
    ...
    Last login: Mon Nov 18 18:28:42 2019 from ::1

Additional resources

  • ssh-keygen(1) and ssh-copy-id(1) man pages

1.4. Using SSH keys stored on a smart card

Red Hat Enterprise Linux 8 enables you to use RSA and ECDSA keys stored on a smart card on OpenSSH clients. Use this procedure to enable authentication using a smart card instead of using a password.

Prerequisites

  • On the client side, the opensc package is installed and the pcscd service is running.

Procedure

  1. List all keys provided by the OpenSC PKCS #11 module including their PKCS #11 URIs and save the output to the keys.pub file:

    $ ssh-keygen -D pkcs11: > keys.pub
    $ ssh-keygen -D pkcs11:
    ssh-rsa AAAAB3NzaC1yc2E...KKZMzcQZzx pkcs11:id=%02;object=SIGN%20pubkey;token=SSH%20key;manufacturer=piv_II?module-path=/usr/lib64/pkcs11/opensc-pkcs11.so
    ecdsa-sha2-nistp256 AAA...J0hkYnnsM= pkcs11:id=%01;object=PIV%20AUTH%20pubkey;token=SSH%20key;manufacturer=piv_II?module-path=/usr/lib64/pkcs11/opensc-pkcs11.so
  2. To enable authentication using a smart card on a remote server (example.com), transfer the public key to the remote server. Use the ssh-copy-id command with keys.pub created in the previous step:

    $ ssh-copy-id -f -i keys.pub username@example.com
  3. To connect to example.com using the ECDSA key from the output of the ssh-keygen -D command in step 1, you can use just a subset of the URI, which uniquely references your key, for example:

    $ ssh -i "pkcs11:id=%01?module-path=/usr/lib64/pkcs11/opensc-pkcs11.so" example.com
    Enter PIN for 'SSH key':
    [example.com] $
  4. You can use the same URI string in the ~/.ssh/config file to make the configuration permanent:

    $ cat ~/.ssh/config
    IdentityFile "pkcs11:id=%01?module-path=/usr/lib64/pkcs11/opensc-pkcs11.so"
    $ ssh example.com
    Enter PIN for 'SSH key':
    [example.com] $

    Because OpenSSH uses the p11-kit-proxy wrapper and the OpenSC PKCS #11 module is registered to PKCS#11 Kit, you can simplify the previous commands:

    $ ssh -i "pkcs11:id=%01" example.com
    Enter PIN for 'SSH key':
    [example.com] $

If you skip the id= part of a PKCS #11 URI, OpenSSH loads all keys that are available in the proxy module. This can reduce the amount of typing required:

$ ssh -i pkcs11: example.com
Enter PIN for 'SSH key':
[example.com] $

Additional resources

1.5. Making OpenSSH more secure

The following tips help you to increase security when using OpenSSH. Note that changes in the /etc/ssh/sshd_config OpenSSH configuration file require reloading the sshd daemon to take effect:

# systemctl reload sshd
Important

The majority of security hardening configuration changes reduce compatibility with clients that do not support up-to-date algorithms or cipher suites.

Disabling insecure connection protocols

  • To make SSH truly effective, prevent the use of insecure connection protocols that are replaced by the OpenSSH suite. Otherwise, a user’s password might be protected using SSH for one session only to be captured later when logging in using Telnet. For this reason, consider disabling insecure protocols, such as telnet, rsh, rlogin, and ftp.

Enabling key-based authentication and disabling password-based authentication

  • Disabling passwords for authentication and allowing only key pairs reduces the attack surface and it also might save users’ time. On clients, generate key pairs using the ssh-keygen tool and use the ssh-copy-id utility to copy public keys from clients on the OpenSSH server. To disable password-based authentication on your OpenSSH server, edit /etc/ssh/sshd_config and change the PasswordAuthentication option to no:

    PasswordAuthentication no

Key types

  • Although the ssh-keygen command generates a pair of RSA keys by default, you can instruct it to generate ECDSA or Ed25519 keys by using the -t option. The ECDSA (Elliptic Curve Digital Signature Algorithm) offers better performance than RSA at the equivalent symmetric key strength. It also generates shorter keys. The Ed25519 public-key algorithm is an implementation of twisted Edwards curves that is more secure and also faster than RSA, DSA, and ECDSA.

    OpenSSH creates RSA, ECDSA, and Ed25519 server host keys automatically if they are missing. To configure the host key creation in RHEL 8, use the sshd-keygen@.service instantiated service. For example, to disable the automatic creation of the RSA key type:

    # systemctl mask sshd-keygen@rsa.service
  • To exclude particular key types for SSH connections, comment out the relevant lines in /etc/ssh/sshd_config, and reload the sshd service. For example, to allow only Ed25519 host keys:
# HostKey /etc/ssh/ssh_host_rsa_key
# HostKey /etc/ssh/ssh_host_ecdsa_key
HostKey /etc/ssh/ssh_host_ed25519_key

Non-default port

  • By default, the sshd daemon listens on TCP port 22. Changing the port reduces the exposure of the system to attacks based on automated network scanning and thus increase security through obscurity. You can specify the port using the Port directive in the /etc/ssh/sshd_config configuration file.

    You also have to update the default SELinux policy to allow the use of a non-default port. To do so, use the semanage tool from the policycoreutils-python-utils package:

    # semanage port -a -t ssh_port_t -p tcp port_number

    Furthermore, update firewalld configuration:

    # firewall-cmd --add-port port_number/tcp
    # firewall-cmd --runtime-to-permanent

    In the previous commands, replace port_number with the new port number specified using the Port directive.

No root login

  • If your particular use case does not require the possibility of logging in as the root user, you should consider setting the PermitRootLogin configuration directive to no in the /etc/ssh/sshd_config file. By disabling the possibility of logging in as the root user, the administrator can audit which users run what privileged commands after they log in as regular users and then gain root rights.

    Alternatively, set PermitRootLogin to prohibit-password:

    PermitRootLogin prohibit-password

    This enforces the use of key-based authentication instead of the use of passwords for logging in as root and reduces risks by preventing brute-force attacks.

Using the ⁠X Security extension

  • The X server in Red Hat Enterprise Linux clients does not provide the X Security extension. Therefore, clients cannot request another security layer when connecting to untrusted SSH servers with X11 forwarding. Most applications are not able to run with this extension enabled anyway.

    By default, the ForwardX11Trusted option in the /etc/ssh/ssh_config.d/05-redhat.conf file is set to yes, and there is no difference between the ssh -X remote_machine (untrusted host) and ssh -Y remote_machine (trusted host) command.

    If your scenario does not require the X11 forwarding feature at all, set the X11Forwarding directive in the /etc/ssh/sshd_config configuration file to no.

Restricting access to specific users, groups, or domains

  • The AllowUsers and AllowGroups directives in the /etc/ssh/sshd_config configuration file server enable you to permit only certain users, domains, or groups to connect to your OpenSSH server. You can combine AllowUsers and AllowGroups to restrict access more precisely, for example:

    AllowUsers *@192.168.1.*,*@10.0.0.*,!*@192.168.1.2
    AllowGroups example-group

    The previous configuration lines accept connections from all users from systems in 192.168.1.* and 10.0.0.* subnets except from the system with the 192.168.1.2 address. All users must be in the example-group group. The OpenSSH server denies all other connections.

    Note that using whitelists (directives starting with Allow) is more secure than using blacklists (options starting with Deny) because whitelists block also new unauthorized users or groups.

Changing system-wide cryptographic policies

  • OpenSSH uses RHEL system-wide cryptographic policies, and the default system-wide cryptographic policy level offers secure settings for current threat models. To make your cryptographic settings more strict, change the current policy level:

    # update-crypto-policies --set FUTURE
    Setting system policy to FUTURE
  • To opt-out of the system-wide crypto policies for your OpenSSH server, uncomment the line with the CRYPTO_POLICY= variable in the /etc/sysconfig/sshd file. After this change, values that you specify in the Ciphers, MACs, KexAlgoritms, and GSSAPIKexAlgorithms sections in the /etc/ssh/sshd_config file are not overridden. Note that this task requires deep expertise in configuring cryptographic options.
  • See Using system-wide cryptographic policies in the RHEL 8 Security hardening title for more information.

Additional resources

  • sshd_config(5), ssh-keygen(1), crypto-policies(7), and update-crypto-policies(8) man pages

1.6. Connecting to a remote server using an SSH jump host

Use this procedure for connecting to a remote server through an intermediary server, also called jump host.

Prerequisites

  • A jump host accepts SSH connections from your system.
  • A remote server accepts SSH connections only from the jump host.

Procedure

  1. Define the jump host by editing the ~/.ssh/config file, for example:

    Host jump-server1
      HostName jump1.example.com
  2. Add the remote server jump configuration with the ProxyJump directive to ~/.ssh/config, for example:

    Host remote-server
      HostName remote1.example.com
      ProxyJump jump-server1
  3. Connect to the remote server through the jump server:

    $ ssh remote-server

    The previous command is equivalent to the ssh -J jump-server1 remote-server command if you omit the configuration steps 1 and 2.

Note

You can specify more jump servers and you can also skip adding host definitions to the configurations file when you provide their complete host names, for example:

$ ssh -J jump1.example.com,jump2.example.com,jump3.example.com remote1.example.com

Change the host name-only notation in the previous command if the user names or SSH ports on the jump servers differ from the names and ports on the remote server, for example:

$ ssh -J johndoe@jump1.example.com:75,johndoe@jump2.example.com:75,johndoe@jump3.example.com:75 joesec@remote1.example.com:220

Additional resources

  • ssh_config(5) and ssh(1) man pages

1.7. Additional resources

For more information on configuring and connecting to OpenSSH servers and clients on Red Hat Enterprise Linux, see the resources listed below.

Installed documentation

  • sshd(8) man page documents available command-line options and provides a complete list of supported configuration files and directories.
  • ssh(1) man page provides a complete list of available command-line options and supported configuration files and directories.
  • scp(1) man page provides a more detailed description of the scp utility and its usage.
  • sftp(1) man page provides a more detailed description of the sftp utility and its usage.
  • ssh-keygen(1) man page documents in detail the use of the ssh-keygen utility to generate, manage, and convert authentication keys used by ssh.
  • ssh-copy-id(1) man page describes the use of the ssh-copy-id script.
  • ssh_config(5) man page documents available SSH client configuration options.
  • sshd_config(5) man page provides a full description of available SSH daemon configuration options.
  • update-crypto-policies(8) man page provides guidance on managing system-wide cryptographic policies
  • crypto-policies(7) man page provides an overview of system-wide cryptographic policy levels

Online documentation