FTP client running on a mainframe fails when connecting via SSL

Solution Verified - Updated -

Environment

  • Red Hat Enterprise Linux (RHEL) 7
  • Red Hat Enterprise Linux (RHEL) 6
  • Red Hat Enterprise Linux (RHEL) 5
  • Red Hat Enterprise Linux (RHEL) 4
  • vsftpd
  • FTP client on an IBM mainframe zOS

Issue

  • An FTP client running on a mainframe fails to connect to a vsftpd server via SSL. The client replies "410 (SSL message format is incorrect)".
  • An FTP client running on a mainframe fails to connect to a vsftpd server via SSL. The client replies "SSL connection failed; session reuse required; see require_ssl_reuse option"

Resolution

  1. The issue is caused by SSL incompatibility between zOS and Linux implementations. The issue can be fixed by switching the following vsftpd option to no. Per man vsftpd.conf

    ssl_request_cert
          If  enabled,  vsftpd  will  request  (but  not  necessarily  require; see require_cert) a certificate on
          incoming SSL connections. Normally this should not cause  any trouble  at  all,  but IBM zOS seems to have
          issues.  (New in v2.0.7).

          Default: YES

    This option has been backported in RHEL5 with vsftpd-2.0.5-23

  2. Unfortunately, the issue can persist even after this configuration change. The client might still fail while getting a file, returning the following message:

    522 SSL connection failed; session reuse required; see require_ssl_reuse option

    In that situation we may want to disable also that require_ssl_reuse option:

    require_ssl_reuse
      If set to yes, all SSL data connections are required to exhibit SSL session reuse (which proves that they know the same 
      master secret as the  control  channel).  Although  this  is a secure default, it may break many FTP clients, so you may 
      want to disable it. For a discussion of the consequences, see http://scarybeastsecurity.blogspot.com/2009/02/vsftpd-210-released.html (Added in v2.1.0).

      Default: YES

    This option is available from RHEL6 with vsftpd-2.2.2

  • Examples of the two above parameters to set in /etc/vsftpd/vsftpd.conf:

    ssl_request_cert=no
require_ssl_reuse=no

Root Cause

The described issue is caused by incompatibility of OpenSSL implementations between the IBM mainframe zOS ftp client and Linux implementations.

See also the following references:

Diagnostic Steps

  • Mainframe FTP clients fail when trying to create an SSL session with a vsftpd server running on a RHEL system. The log of the FTP client looks like the following snippet:

    FC0914 authServer: secure_socket_open()
FC0981 authServer: secure_socket_init()
FC0994 authServer: secure_socket_init failed with rc = 410 (SSL message format is incorrect)
FC1349 endSecureConn: entered
EZA2897I Authentication negotiation failed
FC1381 endSecureEnv: entered
CZ0655 SETCEC code = 17
EZA2898I Unable to successfully negotiate required authentication
EZA1460I Command:
EZA1618I Unknown command: 'testftps'
EZA1619I For a list of the available commands, say HELP
EZA1460I Command:
EZA1736I popkin22
EZA1618I Unknown command: 'popkin22'

  • This issue on the client side causes exiting vsftpd child processes handling the mainframe FTP client with the following message, which can be found in the vsftpd strace log:

    12213 write(0, "500 OOPS: ", 10)        = 10
12213 write(0, "error:14094417:SSL routines:SSL3"..., 73) = 73

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments