Red Hat Ceph Cluster 5 Error EINVAL: Can't communicate with remote host 127.0.0.1, possibly because python3 is not installed there: cannot send (already closed?)

Solution Verified - Updated -

Environment

  • Red Hat Ceph Storage 5

Issue

  • adding a host to the cluster through Red Hat Ceph Storage Orchestrator fails with

    Error EINVAL: Can't communicate with remote host 127.0.0.1, possibly because python3 is not installed there: cannot send (already closed?)

Resolution

  • ensure that the remote host is properly configured with passwordless sudo

    $ export CEPHUSER=cephorch
$ echo "${CEPHUSER} ALL=(ALL) NOPASSWD: ALL" > /etc/sudoers.d/cephorch.conf

  • verify the sudo configuration

    $ export CEPHUSER=cephorch
$ runuser -u ${CEPHUSER} -- /usr/bin/sudo -l
Matching Defaults entries for cephorch on node1.example.com:
    !visiblepw, always_set_home, match_group_by_gid, always_query_group_plugin, env_reset, env_keep="COLORS DISPLAY HOSTNAME HISTSIZE KDEDIR
    LS_COLORS", env_keep+="MAIL PS1 PS2 QTDIR USERNAME LANG LC_ADDRESS LC_CTYPE", env_keep+="LC_COLLATE LC_IDENTIFICATION LC_MEASUREMENT
    LC_MESSAGES", env_keep+="LC_MONETARY LC_NAME LC_NUMERIC LC_PAPER LC_TELEPHONE", env_keep+="LC_TIME LC_ALL LANGUAGE LINGUAS     _XKB_CHARSET
    XAUTHORITY", secure_path=/sbin\:/bin\:/usr/sbin\:/usr/bin

User cephorch may run the following commands on node1.example.com:
    (ALL) NOPASSWD: ALL

Root Cause

  • handling of password prompt with sudo in Red Hat Ceph Storage orchestrator isn't supported
  • Orchestrator ssh engine execnet closes the I/O accordingly and returns the generic Exception error message for closed I/O handles.

Diagnostic Steps

  • login to the host to be added

  • verify the sudo configuration

    $ export CEPHUSER=cephorch
$ runuser -u ${CEPHUSER} -- /usr/bin/sudo -l
Matching Defaults entries for cephorch on node1.example.com:
    !visiblepw, always_set_home, match_group_by_gid, always_query_group_plugin, env_reset, env_keep="COLORS DISPLAY HOSTNAME HISTSIZE KDEDIR
    LS_COLORS", env_keep+="MAIL PS1 PS2 QTDIR USERNAME LANG LC_ADDRESS LC_CTYPE", env_keep+="LC_COLLATE LC_IDENTIFICATION LC_MEASUREMENT
    LC_MESSAGES", env_keep+="LC_MONETARY LC_NAME LC_NUMERIC LC_PAPER LC_TELEPHONE", env_keep+="LC_TIME LC_ALL LANGUAGE LINGUAS     _XKB_CHARSET
    XAUTHORITY", secure_path=/sbin\:/bin\:/usr/sbin\:/usr/bin

User cephorch may run the following commands on node1.example.com:
    (ALL) NOPASSWD: ALL
  • ensure NOPASSWD is configured for the user

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments