kernel: nfs: server not responding, still trying

Solution In Progress - Updated -

Environment

  • Red Hat Enterprise Linux 6.3

Issue

  • NFS client issue occurs on all servers.
kernel: nfs: server not responding, still trying

Root Cause

From the RPC debug found below errors.

Sep 13 15:45:06 node1 kernel: RPC:       xs_tcp_send_request(120) = -110
Sep 13 15:45:06 node1 kernel: RPC:       sendmsg returned unrecognized error 110
Sep 13 15:45:06 node1 kernel: RPC:       xs_tcp_state_change client ffff88101728a000...
Sep 13 15:45:06 node1 kernel: RPC:       state 7 conn 0 dead 0 zapped 1 sk_shutdown 3
Sep 13 15:45:06 node1 kernel: RPC:       disconnected transport ffff88101728a000
Sep 13 15:45:06 node1 kernel: RPC:  3826 call_transmit_status (status -110)
Sep 13 15:45:06 node1 kernel: RPC:       wake_up_next(ffff88101728a190 "xprt_sending")
Sep 13 15:45:06 node1 kernel: RPC:  3827 __rpc_wake_up_task (now 5669699467)
Sep 13 15:45:06 node1 kernel: RPC:  3827 removed from queue ffff88101728a190 "xprt_sending"
Sep 13 15:45:06 node1 kernel: RPC:       __rpc_wake_up_task done
Sep 13 15:45:06 node1 kernel: RPC:  3826 call_status (status -110)

error 110 is connection timeout from #define ETIMEDOUT 110 /* Connection timed out */

Also we can see error 11

Sep 13 15:45:06 node1 kernel: RPC:  3827 call_connect_status (status -11) which is try again.

Above errors are indicating network connectivity following which we have RPC time out and retransmissions.

Diagnostic Steps

Output of rpcinfo -p when the issue happens.

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments