Cluster fails to fence a node with fence_scsi and logs report "Failed: Cannot open file '/var/run/cluster/fence_scsi.key'" in a RHEL 7 Update 1 High Availability cluster

Solution Unverified - Updated -

Issue

  • If I reboot a node, after it rejoins then if it needs to fence another node it fails with an error in /var/log/messages saying it couldn't open /var/run/cluster/fence_scsi.key
Raw
Sep 15 14:22:08 rhel7-node1 stonith-ng[2950]: error: log_operation: Operation 'reboot' [3074] (call 2 from stonith_admin.3047) for host 'rhel7-node2.example.com' with device 'scsi' returned: -201 (Generic Pacemaker error)
Sep 15 14:22:08 rhel7-node1 stonith-ng[2950]: warning: log_operation: scsi:3074 [ WARNING:root:Parse error: Ignoring unknown option 'port=rhel7-node2.example.com' ]
Sep 15 14:22:08 rhel7-node1 stonith-ng[2950]: warning: log_operation: scsi:3074 [  ]
Sep 15 14:22:08 rhel7-node1 stonith-ng[2950]: warning: log_operation: scsi:3074 [ ERROR:root:Failed: Cannot open file "/var/run/cluster/fence_scsi.key" ]
Sep 15 14:22:08 rhel7-node1 stonith-ng[2950]: warning: log_operation: scsi:3074 [  ]
Sep 15 14:22:08 rhel7-node1 stonith-ng[2950]: warning: log_operation: scsi:3074 [ Failed: Cannot open file "/var/run/cluster/fence_scsi.key" ]
Sep 15 14:22:08 rhel7-node1 stonith-ng[2950]: warning: log_operation: scsi:3074 [  ]
Sep 15 14:22:08 rhel7-node1 stonith-ng[2950]: warning: log_operation: scsi:3074 [ ERROR:root:Please use '-h' for usage ]
Sep 15 14:22:08 rhel7-node1 stonith-ng[2950]: warning: log_operation: scsi:3074 [  ]
Sep 15 14:22:08 rhel7-node1 stonith-ng[2950]: warning: log_operation: scsi:3074 [ Please use '-h' for usage ]
Sep 15 14:22:08 rhel7-node1 stonith-ng[2950]: warning: log_operation: scsi:3074 [  ]
Sep 15 14:22:09 rhel7-node1 stonith-ng[2950]: notice: stonith_choose_peer: Couldn't find anyone to fence rhel7-node2.example.com with <any>
Sep 15 14:22:09 rhel7-node1 stonith-ng[2950]: error: remote_op_done: Operation reboot of rhel7-node2.example.com by <no-one> for stonith_admin.3047@rhel7-node1.example.com.49e1b746: No route to host
  • Unfencing isn't creating files /var/run/cluster/fence_scsi.key and /var/run/cluster/fence_scsi.dev
  • My cluster can't fence
  • I'm using fence_scsi and pcs stonith fence returns no route to host, but only after I've rebooted a node and rejoined it to the cluster
Raw
# pcs stonith fence rhel7-node2.example.com
Error: unable to fence 'rhel7-node2.example.com'
Command failed: No route to host

Environment

  • Red Hat Enterprise Linux (RHEL) 7 with the High Availability Add On
  • fence-agents-scsi releases starting with 4.0.11-11.el7_.1 up to (but not including) 4.0.11-13.el7_1.2
  • One or more stonith devices configured to use fence_scsi

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content