How can I diagnose the cause of scsi reservation conflicts in a RHEL cluster using fence_scsi?

Solution Verified - Updated -

Issue

  • Cluster node logs "scsi reservation conflict" during bootup and shared storage is inaccessible or GFS/GFS2 file system withdraws:
kernel: sd 4:0:0:9: reservation conflict
kernel: sd 2:0:0:9: reservation conflict
kernel: sd 2:0:0:9: reservation conflict
kernel: sd 4:0:0:9: reservation conflict
kernel: sd 4:0:0:9: reservation conflict
kernel: sd 2:0:0:9: reservation conflict
kernel: sd 2:0:0:9: reservation conflict
kernel: GFS: fsid=isgruapp7n-prod:amgprod_gfs.0: fatal: I/O error
kernel: GFS: fsid=isgruapp7n-prod:amgprod_gfs.0:   block = 86382275
kernel: GFS: fsid=isgruapp7n-prod:amgprod_gfs.0:   function = gfs_logbh_wait
kernel: GFS: fsid=isgruapp7n-prod:amgprod_gfs.0:   file = /builddir/build/BUILD/gfs-kmod-0.1.34/_kmod_build_/src/gfs/dio.c, line = 816
kernel: GFS: fsid=isgruapp7n-prod:amgprod_gfs.0:   time = 1359814274
kernel: GFS: fsid=isgruapp7n-prod:amgprod_gfs.0: about to withdraw from the cluster
kernel: GFS: fsid=isgruapp7n-prod:amgprod_gfs.0: telling LM to withdraw
kernel: GFS: fsid=isgruapp7n-prod:amgprod_gfs.0: withdrawn
  • The cluster nodes are reporting reservation conflicts errors:
kernel: sd 2:0:5:0: reservation conflict
kernel: sd 2:0:5:0: SCSI error: return code = 0x00000018
kernel: end_request: I/O error, dev sdq, sector 423224328
kernel: device-mapper: multipath: Failing path 65:0.
kernel: sd 2:0:5:0: reservation conflict
kernel: sd 2:0:5:0: SCSI error: return code = 0x00000018
kernel: end_request: I/O error, dev sdq, sector 423224336
  • GFS mount issue on 2 node RH cluster. Mounting GFS filesystems:
/sbin/mount.gfs: error mounting /dev/mapper/SMC_VG-smcdata on /isg/smc: No such file or directory

Environment

  • Red Hat Enterprise Linux (RHEL); including
    • Red Hat Enterprise Linux 5 (with the High Availability or Resilient Storage Add On)
    • Red Hat Enterprise Linux 6 (with the High Availability or Resilient Storage Add On)
  • Red Hat High Availability Cluster with 2 or more nodes.
  • One or more cluster nodes configured to use fence_scsi in /etc/cluster/cluster.conf:
$ grep fence_scsi /etc/cluster/cluster.conf
           <fencedevice agent="fence_scsi" name="myfencing"/>

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content