Why only volumes attached into SAN are unmounted after reboot in RHEL 7.1 ?
Issue
Since updating firmware of HP, only volume attached into SAN are unmounted
when system reboot. it was tested twice. but, same symptoms happened. Fortunately, these volumes
are mounted well using "mount -a" command after system booted normally.
- Time to be rebooted
Jan 8 17:52:17 localhost rsyslogd: [origin software="rsyslogd"
swVersion="7.4.7" x-pid="13785" x-info="http://www.rsyslog.com"] exiting on
signal 15.
Jan 8 18:06:00 localhost rsyslogd: [origin software="rsyslogd"
swVersion="7.4.7" x-pid="6987" x-info="http://www.rsyslog.com"] start
Jan 8 18:05:43 localhost journal: Runtime journal is using 8.0M (max allowed
4.0G, trying to leave 4.0G free of 251.7G available 창<86><92> current limit
4.0G).
Jan 8 18:05:43 localhost kernel: Initializing cgroup subsys cpuset
Jan 8 18:05:43 localhost kernel: Initializing cgroup subsys cpu
Jan 8 18:05:43 localhost kernel: Initializing cgroup subsys cpuacct
- Time to be unmounted unexpectedly
Jan 8 18:05:56 localhost systemd: Starting Import network configuration from
initramfs...
Jan 8 18:05:56 localhost systemd: Unmounting /unify...
Jan 8 18:05:56 localhost systemd: Unmounting /sise_bt...
Jan 8 18:05:56 localhost systemd: Mounting /PATROL...
Jan 8 18:05:56 localhost systemd: Unmounting /test_bt...
Jan 8 18:05:56 localhost systemd: Unmounting /data_bt...
Jan 8 18:05:56 localhost systemd: var.mount: Directory /var to mount over is
not empty, mounting anyway.
Jan 8 18:05:56 localhost systemd: Mounting /var...
Jan 8 18:05:56 localhost kernel: XFS (dm-14): Mounting V4 Filesystem
- It is doubt following log. it looks multipathd process could be issue or bug. because only volumes attached into SAN unmounted unexpectedly.
Jan 8 18:05:53 localhost systemd: PID 3296 read from file
/var/run/multipathd/multipathd.pid does not exist or is a zombie.
- Also, there could be problematical logs regarding LVM and Filesystem as below.
Jan 8 18:05:53 localhost lvm: Found duplicate PV
vs3W2S3N2GUurlgASt9FAPbo3sfzdwZv: using /dev/sdd not /dev/sdbm
Jan 8 18:05:55 localhost kernel: device-mapper: table: 253:85: linear:
dm-linear: Device lookup failed
Jan 8 18:05:55 localhost lvm: Internal error: Performing unsafe table load
while 1 device(s) are known to be suspended: (253:85)
- From the following log, we checked /var partition was mounter later than volumes of SAN. it could be a problem that multipathd.pid is not writable.
Jan 8 18:05:53 localhost systemd: PID 3296 read from file
/var/run/multipathd/multipathd.pid does not exist or is a zombie.
Jan 8 18:05:56 localhost systemd: Unmounting /unify...
Jan 8 18:05:56 localhost systemd: Unmounting /sise_bt...
Jan 8 18:05:56 localhost systemd: Mounting /PATROL...
Jan 8 18:05:56 localhost systemd: Unmounting /test_bt...
Jan 8 18:05:56 localhost systemd: Unmounting /data_bt...
Jan 8 18:05:56 localhost systemd: var.mount: Directory /var to mount over is
not empty, mounting anyway.
Jan 8 18:05:56 localhost systemd: Mounting /var...
- As the result that we checked all logs from 2015 May 29 14:03:54 to 2016 Jan 8 19:22:37, the period of time that issue occurred is when kernel was updated from RHEL 7.1 to RHEL 7.2.
messages:Jan 8 18:05:53 localhost systemd: PID 3296 read from file
/var/run/multipathd/multipathd.pid does not exist or is a zombie.
messages-20160101:Dec 9 17:06:10 localhost systemd: PID 3289 read from file
/var/run/multipathd/multipathd.pid does not exist or is a zombie.
messages-20160101:Dec 9 17:22:40 localhost systemd: PID 3291 read from file
/var/run/multipathd/multipathd.pid does not exist or is a zombie.
messages-20160101:Dec 9 17:28:33 localhost systemd: PID 3293 read from file
/var/run/multipathd/multipathd.pid does not exist or is a zombie.
messages-20160101:Dec 9 17:46:18 localhost systemd: PID 3311 read from file
/var/run/multipathd/multipathd.pid does not exist or is a zombie.
messages-20160101:Dec 9 17:57:50 localhost systemd: PID 3269 read from file
/var/run/multipathd/multipathd.pid does not exist or is a zombie.
How can we diagnosis this issue and fix it? Is it known issue or the certain bug?
Environment
Red Hat Enterprise Linux 7.1
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.
Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.
