SUSE Linux Enterprise Server 12
A system was upgraded from SLES 11 SP4 to SLES 12 during shutdown the following error can be seen:
dracut: Disassembling device-mapper devices , Command failed
this error can be seen on s390x but also on x86 when enabling debugging during shutdown. Detailed output:
Apr 25 16:32:38 linux-aoie systemd: Unmounting Runtime Directory...Apr 25 16:32:38 linux-aoie systemd: Unmounted Lock Directory.Apr 25 16:32:38 linux-aoie umount: umount: /var/run: target is busyApr 25 16:32:38 linux-aoie umount: (In some cases useful info abouApr 25 16:32:38 linux-aoie umount: use the device is found by lsoApr 25 16:32:38 linux-aoie systemd: var-run.mount: Mount process exited, codeApr 25 16:32:38 linux-aoie systemd: Failed unmounting Runtime Directory.
To fix such problem we need to make sure /var/run is pointing to /run.
This can be done via two method:
1) Interactive by booting the system from SLES DVD and select "Rescue Mode"
then mounting the root disk to /mntthen remove /mnt/var/runnext, create a link /mnt/var/run -> /mnt/run
Note in the case /var is separate volume then mount var instead of / and adjust above paths
2) Automated by using convertfs dracut module:
run the following command :
dracut --add convertfs -f
add the following two kernel command line parameters to the kernel Command Line (use 'yast bootloader'):
rd.convertfs rootflags=rw systemctl reboot
After doing above, the result should look something similar to below :
linux-aoie# ll -ld /var/run*lrwxrwxrwx 1 root root 6 May 13 09:22 /var/run -> ../rundrwxr-xr-x 22 root root 4096 Apr 30 11:05 /var/run.runmove~
And the var-run.mount unit file should not be running:
linux-aoie# systemctl status var-run.mountâ var-run.mount - Runtime DirectoryLoaded: loaded (/usr/lib/systemd/system/var-run.mount; static; vendor preset: disabled)Active: inactive (dead
The cause of this issue is duplicated tmpfs Filesystem. The upgraded system end up with two run-time directories:
linux-aoie# mount | grep runtmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)tmpfs on /var/run type tmpfs (rw,nosuid,nodev,mode=755)
In SLES 11 SP4 /var/run is used as tmpfs ; In SLES 12 /run is used as tmpfs. /var/run is a link to /run:
linux-aoie# ll /var/runlrwxrwxrwx 1 root root 4 May 2 2017 /var/run -> /run
But when /var/run is a directory (like with SLES 11), var-run.mount systemd unit gets activated and mounts /var/run as tmpfs
linux-aoie# systemctl status var-run.mountâ var-run.mount - Runtime DirectoryLoaded: loaded (/usr/lib/systemd/system/var-run.mount; static; vendor preset: disabled)Active: active (mounted) since Thu 2019-04-25 16:39:46 CEST; 13min agoWhere: /var/runWhat: tmpfsProcess: 678 ExecMount=/usr/bin/mount /run /var/run -t bind -o bind (code=exited, status=0/SUCCESS)Tasks: 0 (limit: 512)
This could be seen also in journalctl output:
Apr 25 16:39:46 linux-aoie systemd: var-run.mount: Directory /var/run to mount over is not empty, mounting anyway.Apr 25 16:39:46 linux-aoie systemd: Mounting Runtime Directory...Apr 25 16:39:46 linux-aoie systemd: Mounted Runtime Directory.
The following lines prevent systemd from mounting /var/run as tmpfs but systemd still mounting /var/run as tmpfs because its a directory.
linux-aoie# grep Condition /usr/lib/systemd/system/var-run.mountConditionPathIsDirectory=/var/runConditionPathIsSymbolicLink=!/var/run
Above is a workaround for this problem. And should fix the issue. However SUSE Engineering is working on a fix that might be included in SLES12 SP5 DVD.