Environment
Novell Open Enterprise Server 11 (OES 11) Linux Support Pack 2
Novell Cluster Services
Novell Cluster Services
Situation
In a Novell Open Enterprise Server 11 SP2 cluster environment, when removing and adding LUN's without rebooting the server as officially documented in the SLES11 SP3 Storage Administration guide, it was observed that nlvm list devices output at times was not listing all the devices the administrator would expect to see.
For example :
Remove 2 LUNs and added 1 new LUN to a OES11 SP2 cluster.
Perform a "rescan-scsi-bus.sh -r".
Than /etc/multipath/bindings was modified as below (since we are using user_friendly_names) :
Results :
Next, initiate a subsequent multipathd -k reconfigure command on the remaining servers ion the node ServerC.
This time nlvm lists devices shows the device GR53_1 again..
For example :
Remove 2 LUNs and added 1 new LUN to a OES11 SP2 cluster.
Perform a "rescan-scsi-bus.sh -r".
This found 2 new devices - removed 4 devices. Devices = Paths, 2 Paths per LUN, 1 new LUN, 2 LUNs removed.
Than /etc/multipath/bindings was modified as below (since we are using user_friendly_names) :
rename mpatha -> GR52_1_MOVENext, perform a multipathd -k reconfigure so that multipath reads the configuration changes in the bindings file.
remove GR53_1 (devices that were removed during rescan .. -r already)
remove GR53_2 (devices that were removed during rescan .. -r already)
rename GR53_1_MOVE GR53_1
Results :
On ServerA all looks good.
On ServerB/C/D, initiate "nlvm list devices" does NOT show device GR53_1
Next, initiate a subsequent multipathd -k reconfigure command on the remaining servers ion the node ServerC.
This time nlvm lists devices shows the device GR53_1 again..
Resolution
And update was made to multipath-tools and kpartx which is confirmed to resolve the problem.
Cause
The libnlvm code actually looks at the /dev/mapper/* directory for multipath devices. This is because NLVM requires a devnode (an object under /dev/) to access the device.
In this scenario, it was determined the object does exist in the device mapper as shown by dmsetup ls, however, the multipath software (which uses kpartx) never created the devnode in /dev/mapper so NLVM could not find the device.
In this scenario, it was determined the object does exist in the device mapper as shown by dmsetup ls, however, the multipath software (which uses kpartx) never created the devnode in /dev/mapper so NLVM could not find the device.