Environment
Open Enterprise Server 2018 (OES 2018) Linux
Situation
Problem Statement:
Installation of OES 2018 fails on
路 HPE Proliant DL380 Gen 10 machine
路 Lenovo ThinkSystem SR550
Installation Matrix
Hardware / Deployment |
OES 2018 FCS |
Lenovo ThinkSystem SR550 with XClarity Provisioning |
Not Working |
Lenovo ThinkSystem SR550 without XClarity Provisioning |
Working |
HPE Proliant DL380 Gen 10 with Intelligent Provisioning |
Not Working |
HPE Proliant DL380 Gen 10 without Intelligent Provisioning |
Working |
Resolution
Lenovo ThinkSystem SR550
We start with downloading the ISO from SUSE's Solid Driver Program for Lenovo ThinkSystem SR550. After that, either extract or mount the ISO:
$ sudo mount lenovo_servers_thinksystem-sle12sp2-x86_64-1.0.iso /mnt/
mount: /mnt: WARNING: device write-protected, mounted read-only.
After mounting the ISO, we need to extract all kernel module packages (*kmp*.rpm):
$ unrpm /mnt/suse/x86_64/*kmp*.rpm
If you do not have the unrpm command in your $PATH then you can use rpm2cpio command as well:
$ rpm2cpio /mnt/suse/x86_64/*kmp*.rpm | cpio -idv
NOTE: The above commands will extract the *.ko files in the current working directory. Verify the directory whether above command extracted all required .ko files to directory
Path lib/modules/4.4.21-69-default/updates/ should contain following .ko files
bnxt_en.ko i40e.ko
i40evf.ko i40iw.ko
ixgbe.ko
ixgbevf.ko mega-
raid_sas.ko
mpt3sas.ko
If required .ko files are not present in the directory. You have to extract individual kernel module package rpm files one by one
broadcom-bnxt_en-kmp-default-1.7.9_k4.4.21_69-2.1.x86_64
intel-i40e-kmp-default-2.0.23_k4.4.21_69-4.1.x86_64
intel-i40evf-kmp-default-2.0.22_k4.4.21_69-1.1.x86_64
intel-ixgbe-kmp-default-5.0.7_k4.4.21_69-14.1.x86_64
intel-ixgbevf-kmp-default-4.0.5_k4.4.21_69-1.1.x86_64
lsi-megaraid_sas-kmp-default-07.700.26.00_k4.4.21_69-1.1.x86_64
lsi-mpt3sas-kmp-default-20.00.00.00_k4.4.21_69-1.3.x86_64
$ rpm2cpio /mnt/suse/x86_64/<rpm-name.rpm> | cpio 鈥搃dv
Next step is to create a DUD.
mkdud command
install mkdudpackage if it is not already installed on your system. On SLES12 SPx, mkdud packages is available via Package Hub repositories
SuSE Package Hub.
We start with downloading the ISO from SUSE's Solid Driver Program for Lenovo ThinkSystem SR550. After that, either extract or mount the ISO:
$ sudo mount lenovo_servers_thinksystem-sle12sp2-x86_64-1.0.iso /mnt/
mount: /mnt: WARNING: device write-protected, mounted read-only.
After mounting the ISO, we need to extract all kernel module packages (*kmp*.rpm):
$ unrpm /mnt/suse/x86_64/*kmp*.rpm
If you do not have the unrpm command in your $PATH then you can use rpm2cpio command as well:
$ rpm2cpio /mnt/suse/x86_64/*kmp*.rpm | cpio -idv
NOTE: The above commands will extract the *.ko files in the current working directory. Verify the directory whether above command extracted all required .ko files to directory
Path lib/modules/4.4.21-69-default/updates/ should contain following .ko files
bnxt_en.ko i40e.ko
i40evf.ko i40iw.ko
ixgbe.ko
ixgbevf.ko mega-
raid_sas.ko
mpt3sas.ko
If required .ko files are not present in the directory. You have to extract individual kernel module package rpm files one by one
broadcom-bnxt_en-kmp-default-1.7.9_k4.4.21_69-2.1.x86_64
intel-i40e-kmp-default-2.0.23_k4.4.21_69-4.1.x86_64
intel-i40evf-kmp-default-2.0.22_k4.4.21_69-1.1.x86_64
intel-ixgbe-kmp-default-5.0.7_k4.4.21_69-14.1.x86_64
intel-ixgbevf-kmp-default-4.0.5_k4.4.21_69-1.1.x86_64
lsi-megaraid_sas-kmp-default-07.700.26.00_k4.4.21_69-1.1.x86_64
lsi-mpt3sas-kmp-default-20.00.00.00_k4.4.21_69-1.3.x86_64
$ rpm2cpio /mnt/suse/x86_64/<rpm-name.rpm> | cpio 鈥搃dv
Next step is to create a DUD.
mkdud command
install mkdudpackage if it is not already installed on your system. On SLES12 SPx, mkdud packages is available via Package Hub repositories
SuSE Package Hub.
To install the package on your system, first activate this product on your registered system:
SUSECONNECT -P PackageHub/12.2/x86_64
SUSECONNECT -P PackageHub/12.2/x86_64
After this product is activated on your system, you can install the package:
zypper install mkdud
zypper install mkdud
Creating a DUD
In the DUD, we need to include following:
路 Individual kernel modules, i.e., *.ko files so that they override corresponding kernel modules inside the installation system
路 Each *kmp*.rpm package so that they will be installed on the target system
路 Package signing GPG key
mkdud will do all of this for us in a single command:
$ mkdud --create thinksystem-drivers.dud --dist sle12 -i repo ./lib/modules/4.4.21-69-default/updates/*.ko
/mnt/suse/x86_64/*kmp*.rpm /mnt/gpg-pubkey-c2bea7e6-4c2de264.asc
=== Update #1 ===
[SLE 12 (x86_64)] Name:
lib/modules/4.4.21-69-default/updates/bnxt_en.ko .x86_64
lib/modules/4.4.21-69-default/updates/i40e.ko .x86_64
lib/modules/4.4.21-69-default/updates/i40evf.ko .x86_64
lib/modules/4.4.21-69-default/updates/i40iw.ko .x86_64
lib/modules/4.4.21-69-default/updates/ixgbe.ko .x86_64
lib/modules/4.4.21-69-default/updates/ixgbevf.ko .x86_64
lib/modules/4.4.21-69-default/updates/megaraid_sas.ko .x86_64
lib/modules/4.4.21-69-default/updates/mpt3sas.ko .x86_64
broadcom-bnxt_en-kmp-default-1.7.9_k4.4.21_69-2.1.x86_64 Fri Mar 17 17:55:23 2017
intel-i40e-kmp-default-2.0.23_k4.4.21_69-4.1.x86_64 Fri Jun 30 12:45:01 2017
intel-i40evf-kmp-default-2.0.22_k4.4.21_69-1.1.x86_64 Tue Apr 11 21:59:34 2017
intel-ixgbe-kmp-default-5.0.7_k4.4.21_69-14.1.x86_64 Fri Apr 14 16:59:49 2017
intel-ixgbevf-kmp-default-4.0.5_k4.4.21_69-1.1.x86_64 Fri Apr 14 17:31:22 2017
lsi-megaraid_sas-kmp-default-07.700.26.00_k4.4.21_69-1.1.x86_64 Thu May 4 11:14:14 2017
lsi-mpt3sas-kmp-default-20.00.00.00_k4.4.21_69-1.3.x86_64 Tue Jan 10 20:34:21 2017
ID:
0754d750-8bf6-450b-aaf7-88fd5a17ace9
Packages:
broadcom-bnxt_en-kmp-default-1.7.9_k4.4.21_69-2.1.x86_64.rpm (Fri Mar 17 17:55:23 2017)
intel-i40e-kmp-default-2.0.23_k4.4.21_69-4.1.x86_64.rpm (Fri Jun 30 12:45:01 2017)
intel-i40evf-kmp-default-2.0.22_k4.4.21_69-1.1.x86_64.rpm (Tue Apr 11 21:59:34 2017)
intel-ixgbe-kmp-default-5.0.7_k4.4.21_69-14.1.x86_64.rpm (Fri Apr 14 16:59:49 2017)
intel-ixgbevf-kmp-default-4.0.5_k4.4.21_69-1.1.x86_64.rpm (Fri Apr 14 17:31:22 2017)
lsi-megaraid_sas-kmp-default-07.700.26.00_k4.4.21_69-1.1.x86_64.rpm (Thu May 4 11:14:14 2017)
lsi-mpt3sas-kmp-default-20.00.00.00_k4.4.21_69-1.3.x86_64.rpm (Tue Jan 10 20:34:21 2017)
- install methods: repo (repo priority 50)
Modules:
bnxt_en.ko (.x86_64)
i40e.ko (.x86_64)
i40evf.ko (.x86_64)
i40iw.ko (.x86_64)
ixgbe.ko (.x86_64) ixgbevf.ko (.x86_64) megaraid_sas.ko(.x86_64) mpt3sas.ko (.x86_64)
Scripts:
2 x update.pre, update.post2 RPM Public Keys:
gpg-pubkey-c2bea7e6-4c2de264.asc
(PLDP Signing Key <pldp@suse.de> [expires: 2024-05-02])
In the DUD, we need to include following:
路 Individual kernel modules, i.e., *.ko files so that they override corresponding kernel modules inside the installation system
路 Each *kmp*.rpm package so that they will be installed on the target system
路 Package signing GPG key
mkdud will do all of this for us in a single command:
$ mkdud --create thinksystem-drivers.dud --dist sle12 -i repo ./lib/modules/4.4.21-69-default/updates/*.ko
/mnt/suse/x86_64/*kmp*.rpm /mnt/gpg-pubkey-c2bea7e6-4c2de264.asc
=== Update #1 ===
[SLE 12 (x86_64)] Name:
lib/modules/4.4.21-69-default/updates/bnxt_en.ko .x86_64
lib/modules/4.4.21-69-default/updates/i40e.ko .x86_64
lib/modules/4.4.21-69-default/updates/i40evf.ko .x86_64
lib/modules/4.4.21-69-default/updates/i40iw.ko .x86_64
lib/modules/4.4.21-69-default/updates/ixgbe.ko .x86_64
lib/modules/4.4.21-69-default/updates/ixgbevf.ko .x86_64
lib/modules/4.4.21-69-default/updates/megaraid_sas.ko .x86_64
lib/modules/4.4.21-69-default/updates/mpt3sas.ko .x86_64
broadcom-bnxt_en-kmp-default-1.7.9_k4.4.21_69-2.1.x86_64 Fri Mar 17 17:55:23 2017
intel-i40e-kmp-default-2.0.23_k4.4.21_69-4.1.x86_64 Fri Jun 30 12:45:01 2017
intel-i40evf-kmp-default-2.0.22_k4.4.21_69-1.1.x86_64 Tue Apr 11 21:59:34 2017
intel-ixgbe-kmp-default-5.0.7_k4.4.21_69-14.1.x86_64 Fri Apr 14 16:59:49 2017
intel-ixgbevf-kmp-default-4.0.5_k4.4.21_69-1.1.x86_64 Fri Apr 14 17:31:22 2017
lsi-megaraid_sas-kmp-default-07.700.26.00_k4.4.21_69-1.1.x86_64 Thu May 4 11:14:14 2017
lsi-mpt3sas-kmp-default-20.00.00.00_k4.4.21_69-1.3.x86_64 Tue Jan 10 20:34:21 2017
ID:
0754d750-8bf6-450b-aaf7-88fd5a17ace9
Packages:
broadcom-bnxt_en-kmp-default-1.7.9_k4.4.21_69-2.1.x86_64.rpm (Fri Mar 17 17:55:23 2017)
intel-i40e-kmp-default-2.0.23_k4.4.21_69-4.1.x86_64.rpm (Fri Jun 30 12:45:01 2017)
intel-i40evf-kmp-default-2.0.22_k4.4.21_69-1.1.x86_64.rpm (Tue Apr 11 21:59:34 2017)
intel-ixgbe-kmp-default-5.0.7_k4.4.21_69-14.1.x86_64.rpm (Fri Apr 14 16:59:49 2017)
intel-ixgbevf-kmp-default-4.0.5_k4.4.21_69-1.1.x86_64.rpm (Fri Apr 14 17:31:22 2017)
lsi-megaraid_sas-kmp-default-07.700.26.00_k4.4.21_69-1.1.x86_64.rpm (Thu May 4 11:14:14 2017)
lsi-mpt3sas-kmp-default-20.00.00.00_k4.4.21_69-1.3.x86_64.rpm (Tue Jan 10 20:34:21 2017)
- install methods: repo (repo priority 50)
Modules:
bnxt_en.ko (.x86_64)
i40e.ko (.x86_64)
i40evf.ko (.x86_64)
i40iw.ko (.x86_64)
ixgbe.ko (.x86_64) ixgbevf.ko (.x86_64) megaraid_sas.ko(.x86_64) mpt3sas.ko (.x86_64)
Scripts:
2 x update.pre, update.post2 RPM Public Keys:
gpg-pubkey-c2bea7e6-4c2de264.asc
(PLDP Signing Key <pldp@suse.de> [expires: 2024-05-02])
Please make sure that the output of mkdud matches the above. Important entries to check are:
Packages
Modules
RPM Public Keys
Packages
Modules
RPM Public Keys
Proceed with next step ONLY after verifying the contents of the DUD as mentioned above.
NOTE: Contents of the DUD can be viewed at any point of time by running:
$ mkdud --show </path/to/file.dud>
NOTE: Contents of the DUD can be viewed at any point of time by running:
$ mkdud --show </path/to/file.dud>
Prepare an installation media (CD/DVD or USB drive) containing OES 2018 product. Optionally, copy the DUD created above in another USB drive or insert it via Lenovo XClarity Controller's Remote Disk on Card (RDOC) feature.
Convert DUD to ISO for RDOCRemote Disk on Card (RDoC) feature only supports file ISO or IMG formats. For this, we can easily create an ISO image containing our DUD by running:
$ mkisofs -J -R -o dud.iso thinksystem-drivers.dud
Setting input-charset to 'UTF-8' from locale.
Total translation table size: 0
Total rockridge attributes bytes: 288
Total directory byes: 0
Path table size (byte): 10
Max brk space used 0
2977 extents written ($ MB
Convert DUD to ISO for RDOCRemote Disk on Card (RDoC) feature only supports file ISO or IMG formats. For this, we can easily create an ISO image containing our DUD by running:
$ mkisofs -J -R -o dud.iso thinksystem-drivers.dud
Setting input-charset to 'UTF-8' from locale.
Total translation table size: 0
Total rockridge attributes bytes: 288
Total directory byes: 0
Path table size (byte): 10
Max brk space used 0
2977 extents written ($ MB
Once both OES 2018 installation media and DUD is presented to the system, reboot the system and press F12 key to choose the boot device. Boot off of the OES 2018 installation media. You should now see the UEFI GRUB menu:
From the Boot menu select 鈥淚nstallation鈥 and Press the e key to Edit the menu item. Scroll down to the linuxefi line and append the following: dud=disk:/thinksystem-drivers.dud
The menu entry should look like this:
Setparams 鈥業nstallation鈥
Set gfxpayload=keep
Echo 鈥楲oading kernel 鈥︹
Linuxefi /boot/x86_64/loader/linux splash=silent dud=disk:/thinksystem-drivers.dud
Echo 鈥楲oading initial ramdisk 鈥︹
Initrdefi /boot/x86_64/loader/initrd
Press Ctrl+x or F10 key to start the installation process. If you press F2 or Esc key during the bootsplash screen, then you should now see that the DUD is being extracted and applied in the installation system. If you do NOT see this part, then please reboot and double check if a disk containing the thinksystem-drivers.dud is presented to the system - either via a USB storage or via XClarity Controller's RDOC.
Reading driver update: disk:/thinksystem-drivers.dud
Loading file:/mounts/mp_0002/thinksystem-drivers.dud - 100%
Drivers Update: lib/modules/4.4.21-69-default/updates/bnxt_en.ko .x86_64
Drivers Update: lib/modules/4.4.21-69-default/updates/i40e.ko .x86_64
Drivers Update: lib/modules/4.4.21-69-default/updates/i40evf.ko .x86_64
Drivers Update: lib/modules/4.4.21-69-default/updates/i40iw.ko .x86_64
Drivers Update: lib/modules/4.4.21-69-default/updates/ixgbe.ko .x86_64
Drivers Update: lib/modules/4.4.21-69-default/updates/ixgbevf.ko .x86_64
Drivers Update: lib/modules/4.4.21-69-default/updates/megaraid_sas.ko .x86_64
Drivers Update: lib/modules/4.4.21-69-default/updates/mpt3sas.ko .x86_64
Drivers Update: broadcom-bnxt_en-kmp-default-1.7.9_k4.4.21_69-2.1.x86_64 Fri Mar 17 17:55:23 2017
Drivers Update: intel-i40e-kmp-default-2.0.23_k4.4.21_69-4.1.x86_64 Fri Jun 30 12:45:01 2017
Drivers Update: intel-i40evf-kmp-default-2.0.22_k4.4.21_69-1.1.x86_64 Tue Apr 11 21:59:34 2017
Drivers Update: intel-ixgbe-kmp-default-5.0.7_k4.4.21_69-14.1.x86_64 Fri Apr 14 16:59:49 2017
Drivers Update: intel-ixgbevf-kmp-default-4.0.5_k4.4.21_69-1.1.x86_64 Fri Apr 14 17:31:22 2017
Drivers Update: lsi-megaraid_sas-kmp-default-07.700.26.00_k4.4.21_69-1.1.x86_64 Thu May 4 11:14:14 2017
Drivers Update: lsi-mpt3sas-kmp-default-20.00.00.00_k4.4.21_69-1.3.x86_64 Tue Jan 10 20:34:21 2017
Once YaST is started and the installation process begins, you should now see the DUD as an add-on product added in the 鈥淎dd-On Product Installation鈥 page as shown below:
Driver Update 0 dir:///update/000/repo,Directory:/
After clicking "Next" and selecting the system role, you should now be presented with the "Suggested Partitioning" screen which should be partitioning your physical drive (or VDisk of your RAID controller, if configured):
Continue with installing OES 2018 on the Lenovo system.
NOTE: On a couple of instances, system hangs after YaST has finished installing OES 2018. System had to rebooted to continue with the second stage of OES 2018 configuration.
HPE Proliant DL380 Gen 10
There is no need to create a DUD for HPE Proliant DL380 Gen10 servers. But in case there is a need, please check SUSE Solid Driver Program for HPE Gen10 Systems to check if there is a new installation kit available for your hardware configuration. If available, please follow installation instructions available in the download page.
OES 2018 cannot be installed using HPE's Intelligent Provisioning (IP) and must be installed using one of the following methods:
路 USB or DVD drive
路 Virtual Media attached through Integrated Lights Out (iLO) Management portal
The process for installing OES 2018 using either of these methods is fairly similar.
1. Boot the system and during POST screen, press F11 key to enter Boot Menu.
2. Once the boot menu shows up, either select "Rear USB 1 or "iLO Virtual USB 3" depending on how you have presented the OES 2018 installation media to the system.
3. Wait for the GRUB screen to show up and then press Enter key after selecting "Installation" option from the menu. Proceed with usual installation process of OES 2018. You should be able to see virtual disks created in RAID configuration as hard disks for installing OES. If you do not see any disks, then please reboot into "Intelligent Provisioning" and check the RAID configuration.
The menu entry should look like this:
Setparams 鈥業nstallation鈥
Set gfxpayload=keep
Echo 鈥楲oading kernel 鈥︹
Linuxefi /boot/x86_64/loader/linux splash=silent dud=disk:/thinksystem-drivers.dud
Echo 鈥楲oading initial ramdisk 鈥︹
Initrdefi /boot/x86_64/loader/initrd
Press Ctrl+x or F10 key to start the installation process. If you press F2 or Esc key during the bootsplash screen, then you should now see that the DUD is being extracted and applied in the installation system. If you do NOT see this part, then please reboot and double check if a disk containing the thinksystem-drivers.dud is presented to the system - either via a USB storage or via XClarity Controller's RDOC.
Reading driver update: disk:/thinksystem-drivers.dud
Loading file:/mounts/mp_0002/thinksystem-drivers.dud - 100%
Drivers Update: lib/modules/4.4.21-69-default/updates/bnxt_en.ko .x86_64
Drivers Update: lib/modules/4.4.21-69-default/updates/i40e.ko .x86_64
Drivers Update: lib/modules/4.4.21-69-default/updates/i40evf.ko .x86_64
Drivers Update: lib/modules/4.4.21-69-default/updates/i40iw.ko .x86_64
Drivers Update: lib/modules/4.4.21-69-default/updates/ixgbe.ko .x86_64
Drivers Update: lib/modules/4.4.21-69-default/updates/ixgbevf.ko .x86_64
Drivers Update: lib/modules/4.4.21-69-default/updates/megaraid_sas.ko .x86_64
Drivers Update: lib/modules/4.4.21-69-default/updates/mpt3sas.ko .x86_64
Drivers Update: broadcom-bnxt_en-kmp-default-1.7.9_k4.4.21_69-2.1.x86_64 Fri Mar 17 17:55:23 2017
Drivers Update: intel-i40e-kmp-default-2.0.23_k4.4.21_69-4.1.x86_64 Fri Jun 30 12:45:01 2017
Drivers Update: intel-i40evf-kmp-default-2.0.22_k4.4.21_69-1.1.x86_64 Tue Apr 11 21:59:34 2017
Drivers Update: intel-ixgbe-kmp-default-5.0.7_k4.4.21_69-14.1.x86_64 Fri Apr 14 16:59:49 2017
Drivers Update: intel-ixgbevf-kmp-default-4.0.5_k4.4.21_69-1.1.x86_64 Fri Apr 14 17:31:22 2017
Drivers Update: lsi-megaraid_sas-kmp-default-07.700.26.00_k4.4.21_69-1.1.x86_64 Thu May 4 11:14:14 2017
Drivers Update: lsi-mpt3sas-kmp-default-20.00.00.00_k4.4.21_69-1.3.x86_64 Tue Jan 10 20:34:21 2017
Once YaST is started and the installation process begins, you should now see the DUD as an add-on product added in the 鈥淎dd-On Product Installation鈥 page as shown below:
Driver Update 0 dir:///update/000/repo,Directory:/
After clicking "Next" and selecting the system role, you should now be presented with the "Suggested Partitioning" screen which should be partitioning your physical drive (or VDisk of your RAID controller, if configured):
Continue with installing OES 2018 on the Lenovo system.
NOTE: On a couple of instances, system hangs after YaST has finished installing OES 2018. System had to rebooted to continue with the second stage of OES 2018 configuration.
HPE Proliant DL380 Gen 10
There is no need to create a DUD for HPE Proliant DL380 Gen10 servers. But in case there is a need, please check SUSE Solid Driver Program for HPE Gen10 Systems to check if there is a new installation kit available for your hardware configuration. If available, please follow installation instructions available in the download page.
OES 2018 cannot be installed using HPE's Intelligent Provisioning (IP) and must be installed using one of the following methods:
路 USB or DVD drive
路 Virtual Media attached through Integrated Lights Out (iLO) Management portal
The process for installing OES 2018 using either of these methods is fairly similar.
1. Boot the system and during POST screen, press F11 key to enter Boot Menu.
2. Once the boot menu shows up, either select "Rear USB 1 or "iLO Virtual USB 3" depending on how you have presented the OES 2018 installation media to the system.
3. Wait for the GRUB screen to show up and then press Enter key after selecting "Installation" option from the menu. Proceed with usual installation process of OES 2018. You should be able to see virtual disks created in RAID configuration as hard disks for installing OES. If you do not see any disks, then please reboot into "Intelligent Provisioning" and check the RAID configuration.