Environment
The purpose of this document is to provide additional background
information on the relationships between the various daemons that hook into NSS, and to provide support
with a set of steps in aiding their troubleshooting efforts, so to
provide support and engineering with more appropriate details when opening support incidents or when filing bug
reports.
Situation
We need to understand how the various modules and dependencies
between User and Kernel space are related to the NSS file
system
In User space, SMS hooks into libzapi,
- in Kernel space (through VFS) libzapi hooks into zapi
In User space, LUM hooks into ndpapp,
- in Kernel space (through VFS) ndpapp makes system calls into ndpmod
In User space, the eDir part of NDSD hooks into ncpid, and ncpid hooks into ndpapp,
- in Kernel space (through VFS) ndpapp makes system calls into ndpmod
In User space, the NCP part of NDSD hooks into both adminusd and libnebus,
- in Kernel space (through VFS) adminusd makes system calls into both LSA& admindrv
- in Kernel space (through VFS) libnebus makes system calls into nebdrv
In User space, the NCS modules hook into adminfsd
- in Kernel space (through VFS) adminfsd makes system calls into adminfs
- in Kernel space (through VFS) adminfsd makes system calls into adminfsdrv
All NSS utilities that run in User space (through VFS), hook intoLSA that runs in Kernel space
In User space, SMS hooks into libzapi,
- in Kernel space (through VFS) libzapi hooks into zapi
In User space, LUM hooks into ndpapp,
- in Kernel space (through VFS) ndpapp makes system calls into ndpmod
In User space, the eDir part of NDSD hooks into ncpid, and ncpid hooks into ndpapp,
- in Kernel space (through VFS) ndpapp makes system calls into ndpmod
In User space, the NCP part of NDSD hooks into both adminusd and libnebus,
- in Kernel space (through VFS) adminusd makes system calls into both LSA& admindrv
- in Kernel space (through VFS) libnebus makes system calls into nebdrv
In User space, the NCS modules hook into adminfsd
- in Kernel space (through VFS) adminfsd makes system calls into adminfs
- in Kernel space (through VFS) adminfsd makes system calls into adminfsdrv
All NSS utilities that run in User space (through VFS), hook intoLSA that runs in Kernel space
Resolution
References
(1) NDPAPP
NDPAPP facilitates any eDir & LUM interactions with the NSS file system.
When ndpapp (the kernel module) comes up we create a device called /dev/ndp
When accessing a file on the NSS file system, permissions are checked by contacting LUM for the FDN name for the user, than eDir will be contacted for the GUID
Note1: Typically we only see one ndpapp thread running
Note2: See section 34.6.4 Logging Communications between NSS and eDirectory, NICI, or Linux User management for hints on troubleshooting such communication issues.
(2) NCP2NSS
ncp2nss facilitates any changes that are requested from an eDirectory perspective to the file system and vice-versa. ncp2nss propagates changes for the file system using ndpapp.
Note: when you run the gstack command on a 'unpatched' OES2 SP1 Linux server, due to an NCP bug, you will kill all(!) ncp2nss threads. Since we have crashed kernel type threads, the nett result of this is that when for example you are than trying to un-mount a volume, this will hang the system in such a way that you will need to reboot the server in order to recover from this.
All threads that run in user space, also have a corresponding thread in kernel space
echo 1> /proc/sys/kernel/sysrq
echo t> /proc/sysrq-trigger (where t is processes)
As a side-note:
In a cluster will see different threads doing these functions (python threads) and never adminusd. If you see this, please get the corresponding kernel stack output.
Remember at all times you are looking at a stack dump however, and not a stack walk. What you are looking for are threads in a <defunc> state. If a thread cannot be interrupted (threaded) they need to be awaken by themselves.
(1) NDPAPP
NDPAPP facilitates any eDir & LUM interactions with the NSS file system.
When ndpapp (the kernel module) comes up we create a device called /dev/ndp
When accessing a file on the NSS file system, permissions are checked by contacting LUM for the FDN name for the user, than eDir will be contacted for the GUID
Note1: Typically we only see one ndpapp thread running
Note2: See section 34.6.4 Logging Communications between NSS and eDirectory, NICI, or Linux User management for hints on troubleshooting such communication issues.
(2) NCP2NSS
ncp2nss facilitates any changes that are requested from an eDirectory perspective to the file system and vice-versa. ncp2nss propagates changes for the file system using ndpapp.
Note: when you run the gstack command on a 'unpatched' OES2 SP1 Linux server, due to an NCP bug, you will kill all(!) ncp2nss threads. Since we have crashed kernel type threads, the nett result of this is that when for example you are than trying to un-mount a volume, this will hang the system in such a way that you will need to reboot the server in order to recover from this.
All threads that run in user space, also have a corresponding thread in kernel space
echo 1> /proc/sys/kernel/sysrq
echo t> /proc/sysrq-trigger (where t is processes)
As a side-note:
In a cluster will see different threads doing these functions (python threads) and never adminusd. If you see this, please get the corresponding kernel stack output.
Remember at all times you are looking at a stack dump however, and not a stack walk. What you are looking for are threads in a <defunc> state. If a thread cannot be interrupted (threaded) they need to be awaken by themselves.