Environment
Novell eDirectory 8.8 for Linux
SUSE Linux
Situation
When troubleshooting ndsd utilization issues it can be very helpful to grab a stack trace of the ndsd process in order to see what the threads are doing. On linux and specifally SUSE a script called "gstack" is included with the gdb package which makes the task very simple. The purpose of this document is to describe how to use gstack to grab a stack trace of the ndsd process.
Resolution
1. First ensure that gdb is installed on the server.
In order to accomplish this a couple of things could be done.
a. Type "which gdb" should display the path containing the gdb binary.
b.
If this command does not show a path togdb, then try an "rpm -qa |
grep -i gdb". This should result in a package starting with "gdb-x.x*".
If
the package is not installed, install it using rpm or yast. This is
included on the SUSE disk. GDB can also be download from the gdb home page
2. To grab a stack trace, use "gstack <ndsd_pid">> gstack.log
To get the ProcessID/PID for ndsd could use "ps -C ndsd"
Example:
ps -C ndsd
PID TTY TIME CMD
4323 ? 00:01:22 ndsd
gstack 4323 > gstack.log
Threads could change quite a bit in cpu time, so it would be best to grab 4 or 5 gstacks with in 10 second intervals during high utilization to have something to compare with.
Example command:
CMD="gstack 4323"; for i in $(seq 5); do $CMD >> gstack_$i; sleep 10; done
This should create the files "gstack_1 gstack_2 gstack_3 gstack_4 gstack_5"
Example command:
CMD="gstack 4323"; for i in $(seq 5); do $CMD >> gstack_$i; sleep 10; done
This should create the files "gstack_1 gstack_2 gstack_3 gstack_4 gstack_5"
It
is out of the scope of this TID to explain how to read a stack trace or
gstack, but this can be valuable information for Novell Technical
Support.
Additional Information
While troubleshooting utilization, with ndsd it might be a good idea to automate taking gstack until the issue is resolved. Something like the following procedure could be used:
As root, type crontab -e and add the following to the crontab to grab a gstack every 5 minutes.
*/5 * * * * root /usr/bin/gstack `pgrep ndsd` > /root/gstack-ndsd_`echo $HOSTNAME`_`date +\%Y-\%m-\%d_\%H-\%M-\%S`.txt
(must escape the % in crontab with a \ for this to work)
Next restart cron by performing "rccron restart"
This will result with in logs with the following syntax as an example:
gstack-14:35:01.txt
gstack-14:35:01.txt