High CPU utilization seen on server at all hours

  • 3363182
  • 08-Nov-2007
  • 26-Apr-2012

Environment

Novell NetWare 5.1 Support Pack 6
Novell NetWare 5.1 Support Pack 5
Novell Directory Services 7
NLS:List Of Handles has a high number of values
NLS:Cert Peak Used Pool has a high number of values
NLS:updateEntry and NLS:updateJanitor are seen as busiest threads in MONITOR.NLM

Situation

Applied Support pack 6
Applied latest Novell Licensing Services (NLS) code, currently found in NLS603FT.EXE
Applied latest DS code for NDS 7.x, currently found FDS762C.EXE
High CPU utilization seen on server at all hours

Resolution

By applying the latest DS code to this server we where able to take advantage of a new -NLC switch which removes high valued attributes. Running DSREPAIR.NLM with this switch removed the NLS:Cert Peak Used Pool attributed, however it does not remove the NLS:List Of Handles. While CPU utilization was reduced, it did not return to normal levels. By running DSREPAIR -CVxxxx (DS 7.x does not allow the search for high valued attributes via DSBROWSE.NLM) we where able to identify the licensing certificates with the high valued attributes. The customer deleted and recreated these and CPU utilization on the server dropped back to normal levels.
In order to make sure the problem does not reappear, you should make sure that you are running the latest NLS code and that you have set the following SET PARAMETERS

SET STORE NETWARE 5 CONN SCL MLA USAGE IN NDS = OFF

SET NLS REPORT DATA=OFF


Applying Support pack 4 or later will prevent the NLS:List Of Handles attribute from growing

Additional Information

The high utilization was being by the synchronization of the attributes NLS:List Of Handles and NLS:Cert Peak Used Pool
(NLS uses an attribute of license certificates called "List of Handles” to keep track of license unit consumption for individual user objects in each tree (NetWare 6.x) or for connections to specific servers (NetWare 5.x). Periodically NLS checks these handles for a number of reasons: 1.) A user logs in to the tree either for the first time or from an additional location 2.) A timeout value is encountered causing NLS to verify that active connections have valid licenses. To verify that the correct number of license units are consumed, NLS first checks the server's cache and then the actual eDirectory database. If a handle is not found for the specified user object, then an attempt is made to obtain a handle and grant a certificate unit to the user object in question. If this process fails, then access is denied. There was an issue with NLS that began in the NetWare 5.1 days where NLS would search for a handle and quit searching before the end of the list was reached. In this condition, an additional handle for the same object was issued. This could happen repeatedly, cause the NLS:List of Handles to grow so large that eDirectory replication causes high utilization on the affected servers. In some cases, this utilization could even result in CPU Hog Timer abends or cause licensing to fail to respond, etc.


More information on how to resolve high CPU utilization issues related to DS can be found in the following TIDs

DS High Utilization

High Utilization in DS.NLM

High Utilization on the server caused by NDS Synchronization



Formerly known as TID# 10089387