Middle Tier stops responding - DHOST high utilization

  • 3009924
  • 09-Oct-2006
  • 26-Apr-2012

Environment


Novell eDirectory 8.7.3.8 for Windows 2000
Novell eDirectory 8.7.3.8 for Windows 2003
Novell ZENworks Desktop Management 7 - ZDM7 Middle Tier - XTier
Novell ZENworks for Desktops 4 - ZfD4 Middle Tier
Microsoft Windows Server 2003 Enterprise Edition
Microsoft Windows 2000 Server

Situation

High utilization with the DHOST module on a Windows eDirectory server
Loss of Virtual Memory
DHOST unloads
Middle Tier stops responding
Within iMonitor a buildup of inbound connections within eDirectory can be seen that coorelate to the reduction in virtual memory.
Spikes occur every 30 minutes in thread and handle usage within the DHOST service as seen in Perfmon.

Resolution

These symptoms began to show up after applying eDirectory 8.7.3.7. This problem has only been seen within a busy ZEN XTier environment.

These problems result from two different issues:

1. In regards to the spikes every 30 minutes, there is a problem in the way eDirectory grabs a schema lock on the database when the purge process began. At the end of the purge process eDirectory must update the SEV table for all existing connections. A Read-Writer lock was being used to perform this task. The code was changed to use a Mutex instead.

2. The loss of virtual memory, high utilization and build up of inbound connections is due to a defect within NCPEngine that is exposed by a change introduced in eDirectory 8.7.3.7. This issue persists in eDirectory 8.7.3.8 as well. In Windows each connection requires its own thread and each thread requires a portion of virtual memory. In a heavily used XTier environment the inbound connections from the XTier server to the Windows eDirectory server results in a buildup of inbound connections. This, in turn, results in a complete consumption of virtual memory. Once virtual memory runs down to a small amount DHOST services will unload.

Additional Information

This has been resolved in the patch edir8738ftf_1.exe.