Memory problems on NetWare servers running Audit instrumentation

  • 7005385
  • 24-Feb-2010
  • 14-Jan-2014

Environment

Novell NetWare 6.5
Novell Audit 2.0.2 Platform Agent
NetIQ Sentinel

Situation

Memory leak
Memory hog
Error:  Cache memory allocator out of available memory
Error:  Short-term memory allocator out of available memory
Novell Remote Manager (NRM/NoRM) shows low memory
MONITOR.NLM shows low memory
SEG.NLM shows low memory
Server hang

Resolution

There are number of different causes of this problem and hence a number of resolutions.  For best results they must all be applied.
  • NetWare patches
    • NSS Update for NetWare 6.5 Support Pack 8 1.0 or later- available from https://download.novell.com/patch/finder (Note that NetWare 6.5 SP8 is a prerequisite to this patch)
    • Audit instrumentation 2.0.2 Build 62 or later (LCACHE.NLM, LOGEVENT.NLM and AUDITNW.NLM) - available directly from Novell Services
  • Minimum parameters in SYS:\ETC\LOGEVENT.CFG
LogHost=<sentinel host IP>
LogEnginePort=<sentinel host port>
LogReconnectInterval=60
LogCacheLimitAction=roll cache

The Audit instrumentation can be sensitive to latency, such as slow network or an overloaded Sentinel server, and it is not unusual to see AUDITNW.NLM memory usage increase as latency increases.  AUDITNW.NLM memory usage should drop when the period of latency drops.  However, if there are circumstances where there are permanently slow networks, AUDITNW.NLM may consume more and more memory. 
  • If, in spite of all the above fixes being applied, AUDITNW.NLM memory continues to rise, add the following parameter to the existing SYS:\ETC\LOGEVENT.CFG
LogForceCaching=Y

Additional Information

The following steps can be used to troubleshoot before applying the fixes and to monitor after applying the fixes:
  • On the NetWare server, the Audit instrumentation keeps its cache files in SYS:ETC\logcache andSYS:ETC\logcache\backup - the contents of these directories will cycle but should not keep on growing
    • If the number of cache files grows, unload (AUDITSTP) and reload (AUDITAGT) the Audit Instrumentation
  • On Sentinel, use the new (in Connector v6.8) Queue feature in the Connector to ensure that the queue does not fill up.  The queue can be monitored as follows:
Event Source Management ->
Edit Event Source Server ->
Manage Queue



Note that this screen doesn't update in real-time
  • In Sentinel, set up an Active View to monitor the Events Per Second throughput to ensure that the server is not consistently running at a high (> 300eps) event rate; if it is, another event server may be required

  • Monitor the Sentinel server's performance to ensure that is coping with the load
    • On Linux use top or similar
    • On Windows use Task Manager or similar

Feedback service temporarily unavailable. For content questions or problems, please contact Support.