Environment
Novell NetWare 6.5
Novell Audit 2.0.2 Platform Agent
NetIQ Sentinel
Novell Audit 2.0.2 Platform Agent
NetIQ Sentinel
Situation
Memory leak
Memory hog
Error: Cache memory allocator out of available memory
Error: Short-term memory allocator out of available memory
Novell Remote Manager (NRM/NoRM) shows low memory
MONITOR.NLM shows low memory
SEG.NLM shows low memory
Server hang
Memory hog
Error: Cache memory allocator out of available memory
Error: Short-term memory allocator out of available memory
Novell Remote Manager (NRM/NoRM) shows low memory
MONITOR.NLM shows low memory
SEG.NLM shows low memory
Server hang
Resolution
There are number of different causes of this problem and hence a number of resolutions. For best results they must all be applied.
The Audit instrumentation can be sensitive to latency, such as slow network or an overloaded Sentinel server, and it is not unusual to see AUDITNW.NLM memory usage increase as latency increases. AUDITNW.NLM memory usage should drop when the period of latency drops. However, if there are circumstances where there are permanently slow networks, AUDITNW.NLM may consume more and more memory.
- NetWare patches
- NSS Update for NetWare 6.5 Support Pack 8 1.0 or later- available from https://download.novell.com/patch/finder (Note that NetWare 6.5 SP8 is a prerequisite to this patch)
- Audit instrumentation 2.0.2 Build 62 or later (LCACHE.NLM, LOGEVENT.NLM and AUDITNW.NLM) - available directly from Novell Services
- Sentinel patches
- Sentinel 6.1 SP1 HOTFIX2 or later - available from https://dl.netiq.com
- Audit Connector Version 6r8 or later - available from https://support.novell.com/products/sentinel/secure/sentinel61.html
- Novell NetWare Collector Version 6.1r4 or later - available from https://support.novell.com/products/sentinel/secure/sentinel61.html
- Minimum parameters in SYS:\ETC\LOGEVENT.CFG
LogHost=<sentinel host IP>
LogEnginePort=<sentinel host port>
LogReconnectInterval=60
LogCacheLimitAction=roll cache
LogEnginePort=<sentinel host port>
LogReconnectInterval=60
LogCacheLimitAction=roll cache
The Audit instrumentation can be sensitive to latency, such as slow network or an overloaded Sentinel server, and it is not unusual to see AUDITNW.NLM memory usage increase as latency increases. AUDITNW.NLM memory usage should drop when the period of latency drops. However, if there are circumstances where there are permanently slow networks, AUDITNW.NLM may consume more and more memory.
- If, in spite of all the above fixes being applied, AUDITNW.NLM memory continues to rise, add the following parameter to the existing SYS:\ETC\LOGEVENT.CFG
LogForceCaching=Y
Additional Information
The following steps can be used to troubleshoot before applying the fixes and to monitor after applying the fixes:
- On NetWare, use SEG.NLM's Trend page (<F6> from the main menu) to monitor memory usage; SEG.NLM is available from https://www.novell.com/coolsolutions/tools/14445.html
- On the NetWare server, the Audit instrumentation keeps its cache files in SYS:ETC\logcache andSYS:ETC\logcache\backup - the contents of these directories will cycle but should not keep on growing
- If the number of cache files grows, unload (AUDITSTP) and reload (AUDITAGT) the Audit Instrumentation
- On Sentinel, use the new (in Connector v6.8) Queue feature in the Connector to ensure that the queue does not fill up. The queue can be monitored as follows:
Event Source Management ->
Edit Event Source Server ->
Manage Queue
Note that this screen doesn't update in real-time
Edit Event Source Server ->
Manage Queue
Note that this screen doesn't update in real-time
- In Sentinel, set up an Active View to monitor the Events Per Second throughput to ensure that the server is not consistently running at a high (> 300eps) event rate; if it is, another event server may be required
- Monitor the Sentinel server's performance to ensure that is coping with the load
- On Linux use top or similar
- On Windows use Task Manager or similar