Large numbers (Hundreds of Thousands) of Active alarms cause high utilization on the Site server.

  • 3950346
  • 16-May-2007
  • 30-Apr-2012

Environment

Novell ZENworks for Servers.
Novell ZENworks for Servers 3.0.
Novell ZENworks 6.5 Server Management Support Pack 2 - ZSM6.5 SP2
Novell ZENworks 7 Server Management Support Pack 1 - ZSM7 SP1
Novell Management and Monitoring Services.
Novell Alarm Management.

Situation

Large numbers (Hundreds of Thousands) of Active alarms cause high utilization on the Site server.
The high utilization fluctuates up to 100%, but the server is still responsive.
Large numbers (Hundreds of Thousands) of Active alarms slows down Consoleone.
The Java Alarm thread is the busiest thread.
ConsoleOne screen refreshes become slower as Active alarms increase.
ConsoleOne screen refreshes will start to take hours.
The NetWare Server Management Agent default configuration is the desired configuration for hundreds of servers in an Enterprise environment (meaning many alarms are being sent on a daily basis).
Active alarms are not being 'Handled' on the Active alarm screen.
Active alarms are arriving too freguently to be 'Handled' on the Active alarm screen.
Error: "Error decoding Packet: Snmp2.SnmpException: Parse Auth: Incorrect VarBind"

Resolution

Filter unneeded, unnecessary Alarms (determined by the environment and business needs) from being sent by the NetWare Server Management Agent by editing theNWTRAP.CFG& NDSTRAP.CFG and adding the READ_CFG switch to the load lines in NMA5.NCF (ex: LOAD NWTRAP READ_CFG,LOAD NDSTRAP.CFG) so the new configuration in the .CFG files are read on each load of NMA5.NCF.

The edited, custom NWTRAP.CFG, NDSTRAP.CFG, & NMA5.NCF files should be pushed out to all NetWare servers with the full Server Management Agent installed. Run UNNMA5.NCF and then NMA5.NCF to activate the changes. With NMA5.NCF modules running, use the file server HELP command to see the NWTRAP and NDSTRAP command line options. NWTRAP LIST ENABLED and NDSTRAP LIST ENABLED will confirm what alarms are truely active. When editing the NWTRAP.CFG andNDSTRAP.CFG, use only the '#' sign for remarking out lines.
As potentially a short term solution, to filter what Alarms ConsoleOne sees in the Active Alarm screen of the potential thousands of unwanted alarms by each server that are of Severity= Unknown, Informational, & Minor Alert (or any Alarm for that matter) do the following:

1. Select the Site Server object, right mouse click, select Properties.
2. Select the 'Alarm Disposition' TAB.
3. From the Alarm Template table, select the rows where Severity= Unknown, Informational, or Minor and click on Edit Button, select Edit and the Alarm Disposition dialog will open.
4. Select 'Other Configuration' TAB.
5. Uncheck the 'Archive' and 'Show On Ticker Bar' check boxes, click OK, then Apply.

After this, Alarms of Severity Unknown, Informational, & Minor Alert will be ignored by the ZfS3 system in ConsoleOne. The high utilization on the Java Alarm thread will still remain, but the server should still be responsive and available. Active alarms that have already been received will still have to be 'Handled' in the Active Alarm screen.

If there are to many Alarms in the database already, restoring the empty database files ( \ZENWORKS\MMS\EMPTYDB ) that was an option to create during the install, can be restored. To restore the \EMPTYDB\ files, unload all ZFS modules, including MGMTDBS.NCF, move all the \ZENWORKS\MMS\DB\*.* files to a holding directory and replace with a copy of all the \ZENWORKS\MMS\EMPTYDB\*.* files.
Handled Alarms' will still be in the database till the Alarm Purge Process (AMPURGE.PROPERTIES& SLOADER.PROPERTIES, see on-line documentation ) purges them. They can be manually deleted from the Alarm History.
This has been reported to development. Automated options for'Handling' of some or all Active alarms have been requested and are being considered for a future product release.
Additional information:
How to Use the Alarm Management System of ZENworks for Servers 3.
New info (12/12/2003): The first time NWTRAP.NLM and NDSTRAP.NLM are loaded (NMA5.NCF) after the very first install of the NetWare Server Management Agent, the NWTRAP.CFG & NDSTRAP.CFG files are read (if exist) and will use this configuration. The information from the .CFG files is entered in the NetWare registry at \My Server\Software\Novell\NMA\NWTrap& \NDSTrap.

The next time NWTRAP.NLM& NDSTRAP.NLM are loaded (NMA5.NCF) the configuration is read in the registry and not from the .CFG files. Any subsequent changes made to the .CFG files must be updated to the registry by using the READ_CFG command for bothNWTRAP.NLM& NDSTRAP.NLM. This can be done dynamically at the server console prompt if NMA5.NCF is running. The READ_CFG parameter can also be added in the NMA5.NCF file, but only if the.CFG is wanted to be read every time NMA5.NCF runs.

So what does this mean in regards to ZFS3SP2.EXE as it installs new, more restrictive NWTRAP.CFG& NDSTRAP.CFG?

If you install Service Pack 2 on an existing ZfS installation (or upgrade an existing environment to ZfS 3.0.2) the support pack will copy the new .CFG files to the server. Due to the fact that the registry is already the configuration of the NWTRAP, the new .CFG files won't be used. To resolve this, use READ_CFG once so that the new settings get applied.

On a new installation with ZfS 3.0.2 the modified .CFG files are used the first time NWTRAP.NLM& NDSTRAP.NLM are loaded withNMA5.NCF.

Additional Information

The NetWare SNMP Alarms are not being filtered.
The Server Management Agent is the default configuration for NWTRAP.NLM & NWTRAP.CFG and NDSTRAP.NLM & NDSTRAP.CFG and NMA5.NCF
(meaning many alarms are being sent on a daily basis).

Error: "Error decoding Packet: Snmp2.SnmpException: Parse Auth: Incorrect VarBind

Packet from: x.x.x.x : 161

DATA:

30 50 02 01 00 04 05 64 68 73 62 62 a2 44 02 02 01 bb 02 03
03 00 01 02 01 38 30 0c 06 08 2b 06 01 02 01 11 01 01 05 00
30 0c 06 08 2b 06 01 02 01 11 01 02 05 00 30 0c 06 08 2b 06
01 02 01 11 01 03 05 00 30 0c 06 08 2b 06 01 02 01 11 04 02
05 00

java.lang.OutOfMemoryError
at java.lang.StringBuffer.expandCapacity(Unknown Source)
at java.lang.StringBuffer.append(Unknown Source)
at java.lang.StringBuffer.append(Unknown Source)
at com.novell.managewise.discovery.agentManager.SNMPQueryModule.buildPro
tocolProperties(SNMPQueryModule.java, Compiled Code)
at com.novell.managewise.discovery.agentManager.SNMPQueryModule.queryOid
s(SNMPQueryModule.java, Compiled Code)
at com.novell.managewise.discovery.agentManager.AgentManager.processNDSN
ameObject(AgentManager.java, Compiled Code)"


Formerly known as TID# 10074368

Change Log

23 Oct. 2008 Alan Ehlert, couple of updates ('to' changed to 'too', and '(meaning many alarms are being sent on a daily basis) added' added, moved error data to Additional Information section, font changes.