SPC Verizon SPC_AlarmAgent SNMP trap storm

  • KM01400759
  • 11-Feb-2015
  • 11-Feb-2015

This document has not been formally reviewed for accuracy and is provided "as is" for your convenience.

Summary

AdminAgent failed on parsing particular lines in the AdminAgent logfile

Question

Case ID: 4636736540  
 SNMP trap storm reported.   
 SPC_AlarmAgent encapsulator that is monitoring the AdminAgent is failing on parsing particular lines in the AdminAgent logfile. 
  
 Here are some of the errors as seen.
 
 08/15/2011 20:20:01.140 GMT AdminAgent (08.121.01) ERROR: Record skipped - problem parsing entry 4 of record: "Date=08/15/2011 20:15:01.867 GMT EventId=01.013.04 Severity=WARNING message=/opt/SIUS1FE1/bin/SIUJava stdout output on flush: at java.lang.Object.wait(Object.java:485"
 08/15/2011 20:20:01.140 GMT AdminAgent (08.094.09) WARNING: RecordEncapsulator.getNME() Received a ParserException: Error in Parsing Record:Date=08/15/2011 20:15:01.867 GMT EventId=01.013.04 Severity=WARNING Message=/opt/SIU S1FE1/bin/SIUJava stdout output on flush:at java.lang.Object.wait Object.java:485 

Answer

Configuration change was made in SPC 2.5 to update process.   
The AdminAgent starts to capture stdout and stderr.  When an IUM server writes to its stdout, it is captured in the AdminAgent logfile and logged at WARNING level. AlarmAgent is not expecting this kind of logs, so it doesn’t know how to parse the records and you see a WARNING and ERROR for each record when AlarmAgent tries to parse.