MsgQueue grows and messages arrive delayed

  • KM523948
  • 21-Nov-2008
  • 21-Nov-2008

Summary

\Program Files\HP OpenView\Data\shared\tmp\queues\MsgQueue grows and the messages in the active message browser come in with a huge delay, up to several hours

Error

On an HP Openview Operations for Windows (OVOW) 7.5 server with the HP Network Node Manager (NNM) integration done via "SNMP Interceptor" policies the file

%OvDataDir%\\shared\\queues\\MsgQueue

grows and the messages in the active message browser come in with a huge delay, in some cases up to several hours.

A trace of OvEpMessageActionServer (OvOW Message Action Server), in tracemon OvEpMsgActSrv, showed the following for several different nodes:

FindNodeInternal() called: Trying to lookup node (name: <node name>, agentID:  and IP: '0'.
Lookup progress: Found NO node in name-map (name: '<node name>').
Lookup progress: Found NO node in name-map (short name: '<node name>').
FindNodeInternal() was NOT successful. No node was found matching given parameters (agent id: '', name: '<node name>'and ip: '0x0'.
End:   COvEpNodeCache::FindNodeInternal
Non-zero result returned from COvEpNodeCache::FindNodeInternal: (ITF38659I) Error 0x00049703 (SUCCESS, Facility: ITF, Code: 38659)

 

Cause

The "SNMP Interceptor" policy showed that the option "Unmatched events" was set to "are sent to the active message browser", meaning that all
messages not matched in the policy would pass dispite not matched in the policy.

As several thousands events a day are generated by the NNM server, these events passed without matching the policy rules. This meant that the OVOW server was too busy processing these unmatched events to match a node in the "Nodes" tree. Since these nodes do not exist in the Nodes tree the results as seen in the trace are generated.

Fix

When the option "Unmatched events" is set to "are sent to the active message browser" was changed to "are ignored" all was working fine again and the message load on the OVOW server returned to normal.