MultiFire, Alert Escalation, Back To Normal, and Self Correction Delay Configuration

  • 7019882
  • 30-Nov-2011
  • 07-Aug-2017

Environment

Redline 3.5.1 build 1011 or higher

Situation

The purpose of this document is to provide a detailed overview of the Redline alert configurations.  Specifically, this document is going to address the common questions associated with MultiFire, Alert Escalation, Back To Normal, and Self Correction Delay.

Resolution

The MultiFire Alert, Alert Escalation, Back to Normal Alert, and Self Correction Delay are features that can help administrators more effectively manage thresholds and recommendations in their system and minimize false alerts. These settings can be enabled on a global level or individual agent level.  Let’s examine the following threshold/recommendation settings closer. MultiFire Alert:  MultiFire is a configuration option on an alert.  It provides a way for an administrator to be continuously reminded of a problem that does not get resolved.  An alert using MultiFire will send out a notification (Email, Broadcast, SNMP, API) at a defined interval until the alert is solved.  The MultiFire interval is set at the Control Center in the Configure | Analyzer page.   The MultiFire interval is a global setting and applies to all MultiFire events.  The interval cannot be set to a different length of time for each alert.
  • For step-by-step instruction on how to set MultiFire alerts, click here.
Alert Escalation:Alert Escalation provides the administrator with a way to increase the severity of an alert over time.   Alert Escalation works in conjunction with the MultiFire alert option which also must be enabled in order to function.  With each MultiFire interval, Alert Escalation raises the severity of an alert one level.  For instance, if an agent is not responding, the first alert may be set to a very low severity, (Informational).  When the MultiFire interval is reached, the alert level will be raised to the next higher severity, (Minor).  In this example, if the alert has not been solved over four MultiFire intervals, the alert will cycle from Informational, to Minor, to Problem, until finally a Critical alert will be generated.  If an alert reaches Critical severity it will remain there until marked solved.  Any alerts configured to use Alert Escalation will escalate to Critical severity if they are not resolved within the appropriate intervals.  You may set the starting severity of an alert, but you cannot set the ceiling.
  • For step-by-step instruction on how to setup Alert Escalation, click here.
Back to Normal Alert:  Back to Normal Alert is a feature designed to automatically manage alerts that occasionally exceed their conditions and then return to normal shortly thereafter.  An example of this may be a mail queue from an email blast, or number of threads in use on an agent.  When enabled, a Back to Normal type of notification is generated through the alert system when the condition returns to normal.  The alert is marked solved with "BackToNormal" as the user and it is removed from the Alerts page.  This is done without any need for administrator intervention.  The email address assigned to the alert will receive an email containing the string "BackToNormal"  indicating the alert has been resolved.
  • For step-by-step instruction on how to set Back to Normal, click here.
Self Correction Delay:  Self Correction Delay is designed for monitored agents that vary in their response time.  This is an ideal feature to use when dealing with a remote post office on a slow WAN.  The slow response can often trigger a false alert.  To minimize a potential false alert, Redline will forgo notification (the alert still fires) for the next reporting cycle (2 minutes by default).  If the situation resolves in the next cycle, the alert will be automatically resolved, and no notification will be sent.  The record of the alert remains stored in the database, and will be listed as solved by "SelfCorrected".  However, if two sequential cycles are missed, a notification will be sent and the alert will be handled according to it's normal settings.
  • For step-by-step instruction on how to setup Self Correction Delay, click here.

 

Below is a graph to help visualize what actions will be taken when each of these options are enabled.  They may be combined together with one rule; Alert Escalation must always run with MultiFireMultiFire does not require Alerts Escalation however.  Each of these options are contingent upon the alert itself being enabled.

Additional Information

This article was originally published in the GWAVA knowledgebase as article ID 2024.