Environment
Situation
Sentinel is running with high event rate (>350 eps). I am
seeing event insertion performance issue. What can I do to improve
the performance?
Resolution
If you are using Oracle database, please refer to
"Setting Up the Oracle Call Interface (OCI) Event Insertion Strategy”
If you are using Microsoft SQL Server database, please refer to session
"Setting Up the Active Data Objects (ADO) Event Insertion Strategyâ€.
Setting Up the Oracle Call Interface (OCI) Event Insertion Strategy
Sentinel 5.1 provides a framework for plugging in different strategies to insert events into the database. Sentinel 5.1 provides two strategies to insert events into the Oracle database
ØJDBCLoadStrategy
ØOCILoadStrategy
The strategy to be used for inserting events is governed by the insert.strategy property of the EventStoreService component in das_binary.xml. The JDBC strategy is the default strategy configured out of the box.
The OCI strategy is a native insert strategy for faster event insertion. This strategy requires the Oracle OCI libraries be installed on the machine running the DAS component. The OCI strategy must be used in configurations where a high event rate is expected.
The number of events to be grouped together for insertion into the database is governed by the insert.batchsize property. This insert.batchsize property is used by all the event insert strategies.
To change Sentinel’s Event Insertion strategy from the default JDBC Insertion Strategy to the OCI Insertion Strategy, there are a few steps that need to be performed.
Changing Event Insertion strategy from JDBC to OCI Insertion Strategy
1.Ensure the Oracle OCI libraries are installed on the machine running the e-Security DAS component. You will need to know the path to ORACLE_HOME in the following steps.
2.Log into the machine from step 1 as the esecadm user.
3.Create a".profile†file in the esecadm user’s home dir. Put the following text in that file (modify the path to ORACLE_HOME to match your installation):
For 32 bit Oracle installation:
ORACLE_HOME=/build/home/oracle/OraHome
export ORACLE_HOME LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$ORACLE_HOME/lib
export LD_LIBRARY_PATH
For 64 bit Oracle installation,
ORACLE_HOME=/build/home/oracle/OraHome
export ORACLE_HOME LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$ORACLE_HOME/lib32
export LD_LIBRARY_PATH
4.Open the $ESEC_HOME/sentinel/config/das_binary.xml file for editing in any text editor.
5.Do a search on the following text:JDBCLoadStrategy
6.Change that text to:OCILoadStrategy
7.Save this change to the das_binary.xml file.
8.Restart the DAS Binary application. (Restarting DAS Binary can most easily be done by performing a "ps –ef | grep DAS_Binary†to get the process ID, killing that process, and then letting the Sentinel Watchdog automatically restart the process.) Once DAS Binary has been restarted, the $ESEC_HOME/sentinel/lib/libocievent.so library will be loaded and used to perform the Event insertions into the database via OCI.
Additional OCI Event Insertion Options
In addition to specifying the "OCILoadStrategy†in the das_binary.xml file, there are several other OCI-related options that can also be configured.
Øinsert.batchsize – This setting allows you to configure the maximum number of Events to insert into the database at a time.
Øinsert.oci.workerCount – This setting allows you to configure the number of threads being used to insert Event data into the database.??insert.oci.queueWaitTime – This setting specifies the max time in seconds to wait before inserting the data from the inbound queue into the database. Whenever a full "batchsize†of events is received, the entire batch is inserted. But if the inbound flow of events is slow, the queue wait time is used to determine when to do the database insertion (even if a full batch of events has not yet been received).
Øinsert.oci.highWatermark – The inbound Event queue’s high water mark.
Øinsert.oci.lowWatermark – The inbound Event queue’s low watermark.
Øinsert.oci.optimizationFlag – Optimization flag. "on†or "offâ€.
OCI Debugging Tips
The OCI interface will log messages to the $ESEC_HOME/sentinel/log/ocievent.log file. Initial messages written to the log file should include success (or fail) database connection messages… This is a good place to check to verify that the OCI library was loaded and configured correctly.
The OCI interface will also log errors to the das_binary log file located in the $ESEC_HOME/sentinel/log directory. Errors logged to the das_binary log file include failures to locate/load the libocievent.so library, failures to connect to the database, and failures to insert Events/Event Associations.
If error messages indicate that the "libocievent.so†file is not being located or loaded, then there are three things to check:
1.Make sure the Oracle OCI libraries are installed.
2.Make sure that the "libocievent.so†file is located in the $ESEC_HOME/sentinel/lib directory.
3.Make sure that the $ESEC_HOME/sentinel/lib directory is in the "esecadm†user’s "LD_LIBRARY_PATHâ€. If not, you can update the LD_LIBRARY_PATH in the "esecadm†user’s .profile
4.Make sure that the environment variables ORACLE_HOME and LD_LIBRARY_PATH are updated properly in esecadm’s user environment variables as described in the section "Changing Event Insertion strategy from JDBC to OCI Insertion Strategyâ€.
Setting Up the Active Data Objects (ADO) Event Insertion Strategy
Sentinel 5.1 provides a framework for plugging in different strategies to insert events into the database. Sentinel 5.1 provides two strategies to insert events into the MS SQL database:
ØJDBCLoadStrategy
ØADOLoadStrategy
The strategy to be used for inserting events is governed by the insert.strategy property of the EventStoreService component in das_binary.xml.
The JDBC strategy is the default strategy configured out of the box.
The ADO strategy is a native insert strategy for faster event insertion. This strategy requires the additional Windows packages be installed on the machine running the DAS component. See the section below for information on what packages must be installed. The ADO strategy must be used in configurations where a high event rate is expected.
The number of events to be grouped together for insertion into the database is governed by the insert.batchsize property. This insert.batchsize property is used by all the event insert strategies.
The sections below describe how to switch to ADO load strategies.
Prerequisites for ADOLoadStrategy
The ADO native connector needs the .net framework and the J# redistributable package to be installed on the machine running DAS Binary.
NOTE:You will need to uninstall any older versions of the .net framework and the J# redistributable package and install the versions listed in the following order.
Microsoft .NET Framework Version 2.0 Redistributable Package (x86)
Microsoft Visual J# Version 2.0 Redistributable Package
Setting up ADO Load Event Insertion Strategy
To change Sentinel’s Event Insertion strategy from the default JDBC Insertion Strategy to the ADO Insertion Strategy, there are a few steps that need to be performed.
Changing from JDB Insertion Strategy to ADO Insertion Strategy
1.Using a text editor, open %ESEC_HOME%\sentinel\config\das_binary.xml.
2.Do a search on the following text:
JDBCLoadStrategy
3.Change that text to:
ADOLoadStrategy
4.Save this change to the das_binary.xml file.
5.Restart the DAS Binary application.
Once DAS Binary has been restarted, the %ESEC_HOME%\Sun-1.4.2\bin\ EventInsert.dll and EventJNICLIBridge.dll will be loaded and used to perform the Event insertions into the database via ADO.
ADODebugging Tips
The ADO interface will only log error messages to the %ESEC_HOME%\sentinel\log\ADOEventStoreError.log file. Initial error messages written to the log file may include database connection failed messages. This file will also log exceptions that occur while inserting events into the database. Please note: only Errors are logged to this file,
To verify ADO connected and loaded properly please check the das_binary log file located in the %ESEC_HOME%\sentinel\log directory.
The ADO interface also logs errors to the das_binary log file located in the %ESEC_HOME%\sentinel\log directory. Errors logged to the das_binary log file include failures to locate/load the EventJNICLIBridge.dll, failures to connect to the database and failures to insert Events/Event Associations.
If error messages indicate that the native connectors have not been loaded properly, check the following:
ØMake sure that the machine has the right version of .net framework and J# redistributable package installed.
ØMake sure that the "EventJNICLIBridge.dll†and the "EventInsert.dll†files are located in the %ESEC_HOME%\Sun-1.4.2\bin\ directory.