Environment
Sentinel 6.1.x Sentinel Control Center
Sentinel 6.0.x Sentinel Control Center
Sentinel 6.0.x Sentinel Control Center
Sentinel 5.1.3 Sentinel Control Center
Sentinel Core services
Sentinel Core services
Situation
Sentinel Control Center becomes unresponsive.
Sentinel Control Center Gui stops displaying, screen turns
white.
After logging in to Sentinel Control Center, Active views'
most recent data is delayed for a few minutes, and some times there
is gray area in between chart data.
Active Views show messages like "Lost connection to server, attempting to reconnect..." or "Active View is disconnected from the server".
Data seems to be written to the database correctly.
Resolution
First step is to be sure you are running this latest Support
Pack and hotfixes for Sentinel. If this does not resolve the
problem, then continue with the following recommendations:
If the systim is not CPU and memory bound, and there is no
network problem, this problem could be due to disk IO issues caused
by the Antivirus software on the machine. This slow disk IO
preventes Active Views from caching events in a timely manner which
causes the charts to fall behind and eventually lose connectivity
altogether. The slow disk IO also indirectly affectes other parts
of Sentinel, causing strange behavior with ESM and other GUI
components.
The resolution is to exclude the directories Sentinel uses for caching from the Antivirus configuration.
1. Directories that need to be excluded from anti virus:
It is preferred to have %esec_home% directory excluded from antivirus configuration. If this is not possible, please make sure at least these three subdirectories are excluded:
%esec_home%\data
%esec_home%\log
%esec_home%\3rdparty
2. Modify configuration.xml so that all tmp directories point to %esec_home%\data\tmp - see below for example. Please ensure that the %esec_home%\data\tmp directory exists.
< process component="DAS" depends="UNIX Communication Server,Windows Communication Server" image=""$(ESEC_JAVA_HOME)/java" -server -Dsrv_name=DAS_Query -Xmx160m -Xms64m -XX:+UseParallelGC -Xss136k -Xrs -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=../log/DAS_Query.hprof -Duser.language=en -Djava.net.preferIPv4Stack=true -Dfile.encoding=UTF8 -Desecurity.cache.directory=../data/das_query.cache -Desecurity.dataobjects.config.file=/xml/BaseMetaData.xml,/xml/WorkflowMetaData.xml -Djava.util.logging.config.file=../config/das_query_log.prop -Djava.security.auth.login.config=../config/auth.login -Djava.security.krb5.conf=../config/krb5.conf -Desecurity.execution.config.file=../config/execution.properties -Dcom.esecurity.configurationfile=../config/configuration.xml-Djava.io.tmpdir=../data/tmp -jar../lib/ccsbase.jar ..//config//das_query.xml" min_instances="1" name="DAS_Query" post_startup_delay="20" type="container" working_directory="$(ESEC_HOME)/data"/>
< process component="DAS" depends="UNIX Communication Server,Windows Communication Server" image=""$(ESEC_JAVA_HOME)/java" -server -Dsrv_name=DAS_Binary -Xmx200m -Xms64m -XX:+UseParallelGC -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=../log/DAS_Binary.hprof -Xss136k -Xrs -Duser.language=en -Djava.net.preferIPv4Stack=true -Dfile.encoding=UTF8 -Desecurity.cache.directory=../data/das_binary.cache -Desecurity.dataobjects.config.file=/xml/BaseMetaData.xml -Djava.util.logging.config.file=../config/das_binary_log.prop -Dcom.esecurity.configurationfile=../config/configuration.xml -Djava.security.auth.login.config=../config/auth.login -Djava.security.krb5.conf=../config/krb5.conf-Djava.io.tmpdir=../data/tmp -jar../lib/ccsbase.jar ..//config//das_binary.xml" min_instances="1" name="DAS_Binary" post_startup_delay="20" type="container" working_directory="$(ESEC_HOME)/data"/>
< process component="DAS" depends="UNIX Communication Server,Windows Communication Server" image=""$(ESEC_JAVA_HOME)/java" -server -Dsrv_name=DAS_Aggregation -Xmx160m -Xms64m -XX:+UseParallelGC -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=../log/DAS_Aggregation.hprof -Xss136k -Xrs -Duser.language=en -Djava.net.preferIPv4Stack=true -Dfile.encoding=UTF8 -Desecurity.cache.directory=../data/das_aggregation.cache -Desecurity.dataobjects.config.file=/xml/BaseMetaData.xml -Djava.util.logging.config.file=../config/das_aggregation_log.prop -Dcom.esecurity.configurationfile=../config/configuration.xml -Djava.security.auth.login.config=../config/auth.login -Djava.security.krb5.conf=../config/krb5.conf-Djava.io.tmpdir=../data/tmp -jar../lib/ccsbase.jar ..//config//das_aggregation.xml" min_instances="1" name="DAS_Aggregation" post_startup_delay="20" type="container" working_directory="$(ESEC_HOME)/data"/>
< process component="DAS" depends="UNIX Communication Server,Windows Communication Server" image=""$(ESEC_JAVA_HOME)/java" -server -Dsrv_name=DAS_RT -Xmx200m -Xms64m -XX:+UseParallelGC -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=../log/DAS_RT.hprof -Xss136k -Xrs -Duser.language=en -Djava.net.preferIPv4Stack=true -Dfile.encoding=UTF8 -Desecurity.cache.directory=../data/das_rt.cache -Desecurity.dataobjects.config.file=/xml/BaseMetaData.xml -Djava.util.logging.config.file=../config/das_rt_log.prop -Dcom.esecurity.configurationfile=../config/configuration.xml -Djava.security.auth.login.config=../config/auth.login -Djava.security.krb5.conf=../config/krb5.conf-Djava.io.tmpdir=../data/tmp -jar../lib/ccsbase.jar ..//config//das_rt.xml" min_instances="1" name="DAS_RT" post_startup_delay="20" type="container" working_directory="$(ESEC_HOME)/data"/>
< process component="DAS" depends="UNIX Communication Server,Windows Communication Server" image=""$(ESEC_JAVA_HOME)/java" -server -Dsrv_name=DAS_iTRAC -Xmx80m -Xms64m -XX:+UseParallelGC -Xss136k -Xrs -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=../log/DAS_iTRAC.hprof -Duser.language=en -Djava.net.preferIPv4Stack=true -Dfile.encoding=UTF8 -Desecurity.cache.directory=../data/das_itrac.cache -Desecurity.dataobjects.config.file=/xml/BaseMetaData.xml,/xml/ActMetaData.xml,/xml/WorkflowMetaData.xml -Djava.util.logging.config.file=../config/das_itrac_log.prop -Desecurity.execution.config.file=../config/execution.properties -Dcom.esecurity.configurationfile=../config/configuration.xml -Djava.security.auth.login.config=../config/auth.login -Djava.security.krb5.conf=../config/krb5.conf -Desecurity.remote.timeout=180-Djava.io.tmpdir=../data/tmp -jar../lib/ccsbase.jar ..//config//das_itrac.xml" min_instances="1" name="DAS_iTRAC" post_startup_delay="20" type="container" working_directory="$(ESEC_HOME)/data"/>
< process component="COMM_SERVER" depends="UNIX Communication Server,Windows Communication Server" image=""$(ESEC_JAVA_HOME)/java" -server -Dsrv_name=DAS_Proxy -Xmx80m -Xms64m -XX:+UseParallelGC -Xss136k -Xrs -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=../log/DAS_Proxy.hprof -Duser.language=en -Djava.net.preferIPv4Stack=true -Dfile.encoding=UTF8 -Desecurity.cache.directory=../data/das_proxy.cache -Desecurity.dataobjects.config.file=/xml/BaseMetaData.xml -Djava.util.logging.config.file=../config/das_proxy_log.prop -Desecurity.execution.config.file=../config/execution.properties -Dcom.esecurity.configurationfile=../config/configuration.xml -Djava.security.auth.login.config=../config/auth.login -Djava.security.krb5.conf=../config/krb5.conf -Desecurity.remote.timeout=180-Djava.io.tmpdir=../data/tmp -jar../lib/ccsbase.jar ..//config//das_proxy.xml" min_instances="1" name="DAS_Proxy" post_startup_delay="20" remote_control="false" type="container" working_directory="$(ESEC_HOME)/data"/>
< !-- correlation_engine -->
< process component="CORRELATION" depends="UNIX Communication Server,Windows Communication Server" image=""$(ESEC_JAVA_HOME)/java" -server -Dsrv_name=Correlation_Engine -Xmx200m -Xms64m -XX:+UseParallelGC -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=../log/Correlation_Engine.hprof -Xss136k -Xrs -Duser.language=en -Djava.net.preferIPv4Stack=true -Dfile.encoding=UTF8 -Desecurity.cache.directory=../data/correlation_engine.cache -Desecurity.dataobjects.config.file=/xml/BaseMetaData.xml -Djava.util.logging.config.file=../config/correlation_engine_log.prop -Desecurity.execution.config.file=../config/execution.properties -Dcom.esecurity.configurationfile=../config/configuration.xml -Djava.security.auth.login.config=../config/auth.login -Djava.security.krb5.conf=../config/krb5.conf-Djava.io.tmpdir=../data/tmp -jar../lib/ccsbase.jar ..//config//correlation_engine.xml" min_instances="1" name="Correlation_Engine" post_startup_delay="20" type="container" working_directory="$(ESEC_HOME)/data"/>
< process component="COLLECTOR_MANAGER" depends="UNIX Communication Server,Windows Communication Server,DAS_Query,DAS_Binary" image=""$(ESEC_JAVA_HOME)/java" -Xdebug -Xnoagent -Xrunjdwp:transport=dt_socket,address=12345,server=y,suspend=n -server -Dsrv_name=Collector_Manager -Xmx200m -Xms64m -XX:+UseParallelGC -Xss136k -Xrs -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=../log/Collector_Manager.hprof -XX:NewRatio=2 -Duser.language=en -Djava.net.preferIPv4Stack=true -Dfile.encoding=UTF8 -Desecurity.cache.directory=../data/collector_mgr.cache -Desecurity.dataobjects.config.file=/xml/BaseMetaData.xml,/xml/AgentManagerMetaData.xml -Desecurity.router.config.file=../config/event-router.properties -Dcom.esecurity.configurationfile=../config/configuration.xml -Djava.util.logging.config.file=../config/collector_mgr_log.prop-Djava.io.tmpdir=../data/tmp -jar../lib/ccsbase.jar ..//config//collector_mgr.xml" min_instances="1" name="Collector_Manager" post_startup_delay="0" remote_control="true" type="container" working_directory="$(ESEC_HOME)/data"/>
3. Modify control_center.ja to include the following line to modify the temp directory for Sentinel Control Center
-Djava.io.tmpdir="\data\tmp"
Note: you need to fill in the with specifics in
your environment
Currently, there might not be ways to change memory settings for control_center.exe, if you want to run Sentinel Control Center with high memory settings, you will have to run the control_center.bat file.
The resolution is to exclude the directories Sentinel uses for caching from the Antivirus configuration.
1. Directories that need to be excluded from anti virus:
It is preferred to have %esec_home% directory excluded from antivirus configuration. If this is not possible, please make sure at least these three subdirectories are excluded:
%esec_home%\data
%esec_home%\log
%esec_home%\3rdparty
2. Modify configuration.xml so that all tmp directories point to %esec_home%\data\tmp - see below for example. Please ensure that the %esec_home%\data\tmp directory exists.
< process component="DAS" depends="UNIX Communication Server,Windows Communication Server" image=""$(ESEC_JAVA_HOME)/java" -server -Dsrv_name=DAS_Query -Xmx160m -Xms64m -XX:+UseParallelGC -Xss136k -Xrs -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=../log/DAS_Query.hprof -Duser.language=en -Djava.net.preferIPv4Stack=true -Dfile.encoding=UTF8 -Desecurity.cache.directory=../data/das_query.cache -Desecurity.dataobjects.config.file=/xml/BaseMetaData.xml,/xml/WorkflowMetaData.xml -Djava.util.logging.config.file=../config/das_query_log.prop -Djava.security.auth.login.config=../config/auth.login -Djava.security.krb5.conf=../config/krb5.conf -Desecurity.execution.config.file=../config/execution.properties -Dcom.esecurity.configurationfile=../config/configuration.xml-Djava.io.tmpdir=../data/tmp -jar../lib/ccsbase.jar ..//config//das_query.xml" min_instances="1" name="DAS_Query" post_startup_delay="20" type="container" working_directory="$(ESEC_HOME)/data"/>
< process component="DAS" depends="UNIX Communication Server,Windows Communication Server" image=""$(ESEC_JAVA_HOME)/java" -server -Dsrv_name=DAS_Binary -Xmx200m -Xms64m -XX:+UseParallelGC -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=../log/DAS_Binary.hprof -Xss136k -Xrs -Duser.language=en -Djava.net.preferIPv4Stack=true -Dfile.encoding=UTF8 -Desecurity.cache.directory=../data/das_binary.cache -Desecurity.dataobjects.config.file=/xml/BaseMetaData.xml -Djava.util.logging.config.file=../config/das_binary_log.prop -Dcom.esecurity.configurationfile=../config/configuration.xml -Djava.security.auth.login.config=../config/auth.login -Djava.security.krb5.conf=../config/krb5.conf-Djava.io.tmpdir=../data/tmp -jar../lib/ccsbase.jar ..//config//das_binary.xml" min_instances="1" name="DAS_Binary" post_startup_delay="20" type="container" working_directory="$(ESEC_HOME)/data"/>
< process component="DAS" depends="UNIX Communication Server,Windows Communication Server" image=""$(ESEC_JAVA_HOME)/java" -server -Dsrv_name=DAS_Aggregation -Xmx160m -Xms64m -XX:+UseParallelGC -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=../log/DAS_Aggregation.hprof -Xss136k -Xrs -Duser.language=en -Djava.net.preferIPv4Stack=true -Dfile.encoding=UTF8 -Desecurity.cache.directory=../data/das_aggregation.cache -Desecurity.dataobjects.config.file=/xml/BaseMetaData.xml -Djava.util.logging.config.file=../config/das_aggregation_log.prop -Dcom.esecurity.configurationfile=../config/configuration.xml -Djava.security.auth.login.config=../config/auth.login -Djava.security.krb5.conf=../config/krb5.conf-Djava.io.tmpdir=../data/tmp -jar../lib/ccsbase.jar ..//config//das_aggregation.xml" min_instances="1" name="DAS_Aggregation" post_startup_delay="20" type="container" working_directory="$(ESEC_HOME)/data"/>
< process component="DAS" depends="UNIX Communication Server,Windows Communication Server" image=""$(ESEC_JAVA_HOME)/java" -server -Dsrv_name=DAS_RT -Xmx200m -Xms64m -XX:+UseParallelGC -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=../log/DAS_RT.hprof -Xss136k -Xrs -Duser.language=en -Djava.net.preferIPv4Stack=true -Dfile.encoding=UTF8 -Desecurity.cache.directory=../data/das_rt.cache -Desecurity.dataobjects.config.file=/xml/BaseMetaData.xml -Djava.util.logging.config.file=../config/das_rt_log.prop -Dcom.esecurity.configurationfile=../config/configuration.xml -Djava.security.auth.login.config=../config/auth.login -Djava.security.krb5.conf=../config/krb5.conf-Djava.io.tmpdir=../data/tmp -jar../lib/ccsbase.jar ..//config//das_rt.xml" min_instances="1" name="DAS_RT" post_startup_delay="20" type="container" working_directory="$(ESEC_HOME)/data"/>
< process component="DAS" depends="UNIX Communication Server,Windows Communication Server" image=""$(ESEC_JAVA_HOME)/java" -server -Dsrv_name=DAS_iTRAC -Xmx80m -Xms64m -XX:+UseParallelGC -Xss136k -Xrs -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=../log/DAS_iTRAC.hprof -Duser.language=en -Djava.net.preferIPv4Stack=true -Dfile.encoding=UTF8 -Desecurity.cache.directory=../data/das_itrac.cache -Desecurity.dataobjects.config.file=/xml/BaseMetaData.xml,/xml/ActMetaData.xml,/xml/WorkflowMetaData.xml -Djava.util.logging.config.file=../config/das_itrac_log.prop -Desecurity.execution.config.file=../config/execution.properties -Dcom.esecurity.configurationfile=../config/configuration.xml -Djava.security.auth.login.config=../config/auth.login -Djava.security.krb5.conf=../config/krb5.conf -Desecurity.remote.timeout=180-Djava.io.tmpdir=../data/tmp -jar../lib/ccsbase.jar ..//config//das_itrac.xml" min_instances="1" name="DAS_iTRAC" post_startup_delay="20" type="container" working_directory="$(ESEC_HOME)/data"/>
< process component="COMM_SERVER" depends="UNIX Communication Server,Windows Communication Server" image=""$(ESEC_JAVA_HOME)/java" -server -Dsrv_name=DAS_Proxy -Xmx80m -Xms64m -XX:+UseParallelGC -Xss136k -Xrs -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=../log/DAS_Proxy.hprof -Duser.language=en -Djava.net.preferIPv4Stack=true -Dfile.encoding=UTF8 -Desecurity.cache.directory=../data/das_proxy.cache -Desecurity.dataobjects.config.file=/xml/BaseMetaData.xml -Djava.util.logging.config.file=../config/das_proxy_log.prop -Desecurity.execution.config.file=../config/execution.properties -Dcom.esecurity.configurationfile=../config/configuration.xml -Djava.security.auth.login.config=../config/auth.login -Djava.security.krb5.conf=../config/krb5.conf -Desecurity.remote.timeout=180-Djava.io.tmpdir=../data/tmp -jar../lib/ccsbase.jar ..//config//das_proxy.xml" min_instances="1" name="DAS_Proxy" post_startup_delay="20" remote_control="false" type="container" working_directory="$(ESEC_HOME)/data"/>
< !-- correlation_engine -->
< process component="CORRELATION" depends="UNIX Communication Server,Windows Communication Server" image=""$(ESEC_JAVA_HOME)/java" -server -Dsrv_name=Correlation_Engine -Xmx200m -Xms64m -XX:+UseParallelGC -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=../log/Correlation_Engine.hprof -Xss136k -Xrs -Duser.language=en -Djava.net.preferIPv4Stack=true -Dfile.encoding=UTF8 -Desecurity.cache.directory=../data/correlation_engine.cache -Desecurity.dataobjects.config.file=/xml/BaseMetaData.xml -Djava.util.logging.config.file=../config/correlation_engine_log.prop -Desecurity.execution.config.file=../config/execution.properties -Dcom.esecurity.configurationfile=../config/configuration.xml -Djava.security.auth.login.config=../config/auth.login -Djava.security.krb5.conf=../config/krb5.conf-Djava.io.tmpdir=../data/tmp -jar../lib/ccsbase.jar ..//config//correlation_engine.xml" min_instances="1" name="Correlation_Engine" post_startup_delay="20" type="container" working_directory="$(ESEC_HOME)/data"/>
< process component="COLLECTOR_MANAGER" depends="UNIX Communication Server,Windows Communication Server,DAS_Query,DAS_Binary" image=""$(ESEC_JAVA_HOME)/java" -Xdebug -Xnoagent -Xrunjdwp:transport=dt_socket,address=12345,server=y,suspend=n -server -Dsrv_name=Collector_Manager -Xmx200m -Xms64m -XX:+UseParallelGC -Xss136k -Xrs -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=../log/Collector_Manager.hprof -XX:NewRatio=2 -Duser.language=en -Djava.net.preferIPv4Stack=true -Dfile.encoding=UTF8 -Desecurity.cache.directory=../data/collector_mgr.cache -Desecurity.dataobjects.config.file=/xml/BaseMetaData.xml,/xml/AgentManagerMetaData.xml -Desecurity.router.config.file=../config/event-router.properties -Dcom.esecurity.configurationfile=../config/configuration.xml -Djava.util.logging.config.file=../config/collector_mgr_log.prop-Djava.io.tmpdir=../data/tmp -jar../lib/ccsbase.jar ..//config//collector_mgr.xml" min_instances="1" name="Collector_Manager" post_startup_delay="0" remote_control="true" type="container" working_directory="$(ESEC_HOME)/data"/>
3. Modify control_center.ja to include the following line to modify the temp directory for Sentinel Control Center
-Djava.io.tmpdir="
Note: you need to fill in the
Currently, there might not be ways to change memory settings for control_center.exe, if you want to run Sentinel Control Center with high memory settings, you will have to run the control_center.bat file.