Environment
NetIQ Sentinel 7.3
NetIQ Sentinel 7.3.1
Situation
This problem can present several symptoms:
- The server may fail to fully initialize after the upgrade to 7.3.1
- Even if it fully initializes, it is not possible to edit a rule that has already been created. A generic REST error is displayed when trying to do so.
- It is not possible to delete the rule either. The system doesn't complain, but the rule doesn't get deleted either.
What is consistent on all occasions is that in the server0.*.log files there is an entry like this one:
Mon Aug 17 13:08:24 CEST 2015|SEVERE|Container Startup Thread|esecurity.base.ccs.proxy.ContainerProxy$1.uncaughtException
Uncaught exception
java.lang.StackOverflowError
at antlr.TokenBuffer.LA(TokenBuffer.java:80)
at antlr.LLkParser.LA(LLkParser.java:52)
at esecurity.ccs.correlation.impl.tpm.grammar.RuleParser.boolean_primary(RuleParser.java:920)
...
- The server may fail to fully initialize after the upgrade to 7.3.1
- Even if it fully initializes, it is not possible to edit a rule that has already been created. A generic REST error is displayed when trying to do so.
- It is not possible to delete the rule either. The system doesn't complain, but the rule doesn't get deleted either.
What is consistent on all occasions is that in the server0.*.log files there is an entry like this one:
Mon Aug 17 13:08:24 CEST 2015|SEVERE|Container Startup Thread|esecurity.base.ccs.proxy.ContainerProxy$1.uncaughtException
Uncaught exception
java.lang.StackOverflowError
at antlr.TokenBuffer.LA(TokenBuffer.java:80)
at antlr.LLkParser.LA(LLkParser.java:52)
at esecurity.ccs.correlation.impl.tpm.grammar.RuleParser.boolean_primary(RuleParser.java:920)
...
Resolution
This error will normally occur if the correlation rule that is having the problem has a lot of conditions in parentheses one after the other. For example a rule that starts like this:
filter(((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((e.dip match subnet ... will have over 70 levels of recursion, which result in a considerable amount of Stack memory space.
This issue is still under investigation by Sentinel engineering, but the current suspicion is that the upgrade from Java 7 to Java 8 that occurs when migrating to 7.3.1 is causing more memory to be consumed for the same rule. For this reason, a rule that was valid in 7.3 is now causing the stack to be filled in 7.3.1.
To avoid this problem, it is possible to increase the amount of memory allocated to the Java stack by modifying the -Xss parameter. This can be accomplished by modifying this line in the server.conf file in the /etc/opt/novell/sentinel/config folder:
wrapper.java.additional.20=-Xss228k
Increase it to -Xss512k or higher depending on your needs. After the modification, the sentinel service needs to be restarted.
By increasing this value, the heap size can grow to the amount of memory specified and it should be possible to load and edit the correlation rule without problems.
filter(((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((e.dip match subnet ... will have over 70 levels of recursion, which result in a considerable amount of Stack memory space.
This issue is still under investigation by Sentinel engineering, but the current suspicion is that the upgrade from Java 7 to Java 8 that occurs when migrating to 7.3.1 is causing more memory to be consumed for the same rule. For this reason, a rule that was valid in 7.3 is now causing the stack to be filled in 7.3.1.
To avoid this problem, it is possible to increase the amount of memory allocated to the Java stack by modifying the -Xss parameter. This can be accomplished by modifying this line in the server.conf file in the /etc/opt/novell/sentinel/config folder:
wrapper.java.additional.20=-Xss228k
Increase it to -Xss512k or higher depending on your needs. After the modification, the sentinel service needs to be restarted.
By increasing this value, the heap size can grow to the amount of memory specified and it should be possible to load and edit the correlation rule without problems.