Huge log files of gigabytes where created on the Linux Access Gateway in less then a day.

  • 7005293
  • 04-Feb-2010
  • 26-Apr-2012

Environment

Novell Access Manager 3 Linux Access Gateway
Novell Access Manager 3.1 Linux Access Gateway
Novell Access Management 3.1 SP1 IR3 applied

Situation

Customer reported slow and in-responsive LAG boxes.
The catalina.out file showed the following:
INFO: queuing alert: Severe: Log file(s) have crossed 10000263090 bytes!
After a couple of those warnings with increasing size of the log the following was seen:
SEVERE: All threads (150) are currently busy, waiting. Increase maxThreads (150) or check the servlet status.
When those huge log files where log rotated they caused the box to go into high utilization and consume all of the threads making the box become in-responsive.
Under/var/opt/novell/tomcat5/webapps/nesp/WEB-INF/logs it was seen that huge log files in XML format existed.
The Access Manager configuration showed under the IDP, General, Logging page that only the component Application was set to Info.

Customer had a lot of policies and all the policy evaluation info was logged causing the log file to grow very fast to gigabytes in size.

Resolution

Changing the Log level to Off stopped the excessive logging.

Additional Information

Component file logging is more verbose than audit logging.
It increases processing load, and on a day-to-day basis, it should be enabled only to log error conditions and system warnings.
If a specific problem occurs, component file logging can be set to info or config to gather the information needed to isolate and repair the detected problem.
When the problem is resolved, component file logging should be reconfigured to log only error conditions and system warnings.