Retain Server Errors with Too Many Open Files

  • 7021442
  • 15-Sep-2017
  • 11-May-2019

Environment

Retain Unified Archiving
SUSE Linux Enterprise Server 12

Situation

Retain server fails with error messages in the log files.
00:23:23, 526[RTWQuartzScheduler_Archive_Worker-1] [ERROR] DefaultCAAPIErrorHandleStrategy: reportError: processMailbox :: com.gwava.workerutils.DefaultCAAPIErrorHandleStrategy.returnCaller:55 :: java.lang.IllegalStateException: 
java.io.FileNotFoundException: /opt/beginfinite/retain/RetainWorker2/WEB-INF/lib/EasyGoogleApps.jar (Too many open files)
00:23:23, 526[RTWQuartzScheduler_Archive_Worker-1] [ERROR] StandardErrorHandleStrategy: reportError: WorkerJob :: com.gwava.caapi.process.RetainDredger.processMailboxes:429 :: com.gwava.archive.exceptions.RetainWorkerException: com.gwava.archive.exceptions.RetainWorkerException: java.lang.IllegalStateException:
java.io.FileNotFoundException: /opt/beginfinite/retain/RetainWorker2/WEB-INF/lib/EasyGoogleApps.jar (Too many open files),

Resolution

Increase the number of allowed open files for the Retain systemd process.

1. Login as root on the SLES12 server with Retain
2. Run mkdir -p /etc/systemd/system/retain.service.d
3. Create /etc/systemd/system/retain.service.d/override.conf
4. Add the following to the override.conf file:
[Service]
LimitNOFILE=60000
5. Save and quit
6. Run systemctl deamon-reload

Confirm the Retain daemon has more files descriptors available:

# systemctl show retain.service | grep LimitNOFILE
LimitNOFILE=60000

Cause

The number of allowable open files for the Retain systemd process was insufficient.