Filr Appliance - Out of disk space

  • 7020997
  • 20-Jun-2017
  • 27-Aug-2018

Environment

Novell Filr 3.1 Appliance
Novell Filr 3.1 Administration

Situation

Various problems can result from running out of disk space on the Filr Appliance, such as:

1. When attempting to access the appliance via port 9443 or 8443:
Error: HTTP Status 500 

2. When attempting to view the details of a file, or preview a file:
Error: I/O error: Could not create directory [/vastorage/filr/cachefilestore/converted_html_files/kablink/<additional path to file>

Resolution

The Filr service is stopped automatically if disk usage on the root drive goes above 90%. When this happens, /var/log/messages contains an entry about this.

Identify the disk space configuration and usage

Check storage using the command
df -m
or
df -h

Check inodes using the command
df -i


Expand the disk space available

NOTE: Ensure that you have a reliable backup before proceeding, as errors occurring when the disk is expanded can render the system unusable.

Important: The boot partition cannot be expanded.

To expand the storage space allocated for the /vastorage and /var partitions, do the following:
Use the tools provided by your virtualization platform vendor to expand the virtual disks that contain the partitions your are expanding.

Once you can again gain access to the Filr port 9443 Appliance Console, use the Appliance Configuration "Storage" tool to expand the partition(s) as needed. When expanding the disk from Filr port 9443 Appliance Console, please make sure all terminals to appliance are closed (including VMWare console).

Another method is to delete files directly from a terminal session on the Filr Appliance. This can be useful for deleting large log files, etc. 

To view the largest 10 files on the device, use this command:
find /var -printf '%s %p\n'| sort -nr | head -10

To view all files in ascending order of size, use this command:
find . -type f  -exec du -h {} + | sort -h

Cause

Insufficient disk space can be caused by enabling "Content Indexing" at the Net Folder Server level when creating a Net Folder Server (Port 8443 Filr Admin Console > Management > Net Folder Servers). Context Indexing indexes any text within all files in order to be able to search within the content of files. This process involves writing temporary files on the Filr appliance, which get cleaned up about once a day. This can cause the disk to become full, resulting a failure of the Filr server before it has a chance to clean up the temporary files.

For this reason, the Filr Administrative User Interface Reference (8.1.2 Creating a Net Folder Server) states:

IMPORTANT: Enabling indexing at the Net Folder Server level is not usually a best practice. Rather, indexing should be confined to Net Folders. This is why disk space planning on the Planning Worksheets is done at the Net Folder level. A possible exception to this rule is enabling content indexing of Home Net Folder Servers.

Additional Information

If excessive Net Folder indexing results in running out of iNodes (as displayed by "df -i"), you can delete the  files  under /vastorage/filr/cachefilestore/kablink/ (but not the kablink directory itself), which should temporarily resolve the problem. Disabling the file indexing (or reducing indexing by limiting it to smaller Net Folders) may prevent a recurrence.