Speeding Up Reload Standard Backup Jobs

Speeding Up Reload Standard Backup Jobs

Reload's Standard (Daily Incremental) backup jobs can often be tuned to increase the speed of the backup.

There is one setting for Standard Backups that if finely tuned, will often speed up Reload Standard Backups. Sometimes by a magnitude of 500% faster.

How Reload Backs Up a Post Office

First, read this explanation of how Reload backs up GroupWIse post offices. A GroupWise post office has databases and BLOB files (the contents of the OFFILES directory) which all need to be backed up.

Reload can simultaneously backup the databases and BLOB files. There are many BLOB files that need to be correlated and backed up, but they are spread amongst 250+ directories. The process of backing up and correlating BLOB files requires that each BLOB file is considered and then backed up or verified as being backed up.

On a typical GroupWise post office there can be hundreds of thousands, and even millions of BLOB files that need to be correlated during each backup job. The correlation process takes a lot of time, although not a lot of data is necessarily being transferred. 

Reload is uniquely designed so that it can actually create simultaneous instances of DBCOPY (the underlying Novell GroupWise backup utility in Reload) to process a specific portion of the BLOBS files off of the OFFILES directories. By default, Reload is designed to run 2 simultaneous instances of DBCOPY for backing up the contents of the OFFILES directory. Each instance is processing about 125 of the directories off of the OFFILES directory.

Reload is engineered to allow for up to 10 simultaneous processes of DBCOPY that are individually processing 1/10th of the BLOBS file directories off of the OFFILES directory of a post office. These simultaneous instances of DBCOPY in Reload are called:

"Reload BLOBS Threads" . . . in Reload Web Administration

or

"BLOB Sync Threads" . . . in Reload Console Administration

The Reload BLOBS Threads are not to be confused with DBCOPY's built in threading ability, which although useful for backing up databases, is not all that useful for backing up the BLOBS files.

The very first time Reload backs up the BLOBS Threads from a post office, Reload will not use more than 2 BLOBS threads, even if it is configured to use more. The reason for this is that the first time Reload is backing up a post office, every BLOB file is being replicated. So spinning up more than 2 BLOBS threads is generally not necessary, because it can cause excessive disk contention which will effectively slow down the backup speed. However, after the initial backup, increasing the BLOBS Threads can greatly speed up Reload Standard Backup jobs.

Calculating The Proper BLOBS Threads Amount
As explained earlier, in order for Reload to backup a post office, it must back up two different types of files:

1.  Database Files

and

2.  BLOB Files

In Reload's Event Log, the processes that back up databases logs to the Event Log as "GRE_DBS". Below are a couple of lines that were copied from a Reload Server's Event Log: 

DATE: Wed_Sep_18 TIME: 15:32:20 - [GRE_DBS] OFUSER Backup Start Time: 15:29:53
DATE: Wed_Sep_18 TIME: 15:35:07 - [GRE_DBS] OFMSG Backup Start Time: 15:32:29
DATE: Wed_Sep_18 TIME: 15:35:14 - [GRE_DBS] Completed Database Portion of Backup
DATE: Wed_Sep_18 TIME: 15:35:14 - [GRE_DBS] Total Run Time: 5 Minutes 55 Seconds

In the Event Log, the process that backs up the BLOBS (OFFILES directory) contents is called "GRE_BLOBS".

DATE: Wed_Sep_18 TIME: 15:39:36 - [GRE_BLOBS] Finished BLOBS Portion of Backup
DATE: Wed_Sep_18 TIME: 15:39:36 - [GRE_BLOBS] Total Run Time: 8 Minutes, 29 Seconds

As a general rule-of-thumb for tuning the Reload BLOBS Threads, until in the Reload Event Log you can observe that the GRE_BLOBS portion of the backup finishes before the GRE_DBS portion of the backup, the Reload BLOBS Threads should be increased.Once you have increased the Reload BLOBS Threads to a maximum of 10, even if the GRE_BLOBS processes do not finish before the GRE_DBS process, you have tuned your Reload server to the maximum amount of BLOBS Threads possible. Reload does not support more than 10 Reload BLOBS Threads because testing has shown that increasing threads beyond 10 actually gives diminishing returns. Also, allocating more Reload BLOBS Threads than what is needed can also slow down backups.

The following screenshot is taken from Reload Web Administration, from the in-line documentation. This explanation may also help to further understand Reload BLOBS Threads.

Changing Reload BLOBS Threads
Following are the steps for changing the Reload BLOBS Threads setting in Reload:

Reload Web Administration

1.  Select the Reload Post Office Profile that you want to configure.

2.  Select the Configure tab

3.  Choose the Configure Backup Job Settings panel

4.  Choose OFFILES Backup Settings

5.  Select the setting: Reload BLOBS Threads and modify the threads as per this recommendations made earlier in this article.

Reload Console Administration

1.  Choose Profiles (Administer Profiles)

2.  Choose POST OFFICE PROFILE

3.  Choose the Reload Post Office profile you would like to administer

4.  Choose Standard (Configure Standard Daily Backup Jobs)

5.  Choose BLOBS (Standard Backup BLOBS OFFILES Threads Configuration)

6.  Choose BLOBS-Threads (Number of BLOBS Sync Threads), and modify the threads as per this recommendations made earlier in this article.

Final Notes
There is one other setting that can effect the speed of Reload Standard Backup Jobs. It is a setting called "High Performance Backups". This setting is automatically enabled, and for 98% of customers having this setting enabled is desirable. Since this setting rarely needs to be disabled, it is tucked away under the Advanced menu under Standard Backups configuration of a post office profile, only in Reload Console Administration.

If you are curious if this setting is enabled for a Reload Post Office profile, then look at the Event Log for that Post Office Profile, and look for a line that reads as follows:

DATE: Wed_Sep_18 TIME: 15:35:14 - [GRE_DBS] High Performance Backup: Enabled

This article was originally published in the GWAVA knowledgebase as article ID 2213