WAN Backup Troubleshooting

  • 7019323
  • 12-Nov-2009
  • 29-Aug-2017

Environment


Reload (all versions)

Situation


A Reload post office profile used to backup regularly, but it does not perform a backup any longer.

A Reload post office profile cannot get a backup.

NOTE: This is most likely to occur when the post office is across a slow WAN link from the Reload server. The basic premise of this article is that the problems getting a backup is because of the fact that the backup is being performed over a WAN link. Following this article will help many customers to get Standard Backup Job, however some customers will not be able to. If taking the steps in this article do not allow a Standard Backup Job to run, then you should strongly consider putting a Reload server in close LAN proximity to the post office in question.


Resolution

[ Discussion ]

This kind of a problem is generally due to the fact that the WAN link is unreliable or just too slow to feasibly pull a backup over with Reload. Reload does require that approximately 12% of the size of the post office is replicated. So if a post office is 100 gigabytes, then Reload is attempting to replicate approximately 12 gigabytes of data. Consider the speed of your WAN links, and determine if they will support this kind of a data transfer.

There are things that you can do in Reload to try and troubleshoot backups over WANs, and to design Reload's Standard backup jobs to be more sparing on WAN usage. These steps will cause Reload to use the least amount of bandwidth at any given moment as is possible. These steps are meant for WAN-based backups, not LAN based generally. The net effect of these changes is that only one thread is used and so the speed of the backup time will be slower, but hopefully the backup will complete, which is the primary goal.

[ Standard Backup Configuration Steps ]

Configure Reload Standard Backups to use the least amount of WAN resources at any given time during a Standard Backup Job for this particular profile. NOTE: Only make these changes for the Reload post office profile for which you are trying to get a completed Standard backup job.  

1. Edit the post office profile in Reload Console Administration

2. Select Standard

3. Select BLOBS

4. Select DBCOPY-Threads and set the DBCOPY Threads to 1

5. Select BLOBS-Threads and set the BLOB Sync Threads to 1

6. Select the < BACK > button

7. Back at the Standard Backup Main Menu select Database

8. Select DBCOPY-Threads and change the DBCOPY Threads to 1

9. Select the < BACK > button

11. Back at the Standard Backup Main Menu select SmartPurge

12. Select Pre-Processing and turn the Pre-processing routines to [ Disabled ]

13. Select the < BACK > button

14. Back at the Standard Backup Main Menu select Advanced

15. Select  High and set High Performance Standard Backups to [ Disabled ] 

Now with these settings in place Reload Standard Backups for the profile that you changed are designed to:

- Never establish more than one connection to the server with the Post Office on it

- Pull the least amount of data at any given time via only one instance of DBCOPY

- DBCOPY will pull the least amount of data because it is only using one thread

[ Reload Server System Configuration Steps ]

The Reload server can be configured to allow "Concurrent Standard Backup Jobs". Turn this feature off, so that no other jobs run during the time of the Standard backup job of the profile you are trying to get a backup for. This feature can be disabled in Reload Console Administration. From the Main Menu in Reload Console Administation, select Jobs | Configuration and set Concurrent to [ Disabled ]

[ When to Run a Standard Backup Job ]

Try running the Standard backup job during a time in which the WAN link that you are using is likely to be less busy.

[ Reload Server NFS Client Connectivity Configuration Ideas ]

If the post office that you are attempting to backup is on the Linux platform, there are some further fine tuning settings you can make relative to NFS, as long as you are using NFS connectivity vs. NCPFS connectivity. Here is an explanation of where they are and what you might set them to:

1.  Edit the Reload post office profile that you are trying to get a Standard backup job for.

2.  Select the Advanced menu.

3.  As long as the connectivity to the post office is NFS/Linux, then you will see the menu options for MTU and SPEED.  See, "Off-Site Server / Slow or Failing Backups Over Slow WAN Links" for an explanation of the MTU setting.

Since this is a WAN connection, then the speed should be set to "Slow". Changing the speed to Slow configures the NFS client to use the "soft" connection setting vs. the "hard" connection settings. You can read more about these settings on the Internet to understand what these settings are doing.

If - in your research of NFS client options - you would like to experiment with a particular NFS client option, you can have the Reload NFS connectivity module use the setting by adding the setting to the *.conf file of the profile that you are troubleshooting. Here's how you might do this:

1.  Go to the /opt/beginfinite/reload/conf directory, and find the *.conf file for your profile. If the profile is called PO1, then the name of the file is po1.conf

2.  In our example, the profile is called po1. So edit the po1.conf file and look for the line that reads:

CLIENT_NFS_OPTIONS

for example, it might read:

CLIENT_NFS_OPTIONS="tcp,rsize=8192,wsize=8192,hard"

As to what to put into the CLIENT_NFS_OPTIONS area, that is up to you. There are lots of guides on the Internet regarding NFS tuning. Consider searching the Internet for "NFS Client Tuning".

[ Summary ]

If using the methods in this article did allow a Standard backup job to run, then you can consider which settings to re-enable. For example, perhaps you will want to re-enable "Concurrent Standard Backup Jobs" first. If just this one setting can be re-enabled, than you might have a sufficient working environment to get backups. The Standard Backup Job for the one Reload profile in question will always run slowly, but if "Concurrent Standard Backup Jobs" is enabled and works, then at least other profiles will be able to run backup jobs without being negatively impacted by the slower backup job.

Additional Information

This article was originally published in the GWAVA knowledgebase as article ID 1543.