Environment
Novell Data Synchronizer Mobility Pack
Situation
After applying DataSync build 579, attachments are still being
downloaded as text.txt.
Attachments are not downloading right away, but showing up on the device sometime after the email has been delivered.
Emails are not syncing because of attachment backlog.
Attachments are not downloading right away, but showing up on the device sometime after the email has been delivered.
Emails are not syncing because of attachment backlog.
Resolution
Be sure that File Patch 1 for Data Synchronizer 1.2 (Build 579) or
later has been applied first, before proceeding with the steps
below. If after applying File Patch 1, the back log of
attachments has not cleared or attachments are still downloading
with the text.txt format, it could be that Datasync just can't keep
up with the demand. By having a backlog of attachments,
DataSync still delivers the email, but downloads the attachment
after some time, which makes it appear that the attachment failed
to download or did not get converted to a readable
format.
Please perform the following steps to increase the number of threads used by the GroupWise Connector for attachment handling/processing:
Please perform the following steps to increase the number of threads used by the GroupWise Connector for attachment handling/processing:
- Edit the GroupWise connector file located: /etc/datasync/configengine/engines/default/pipelines/pipeline1/connectors/groupwise/connector.xml
- Find and change <numWorkers>4</numWorkers> to a larger number such as 8.
<numWorkers>4</numWorkers>
OR
Add <numAttachmentWorkers>4</numAttachmentsWorkers> inside the <custom> tag section like below
<numWorkers>4</numWorkers>
<numAttachmentWorkers>4</numAttachmentsWorkers> - Save the file.
- Restart DataSync by typing "rcdatasync restart" and press Enter.
Additional Information
Notes:
Novell Data Synchronizer Mobility Pack 1.2 Patch File 1 helps
with additional issues that were seen with downloading and
processing attachments. If a lot of attachments are flowing
to Mobility, it may take some time for DataSync to
process attachments causing a back log of attachments that
may often resulted in emails that quit syncing.
Depending on the size of the backlog and how busy DataSync is, it
can take a while for DataSync to get fully caught up, especially if
users are heavy into sending attachments and are power users.
Hence the need to increase the threads used for attachments.
Changing the <numWorkers> field, will change the amount of worker threads that the GroupWise connector uses of handling messages and attachments. The connector will use half of the threads for attachments, so by using the default value of 4, it will use only 2 threads for attachments. By increasing this to a larger number such as 8, it will allocate 4 of those threads to handling attachments.
The <numAttachmentsWorkers> field is not present by default, so by adding it Data Sync uses the number present in this setting to allocate the number of threads for attachments. So by assigning it 8 or another number, DataSync uses 8 threads purely for handling attachments to download from GroupWise, instead of the default value of 2. While modifiying this setting, numWorkers may not be changed and can be at the default as<numAttachmentsWorkers> field will take precedence
Changing the <numWorkers> field, will change the amount of worker threads that the GroupWise connector uses of handling messages and attachments. The connector will use half of the threads for attachments, so by using the default value of 4, it will use only 2 threads for attachments. By increasing this to a larger number such as 8, it will allocate 4 of those threads to handling attachments.
The <numAttachmentsWorkers> field is not present by default, so by adding it Data Sync uses the number present in this setting to allocate the number of threads for attachments. So by assigning it 8 or another number, DataSync uses 8 threads purely for handling attachments to download from GroupWise, instead of the default value of 2. While modifiying this setting, numWorkers may not be changed and can be at the default as<numAttachmentsWorkers> field will take precedence
In other words, <numWorkers> can be set to the
default of 4, and <numAttachmentsWorkers> field can be
set to 8, allowing DataSync to use 4 threads for normal email
processing and 8 for attachment handling, processing and
conversion. Even with higher values, the back log can still
take a while to clear, but the raised values should allow for
quicker recovery as well as faster future handling.
The changes made to the connector.xml will require a Data Synchronizer restart to take effect. Please take caution will changing any values in any DataSync configuration file, as raising the values too much can actually hurt performance more then increase it. The beefier the box or box with better resources, the higher values can go before performance is impacted, so please adjust values in incremental steps to find values right for the system.
The changes made to the connector.xml will require a Data Synchronizer restart to take effect. Please take caution will changing any values in any DataSync configuration file, as raising the values too much can actually hurt performance more then increase it. The beefier the box or box with better resources, the higher values can go before performance is impacted, so please adjust values in incremental steps to find values right for the system.
Troubleshooting
When logged into the mobility database of DataSync, running commands similar to: "select count(*) from attachments where state = '1';" or "select count(*) from attachments where state = '2';" display large amounts of attachments in the thousands and the number does not appear to be going down much or at all, even after several hours.
When logged into the mobility database of DataSync, running commands similar to: "select count(*) from attachments where state = '1';" or "select count(*) from attachments where state = '2';" display large amounts of attachments in the thousands and the number does not appear to be going down much or at all, even after several hours.
Monitoring the default.pipeline1.mobility-AppInterface.log and grepping for attachmentMonitor, shows attachments being placed in a queue, but searching thedefault.pipeline1.groupwise-Appinterface for the attachments does not show it being downloaded right away, but some time later.