Attachments show as .txt for a while and then get downloaded properly after some time, and improving attachment handling performance

  • 7009702
  • 07-Nov-2011
  • 10-Dec-2013


Novell Data Synchronizer Mobility Pack


After applying DataSync build 579, attachments are still being downloaded as text.txt.
Attachments are not downloading right away, but showing up on the device sometime after the email has been delivered.
Emails are not syncing because of attachment backlog.


Be sure that File Patch 1 for Data Synchronizer 1.2 (Build 579) or later has been applied first, before proceeding with the steps below.  If after applying File Patch 1, the back log of attachments has not cleared or attachments are still downloading with the text.txt format, it could be that Datasync just can't keep up with the demand.  By having a backlog of attachments, DataSync still delivers the email, but downloads the attachment after some time, which makes it appear that the attachment failed to download or did not get converted to a readable format. 

Please perform the following steps to increase the number of threads used by the GroupWise Connector for attachment handling/processing:
    1. Edit the GroupWise connector file located:  /etc/datasync/configengine/engines/default/pipelines/pipeline1/connectors/groupwise/connector.xml
    2. Find and change <numWorkers>4</numWorkers> to a larger number such as 8. 
      Add <numAttachmentWorkers>4</numAttachmentsWorkers> inside the <custom> tag section like below
    3. Save the file.
    4. Restart DataSync by typing "rcdatasync restart" and press Enter.

Additional Information

Novell Data Synchronizer Mobility Pack 1.2 Patch File 1 helps with additional issues that were seen with downloading and processing attachments.  If a lot of attachments are flowing to Mobility, it may take some time for DataSync to process attachments causing a back log of attachments that may often resulted in emails that quit syncing.  Depending on the size of the backlog and how busy DataSync is, it can take a while for DataSync to get fully caught up, especially if users are heavy into sending attachments and are power users.  Hence the need to increase the threads used for attachments.

Changing the <numWorkers> field, will change the amount of worker threads that the GroupWise connector uses of handling messages and attachments.  The connector will use half of the threads for attachments, so by using the default value of 4, it will use only 2 threads for attachments.  By increasing this to a larger number such as 8, it will allocate 4 of those threads to handling attachments.

The <numAttachmentsWorkers> field is not present by default, so by adding it Data Sync uses the number present in this setting to allocate the number of threads for attachments. So by assigning it 8 or another number, DataSync uses 8 threads purely for handling attachments to download from GroupWise, instead of the default value of 2.  While modifiying this setting, numWorkers may not be changed and can be at the default as<numAttachmentsWorkers> field will take precedence
In other words, <numWorkers> can be set to the default of 4, and <numAttachmentsWorkers> field  can be set to 8, allowing DataSync to use 4 threads for normal email processing and 8 for attachment handling, processing and conversion.  Even with higher values, the back log can still take a while to clear, but the raised values should allow for quicker recovery as well as faster future handling.

The changes made to the connector.xml will require a Data Synchronizer restart to take effect.  Please take caution will changing any values in any DataSync configuration file, as raising the values too much can actually hurt performance more then increase it.  The beefier the box or box with better resources, the higher values can go before performance is impacted, so please adjust values in incremental steps to find values right for the system.
When logged into the mobility database of DataSync, running commands similar to: "select count(*) from attachments where state = '1';" or "select count(*) from attachments where state = '2';" display large amounts of attachments in the thousands and the number does not appear to be going down much or at all, even after several hours.

Monitoring the default.pipeline1.mobility-AppInterface.log and grepping for attachmentMonitor, shows attachments being placed in a queue, but searching thedefault.pipeline1.groupwise-Appinterface for the attachments does not show it being downloaded right away, but some time later.