Mobility services crash

  • 7010701
  • 28-Aug-2012
  • 10-Dec-2013

Environment

Novell Data Synchronizer Mobility Pack

Situation

Mobility or Groupwise connector crashes
DataSync Connector Manager shuts down
Defunct python threads in background
DataSync python threads go defunct
No reason for shutdown indicated in corresponding log files
Out of Memory Errors in /var/log/messages

Resolution

The server needs more memory. Adequate server memory depends on the number of users/devices supported by the Mobility server. Current recommendations can be found in the System Requirements section of Data Synchronizer Documentation. However, and at the time of writing this TID, it is recommended that a Mobility server has:
  • 4 GB RAM to support approximately 300 users/devices
  • 8 GB RAM to support approximately 500 users/devices

Cause

Mobility services shut down due to a lack of memory

Additional Information

/var/log/messages contains something similar to the following:
Mobility kernel: [<ffffffff80009af5>] dump_trace+0x65/0x180
Mobility kernel: [<ffffffff8034f056>] dump_stack+0x69/0x73
Mobility kernel: [<ffffffff8009cf20>] oom_kill_process+0xe0/0x220
Mobility kernel: [<ffffffff8009d690>] __out_of_memory+0x50/0xa0
Mobility kernel: [[<ffffffff8009d73e>] out_of_memory+0x5e/0xc0
Mobility kernel: [<ffffffff800a06f9>] __alloc_pages_slowpath+0x429/0x500
Mobility kernel: [<ffffffff800a090a>] __alloc_pages_nodemask+0x13a/0x140
Mobility kernel: [<ffffffff800a379d>] __do_page_cache_readahead+0xdd/0x230
Mobility kernel: [<ffffffff800a390c>] ra_submit+0x1c/0x30
Mobility kernel: [<ffffffff8009ba23>] filemap_fault+0x3d3/0x3e0
Mobility kernel: [<ffffffff800b6817>] __do_fault+0x57/0x6d0
Mobility kernel: [[<ffffffff800ba2a7>] handle_mm_fault+0x257/0x5c0
Mobility kernel: [[<ffffffff8035521b>] do_page_fault+0x21b/0x400
Mobility kernel: [<ffffffff80352d88>] page_fault+0x28/0x30
Mobility kernel: [<00007f28116dc9c0>] 0x7f28116dc9c0
...
Mobility kernel: Out of memory: kill process 2166 (postmaster) score 84411 or a child
Mobility kernel: Killed process 2179 (postmaster)
Mobility kernel: python invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0
Mobility kernel: python cpuset=/ mems_allowed=0

OR

kernel: [1058651.055009] Call Trace:
kernel: [1058651.055018]  [<ffffffff810061dc>] dump_trace+0x6c/0x2d0
kernel: [1058651.055024]  [<ffffffff81394288>] dump_stack+0x69/0x71
kernel: [1058651.055029]  [<ffffffff810b6e7d>] oom_kill_process+0xcd/0x2f0
kernel: [1058651.055032]  [<ffffffff810b74f0>] __out_of_memory+0x50/0xa0
kernel: [1058651.055035]  [<ffffffff810b76d8>] out_of_memory+0x198/0x210
kernel: [1058651.055038]  [<ffffffff810bab8a>] __alloc_pages_slowpath+0x4fa/0x550
kernel: [1058651.055042]  [<ffffffff810bad1a>] __alloc_pages_nodemask+0x13a/0x140
kernel: [1058651.055045]  [<ffffffff810be1fe>] __do_page_cache_readahead+0xce/0x220
kernel: [1058651.055048]  [<ffffffff810be36c>] ra_submit+0x1c/0x30
kernel: [1058651.055051]  [<ffffffff810b5153>] filemap_fault+0x3c3/0x3d0
kernel: [1058651.055055]  [<ffffffff810cd9f7>] __do_fault+0x57/0x520
kernel: [1058651.055058]  [<ffffffff810d2349>] handle_mm_fault+0x199/0x430
kernel: [1058651.055062]  [<ffffffff813999cf>] do_page_fault+0x1bf/0x3e0
kernel: [1058651.055065]  [<ffffffff813972ef>] page_fault+0x1f/0x30
kernel: [1058651.055088]  [<00007f73435cb3e1>] 0x7f73435cb3e1
kernel: [1058651.057514] Out of memory: kill process 14408 (postmaster) score 638405 or a child
kernel: [1058651.057515] Killed process 19960 (postmaster)