NetWare server running NDPS reports short term memory allocation failed, but for huge memory size request.

  • 10060320
  • NOVL39210
  • 07-Feb-2001
  • 04-Aug-2003

Archived Content: This information is no longer maintained and is provided 'as is' for your convenience.

Fact

Novell Distributed Print Services (NDPS) 2.1.5

Novell Distributed Print Services (NDPS) 2.1.4

Novell Distributed Print Services (NDPS) 2.1.3

Novell Distributed Print Services (NDPS) 2.1.2

Novell Distributed Print Services (NDPS) 2.1.1

Novell Distributed Print Services (NDPS) 2.0

Symptom

NetWare server running NDPS reports short term memory allocation failed, but for huge memory size request.

Error: "Short term memory allocator out of available memory."

The following error message is displayed on the server console repeatedly:  Short term memory allocator is out of memory.  XXX attempts to get more memory failed.

Short term memory allocator out of available memory error message on console

Short term cache allocator out of memory

Short term memory allocator is out of memory.

MONITOR.NLM shows NDPSM.NLM or other NDPS module suddenly holding over 800MB of memory (not a gradual increase or slow leak).

Cause

BACKGROUND:

The issue of seeing a attempted memory allocation of 500MB+ has been observed in a few environments.  There are multiple causes to this large memory allocation attempt that have been identified by Novell.  The first issue occurs in relation to NDPS having assembled or received an RPC message which has a zero length fragment.  The second scenario is related to a malformed RPC packet being received by the server.  This should be addressed by the DPRPCNLM.NLM listed in the fix.  The third scenario is where REGSRVR.NLM is requesting the large memory allocation.  This is due to a valid RPC packet being sent from another SRS on the network, but the packet contains bogus information.  This scenario could occur randomly and in some environments could be more prevalent due to the size if the messages being requested and exchanged in some particular environment versus another.  The fourth scenario is when a packet is sent with less than four bytes of data.  This has been resolved with DPRPCNLM.NLM 3.00k PTF 1.03 dated 16JAN2003.  These fixes will also be rolled into NW6SP3.EXE and NW51SP6.EXE, if and when those support packs become available.

Note that in most cases the memory allocations simply fail, because the 500MB+ request is simply more than the short term allocator has to offer.  Other than NDPS reporting back to the sender that there wasn't enough memory to hold the message currently, this condition is relatively benign and doesn't cause operational problems since communication failure is typically handled and retried.  What does cause a more formidable issue is when the server actually has sufficient memory to fulfill the massive request, at which point short term memory allocations start to fail for real because NDPS has exhausted the pool by holding such a large allocation albeit only temporarily.

When the more benign scenario occurs, an error message similar to the following is displayed at the server console:

2-6-2001   12:24:45 pm:    SERVER-5.0-0
  Severity = 5  Locus = 1  Class = 1
  Short term memory allocator is out of memory.
1 attempts to get more memory failed.
request size in bytes 1545143878 from Module NDPSM.NLM

Note that the module cited may be different (e.g. REGSRVR.NLM) depending on the module which was attempting to communicate when the issue occurred.

When the server has enough physical memory to fulfill such a request, no error message citing a huge allocation is generated since the allocation actually succeeds.  The server will likely start reporting short term memory allocation failures for other applications running on the server, but not necessarily NDPS.  Under MONITOR.NLM | System Resources |  Alloc Memory (Bytes) use F3 to sort by "Number in use" and one of the NDPS modules will be a highest / the highest allocator showing over 800MB of memory allocated.

Fix

NOTE:  Any short term memory allocator problems that are experienced by a NetWare 5.0 SP6 server will NOT be addressed because NetWare 5.0 is no longer supported.  You will need to move to a current version of NetWare in order to have this problem looked at.

The issue with a malformed RPC packet being sent has been addressed with DPRPCNLM.NLM v3.00h dated 13NOV2001 or later. Visit Novell Product Updates to find the updated DPRPCNLM.NLM. Additional fixes have been made in DPRPCNLM.NLM, including NW6SP3.EXE and NW51SP6.EXE.  For servers running NW6SP2.EXE or NW51SP5.EXE, please apply DPRPCNLM.NLM dated 16JAN2003.

The issue where  REGSRVR.NLM receives a bogus packet from another SRS and subsequently requests a large memory allocation *should* be addressed in REGSRVR.NLM dated 07NOV2001.  Visit Novell Product Updates to find the updated REGSRVR.NLM. Apply the appropriate NetWare support pack containing the updated file.

DPLSV386.NLM dated 16JAN2002 or later corrects a potential problem where the NDPS library can, under certain circumstances, allocate a large amount of memory against NDPS.  Visit Novell Product Updates to find the updated DPLSV386.NLM. Apply the appropriate NetWare support pack containing the updated file.

NDPSM.NLM dated 29APR2002 or later addresses a potential problem where the NDPS Manager can allocate a large amount of memory.  The above mentioned NDPSM.NLM is available in NDP21P4.EXE. Visit Novell Product Updates to find the updated NDPSM.NLM. Apply the appropriate NetWare support pack containing the updated file.

The above modules (DPRPCNLM.NLM and REGSRVR.NLM) reduce the number of large memory allocation requests being made by the various NDPS modules, but does not eliminate them altogether.  In some environments, the failed large memory allocation requests are actually negative.  The negative number indicates that the requested allocation amount exceeded 2GB in size.  The request is actually charged against the NDPS Manager (NDPSM.NLM).  These continuing large memory allocation requests are currently under investigation by Novell.

Anecdotal evidence seems to indicate that the memory allocations against the NDPS Manager are RPC packets being sent over IPX.  Anecdotal evidence also suggests that the memory allocation attempts do not occur if the clients communicate to the server via IP.  If a server runs NDPS and has IP and IPX bound to it, the Broker and Manager will bind to both interfaces.  If a workstation and a server both have IP and IPX bound to them, it is possible for the client workstations to be talking to NDPS via IPX instead of IP.  Please do not confuse the NetWare connection in Monitor with the type of connection NDPS on the client may have with the Broker or the Manager.  Monitor may show an IP connection to the server for NCP, but only a LAN trace will show what kind of connection the workstation has with the server with regards to NDPS.

There are two things you can do to ensure what protocol the client uses to comunicate with NDPS on the server.  The first is to remove IPX (or IP) from the server.  If that is not possible, you can have NDPS not bind to the IPX interface.  That is done by loading the NDPS Manager and Broker with the /NOIPX switch.  Please see TID 10063683 for the syntax.  The second way is to remove IPX from the workstations.

If you have the latest NDPS modules indicated above and you are experiencing large memory allocations, you may want to open an incident with Novell Technical Support.  If you do open an incident to troubleshoot this issue, you will be asked to load a special troubleshooting module, (ALLOCMON.NLM) that will cause the server to break into the debugger when a large memory allocation request is made.  Dropping the server into the debugger is necessary so a coredump of the server can be obtained.  A LAN trace will also be requested of all traffic to and from the NetWare server up to the time the server breaks into the debugger.  The coredump and LAN trace will be necessary in order to quickly and effectively troubleshoot the issue.