Reflection or InfoConnect Desktop, Rumba Desktop, or Extra! can randomly crash immediately when loaded

  • 7024906
  • 03-Nov-2020
  • 06-Nov-2020

Environment

Reflection Desktop (including Pro, for IBM, or for UNIX and OpenVMS) 16.1
InfoConnect Desktop (including Pro, or for Unisys) 16.1
Rumba Desktop version 9.5 SP1
Rumba Desktop version 9.5
Extra! X-treme 9.5

Situation

This article explains how to troubleshoot apparently random crashes, exhibiting a variety of different symptoms (including the product going to a "Not Responding" state), which can occur infrequently in the specific versions of the products listed above. These crashes occur only at initial load of the product and only appear when the software products are installed on a Microsoft Windows-based Server.

Resolution

Troubleshooting:

To determine if the ATMCRYPTO.DLL module gets loaded at a required memory address, and might be the cause of this issue, use the Process Monitor or Process Explorer tools.

Process Monitor:

        Search in a Process Monitor trace for a “Load Image” Operation for C:\Program Files (x86)\Micro Focus\<product name>\atmcrypto.dll.
        In the Detail column of Process Monitor, look for the Image Base address.  It should be 0x600000 or 0xFB0000, and if it is another value, this would be an invalid address and could be the cause of the crash.

Process Explorer:

        See the following technical article for information on how to use Process Explorer to diagnose this type of issue, and locate where the atmcrypto.dll module gets loaded:  https://support.microfocus.com/kb/doc.php?id=7022938

Solution:

Upgrade to a later version of the Micro Focus product listed below:
  • Reflection Desktop (including Pro, for IBM, or for UNIX and OpenVMS) 16.2 or higher
  • InfoConnect Desktop (including Pro, or for Unisys) 16.2 or higher
  • Rumba Desktop version 10.0 or higher
  • Extra! X-treme 9.6 or higher

Alternative Workaround:

Make a backup copy of the existing ATMCRYPTONF.DLL file; by making a copy to another location and renaming it there, and then copy this file back on top of the existing ATMCRYPTO.DLL to replace it.
The file named ATMCRYPTONF.DLL is a non-FIPS certified version of file ATMCRYPTO.DLL which does not have the memory load address requirement and limitations, but cannot be used if a FIPS connection via TLS is required.

Cause

The apparently random failures of the Micro Focus software products listed above to properly load, can be caused by the incorrect loading of a file called ATMCRYPTO.DLL. The ATMCRYPTO.DLL file is part of the Reflection Security Components (RSC), which are the cryptographic and security modules for the Micro Focus Host Connectivity line of products (which includes Reflection and InfoConnect Desktop, Rumba Desktop, and Extra!). In the specific versions of the products listed above, ATMCRYPTO.DLL may fail to load in an appropriate memory address required for the product, causing a failure of the software to start and load, with a variety of different symptoms occurring.

Additional Information

When running in FIPS (Federal Information Processing Standard) mode, the ATMCRYPTO.DLL performs a self-test to ensure it hasn't been tampered with. A checksum is used during these self-tests and this checksum is located at a specific memory address determined at compile time, which means that the ATMCRYPTO.DLL is required to load at a particular memory address in order to obtain that checksum. This loading is a requirement, regardless of if the DLL is in FIPS Mode or not.   If the module can't load at that address, it will fail, witha  number of possible outcomes. Occasionally, Microsoft Windows will have already loaded a DLL in the location needed, causing the ATMCRYPTO.DLL to fail to load properly.  To prevent this from occurring, it is recommended that an application (or process) load ATMCRYPTO.DLL first, which works most of the time on a Microsoft Windows workstation platform, like Windows 10.  

This workaround is not always possible especially in a Windows Server, Windows Terminal Server, or Citrix Server environment because of the way Microsoft Windows will “optimize” memory used by common DLL files.  This optimization of memory occurs in both Windows Workstation and Servers but is typically not much of an issue on a Windows stand-alone Workstation.,  In theory when a Windows process is loaded, it has its own memory address space, separate from other processes that are currently running.  But if a common DLL across Windows processes gets loaded, especially a Windows System DLL, then the memory used in one process space will also be occupied by this same DLL in the other process spaces, which will happen frequently in the Windows Server environment.  Thus, even though the Micro Focus Host Connectivity products start up their own process, Microsoft Windows may not let them determine where they can load DLLs and Windows may determine that another DLL may need to load in the required memory location.  Thus ATMCRYPTO.DLL is left to find its own spot, which may means the cryptographic module cannot be found by the Reflection Security Components processes which need it, leading to a potential crash.

Feedback service temporarily unavailable. For content questions or problems, please contact Support.