Single sign on fails because Identity Injection randomly fails to inject configured parameters on Linux Access Gateway

  • 7009949
  • 03-Jan-2012
  • 26-Apr-2012

Environment

Novell Access Manager 3.1 Linux Access Gateway
Linux Access Gateway running on SLES11
Novell Access Manager 3.1 Support Pack 3 applied

Situation

Two Linux Access Gateway (LAG) Servers configured in a cluster together with a green status. All services seem to be working fine until users start reporting single sign on issues with an accelerated application. After analysing the setup, only one of the LAGs in the cluster seems be be having issues with identity injection, and the problem does not happen all the time.
In order for Web applications to function properly, Access Manager's Identity Injection should provide the user's DN and password on pages where the user is authenticated.  However, there are situations where either the DN part appears to be omitted, or the password part appears to be omitted.

Resolution

Make sure the minimum memory requirements exist on the LAG and that 2GB are assigned to the Java heap.

Assuming there is 4GB of RAM on the LAG and that the LAG is SLES 11 based, increase java memory from the default 1024 to 2048 using the -Xmx parameter in tomcat5.conf. Save the change and restart novell-tomcat5 on the LAG after making the change below:
 
File:
/var/opt/novell/tomcat5/conf/tomcat5.conf
Example:

CHANGE THIS
JAVA_OPTS="-server -Xmx1024m -Xms512m -Xss128k -XX:+UseConcMarkSweepGC"

TO THIS
JAVA_OPTS="-server -Xmx2048m -Xms2048m -Xss128k -XX:+UseConcMarkSweepGC"

Additional Information

When troubleshooting policy related issues, it is important to
- enable the IDP logging components of 'Web Services Consumer' and 'Web Services Provider' to DEBUG, along with the 'Application' component.
- enable the DEBUG level of 7 and LAGHTTPHEADERS flag in /etc/laglogs.conf on the LAG

After doing this and capturing a valid error, we confirmed that some of the injected headers were blank in the /var/log/laghttpheaders file instead of being filled correctly. The following key exception was also discovered in the catalina.out of the LAG as the Identity Injection policy was being evaluated and where the attributes failed to get injected correctly.
 
<amLogEntry> 2011-11-01T18:53:06Z DEBUG NIDS Application:
Method: SwapHashMap.removeEldestEntry
Thread: http-127.0.0.1-8080-Processor4238
Moving entry to swap file: Key: 0E9886FB6F59E0764599D9E92D58B738 </amLogEntry>
 
<amLogEntry> 2011-11-01T18:53:06Z SEVERE NIDS Application: 501101053: Exception - java.io.IOException: No space left on device
Exception message: "java.io.IOException: No space left on device"
     y, Line: 1201, Method: A
     y, Line: 2142, Method: A
     y, Line: 2047, Method: removeEldestEntry
     LinkedHashMap.java, Line: 410, Method: addEntry
     HashMap.java, Line: 385, Method: put
     y, Line: 2800, Method: put
     y, Line: 3362, Method: add
     y, Line: 2144, Method: addCache
     y, Line: 2663, Method: A
     y, Line: 758, Method: A
     y, Line: 457, Method: getDataWithoutInteraction
     y, Line: 2251, Method: getData
     y, Line: 869, Method: A
     y, Line: 268, Method: A
:
<amLogEntry> 2011-11-01T18:53:06Z DEBUG NIDS Application: AM#501101052: AMDEVICEID#esp-FA301B709CD38016: AMAUTHID#02BCC3B4630074B14E827726332B3D8D: PolicyID#0LK15
761-9044-75N9-10NL-1029LO89748N: NXPESID#5032:  Throwing data unavailable response: Data Item - 7010: Parameter - NEPXurn~3Anovell~3Acredentialprofile~3A2005-03~2
Fcp~3ASecrets~2Fcp~3ASecret~2Fcp~3AEntry~40~40~40~40WSCQSSToken~40~40~40~40~2Fcp~3ASecrets~2Fcp~3ASecret~5Bcp~3AName~3D~22LDAPCredentials~22~5D~2Fcp~3AEntry~5Bcp~
3AName~3D~22UserPassword~22~5D </amLogEntry>
 
This will explain why the user password is not included!  This typically is thrown when running low on disk space OR low java memory. Confirm that the Java heap size on the LAG, as in the resolution above.  
Use the terminal command 'df -h' to see how much space is left on the file system, especially the /tmp file system.
 
In this case, it was an issue of low java memory.