x509 authentication to Identity server fails because server is not requesting certificate

ACcess Manager setup work working correctly with SUpport Pack 2 applied ie. users can access protected resources behind Access Gateways and SSLVPN servers after authenticating to the Identity Server. Multiple authentication methods were setup on the Identity Server including name/password and x509 authentication. After upgrading to Access Manager 3.1 Support Pack 3, the name/password based autehntication continued to work correctly, but the X509 authentication failed. The contract did appear to be executed, but it failed with the error that the "client certificate could not be obtained."

The catalina.out file would report the following exception:

<amLogEntry> 2011-09-01T13:32:16Z DEBUG NIDS Application:
Method: X509Class.validateX509
Thread: http-8443-Processor8
Renegotiating with client for X509 certificates! </amLogEntry>

01-Sep-2011 14:32:16 org.apache.tomcat.util.net.jsse.JSSE14Support synchronousHandshake
INFO: SSL Error getting client Certs javax.net.ssl.SSLHandshakeException: no cipher suites in common
        at com.sun.net.ssl.internal.ssl.Alerts.getSSLException(Alerts.java:174)
        at com.sun.net.ssl.internal.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1649)
        at com.sun.net.ssl.internal.ssl.Handshaker.fatalSE(Handshaker.java:241)
        at com.sun.net.ssl.internal.ssl.Handshaker.fatalSE(Handshaker.java:231)
        at com.sun.net.ssl.internal.ssl.ServerHandshaker.chooseCipherSuite(ServerHandshaker.java:787)
        at com.sun.net.ssl.internal.ssl.ServerHandshaker.clientHello(ServerHandshaker.java:557)
        at com.sun.net.ssl.internal.ssl.ServerHandshaker.processMessage(ServerHandshaker.java:151)
:

Looking at the SSL handshake in a network trace, one can see that the server is sending its certificate, but the certificate required request is missing prior to the Server Hello Done message. As a result the client never tries to send a certificate and the authentication fails.

Make sure that the option to disable SSL renegotiation during the SP3 upgrade on the Identity Server is set to false. One can confirm this by looking at the server.xml file (/var/opt/novell/tomcat5/conf/server.xml) and make sure that the allowUnsafeLegacyRenegotiation="true" is enabled as shown below:

<Connector NIDP_Name="connector" URIEncoding="utf-8" acceptCount="0" className="org.apache.coyote.tomcat5.CoyoteConnector" clientAuth="false" debug="0" disableUploadTimeout="true" enableLookups="false" keystoreFile="/opt/novell/devman/jcc/certs/idp/connector.keystore" keystorePass="41TPF5KA5Q0j8c6" maxProcessors="200" maxSpareThreads="50" maxThreads="200" minProcessors="3" minSpareThreads="5"port="8443" sSLImplementation="com.novell.nidp.common.util.net.server.NIDPSSLImplementation" scheme="https" secure="true" sslProtocol="TLS" useBodyEncodingURI="false" useURIValidationHack="false" allowUnsafeLegacyRenegotiation="true" />

There may be cases where the option is missing from the connector in server.xml and it is instead set in the tomcat5 Options file (/var/opt/novell/tomcat5/conf/tomcat5.conf on Linux or set with tomcat5w.exe on Windows). SImply make sure that the option

JAVA_OPTS="${JAVA_OPTS} -Dsun.security.ssl.allowUnsafeRenegotiation=true".

Is set. In both cases, administrator must restart tomcat for changes to take effect.