Access Gateway returns Servererror Null for authenticated user trying to access a protected resource configured for FormFill

  • 7025241
  • 10-Aug-2021
  • 10-Sep-2021

Environment

  • Access Manager 4.5.x
  • Access Manager 5.0

Situation

  • Access Gateway Cluster with multiple nodes

  • Layer 4 switch load balancing user requests to configured Access Gateway Services / Proxy Services.

  • Protected Resource configured to run a FormFill Policy

  • Access Gateway nodes always return the "Serverror Null" in case the users have been switched between cluster nodes. In particular between the proxy service running the Embedded Service provider (NESP) and any proxy service which runs a FormFill or Identity Injection policy which requires to retrieve additional user attributes.

  • The NESP catalina,out of the proxy service which returned the Servererror Null logs:

    • Exception message: "HTTPS hostname wrong:  should be <192.168.100.180>"
    • Status: UnableToLocateUser



Resolution

  • configure the certificate assigned to the NESP and "ESP Mutual SSL"to be the same

Cause

  • In this case the certificate assigned to the reverse proxy and "ESP Mutual SSL" were not the same by manual configuration whch can be tracked in the catalina.out
<amLogEntry> 2021-08-10T08:50:10Z VERBOSE NIDS Application: AM#600105005: AMDEVICEID#esp-86B43A979F03C6E7: AMAUTHID#21bf63a0a02c7bbf30b1a05f78c216ec31e7c4a820934bffddc9b8c26dd36006:  Obtained ip address of cluster member handling this users requests by asking cluster members which one handles this user session. Address: AM#600105005: AMDEVICEID#esp-86B43A979F03C6E7: AMAUTHID#21bf63a0a02c7bbf30b1a05f78c216ec31e7c4a820934bffddc9b8c26dd36006:  </amLogEntry>

<amLogEntry> 2021-08-10T08:50:10Z VERBOSE NIDS Application: AM#600105006: AMDEVICEID#esp-86B43A979F03C6E7: AMAUTHID#21bf63a0a02c7bbf30b1a05f78c216ec31e7c4a820934bffddc9b8c26dd36006:  Must proxy HTTP request to other cluster member. This cluster member: 192.168.100.177, cluster member for this user: 192.168.100.180. </amLogEntry>

<amLogEntry> 2021-08-10T08:50:10Z INFO NIDS Application: AM#500105001: AMDEVICEID#esp-86B43A979F03C6E7: AMAUTHID#21bf63a0a02c7bbf30b1a05f78c216ec31e7c4a820934bffddc9b8c26dd36006:  Forwarding HTTP request to cluster member at URL: https://192.168.100.180:443/nesp/app/soap </amLogEntry>

<amLogEntry> 2021-08-10T08:50:10Z DEBUG NIDS Application:
Method: NIDPProxyableHostNameVerifier.verify
Thread: ajp-nio-127.0.0.1-9009-exec-20
My Certificate: Issuer: O=idpa31_tree, OU=Organizational CA, Subject: O=novell, OU=accessManager, CN=test-connector </amLogEntry>

<amLogEntry> 2021-08-10T08:50:10Z DEBUG NIDS Application:
Method: NIDPProxyableHostNameVerifier.verify
Thread: ajp-nio-127.0.0.1-9009-exec-20
Peer Certificate #0: Issuer: O=idpa31_tree, OU=Organizational CA, Subject: C=DE, L=Duisburg, O=Micro Focus, OU=Technical Services, CN=nesp.kgast.nam.com </amLogEntry>

<amLogEntry> 2021-08-10T08:50:10Z DEBUG NIDS Application:
Method: NIDPProxyableHostNameVerifier.verify
Thread: ajp-nio-127.0.0.1-9009-exec-20
Peer Certificate #1: Issuer: O=idpa31_tree, OU=Organizational CA, Subject: O=idpa31_tree, OU=Organizational CA </amLogEntry>

<amLogEntry> 2021-08-10T08:50:10Z DEBUG NIDS Application:
Method: NIDPProxyableHostNameVerifier.verify
Thread: ajp-nio-127.0.0.1-9009-exec-20
Match NOT Found! </amLogEntry>

<amLogEntry> 2021-08-10T08:50:10Z DEBUG NIDS Application:
Method: NIDPProxyableServlet.A
Thread: ajp-nio-127.0.0.1-9009-exec-20
Proxy: Request: added header: Name: content-length, Value: 825
Proxy: Request: added header: Name: content-type, Value: text/xml
Proxy: Request: switched Host header value from: nesp.kgast.nam.com to: nesp.kgast.nam.com
Proxy: Request: added header: Name: host, Value: nesp.kgast.nam.com
Proxy: Request: added header: Name: connection, Value: Keep-Alive
Proxy: Request: added header: Name: RemoteClientIPAddress, Value: 192.168.100.14
Proxy: Request: added new Via header: HTTP/1.1 192.168.100.177
Proxy: Request: added new custom NIDPProxiedRequest http header: 192.168.100.177;ks5tpdcd4v319;ks5tpdcdefd1a
Proxy: Response: The Cluster Proxy Request List has 0 members!
 </amLogEntry>

<amLogEntry> 2021-08-10T08:50:10Z DEBUG NIDS Application:
Method: NIDPProxyableServlet.myDoGetWithProxy
Thread: ajp-nio-127.0.0.1-9009-exec-20

Exception message: "HTTPS hostname wrong:  should be <192.168.100.180>"
     HttpsClient.java, Line: 649, Method: checkURLSpoofing

Additional Information

Troubleshooting

  • Configure the following Auditing and Logging setting:
    • File Logging Enabled
    • Echo to Console Enabled
    • Component File Logger Levels:
      • Application and Liberty: debug
      • Web Service Provider / Consumer: Info

  • Access Gateway Global Advanced options:
    • NAGGlobalOptions DebugHeaders=on
      Note: this will allow to track a user session using the Access Gateway X-MAG debug cookie within a browser header trace (like fiddler).
      The X-MAG header will include the Access Gateway device ID as the first Parameter



  • Clear up the catalina.out
    • Access Gateway (MAG):  "> /var/opt/novell/nam/logs/mag/tomcat/catalina.out"
    • Identity Provider (IDP):  /var/opt/novell/nam/logs/idp/tomcat/catalina.out

  • Configure your test workstation to:Bold
    • use a hosts file to access / resolve fixed , different  cluster node for the
      • Embedded Service Provider (NESP)
      • Proxy service running the FormFill Policy
    •  run ether the Telerik Fiddler or any Header trace browser plugin like SAML Tracer

  • User session replication / distributtion
    • There is no process with the Access Gateway which will distribute or replicate user session information to all cluster nodes.
    • In case a given user will be switched by the load balancer to a cluster member not owning any information about the user session a proxy request will be initiated to the NESP running on the cluster member owning the user session.
    • Note: there is a discovery process before running the proxy service in order to identify which particular cluster node knows about the existing user.
    • This can be tracked with the above configured logging setting within the catalina.out

      <amLogEntry> 2021-08-10T08:50:10Z VERBOSE NIDS Application: AM#600105005: AMDEVICEID#esp-86B43A979F03C6E7: AMAUTHID#21bf63a0a02c7bbf30b1a05f78c216ec31e7c4a820934bffddc9b8c26dd36006:
        Obtained ip address of cluster member handling this users requests by asking cluster members which one handles this user session. Address: AM#600105005: AMDEVICEID#esp-86B43A979F03C6E7: AMAUTHID#21bf63a0a02c7bbf30b1a05f78c216ec31e7c4a820934bffddc9b8c26dd36006:
      </amLogEntry>

      <amLogEntry> 2021-08-10T08:50:10Z VERBOSE NIDS Application: AM#600105006: AMDEVICEID#esp-86B43A979F03C6E7: AMAUTHID#21bf63a0a02c7bbf30b1a05f78c216ec31e7c4a820934bffddc9b8c26dd36006:
        Must proxy HTTP request to other cluster member. This cluster member: 192.168.100.177, cluster member for this user: 192.168.100.180.
      </amLogEntry>

      <amLogEntry> 2021-08-10T08:50:10Z INFO NIDS Application: AM#500105001: AMDEVICEID#esp-86B43A979F03C6E7: AMAUTHID#21bf63a0a02c7bbf30b1a05f78c216ec31e7c4a820934bffddc9b8c26dd36006: 
      Forwarding HTTP request to cluster member at URL: https://192.168.100.180:443/nesp/app/soap
      </amLogEntry>

    • The proxy request above requires authentication and Certificate validation.
      The certificate assigned to the reverse proxy acting as NESP has to be same
      • on all Access Gateway Cluster nodes
      • SSL Certificate configured as "NESP SSL Mutual"