Error 502 'mal-formed request from origin server'

  • 7001820
  • 07-Nov-2008
  • 26-Apr-2012

Environment

Novell Access Manager 3 Linux Access Gateway
Novell Access Manager 3 Support Pack 4 applied
Proxy setup to communicate with Oracle Application Server on Linux running 10.1.3.3 and 10.1.3.4
Oracle Application Server is SSL enabled

Situation

Linux Access Gateway (LAG) setup to protect multiple protected resources on multiple back end Web servers. Communication to these back end web servers work fine for the most part. Users trying to access an application running on an Oracle Application server always get a '502 Bad Gateway' error with the message:

' Mal-formed request from Origin Server'

By modifying the /etc/laglogs.conf file to include the following information (and restarting the proxy server)

LOG_LEVEL=7
DEBUG_HTTP_HEADERS=1
DEBUG_SOAP_MESSAGES=0

the /var/log/laghttpheaders confirmed that the requests were being sent by the LAG to the back end Oracle server. No responses were coming back ..


----------------------------------------------------------------------------------------
Sending request to webserver 205.90.20.163 for browser request '1'
----------------------------------------------------------------------------------------
GET /oracle/ HTTP/1.1
Connection: close
Host: www.neil.org:443
Accept: */*
Accept-Language: en-US,en-IE;q=0.5
Ua-Cpu: x86
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 2.0.50727; .NET CLR 3.0.04506.648; .NET CLR 3.5.21022)
Cookie: novell_language=en-us
Pragma: no-cache
Via: 1.1 lag129.lab.novell.com (Access Gateway 3.1.0-157)

By looking at the /var/log/ics_dyn.log file, one could confirm that the requests were being sent to the origin server but the Oracle Web server would immediately issue a connection close (TCP reset!) in response. The proxy would retransmit the request 4 more times and when no HTTP response was returned, the proxy would issue the '502 bad Gateway' response to the browser. The entry from the ics_dyn.log file would look as follows:

Sep 12 10:32:41 lag31dub : AM#504503000: AMDEVICEID#ag-: AMAUTHID#0: AMEVENTID#1: connecting to webserver 205.90.20.163:443 74db37de gotcookie use browser's binding
Sep 12 10:32:41 lag31dub : AM#504515000: AMDEVICEID#ag-: AMAUTHID#0: AMEVENTID#0: Connection Established with peer (205.90.20.163), port(443)
Sep 12 10:32:41 lag31dub : AM#504503000: AMDEVICEID#ag-: AMAUTHID#0: AMEVENTID#2: Establish SSL connection to webserver
Sep 12 10:32:41 lag31dub : AM#504518000: AMDEVICEID#ag-: AMAUTHID#0: AMEVENTID#2804267440: Accept any trusted root from webserver
Sep 12 10:32:41 lag31dub : AM#504503000: AMDEVICEID#ag-: AMAUTHID#0: AMEVENTID#1: Sending request to origin server 204.90.20.163:443 (74db37de.74db37de)
Sep 12 10:32:42 lag31dub : AM#504518000: AMDEVICEID#ag-: AMAUTHID#0: AMEVENTID#2804267440: Connection close received
Sep 12 10:32:42 lag31dub : AM#504503000: AMDEVICEID#ag-: AMAUTHID#0: AMEVENTID#1: totalMsgs:84 msg:66:[Mal-formed reply from origin server. Please try your request again.]
Sep 12 10:32:42 lag31dub : AM#504520000: AMDEVICEID#ag-: AMAUTHID#0: AMEVENTID#1: Browser req/resp[55, 0, 0] [timeToResp:-1 respDuration:-1]    curTime:59 ~ServerRequest [auth:0 acl:0 II:0] [rewrite 0 :0 0 0] [origin: 55, 59, 59,0 retry:4 0]


Resolution

Oracle have addressed an issue with their Application Server build on Linux and a patch is available for the 10.1.3.3 and 10.1.3.4 environments. Older versions of their code did not suffer from the same issue.

Oracle patch number is 7210972.