"XML document structures must start and end within the same entity." error

Access Manager 3.2 setup and working fine - users could access protected resources on the Access Gateway Appliance after authenticating to the Identity server. AFter rolling out a new application, users trying to access the protected resource after authenticating would get the following message reported via the browser:

""XML document structures must start and end within the same entity."

The main difference between this application and other applications were the number of authorization policies enabled (27 in total) for this protected resource. As a test, we changed the number of protected resources to one and the application worked fine.

Need to make the following changes to the Apache based appliance:

a) add the packetSize parameter and maxPostSize to the /opt/novell/nam/mag/conf/server.xml to "Connector" element with protocol AJP. For example, the connector should look as shown below:

 <Connector port="9009" enableLookups="false" redirectPort="8443" protocol="AJP/1.3" address="127.0.0.1" minSpareThreads="25" maxThreads="300" backlog="0" connectionTimeout="20000" packetSize="65536" maxPostSize="65536"/>

b) add the following to /etc/opt/novell/apache2/conf/httpd.conf file

ProxyIOBufferSize 65536

Note, you have to give the same size in both the places.

c) restart the following services on the Access Gateway APpliance:

- /etc/init.d/novell-mag restart
- /etc/init.d/novell-apache2 restart

the AJP connector is restricted to 8k by default and the SOAP requests from the proxy to the ESP were getting truncated. Running the /etc/init.d/novell-apache2 service in debug mode (/etc/init.d/novell-apache2start debug) showed the following info in the error_log file:

[Fri Apr 27 09:41:22 2012] [debug] proxy_util.c(1506): AMEVENTID#7377: proxy: ajp: found worker ajp://127.0.0.1:9009/nesp for ajp://127.0.0.1:9009/nesp/app/soap [Fri Apr 27 09:41:22 2012] [debug] mod_proxy.c(1024): Running scheme ajp handler (attempt 0) [Fri Apr 27 09:41:22 2012] [debug] mod_proxy_http.c(2099): proxy: HTTP: declining URL ajp://127.0.0.1:9009/nesp/app/soap [Fri Apr 27 09:41:22 2012] [debug] mod_proxy_ajp.c(727): proxy: AJP: serving URL ajp://127.0.0.1:9009/nesp/app/soap [Fri Apr 27 09:41:22 2012] [debug] proxy_util.c(2024): proxy: AJP: has acquired connection for (127.0.0.1) [Fri Apr 27 09:41:22 2012] [debug] proxy_util.c(2080): proxy: connecting ajp://127.0.0.1:9009/nesp/app/soap to 127.0.0.1:9009 [Fri Apr 27 09:41:22 2012] [debug] proxy_util.c(2206): proxy: connected /nesp/app/soap to 127.0.0.1:9009 [Fri Apr 27 09:41:22 2012] [debug] proxy_util.c(2393): proxy: AJP: backend socket is disconnected. [Fri Apr 27 09:41:22 2012] [info] proxy: AJP: fam 2 socket created to connect to 127.0.0.1 [Fri Apr 27 09:41:22 2012] [debug] ajp_header.c(225): Into ajp_marshal_into_msgb [Fri Apr 27 09:41:22 2012] [debug] ajp_header.c(292): ajp_marshal_into_msgb: Header[0] [Content-Length] = [11737] [Fri Apr 27 09:41:22 2012] [debug] ajp_header.c(292): ajp_marshal_into_msgb: Header[1] [Content-Type] = [text/xml] [Fri Apr 27 09:41:22 2012] [debug] ajp_header.c(292): ajp_marshal_into_msgb: Header[2] [Host] = [tesp.integrysgroup.com] [Fri Apr 27 09:41:22 2012] [debug] ajp_header.c(292): ajp_marshal_into_msgb: Header[3] [Connection] = [Keep-Alive] [Fri Apr 27 09:41:22 2012] [debug] ajp_header.c(514): ajp_marshal_into_msgb: Done [Fri Apr 27 09:41:22 2012] [error] AMEVENTID#7377: <SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"><SOAP-ENV:Body><NXPES Id="7377"><Configure-ag LastUpdateTimestamp="0" PEPName="AGIdentityInjection"><PolicyEnforcementList schemaVersion="2.0" RuleCombiningAlgorithm="DenyOverridesWithPriority" LastModifiedBy="cn=aweber,o=novell" LastModified="1335268575333" IncludedPolicyCategories="">\n <PolicyRef UserInterfaceID="PolicyID_xpemlPEP_AGAuthorization_12702310985660" ExternalElementRef="PolicyID_xpemlPEP_AGAuthorization_12702310985660" ExternalDocRef="ou=xpemlPEP,ou=mastercdn,ou=ContentPublisherContainer,ou=Partition,ou=PartitionsContainer,ou=VCDN_Root,ou=accessManagerContainer,o=novell:romaContentCollectionXMLDoc" ElementRefType="ExternalWithIDRef"/>\n <PolicyRef UserInterfaceID="PolicyID_xpemlPEP_AGAuthorization_12702313923880" ExternalElementRef="PolicyID_xpemlPEP_AGAuthorization_12702313923880" ExternalDocRef="ou=xpemlPEP,ou=mastercdn,ou=ContentPublisherContainer,ou=Partition,ou=PartitionsContainer,ou=VCDN_Root,ou=accessManagerContainer,o=novell:romaContentCollectionXMLDoc" ElementRefType="ExternalWithIDRef"/>\n <PolicyRef UserInterfaceID="PolicyID_xpemlPEP_AGAuthorization_12702320978450" ExternalElementRef="PolicyID_xpemlPEP_AGAuthorization_12702320978450" ExternalDocRef="ou=xpemlPEP,ou=mastercdn,ou=ContentPublisherContainer,ou=Partition,ou=PartitionsContainer,ou=VCDN_Root,ou=accessManagerContainer,o=novell:romaContentCollectionXMLDoc" ElementRefType="ExternalWithIDRef"/>\n <PolicyRef UserInterfaceID="PolicyID_xpemlPEP_AGAuthorization_12702314440830" ExternalElementRef="PolicyID_xpemlPEP_AGAuthorization_12702314440830" ExternalDocRef="ou=xpemlPEP,ou=mastercdn,ou=ContentPublisherContainer,ou=Partition,ou=PartitionsContainer,ou=VCDN_Root,ou=accessManagerContainer,o=novell:romaContentCollectionXMLDoc" ElementRefType="ExternalWithIDRef"/>\n <PolicyRef UserInterfaceID="PolicyID_xpemlPEP_AGAuthorization_12702315445300" ExternalElementRef="PolicyID_xpemlPEP_AGAuthorization_12702315445300" ExternalDocRef="ou=xpemlPEP,ou=mastercdn,ou=ContentPublisherContainer,ou=Partition,ou=PartitionsContainer,ou=VCDN_Root,ou=accessManagerContainer,o=novell:romaContentCollectionXMLDoc" ElementRefType="ExternalWithIDRef"/>\n <PolicyRef UserInterfaceID="PolicyID_xpemlPEP_AGAuthorization_12702315933480" ExternalElementRef="PolicyID_xpemlPEP_AGAuthorization_12702315933480" ExternalDocRef="ou=xpemlPEP,ou=mastercdn,ou=ContentPublisherContainer,ou=Partition,ou=PartitionsContainer,ou=VCDN_Root,ou=accessManagerContainer,o=novell:romaContentCollectionXMLDoc" ElementRefType="ExternalWithIDRef"/>\n <PolicyRef UserInterfaceID="PolicyID_xpemlPEP_AGAuthorization_12702316300400" ExternalElementRef="PolicyID_xpemlPEP_AGAuthorization_12702316300400" ExternalDocRef="ou=xpemlPEP,ou=mastercdn,ou=ContentPublisherContainer,ou=Partition,ou=PartitionsContainer,ou=VCDN_Root,ou=accessManagerContainer,o=novell:romaContentCollectionXMLDoc" ElementRefType="ExternalWithIDRef"/>\n <PolicyRef UserInterfaceID="PolicyID_xpemlPEP_AGAuthorization_12702321627770" ExternalElementRef="PolicyID_xpemlPEP_AGAuthorization_12702321627770" ExternalDocRef="ou=xpemlPEP,ou=mastercdn,ou=ContentPublisherContainer,ou=Partition,ou=PartitionsContainer,ou=VCDN_Root,ou=accessManagerContainer,o=novell:romaContentCollectionXMLDoc" ElementRefType="ExternalWithIDRef"/>\n <PolicyRef UserInterfaceID="PolicyID_xpemlPEP_AGAuthorization_12702154628710" ExternalElementRef="PolicyID_xpemlPEP_AGAuthorization_12702154628710" ExternalDocRef="ou=xpemlPEP,ou=mastercdn,ou=ContentPublisherContainer,ou=Partition,ou=PartitionsContainer,ou=VCDN_Root,ou=accessManagerContainer,o=novell:romaContentCollectionXMLDoc" ElementRefType="ExternalWithIDRef"/>\n <PolicyRef UserInterfaceID="PolicyID_xpemlPEP_AGAuthorization_12702322252100" ExternalElementRef="PolicyID_xpemlPEP_AGAuthorization_12702322252100" ExternalDocRef="ou=xpemlPEP,ou=mastercdn,ou=ContentPublisherContainer,ou=Partition,ou=PartitionsContainer,ou=VCDN_Root,ou=accessManagerContainer,o=novell:romaContentCollectionXMLDoc" ElementRefType="ExternalWithIDRef"/>\n <PolicyRef UserInterfaceID="PolicyID_xpemlPEP_AGAuthorization_12702322747330" ExternalElementRef="PolicyID_xpemlPEP_AGAuthorization_12702322747330" ExternalDocRef="ou=xpemlPEP,ou=mastercdn,ou=ContentPublisherContainer,ou=Partition,ou=PartitionsContainer,ou=VCDN_Root,ou=accessManagerContainer,o=novell:romaContentCollectionXMLDoc" ElementRefType="ExternalWithIDRef"/>\n <PolicyRef UserInterfaceID="PolicyID_xpemlPEP_AGAuthorization_12702323260120" ExternalElementRef="PolicyID_xpemlPEP_AGAuthorization_12702323260120" ExternalDocRef="ou=xpemlPEP,ou=mastercdn,ou=ContentPublisherContainer,ou=Partition,ou=PartitionsContainer,ou=VCDN_Root,ou=accessManagerContainer,o=novell:romaContentCollectionXMLDoc" ElementRefType="ExternalWithIDRef"/>\n <PolicyRef UserInterfaceID="PolicyID_xpemlPEP_AGAuthorization_12702323644190" ExternalElementRef="PolicyID_xpemlPEP_AGAuthorization_12702323644190" ExternalDocRef="ou=xpemlPEP,ou=mastercdn,ou=ContentPublisherContainer,ou=Partition,ou=PartitionsContainer,ou=VCDN_Root,ou=accessManagerContainer,o=novell:romaContentCollectionXMLDoc" ElementRefType="ExternalWithIDRef"/>\n <PolicyRef UserInterfaceID="PolicyID_xpemlPEP_AGAuthorization_1269457096226" ExternalElementRef="PolicyID_xpemlPEP_AGAuthorization_1269457096226" ExternalDocRef="ou=xpemlPEP,ou=mastercdn,ou=ContentPublisherContainer,ou=Partition,ou=PartitionsContainer,ou=VCDN_Root,ou=accessManagerContainer,o=novell:romaContentCollectionXMLDoc" ElementRefType="ExternalWithIDRef"/>\n <PolicyRef UserInterfaceID="PolicyID_xpemlPEP_AGAuthorization_12702155238240" ExternalElementRef="PolicyID_xpemlPEP_AGAuthorization_12702155238240" ExternalDocRef="ou=xpemlPEP,ou=mastercdn,ou=ContentPublisherContainer,ou=Partition,ou=PartitionsContainer,ou=VCDN_Root,ou=accessManagerContainer,o=novell:romaContentCollectionXMLDoc" ElementRefType="ExternalWithIDRef"/>\n <PolicyRef UserInterfaceID="PolicyID_xpemlPEP_AGAuthorization_12702155667360" ExternalElementRef="PolicyID_xpemlPEP_AGAuthorization_12702155667360" ExternalDocRef="ou=xpemlPEP,ou=mastercdn,ou=ContentPublisherContainer,ou=Partition,ou=PartitionsContainer,ou=VCDN_Root,ou=accessManagerContainer,o=novell:romaContentCollectionXMLDoc" ElementRefType="ExternalWithIDRef"/>\n <PolicyRef UserInterfaceID="PolicyID_xpemlPEP_AGAuthorization_12702155999870" ExternalElementRef="PolicyID_xpemlPEP_AGAuthorization_12702155999870" ExternalDocRef="ou=xpemlPEP,ou=mastercdn,ou=ContentPublisherContainer,ou=Partition,ou=PartitionsContainer,ou=VCDN_Root,ou=accessManagerContainer,o=novell:romaContentCollectionXMLDoc" ElementRefType="ExternalWithIDRef"/>\n <PolicyRef UserInterfaceID="PolicyID_xpemlPEP_AGAuthorization_12702156377560" ExternalElementRef="PolicyID_xpemlPEP_AGAuthorization_12702156377560" ExternalDocRef="ou=xpemlPEP,ou=mastercdn,ou=ContentPublisherContainer,ou=Partition,ou=PartitionsContainer,ou=VCDN_Root,ou=accessManagerContainer,o=novell:romaContentCollectionXMLDoc" ElementRefType="ExternalWithIDRef"/>\n <PolicyRef UserInterfaceID="PolicyID_xpemlPEP_AGAuthorization_12702156777190" ExternalElementRef="PolicyID_xpemlPEP_AGAuthorization_12702156777190" ExternalDocRef="ou=xpemlPEP,ou=mastercdn,ou=ContentPublisherContainer,ou=Partition,ou=PartitionsContainer,ou=VCDN_Root,ou=accessManagerContainer,o=novell:romaContentCollectionXMLDoc" ElementRefType="ExternalWithIDRef"/>\n <PolicyRef UserInterfaceID="PolicyID_xpemlPEP_AGAuthorization_1269868071156" ExternalElementRef="PolicyID_xpemlPEP_AGAuthorization_1269868071156" ExternalDocRef="ou=xpemlPEP,ou=mastercdn,ou=ContentPublisherContainer,ou=Partition,ou=PartitionsContainer,ou=VCDN_Root,ou=accessManagerContainer,o=novell:romaContentCollectionXMLDoc" ElementRefType="ExternalWithIDRef"/>\n <PolicyRef UserInterfaceID="PolicyID_xpemlPEP_AGAuthorization_1268405568855" ExternalElementRef="PolicyID_xpemlPEP_AGAuthorization_1268405568855" ExternalDocRef="ou=xpemlPEP,ou=mastercdn,ou=ContentPublisherContainer,ou=Partition,ou=PartitionsContainer,ou=VCDN_Root,ou=ac [Fri Apr 27 09:41:22 2012] [debug] mod_proxy_ajp.c(270): proxy: APR_BUCKET_IS_EOS [Fri Apr 27 09:41:22 2012] [debug] mod_proxy_ajp.c(275): proxy: data to read (max 8186 at 4) [Fri Apr 27 09:41:22 2012] [debug] mod_proxy_ajp.c(290): proxy: got 8186 bytes of data [Fri Apr 27 09:41:22 2012] [debug] ajp_header.c(752): ajp_read_header: ajp_ilink_received 06 [Fri Apr 27 09:41:22 2012] [debug] ajp_header.c(762): ajp_parse_type: got 06 [Fri Apr 27 09:41:22 2012] [debug] ajp_header.c(752): ajp_read_header: ajp_ilink_received 04 [Fri Apr 27 09:41:22 2012] [debug] ajp_header.c(762): ajp_parse_type: got 04 [Fri Apr 27 09:41:22 2012] [debug] ajp_header.c(580): ajp_unmarshal_response: status = 200 [Fri Apr 27 09:41:22 2012] [debug] ajp_header.c(601): ajp_unmarshal_response: Number of headers is = 7 [Fri Apr 27 09:41:22 2012] [debug] ajp_header.c(663): ajp_unmarshal_response: Header[0] [Set-Cookie] = [JSESSIONID=9FA8AEB955172DF098A9C1F724014ECD; Path=/nesp/; Secure; HttpOnly] [Fri Apr 27 09:41:22 2012] [debug] ajp_header.c(663): ajp_unmarshal_response: Header[1] [Set-Cookie] = [UrnNovellNidpClusterMemberId=~03~02fey~1F~1F~00~7B~7C; Path=/nesp] [Fri Apr 27 09:41:22 2012] [debug] ajp_header.c(663): ajp_unmarshal_response: Header[2] [Set-Cookie] = [urn:novell:nidp:cluster:member:id=~03~02fey~1F~1F~00~7B~7C; Path=/nesp] [Fri Apr 27 09:41:22 2012] [debug] ajp_header.c(663): ajp_unmarshal_response: Header[3] [Pragma] = [No-cache] [Fri Apr 27 09:41:22 2012] [debug] ajp_header.c(663): ajp_unmarshal_response: Header[4] [Cache-Control] = [no-cache] [Fri Apr 27 09:41:22 2012] [debug] ajp_header.c(663): ajp_unmarshal_response: Header[5] [Content-Type] = [text/xml] [Fri Apr 27 09:41:22 2012] [debug] ajp_header.c(673): ajp_unmarshal_response: ap_set_content_type done [Fri Apr 27 09:41:22 2012] [debug] ajp_header.c(663): ajp_unmarshal_response: Header[6] [Content-Length] = [284] [Fri Apr 27 09:41:22 2012] [debug] ajp_header.c(752): ajp_read_header: ajp_ilink_received 03 [Fri Apr 27 09:41:22 2012] [debug] ajp_header.c(762): ajp_parse_type: got 03 [Fri Apr 27 09:41:22 2012] [error] AMEVENTID#7377: <SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"><SOAP-ENV:Body><SOAP-ENV:Fault><faultcode>Client</faultcode><faultstring>XML document structures must start and end within the same entity.</faultstring></SOAP-ENV:Fault></SOAP-ENV:Body></SOAP-ENV:Envelope>\n [Fri Apr 27 09:41:22 2012] [debug] ajp_header.c(752): ajp_read_header: ajp_ilink_received 05
This confirmed that it was not a configuration issue but a truncating of the request.