Access Gateway not POSTing entire data received from browser when formfill enabled

Access manager setup and working fine. Users can access protected resources successfully after having authenticated at the Identity (IDP) server. After rolling out a new application however, users started receiving 500 internal errors at the browser rather than the application data itself. This new resource had formfill enabled - disabling formfill did not show up the issue, although single sign on was not possible. Looking at the logs, it appeared that the POST data submitted with the form by the browser was being truncated by the Access Gateway before being sent on to the back end Web server.

Fixed in 3.2.1 IR1a and greater.

The client correctly POSTed the application data to the Access Gateway, but the Access Gateway only submited part of the data. Since the client POST includes a Content-Length header outlining how much data was being sent, the proxy should have waited to retrieve the full content-length buffer of data before transmitting to the Web server. The fix is to keep reading until the full content-length worth of data has been received.

The following set of logs were required to determine the source of the problem:

 - Microsoft STRACE application running on browser: captures the data at the HTTP stack level, before being passed down
to SSL. From the STRACE output, we can see that the POST headers and payload
are exactly what is expected (content-length is 307 bytes)

0000: 50 4f 53 54 20 2f 44 65 6c 74 65 6b 54 43 2f 77   POST /DeltekTC/w
0010: 65 6c 63 6f 6d 65 2e 6d 73 76 3f 6e 6f 76 2d 73   elcome.msv?nov-s
0020: 73 2d 66 66 2d 73 69 6c 65 6e 74 26 6d 61 73 74   s-ff-silent&mast
0030: 65 72 63 64 6e 66 66 2d 64 65 6c 74 65 6b 54 43   ercdnff-deltekTC
0040: 2d 38 33 33 31 30 20 48 54 54 50 2f 31 2e 31 0d   -83310 HTTP/1.1.
0050: 0a 41 63 63 65 70 74 3a 20 69 6d 61 67 65 2f 67   .Accept: image/g
0060: 69 66 2c 20 69 6d 61 67 65 2f 78 2d 78 62 69 74   if, image/x-xbit
:
02e0: 2e 63 6f 6d 0d 0a 43 6f 6e 74 65 6e 74 2d 4c 65   .com..Content-Le
02f0: 6e 67 74 68 3a 20 33 30 39 0d 0a 43 6f 6e 6e 65   ngth: 307..Conne
0300: 63 74 69 6f 6e 3a 20 4b 65 65 70 2d 41 6c 69 76   ction: Keep-Aliv
0310: 65 0d 0a 43 61 63 68 65 2d 43 6f 6e 74 72 6f 6c   e..Cache-Control
0320: 3a 20 6e 6f 2d 63 61 63 68 65 0d 0a 43 6f 6f 6b   : no-cache..Cook
0330: 69 65 3a 20 6c 6c 3d 65 6e 5f 55 53 3b 20 6c 69   ie: ll=en_US; li
0340: 3d 53 53 4f 5f 54 43 5f 44 45 56 3b 20 63 69 3d   =SSO_TC_DEV; ci=
0350: 44 45 56 45 43 44 49 56 3b 20 49 50 43 5a 51 58   DEVECDIV; IPCZQX
0360: 30 33 36 33 35 33 33 66 61 35 3d 30 31 30 30 61   0363533fa5=0100a
0370: 36 30 30 30 61 63 34 30 66 35 37 62 38 64 39 66   6000ac40f57b8d9f
0380: 37 33 35 35 63 62 35 32 33 66 64 61 61 62 37 39   7355cb523fdaab79
0390: 35 66 34 3b 20 5a 4e 50 43 51 30 30 33 2d 33 36   5f4; ZNPCQ003-36
03a0: 33 30 33 35 30 30 3d 37 32 37 39 63 36 61 61 3b   303500=7279c6aa;
03b0: 20 4a 53 45 53 53 49 4f 4e 49 44 3d 50 53 63 6e    JSESSIONID=PScn
03c0: 51 31 42 4a 4a 42 53 46 30 51 72 33 4c 63 4b 50   Q1BJJBSF0Qr3LcKP
03d0: 30 31 35 44 51 6e 42 51 50 63 79 30 79 32 47 64   015DQnBQPcy0y2Gd
03e0: 62 70 5a 72 71 33 31 34 6c 59 47 6b 42 73 79 76   bpZrq314lYGkBsyv
03f0: 21 2d 31 35 31 34 38 30 32 38 31 0d 0a 0d 0a      !-151480281....

0000: 75 6e 69 74 3d 6c 6f 67 69 6e 26 65 76 65 6e 74   unit=login&event
0010: 3d 6c 6f 67 69 6e 26 73 63 72 65 65 6e 57 69 64   =login&screenWid
0020: 74 68 3d 31 30 34 34 26 70 61 73 73 3d 63 38 36   th=104&pass=c86
0030: 39 38 30 63 32 31 35 63 30 38 66 35 35 66 30 62   980c215c08f55f0b
0040: 35 37 37 33 39 65 65 39 62 61 30 62 61 33 38 65   57739ee9ba0ba38e
0050: 63 66 30 31 63 26 70 61 73 73 32 3d 26 61 75 74   cf01c&pass2=&aut
0060: 68 75 73 65 72 3d 53 53 4f 5f 54 43 5f 44 45 56   huser=SSO_TC_DEV
0070: 25 35 42 30 25 32 34 58 25 35 44 44 45 56 45 43   %5B0%24X%5DDEVEC
0080: 44 49 56 26 66 6f 72 67 6f 74 50 61 73 73 77 6f   DIV&forgotPasswo
0090: 72 64 46 6c 3d 66 61 6c 73 65 26 63 68 61 6e 67   rdFl=false&chang
00a0: 65 4c 6f 63 61 6c 65 46 6c 3d 66 61 6c 73 65 26   eLocaleFl=false&
00b0: 75 69 64 3d 53 53 4f 5f 54 43 5f 44 45 56 26 70   uid=SSO_TC_DEV&p
00c0: 61 73 73 46 69 65 6c 64 3d 53 53 4f 5f 54 43 5f   assField=SSO_TC_
00d0: 44 45 56 26 64 6f 6d 3d 44 45 56 45 43 44 49 56   DEV&dom=DEVECDIV
00e0: 26 6c 61 6e 67 75 61 67 65 4c 6f 63 61 6c 65 3d   &languageLocale=
00f0: 65 6e 5f 55 53 26 6e 6f 76 2d 73 73 2d 66 66 2d   en_US&nov-ss-ff-
0100: 4e 70 73 74 3d 6d 61 73 74 65 72 63 64 6e 66 66   Npst=mastercdnff
0110: 2d 64 65 6c 74 65 6b 54 43 2d 38 33 33 31 30 26   -deltekTC-83310&
0120: 4c 6f 67 69 6e 2e 78 3d 35 32 26 4c 6f 67 69 6e   Login.x=52&Login
0130: 2e 79 3d 31 39                                    .y=1


- LAN trace when working: We can see that the POST header and payload span
multiple SSL segments (one segment includes the POST header, and one includes
the POST payload), BUT with one HTTP record in each SSL packet. Packets
from working trace shows the POST header and data being split across
multiple SSL segments.

- LAN trace when failing: As with the working trace, we can see that the POST
header and payload span multiple SSL segments (one segment includes the POST
header, and one includes the POST payload), BUT with two HTTP records in each
SSL packet. Each record shows similar symptoms:

a) record 1 has one byte of data (P) and second
record includes rest of the data (OST /Desltek/...)
b) record 2 has  one byte of data (u) and second
record includes rest of the data (nit=login&event=login....)

The content-length matches the amount of data to be sent ie.  307 bytes. It
would appear that the proxy can handle the POST request coming in with multiple
records, but the POST payload is not handled correctly. We simply proxy the
POST request to the back end Web server with one byte of data (u). Given that
we know the content-length of the POSTed data, and that we have not yet
received all the data, it is a problem with the MAG.