Expiring web pages on origin server not reflected on Linux Access Gateway

Access Manager installed and configured to authenticate users when accessing

protected resources on the Linux Access Gateway (LAG). All worked fine and users

could all access the protected resources on the origin Web server. However, users

noticed that some pages rendered by the proxy appeared outdated. For example:

At 2:28PM, I go to directly to my webserver,
http://oajotstapp08/AccessManager/default.htm.  According to the Firefox plug

in, the page info is:

Modified: Friday, April 13, 2007 11:18:38 AM

Expires:  Tuesday, April 17, 2007 2:29:18 PM

Note: For testing, on the IIS HTTP Headers tab on the virtual directory, I

checked "Enable content expiration" and set "Expire after" to 1 Minute.  The

default is that "Enable content expiration" is not set.

Same page through Access Manager reverse proxy
https://wac-ag1.novell.com/AccessManager/
The page content is out of date and the plug in shows

Modified: Tuesday, April 10, 2007 2:43:36 PM

Expires:  No expiration set

If I press F5 or control/F5 to refresh the page. the plug in shows:

Modified: Friday, April 13, 2007 11:18:38 AM

Expires:  No expiration set

If I close the tab, then type in
https://wac-ag1.dev.novell.com/AccessManager/again, the page content is

out of date again and my page info is back to:

Modified: Tuesday, April 10, 2007 2:43:36 PM

Expires:  No expiration set

No matter what I do, the old page always appears until I force a refresh.  The

Access Manager's cache never seems to get or keep the new page.

In Access Manager, the proxy service's HTTP Options-> Global Cache Options.

Cache freshness is set to the defaults of

HTTP Maximum: 6 minutes

HTTP Default: 2 minutes

HTTP Minimum: 0 seconds

Continue Fill Time: 1 second

HTTP Retries: 4

...and Refresh Request from Browser is set to Refill.

Setting Refresh Request from Browser back to Revalidate (the default) and

applying the changes, makes the April 13th version the one that appears.

If I then make another change, the changed version (April 17) won't come up

even with F5, control/F5 or closing out the tab or anything however, it does

show up if I go directly to my webserver,
http://oajotstapp08/AccessManager/default.htm.

Go to the AG configuration and enable the flag "Allow Pages to be Cached by Browser"

under "HTTP Options" ->"Header Options". This will cause the  proxy to send the "Expires:" header

to browser. Without this flag enabled, there is no need to send the header to

browser as the two cache control headers, no=cache and no-store, will always be injected into the

response from the proxy to the browser.

Here's a snapshot of the HTTP headers when the problem occurs - search for expires and see that the proxy never forwards ...

From Web server ...

GET /sportsdesk/ HTTP/1.1

Host: www.unison.ie

Referer:
http://www.unison.ie/nidp/idff/sso?RequestID=idJyv0CyDJzwnUdE1My3XPcXxxcFk&MajorVersion=1&MinorVersion=2&IssueInstant=2007-04-20T08%3A46%3A04Z&ProviderID=http%3A%2F%2Fwww.mylag.com%3A80%2Fnesp%2Fidff%2Fmetadata&RelayState=http%3A%2F%2Fwww.mylag.com%2Fnesp%2Fapp%2F&consent=urn%3Aliberty%3Aconsent%3Aunavailable&ForceAuthn=false&IsPassive=false&NameIDPolicy=onetime&ProtocolProfile=http%3A%2F%2Fprojectliberty.org%2Fprofiles%2Fbrws-art&AuthnContextStatementRef=%2Furi%2Fanyauthentication
Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg,

application/x-shockwave-flash, */*

Accept-Language: en-us Ua-Cpu: x86

Accept-Encoding: gzip, deflate

User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1)

Cache-Control: no-cache

Cookie: Unison_User=83.141.112.214.109131171028663164

Via: 1.1 www.mylag.com (Access Gateway 3.0.0-85)

HTTP/1.1 200 OK

Date: Fri, 20 Apr 2007 08:46:05 GMT

Server: Apache/1.3.34 (Debian) PHP/4.4.2-1.1

X-Powered-By: PHP/4.4.2-1.1

X-Accelerated-By: PHPA/1.3.3r2

Set-Cookie:

PHPSESSID=200704202da00ff01a0a24311dbff330eaca7b17dc44ac84da87185bc527563086e3a0de094605;

path=/

Expires: Thu, 19 Nov 1981 08:52:00 GMT

Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0

Pragma: no-cache

Last-Modified: Fri, 20 Apr 2007 08:46:05 GMT

Transfer-Encoding: chunked

Content-Type: text/html; charset=iso-8859-1


Proxy server to browser ...


GET /sportsdesk/ HTTP/1.1

Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg,

application/x-shockwave-flash, */*

Referer:
http://idpcluster.lab.novell.com:8080/nidp/idff/sso?RequestID=idJyv0CyDJzwnUdE1My3XPcXxxcFk&MajorVersion=1&MinorVersion=2&IssueInstant=2007-04-20T08%3A46%3A04Z&ProviderID=http%3A%2F%2Fwww.mylag.com%3A80%2Fnesp%2Fidff%2Fmetadata&RelayState=http%3A%2F%2Fwww.mylag.com%2Fnesp%2Fapp%2F&consent=urn%3Aliberty%3Aconsent%3Aunavailable&ForceAuthn=false&IsPassive=false&NameIDPolicy=onetime&ProtocolProfile=http%3A%2F%2Fprojectliberty.org%2Fprofiles%2Fbrws-art&AuthnContextStatementRef=%2Furi%2Fanyauthentication
Accept-Language: en-us UA-CPU: x86

Accept-Encoding: gzip, deflate

User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1)

Host: www.mylag.com

Cookie: Unison_User=83.141.112.214.109131171028663164;

IPCZQX03a36c6c0a=01000000930224c66568735ec73e157f03918f67f6dbb058

Connection: Keep-Alive

Cache-Control: no-cache

HTTP/1.1 200 OK

Date: Fri, 20 Apr 2007 08:46:05 GMT

Server: Apache/1.3.34 (Debian) PHP/4.4.2-1.1

X-Powered-By: PHP/4.4.2-1.1

X-Accelerated-By: PHPA/1.3.3r2

Set-Cookie:

PHPSESSID=200704202da00ff01a0a24311dbff330eaca7b17dc44ac84da87185bc527563086e3a0de094605;

path=/

Last-Modified: Fri, 20 Apr 2007 08:46:05 GMT

Content-Type: text/html; charset=iso-8859-1

Content-Length:       50611

Cache-Control: no-store

Cache-Control: no-cache

Via: 1.1 www.mylag.com (Access Gateway 3.0.0-85)

Because we add the no-cache, no-store cache control headers to the response, there is not need to add the

Expires header and hence it is never sent by the proxy unless the above option is turned off - causing the

no-cache and no-store headers to no longer be injected in responses from the proxy.