2. Understanding the Behavior
Using network monitor was possible to better understand why this happen:
1) The HTTP Header when it works:
- Http: Response, HTTP/1.1, Status Code = 200, URL: /
ProtocolVersion: HTTP/1.1
StatusCode: 200, Ok
Reason: OK
Date: Tue, 13 Oct 2009 15:57:06 GMT
Server: WEBSRV
Expires: Mon, 26 Jul 1997 05:00:00 GMT
Cache-Control: private, no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
P3P: CP="SRV"
Set-Cookie: reg_fb_gate=http%3A%2F%2Fwww.fabrikam.com%2F; path=/; domain=.fabrikam.com
Set-Cookie: reg_fb_ref=http%3A%2F%2Fwww.fabrikam.com%2F; path=/; domain=.fabrikam.com
Set-Cookie: test_cookie=1; path=/; domain=.fabrikam.com
Set-Cookie: lsd=zCI0G; path=/; domain=.fabrikam.com
Connection: close
TransferEncoding: chunked
+ ContentType: text/html; charset=utf-8
ContentEncoding: gzip
HeaderEnd: CRLF
+ chunkSize: 10
- ChunkPayload: HttpContentType = text/html; charset=utf-8
HtmlElement: ヒ
FooterEnd: CRLF
+ chunkSize: 8192
- ChunkPayload: HttpContentType = text/html; charset=utf-8
We can see that the HTTP response says that the following content will be chucked and after that response, destination server sends the other HTTP chucks:
WEBSRV TMG HTTP HTTP:HTTP Payload, URL: /
WEBSRV TMG HTTP HTTP:HTTP Payload, URL: /
WEBSRV TMG HTTP HTTP:HTTP Payload, URL: /
2) HTTP Header when doesn’t work
- Http: Response, HTTP/1.1, Status Code = 200, URL: /
ProtocolVersion: HTTP/1.1
StatusCode: 200, Ok
Reason: OK
Cache-Control: private, no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Expires: Mon, 26 Jul 1997 05:00:00 GMT
P3P: CP="WEBSRV2"
Pragma: no-cache
Set-Cookie: lsd=PQ6kd; path=/; domain=.fabrikam.com
Set-Cookie: reg_fb_gate=http%3A%2F%2Fwww.fabrikam.com%2F; path=/; domain=.fabrikam.com
Set-Cookie: reg_fb_ref=http%3A%2F%2Fwww.fabrikam.com%2F; path=/; domain=.fabrikam.com
Set-Cookie: test_cookie=1; path=/; domain=.fabrikam.com
ContentEncoding: gzip
+ ContentType: text/html; charset=utf-8
X-Cnection: close
Date: Tue, 13 Oct 2009 15:38:13 GMT
ContentLength: 9970
HeaderEnd: CRLF
+ payload: HttpContentType = text/html; charset=utf-8
Notice that the failing server don’t say that the content is chucked, however it is still sending more chucks after that:
WEBSRV2 TMG HTTP HTTP:HTTP Payload, URL: /
WEBSRV2 TMG HTTP HTTP:HTTP Payload, URL: /
WEBSRV2 TMG HTTP HTTP:HTTP Payload, URL: /
Since Chunked Transfer Encoding is a mechanism that allows HTTP messages to be split in several parts the first server is answering correctly, while the second server it is not. According to RFC if a server is using chunked encoding it must set the Transfer-Encoding header to "chunked". In order to compress the content we need to accumulate all the chucks and then compress. When it works, TMG knows that all that content is part of the same HTTP request since it says in the HTTP Response Header; therefore it waits for the entire content, compress and send it back to the client. On the failing server we receive the first answer that doesn’t say that the content is chucked and right after that we receive other chucks, since HTTP Compression is enabled it fails to reassemble all the content since it doesn’t know that they belong to the same content.
3. What can I do to fix it?
The best and most correct thing to do is to contact the administrator for the destination web server and report the problem; they should fix the issue since TMG is acting correctly. However if you want a workaround, that will be disabling the HTTP compression filter.
