HTTPS access through TMG fails from a certain VLAN with a very unusual error: FWX_E_SEQ_ACK_MISMATCH

HTTPS access through TMG fails from a certain VLAN with a very unusual error: FWX_E_SEQ_ACK_MISMATCH

  • Comments 2
  • Likes

In this blog post, I’ll be talking about an interesting problem that I dealt with recently. The problem was that clients running in a certain VLAN were not able to establish HTTPS connections through TMG server. Due to the nature of the network, the clients should be configured as SecureNet clients (my customer cannot configure them as web proxy clients or use TMG client software because these machines are guest machines)

 

I asked for the usual data from our customer to find out what was happening during the problem:

 

- Network trace & HTTPWatch logs from a test client

- TMG data packager from the TMG server

 

1. After receiving the data, I started from client side. What I see on the client side is the client starts failing right after the initial TCP 3-way handshake:

 

Note: Please note that, for privacy purposes all IP addresses have been replaced with random addresses.

 

=> Client network trace:

(ip.addr eq 10.110.233.50 and ip.addr eq 172.16.1.1) and (tcp.port eq 50183 and tcp.port eq 443)

 

453          14:44:32 01.05.2012               27.1395045             0.0000000                               10.110.233.50         172.16.1.1         TCP          TCP: [Bad CheckSum]Flags=......S., SrcPort=50183, DstPort=HTTPS(443), PayloadLen=0, Seq=367515135, Ack=0, Win=8192 ( Negotiating scale factor 0x2 ) = 8192

456          14:44:32 01.05.2012               27.1565249             0.0170204                               172.16.1.1         10.110.233.50         TCP                TCP:Flags=...A..S., SrcPort=HTTPS(443), DstPort=50183, PayloadLen=2, Seq=2180033885 - 2180033888, Ack=367515136, Win=64240 ( Scale factor not supported ) = 64240

457          14:44:32 01.05.2012               27.1565443             0.0000194                               10.110.233.50         172.16.1.1         TCP          TCP: [Bad CheckSum]Flags=...A...., SrcPort=50183, DstPort=HTTPS(443), PayloadLen=0, Seq=367515136, Ack=2180033888, Win=64860 (scale factor 0x0) = 64860

458          14:44:32 01.05.2012               27.1585176             0.0019733                               10.110.233.50         172.16.1.1         TLS          TLS:TLS Rec Layer-1 HandShake: Client Hello.

465          14:44:32 01.05.2012               27.4744962             0.3159786                               10.110.233.50         172.16.1.1         TCP                TCP:[ReTransmit #458] [Bad CheckSum]Flags=...AP..., SrcPort=50183, DstPort=HTTPS(443), PayloadLen=127, Seq=367515136 - 367515263, Ack=2180033888, Win=64860 (scale factor 0x0) = 64860

476          14:44:33 01.05.2012               28.0828875             0.6083913                               10.110.233.50         172.16.1.1         TCP                TCP:[ReTransmit #458] [Bad CheckSum]Flags=...AP..., SrcPort=50183, DstPort=HTTPS(443), PayloadLen=127, Seq=367515136 - 367515263, Ack=2180033888, Win=64860 (scale factor 0x0) = 64860

494          14:44:34 01.05.2012               29.2948179             1.2119304                               10.110.233.50         172.16.1.1         TCP                TCP:[ReTransmit #458]Flags=...AP..., SrcPort=50183, DstPort=HTTPS(443), PayloadLen=127, Seq=367515136 - 367515263, Ack=2180033888, Win=64860 (scale factor 0x0) = 64860

512          14:44:35 01.05.2012               30.3815127             1.0866948                               172.16.1.1         10.110.233.50         TLS          TLS:Continued Data: 2 Bytes

513          14:44:35 01.05.2012               30.3815385             0.0000258                               10.110.233.50         172.16.1.1         TCP                TCP:Flags=...A...., SrcPort=50183, DstPort=HTTPS(443), PayloadLen=0, Seq=367515263, Ack=2180033888, Win=64860 (scale factor 0x0) = 64860

516          14:44:35 01.05.2012               30.5019345             0.1203960                               10.110.233.50         172.16.1.1         TCP                TCP:[ReTransmit #458]Flags=...AP..., SrcPort=50183, DstPort=HTTPS(443), PayloadLen=127, Seq=367515136 - 367515263, Ack=2180033888, Win=64860 (scale factor 0x0) = 64860

535          14:44:37 01.05.2012               31.7018989             1.1999644                               10.110.233.50         172.16.1.1         TCP                TCP:[ReTransmit #458] [Bad CheckSum]Flags=...AP..., SrcPort=50183, DstPort=HTTPS(443), PayloadLen=127, Seq=367515136 - 367515263, Ack=2180033888, Win=64860 (scale factor 0x0) = 64860

568          14:44:38 01.05.2012               33.1872833             1.4853844                               172.16.1.1         10.110.233.50         TLS          TLS:Continued Data: 6 Bytes 

 

- Client cannot connect to remote web site because SSL/TLS negotiation doesn’t succeed because no response from the Web server is received (from client’s perspective)

 

2. Then I decided to check things from TMG server perspective. I first checked the network trace that was collected on the internal interface of TMG server through which the client request was received:

 

6457        14:33:49 01.05.2012               39.8915584             0.0000000                               10.110.233.50         172.16.1.1         TCP                TCP:Flags=......S., SrcPort=50183, DstPort=HTTPS(443), PayloadLen=0, Seq=367515135, Ack=0, Win=8192 ( Negotiating scale factor 0x2 ) = 8192

6461        14:33:49 01.05.2012               39.9079944             0.0164360                               172.16.1.1         10.110.233.50         TCP                TCP:Flags=...A..S., SrcPort=HTTPS(443), DstPort=50183, PayloadLen=0, Seq=2180033885, Ack=367515136, Win=64240 ( Scale factor not supported ) = 64240

6462        14:33:49 01.05.2012               39.9084181             0.0004237                               10.110.233.50         172.16.1.1         TCP                TCP:Flags=...A...., SrcPort=50183, DstPort=HTTPS(443), PayloadLen=0, Seq=367515136, Ack=2180033888, Win=64860 (scale factor 0x0) = 64860

6463        14:33:49 01.05.2012               39.9103936             0.0019755                               10.110.233.50         172.16.1.1         TLS          TLS:TLS Rec Layer-1 HandShake: Client Hello.

6489        14:33:49 01.05.2012               40.2259991             0.3156055                               10.110.233.50         172.16.1.1         TCP                TCP:[ReTransmit #6463]Flags=...AP..., SrcPort=50183, DstPort=HTTPS(443), PayloadLen=127, Seq=367515136 - 367515263, Ack=2180033888, Win=64860 (scale factor 0x0) = 64860

6614        14:33:50 01.05.2012               40.8343158             0.6083167                               10.110.233.50         172.16.1.1         TCP                TCP:[ReTransmit #6463]Flags=...AP..., SrcPort=50183, DstPort=HTTPS(443), PayloadLen=127, Seq=367515136 - 367515263, Ack=2180033888, Win=64860 (scale factor 0x0) = 64860

6806        14:33:51 01.05.2012               42.0457229             1.2114071                               10.110.233.50         172.16.1.1         TCP                TCP:[ReTransmit #6463]Flags=...AP..., SrcPort=50183, DstPort=HTTPS(443), PayloadLen=127, Seq=367515136 - 367515263, Ack=2180033888, Win=64860 (scale factor 0x0) = 64860

6872        14:33:52 01.05.2012               43.1311020             1.0853791                               172.16.1.1         10.110.233.50         TCP                TCP:Flags=...A..S., SrcPort=HTTPS(443), DstPort=50183, PayloadLen=0, Seq=2180033885, Ack=367515136, Win=64240 ( Scale factor not supported ) = 64240

6873        14:33:52 01.05.2012               43.1314010             0.0002990                               10.110.233.50         172.16.1.1         TCP                TCP:Flags=...A...., SrcPort=50183, DstPort=HTTPS(443), PayloadLen=0, Seq=367515263, Ack=2180033888, Win=64860 (scale factor 0x0) = 64860

6891        14:33:52 01.05.2012               43.2517820             0.1203810                               10.110.233.50         172.16.1.1         TCP                TCP:[ReTransmit #6463]Flags=...AP..., SrcPort=50183, DstPort=HTTPS(443), PayloadLen=127, Seq=367515136 - 367515263, Ack=2180033888, Win=64860 (scale factor 0x0) = 64860

7110        14:33:54 01.05.2012               44.4514449             1.1996629                               10.110.233.50         172.16.1.1         TCP                TCP:[ReTransmit #6463]Flags=...AP..., SrcPort=50183, DstPort=HTTPS(443), PayloadLen=127, Seq=367515136 - 367515263, Ack=2180033888, Win=64860 (scale factor 0x0) = 64860

7481        14:33:55 01.05.2012               45.9360680             1.4846231                               172.16.1.1         10.110.233.50         TCP                TCP:Flags=...A.R.., SrcPort=HTTPS(443), DstPort=50183, PayloadLen=0, Seq=2180033886, Ack=367515136, Win=8192 (scale factor 0x0) = 8192 

 

- TMG server doesn’t really send responses back to the client

 

3. Then I decided to check the network trace collected on the external interface of the TMG server:

 

21            14:33:49 01.05.2012               39.6940232             0.0000000                               10.110.235.202       172.16.1.1         TCP                TCP:Flags=......S., SrcPort=55073, DstPort=HTTPS(443), PayloadLen=0, Seq=367515135, Ack=0, Win=8192 ( Negotiating scale factor 0x2 ) = 8192

22            14:33:49 01.05.2012               39.7097097             0.0156865                               172.16.1.1         10.110.235.202       TCP                TCP:Flags=...A..S., SrcPort=HTTPS(443), DstPort=55073, PayloadLen=0, Seq=2180033885, Ack=367515136, Win=64240 ( Scale factor not supported ) = 64240

23            14:33:52 01.05.2012               42.9329903             3.2232806                               172.16.1.1         10.110.235.202       TCP                TCP:Flags=...A..S., SrcPort=HTTPS(443), DstPort=55073, PayloadLen=0, Seq=2180033885, Ack=367515136, Win=64240 ( Scale factor not supported ) = 64240

24            14:33:55 01.05.2012               45.7379523             2.8049620                               172.16.1.1         10.110.235.202       TCP                TCP:Flags=...A.R.., SrcPort=HTTPS(443), DstPort=55073, PayloadLen=0, Seq=2180033886, Ack=367515136, Win=8192 (scale factor 0x0) = 8192

 

- External Web server sends a response to initial TCP 3-way handshake request that is forwarded by TMG, but TMG server doesn’t proceed with the connection

 

4. Then I checked the Web Proxy/Firewall log on the TMG server:

 

01.05.2012 14:33 Denied 10.110.233.50     50183    172.16.1.1 443 0xc0040034 FWX_E_SEQ_ACK_MISMATCH 

01.05.2012 14:33 Denied 10.110.233.50     50183    172.16.1.1 443 0xc0040034 FWX_E_SEQ_ACK_MISMATCH 

01.05.2012 14:33 Denied 10.110.233.50     50183    172.16.1.1 443 0xc0040034 FWX_E_SEQ_ACK_MISMATCH 

 

When I check more details on that error code, I see that we fail because we receive a TCP packet with an invalid sequence number:

 

http://msdn.microsoft.com/en-us/library/ms812624.aspx/

FWX_E_SEQ_ACK_MISMATCH 0xC0040034 A TCP packet was rejected because it has an invalid sequence number or an invalid acknowledgement number.

 

So TMG Server drops the TCP ACK packet (3rd TCP packet in TCP 3-way handshake) coming from the client because it has an invalid TCP ACK number

 

5. The problem was also visible in the ETL trace:

 

… handshake packet is dropped becuase ACK (2180033888) no equal ISN(peer)+1 (2180033886)

Warning:The packet failed TCP sequence validation

… Warning:The packet is dropped because of SEQ_ACK_MISMATCH

 

6. When we check the TMG side network trace we see it there:

 

6457       14:33:49 01.05.2012         39.8915584          0.0000000                            10.110.233.50     172.16.1.1     TCP                TCP:Flags=......S., SrcPort=50183, DstPort=HTTPS(443), PayloadLen=0, Seq=367515135, Ack=0, Win=8192 ( Negotiating scale factor 0x2 ) = 8192

SequenceNumber: 367515135 (0x15E7D5FF)

 

6461       14:33:49 01.05.2012         39.9079944          0.0164360                            172.16.1.1     10.110.233.50     TCP                TCP:Flags=...A..S., SrcPort=HTTPS(443), DstPort=50183, PayloadLen=0, Seq=2180033885, Ack=367515136, Win=64240 ( Scale factor not supported ) = 64240

SequenceNumber: 2180033885 (0x81F0AD5D)

AcknowledgementNumber: 367515136 (0x15E7D600)

 

6462       14:33:49 01.05.2012         39.9084181          0.0004237                            10.110.233.50     172.16.1.1     TCP                TCP:Flags=...A...., SrcPort=50183, DstPort=HTTPS(443), PayloadLen=0, Seq=367515136, Ack=2180033888, Win=64860 (scale factor 0x0) = 64860

SequenceNumber: 367515136 (0x15E7D600)

AcknowledgementNumber: 2180033888 (0x81F0AD60) 

That Acknowledgement number SHOULD HAVE BEEN 2180033886 (0x81F0AD5E)

 

So TMG ignores the rest of the session (like TLS client hello coming from the client machine)

 

6463        14:33:49 01.05.2012               39.9103936             0.0019755                               10.110.233.50         172.16.1.1         TLS          TLS:TLS Rec Layer-1 HandShake: Client Hello.

6489        14:33:49 01.05.2012               40.2259991             0.3156055                               10.110.233.50         172.16.1.1         TCP                TCP:[ReTransmit #6463]Flags=...AP..., SrcPort=50183, DstPort=HTTPS(443), PayloadLen=127, Seq=367515136 - 367515263, Ack=2180033888, Win=64860 (scale factor 0x0) = 64860

6614        14:33:50 01.05.2012               40.8343158             0.6083167                               10.110.233.50         172.16.1.1         TCP                TCP:[ReTransmit #6463]Flags=...AP..., SrcPort=50183, DstPort=HTTPS(443), PayloadLen=127, Seq=367515136 - 367515263, Ack=2180033888, Win=64860 (scale factor 0x0) = 64860

6806        14:33:51 01.05.2012               42.0457229             1.2114071                               10.110.233.50         172.16.1.1         TCP                TCP:[ReTransmit #6463]Flags=...AP..., SrcPort=50183, DstPort=HTTPS(443), PayloadLen=127, Seq=367515136 - 367515263, Ack=2180033888, Win=64860 (scale factor 0x0) = 64860

6872        14:33:52 01.05.2012               43.1311020             1.0853791                               172.16.1.1         10.110.233.50         TCP                TCP:Flags=...A..S., SrcPort=HTTPS(443), DstPort=50183, PayloadLen=0, Seq=2180033885, Ack=367515136, Win=64240 ( Scale factor not supported ) = 64240

6873        14:33:52 01.05.2012               43.1314010             0.0002990                               10.110.233.50         172.16.1.1         TCP                TCP:Flags=...A...., SrcPort=50183, DstPort=HTTPS(443), PayloadLen=0, Seq=367515263, Ack=2180033888, Win=64860 (scale factor 0x0) = 64860

6891        14:33:52 01.05.2012               43.2517820             0.1203810                               10.110.233.50         172.16.1.1         TCP                TCP:[ReTransmit #6463]Flags=...AP..., SrcPort=50183, DstPort=HTTPS(443), PayloadLen=127, Seq=367515136 - 367515263, Ack=2180033888, Win=64860 (scale factor 0x0) = 64860

7110        14:33:54 01.05.2012               44.4514449             1.1996629                               10.110.233.50         172.16.1.1         TCP                TCP:[ReTransmit #6463]Flags=...AP..., SrcPort=50183, DstPort=HTTPS(443), PayloadLen=127, Seq=367515136 - 367515263, Ack=2180033888, Win=64860 (scale factor 0x0) = 64860

7481        14:33:55 01.05.2012               45.9360680             1.4846231                               172.16.1.1         10.110.233.50         TCP                TCP:Flags=...A.R.., SrcPort=HTTPS(443), DstPort=50183, PayloadLen=0, Seq=2180033886, Ack=367515136, Win=8192 (scale factor 0x0) = 8192 

 

7. When I check the client side trace, I see that the ACK number in the TCP ACK packet is really set to the wrong value (2180033888) by the client:

 

  Frame: Number = 6462, Captured Frame Length = 60, MediaType = ETHERNET

+ Ethernet: Etype = Internet IP (IPv4),DestinationAddress:[00-11-22-33-44-55],SourceAddress:[00-12-34-56-78-1B]

+ Ipv4: Src = 10.110.233.50, Dest = 172.16.1.1, Next Protocol = TCP, Packet ID = 24041, Total IP Length = 40

- Tcp: Flags=...A...., SrcPort=50183, DstPort=HTTPS(443), PayloadLen=0, Seq=367515136, Ack=2180033888, Win=64860 (scale factor 0x0) = 64860

    SrcPort: 50183

    DstPort: HTTPS(443)

    SequenceNumber: 367515136 (0x15E7D600)

    AcknowledgementNumber: 2180033888 (0x81F0AD60)

  + DataOffset: 80 (0x50)

  + Flags: ...A....

    Window: 64860 (scale factor 0x0) = 64860

    Checksum: 0x4B5D, Good

    UrgentPointer: 0 (0x0)

 

8. One can think that it’s a problem with TCPIP stack on the client, but when we check the TCP SYN ACK packet (the second TCP packet sent by TMG server before the TCP ACK with wrong sequence number is sent by the client) we see that the client receives that TCP SYN ACK packet with 2 bytes extra data (which is something unusual for a TCP SYN ACK packet - such packets shouldn’t have data just should have TCP header):

 

  Frame: Number = 456, Captured Frame Length = 60, MediaType = ETHERNET

+ Ethernet: Etype = Internet IP (IPv4),DestinationAddress:[00-11-22-33-44-55],SourceAddress:[00-12-34-56-78-1B]

  + DestinationAddress: Microsoft Corporation [00-11-22-33-44-55]

  + SourceAddress: Test Data [00-12-34-56-78-1B]

    EthernetType: Internet IP (IPv4), 2048(0x800)

- Ipv4: Src = 172.16.1.1, Dest = 10.110.233.50, Next Protocol = TCP, Packet ID = 1342, Total IP Length = 46

  + Versions: IPv4, Internet Protocol; Header Length = 20

  + DifferentiatedServicesField: DSCP: 0, ECN: 0

    TotalLength: 46 (0x2E)

    Identification: 1342 (0x53E)

  + FragmentFlags: 0 (0x0)

    TimeToLive: 120 (0x78)

    NextProtocol: TCP, 6(0x6)

    Checksum: 46957 (0xB76D)

    SourceAddress: 194.53.208.72

    DestinationAddress: 10.110.233.50

- Tcp: Flags=...A..S., SrcPort=HTTPS(443), DstPort=50183, PayloadLen=2, Seq=2180033885 - 2180033888, Ack=367515136, Win=64240 ( Scale factor not supported ) = 64240

    SrcPort: HTTPS(443)

    DstPort: 50183

    SequenceNumber: 2180033885 (0x81F0AD5D)

    AcknowledgementNumber: 367515136 (0x15E7D600)

  + DataOffset: 96 (0x60)

  + Flags: ...A..S.

    Window: 64240 ( Scale factor not supported ) = 64240

    Checksum: 0x365C, Good

    UrgentPointer: 0 (0x0)

  - TCPOptions:

   - MaxSegmentSize: 1

      type: Maximum Segment Size. 2(0x2)

      OptionLength: 4 (0x4)

      MaxSegmentSize: 1380 (0x564)

  - TCPPayload: SourcePort = 443, DestinationPort = 50183

     UnknownData: Binary Large Object (2 Bytes)

 

That’s why the TCPIP stack running on the client sends a TCP ACK number which is 2 more than the value that should be:

 

ACK number sent by the client:                                                AcknowledgementNumber: 2180033888 (0x81F0AD60)

The correct ACK number that should have been sent:        AcknowledgementNumber: 2180033886 (0x81F0AD5E)

  

9. And when we check the TCP SYN ACK packet that is leaving the TMG server, we don’t see such an extra 2 bytes:

 

  Frame: Number = 6461, Captured Frame Length = 60, MediaType = ETHERNET

+ Ethernet: Etype = Internet IP (IPv4),DestinationAddress:[00-11-22-33-44-55],SourceAddress:[00-12-34-56-78-1B]

+ Ipv4: Src = 172.16.1.1, Dest = 10.110.233.50, Next Protocol = TCP, Packet ID = 1342, Total IP Length = 44

- Tcp: Flags=...A..S., SrcPort=HTTPS(443), DstPort=50183, PayloadLen=0, Seq=2180033885, Ack=367515136, Win=64240 ( Scale factor not supported ) = 64240

    SrcPort: HTTPS(443)

    DstPort: 50183

    SequenceNumber: 2180033885 (0x81F0AD5D)

    AcknowledgementNumber: 367515136 (0x15E7D600)

  + DataOffset: 96 (0x60)

  + Flags: ...A..S.

    Window: 64240 ( Scale factor not supported ) = 64240

    Checksum: 0x365E, Good

    UrgentPointer: 0 (0x0)

  - TCPOptions:

   - MaxSegmentSize: 1

      type: Maximum Segment Size. 2(0x2)

      OptionLength: 4 (0x4)

      MaxSegmentSize: 1380 (0x564)

 

So that 2 bytes extra data is somehow added to TCP SYN ACK by something else (like the NIC driver on the TMG server, a network device running in between (like Wireless access point etc) or the NIC driver on the client machine

 

SUMMARY:

==========

In summary the HTTPS connectivity problem stems from an issue between the Client and TMG server (including the NIC layer or below on the client and server and the network devices/links in between the two)

 

My customer informed me that the issue was visible with any clients which makes it unlikely that it’s a client side issue. I advised my customer to update NIC drivers on the TMG server side and check the network devices running in the path and upgrade firmwares where possible.

 

Hope this helps

 

Thanks,

Murat

Comments
  • I have a similar issue but also HTTP and HTTPS, still trying to troubleshoot

    we have broadcom drivers with multiple vlans

    working on updating TMG and drivers to highest release

  • Was that issue ever resolved? I have a similar problem.

Your comment has been posted.   Close
Thank you, your comment requires moderation so it may take a while to appear.   Close
Leave a Comment