More than one L2TP VPN connection from behind a NAT device fails with error 809 when TMG 2010 has been configured as a VPN Server

More than one L2TP VPN connection from behind a NAT device fails with error 809 when TMG 2010 has been configured as a VPN Server

  • Comments 2
  • Likes

Introduction

Consider a scenario where a TMG administrator has configured their TMG Server 2010 installed on a Windows Server 2008 R2 for inbound VPN connections. External VPN users (two or more users) are behind a NAT device, which NATs all outbound L2TP VPN traffic. When users try to connect using L2TP VPN connections, only one user from this network can connect at a time. Every connection attempt from another user fails with the following error:

Error message: error 809

The network connection between your computer and the VPN server could not be established because the remote server is not responding. This could be because one of the network devices (e.g., firewalls, NAT, Router etc.) between your computer and the remote server is not configured to allow VPN connections. Please contact your Administrator or your service provider to determine which device may be causing the problem.

Troubleshooting

In this case, since users can connect to VPN as long as they are unique from behind a NAT device, the basic VPN configuration on the TMG Server would not have any problem.

To verify if the issue is specific to L2TP VPN, we configured the TMG VPN Server to allow PPTP. Tests from the client end revealed that we could successfully establish more than one PPTP VPN connections. So the test confirmed that the issue was specific to L2TP VPN connections.

Since in such a scenario, traffic would appear to come from the same external IP, it is normal to think that the TMG server might be dropping the connections as per its Flood Mitigation settings. We did notice some TCP/IP Connection Limit Exceeded errors on the TMG Server in this case. So we created an exception list for the incoming IP address in the Flood Mitigation Settings. Though we see the TCP/IP connection Limit errors go away, this did not resolve our issue.

TMG Live logging shows Initiated and Closed Connections without much detail. Network traces do not help much in this case as the traffic is L2TP and encrypted.

To isolate whether the issue was related to TMG or RRAS, we set up a parallel VPN setup with a Windows Server 2008 R2 machine, configured as RRAS. We were able to reproduce the issue successfully without installing TMG on this machine.

Resolution

The problem in our scenario turned out to be specific to L2TP VPN traffic from behind a NAT device to Windows Server 2008 R2 RRAS Server. This was identified as a problem in Windows Server 2008 R2 and the following KB article addresses this issue. The public hotfix included within the article below updates the Fwpkclnt.sys and Tcpip.sys files:

Only one of the clients that are behind the same NAT device can create L2TP VPN connections to a VPN server that is running Windows Server 2008 R2

Author
Niladri Dasgupta
Support Engineer
Microsoft CSS Forefront TMG Team

Technical Reviewers
Yuri Diogenes
Sr. Support Escalation Engineer
Microsoft CSS Forefront TMG Team

Mohit Kumar
Sr. Support Escalation Engineer
Microsoft CSS Forefront TMG Team

Comments
  • I do not understand the resolution. I am experiencing a similar problem in that we are using the TMG for VPN access. The TMG 2010 is installed on a VM of Server 2008 R2 using Hyper-V. Is it the Server 2008 R2 that is blocking the connections or the TMG? If so, how can I fix it?

  • im getting error that the update is not applicable to your system !

    trying to install it on TMG sp3

Your comment has been posted.   Close
Thank you, your comment requires moderation so it may take a while to appear.   Close
Leave a Comment