Multi-tenant Site-to-Site (S2S) VPN Gateway with Windows Server 2012 R2

Multi-tenant Site-to-Site (S2S) VPN Gateway with Windows Server 2012 R2

  • Comments 6
  • Likes

This blogs describes in detail the concepts of multi-tenant site-to-site (S2S) Virtual Private Network (VPN) introduced in what’s New in 2012 R2: Hybrid Networking blog. In this blog, we will examine how requirements of two organizations Woodgrove Bank, Contoso Ltd are fulfilled by cloud service provider Fabrikam using multi-tenancy capability of Windows Server 2012 R2 RemoteAcccess role.

To provide connectivity from a tenant’s virtual network to any external network such as Internet or tenant’s corporate network, a gateway is required. The gateway should have at least one interface connected to the virtual network and at least another interface connected to external network. In the below diagram a gateway VM dedicated to Contoso connects Contoso virtual network in Fabrikam to external networks. Interface IfCi1 connects to Contoso virtual network in Fabrikam while interface IfCe1 connects to Contoso external network. When packets have to be routed from Contoso external network to Contoso subnet 10.0.0.0/24 in Fabrikam network, they are forwarded to interface IfCe1. TCP/IP stack on Contoso GW VM routes the packets to interface IfCi1 which delivers the packets inside Contoso subnet 10.0.0.0/24. Similarly when packets are to be routed from subnet 10.0.0.0/24 to Contoso external networks they are forwarded to interface IfCi1 and TCP/IP stack on  Contoso GW VM routes the packets to interface IfCe1 to be delivered to Contoso external networks.

image

Similarly another GW VM dedicated for Woodgrove connects Woodgrove virtual network in Fabrikam to external networks. The problem with gateway VM per tenant approach is, the number of gateways increase linearly as the number of tenants increase. Note that for high availability Fabrikam would have to deploy two gateway VMs per tenant. Fabrikam would prefer deploying a gateway that could be shared across multiple tenants similar to deploying multiple tenant virtual networks over a shared physical network using Hyper Network Virtualization (HNV) or VLAN. In this blog we will explain how Fabrikam is able to reduce its gateway infrastructure costs by enabling isolation of routing and connectivity on a shared multi-tenant gateway.

Multi-tenant TCP/IP stack

With Windows multi-tenant TCP/IP stack, it is possible for a single VM to route packets in a compartmentalized manner exclusively between Contoso interfaces (IfCe1 & IfCi1) for Contoso traffic and between Woodgrove interfaces (IfWe1 & IfWi1) for Woodgrove traffic. This is made possible by creating separate routing compartments for Contoso and Woodgrove on a single VM. Routing compartments enable multiple routing entities with overlapping IP addresses to co-exist on the same gateway VM. Packets and network entities including interfaces, IP addresses, route tables, ARP entries etc. of each tenant are isolated by routing compartments. Interfaces and packets in one compartment cannot be seen in another compartment. Isolation of multi-tenant traffic when it enters/leaves the shared gateway VM, is accomplished by extending the routing domain concept of HNV to Windows Server 2012 R2 gateway VM. You can refer to the blog here for more details of multi-tenant stack and its integration with HNV.

image

In the above diagram, there are three compartments in the gateway (GW-VM) viz., default (compartment id 1), Contoso compartment & Woodgrove compartment. All network entities such as interface If1 are by default created in compartment id 1 hence the name default compartment. In addition to the default compartment, there are two other compartments. In this example, interfaces IfCe1 & IfCi1 in Contoso compartment are exclusive for Contoso. This allows Contoso interfaces IfCi1 & IfCe1 to have IP addresses & routes that may overlap with any  interfaces in other compartment. Multi-tenant TCP/IP stack routes packets between IfCi1 & IfCe1 as if only these two interfaces exist in the VM providing Isolation for all Contoso packets. Interface IfCi1 connects Contoso compartment in GW-VM to Contoso virtual network in Fabrikam network.

While the blog here describes multi-tenant gateway integration with HNV virtual networks, the next section describes integration with VLAN. As seen in the above diagram, the gateway runs as a VM and the host’s physical Network Interface Card NIC1 is connected to Fabrikam network. So packets from virtual networks of all tenants hit NIC1, and they are transmitted to interfaces in respective compartments of the gateway VM by v-switch connected to NIC1.

Integration with VLAN

Let’s examine how packets enter IfCi1 from Contoso virtual network. As seen in the above diagram, the gateway runs as a VM and the host’s physical network interface card NIC1 is connected to Fabrikam network. So VLAN packets of all tenants hit NIC1, and they are transmitted through interface IfCi1 to Contoso compartment of the gateway VM by v-switch connected to NIC1. To ensure that that all packets tagged with Contoso VLAN identifier 4000 are transmitted to Contoso interface IfC1, the following configuration is required on the host:

 

PS C:\> Set-VmNetworkAdapterIsolation -VMName GW-VM -VMNetworkAdapterName NIC1 -MultiTenantStack on -IsolationMode Vlan -AllowUntaggedTraffic $true

PS C:\>  Add-VmNetworkAdapterRoutingDomainMapping -VMName GW-VM  -VMNetworkAdapterName NIC1 -Routingdomainid "{12345678-1000-2000-3000-123456780001}"  -Routingdomainname  "Contoso"  -Isolationid  4000 -Isolationname  IfCi1

The first cmdlet enables NIC1 on GW-VM as multi-tenant NIC capable of isolation based on VLAN. The second cmdlet creates compartment for tenant Contoso and interface ifCi1 in  Contoso compartment.

After the above cmdlets are executed, a compartment for Contoso is created in GW-VM. This can be verified by executing the below cmdlet on GW-VM.

PS C:\> Get-NetCompartment

CompartmentId          : 1

CompartmentDescription : Default Compartment

CompartmentGuid        : {xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx}

CompartmentId          : 2

CompartmentDescription : Contoso

CompartmentGuid        : {12345678-1000-2000-3000-123456780001}

The below cmdlet sets the IP address on interface IfCi1 to 10.0.254.249.

PS C:\> New-NetIPAddress –InterfaceAlias IfCi1 –AddressFamily IPv4 –Ipaddress 10.0.254.249 –PrefixLength 29

After this configuration all packets in Contoso virtual network (VLAN id 4000) that are forwarded to 10.0.254.249 are delivered in Contoso compartment via interface IfCi1 by the v-switch on the gateway host. Since interface IfCi1 is created in Contoso compartment, these packets can be routed to other interfaces such as ifCe1 in the same compartment. With similar configuration for tenant Woodgrove, interface IfWi1 is created in Woodgrove compartment with same IP address 10.0.254.249. Thus traffic from different VLANs of Contoso & Woodgrove is routed to their corresponding compartments in the gateway VM through interfaces IfCi1 & IfWi1 respectively (with overlapping IP address) providing isolation. Whether the virtual networks in Fabrikam are deployed using HNV or VLAN, WS 2012 R2 Gateway terminates S2S VPN connections from multiple tenants and enables connectivity to the tenants’ virtual networks.

Packets in Woodgrove compartment can be routed, over S2S VPN to premises of Woodgrove . With WS 2012 R2 RRAS, Border Gateway Protocol (BGP) can be enabled on Interface IfCi1 & IfWi1 even though they have same IP address. On the same GW-VM, there are two BGP listeners on port 179 with the same IP address 10.0.254.249 in two different compartments. The two listeners can learn overlapping routes (e.g. 10.1.1.0/24) from their respective peers using multi-tenancy routing in WS2012 R2. The rest of the blog explains how S2S VPN connectivity is enabled for multiple tenants using single WS2012 R2 gateway VM.

Multi-tenant S2S VPN

Let’s look at the below diagram that depicts a deployment with two tenants Woodgrove and Contoso connecting to their respective virtual networks over S2S VPN from their premises. Woodgrove has two sites in New York (NY) and San Francisco (SFO) with internal subnets 10.1.0.0/24 and 10.2.0.0/24 respectively. From both these sites, Woodgrove connects to its virtual network 10.0.0.0/24 in Fabrikam through WS 2012 R2 VM viz., GW-VM via S2S VPN. Contoso connects its internal network 10.1.0.0/24 in New York to its virtual network 10.0.0.0/24 in Fabrikam through same GW-VM via S2S VPN. Both the tenants dial S2S VPN interface to the same interface If2 (IP address 131.107.10.10) that is connected to Internet.

image

To enable multi-tenancy of RemoteAccess on GW-VM and to enable S2S VPN for Contoso & Woodgrove tenants the following cmdlets need to be executed on GW-VM:

PS C:\> Install-RemoteAccess –MultiTenancy

PS C:\> Enable-RemoteAccessRoutingDomain  -Name Woodgrove –Type VpnS2S              

PS C:\> Enable-RemoteAccessRoutingDomain  -Name Contoso –Type VpnS2S              

To enable S2S connectivity from New York and SFO sites of Woodgrove, S2S VPN interfaces IfWe1 and IfWe2 are configured with below cmdlets.

 PS C:\> Add-VpnS2SInterface –RoutingDomain Woodgrove –Name IfWe1 –Destination 59.10.10.100 -Protocol IKEv2 -AuthenticationMethod PSKOnly -SharedSecret abc -IPv4Subnet 10.1.0.0/24:100 –SourceIpAddress 131.107.10.10                       

PS C:\> Add-VpnS2SInterface –RoutingDomain Woodgrove –Name IfWe2 –Destination 60.10.10.100 -Protocol IKEv2 -AuthenticationMethod PSKOnly -SharedSecret abc -IPv4Subnet 10.2.0.0/24:100 –SourceIpAddress 131.107.10.10              

The below cmdlet configures S2S VPN interface on GW-VM for tenant Contoso to connect from their New York site.

PS C:\> Add-VpnS2SInterface –RoutingDomain Contoso –Name IfCe1 –Destination 59.10.10.10 -Protocol IKEv2 -AuthenticationMethod PSKOnly -SharedSecret abc -IPv4Subnet 10.1.0.0/24:100 –SourceIpAddress 131.107.10.10              

The above cmdlets configure S2S VPN virtual interfaces IfWe1, IfWe2 and IfCe1 on the gateway. These virtual interfaces are created in Contoso or Woodgrove compartment only after S2S VPN is connected. Let’s examine the lifecycle of S2S VPN interface IfCe1 to understand how multi-tenancy of S2S VPN works.

When S2S VPN interface IfCe1 is configured with the above cmdlet, WS2012 R2 RemoteAccess service, plumbs the necessary IPsec policies on GW-VM that allow IPsec/IKEv2 (Internet Key Exchange version2) tunnel connection from 59.10.10.10 (IP address of external interface on gateway in Contoso NY site) to Interface If1 (131.107.10.10) on GW-VM. Similarly, when IfWe1 & IfWe2 are configured, IPsec policies are plumbed to allow IPsec/IKEv2 tunnels from 59.10.10.100 & 60.10.10.100 (IP address of external interface on gateways in Woodgrove NY  & SFO sites) to Interface If1 (131.107.10.10) on GW-VM.

When IPsec/IKEv2 S2S VPN is dialed from 59.10.10.10 it matches the IPsec policy configured for S2S VPN interface IfCe1 on GW-VM. After IKE pre-shared key (PSK) authentication, RemoteAccess service on GW-VM searches the list of S2S interfaces to find one with matching destination IP address 59.10.10.10 and finds interface IfCe1. Once the interface IfCe1 is identified, RoutingDomain value Contoso is retrieved from interface configuration and the interface is created in Contoso compartment. After interface IfCe1 is created in Contoso compartment in addition to already existing interface IfCi1 (note that interface IfCi1 connects GW-VM to virtual network of Contoso), packets can be routed between interface IfCi1 & IfCe1. S2S Interface IfCe1 connects Contoso New York site to the gateway. Thus Contoso virtual network in Fabrikam is connected over S2S VPN to Contoso NY site via S2S VPN. When S2S interface IfCe1 is dialed out from the gateway, RemoteAccess service determines the compartment from RoutingDomain value of interface IfCe1 configuration.

Similarly, S2S interfaces IfWe1 & IfWe2 are created in Woodgrove compartment enabling routing of packets between Woodgrove virtual network and Woodgrove sites in NY & SFO through interface IfWe1 & IfWe2 respectively. Thus routing of packets between multiple tenants’ sites and their respective virtual networks in Fabrikam is enabled on a shared multi-tenant S2S VPN gateway.

In the following sections details of various features of S2S VPN are described.

Authentication of S2S VPN

In the previous section we have seen how tenant is determined in PSK based authentication. Note that authentication plays a key role in identifying the S2S interface for an incoming connection and subsequently determining the compartment/tenant. S2S VPN connections on Fabrikam GW-VM can be authenticated using one of the following methods

  • Pre-Shared Key (PSK)
  • X.501 Certificates
  • Extensible Authentication Protocol (EAP)

This guide describes detailed steps of configuring S2S VPN using pre-shared key authentication. If X.501 certificates are used to authenticate S2S connections, all the incoming connections must chain up to the root certificate of Fabrikam. This guide describes detailed steps of using X.501 certificates for authenticating S2S VPN connections. S2S connections can also be authenticated using Extensible Authentication Protocol (EAP). Please refer to this guide for details on using EAP for S2S VPN authentication.

When an S2S VPN Interface is dialed from the server it is called initiator and when the server accepts incoming connection, it is called responder. IKE/IPsec polices for S2S VPN can be specified at server level or at interface level. For outgoing connections (initiator), the server uses policies and parameters specified at interface level. For non-PSK based authentication, interface level IPsec policies are plumbed only while dialing out the connection. In case of PSK based authentication the IPsec policies corresponding to S2S interface are applied immediately after configuration

For incoming connections (responder), the server can determine the specific interface for which connection is being requested only after authentication. Hence the basis on which an incoming connection is identified is tied to the authentication method.

If the authentication method is EAP(username), S2S interface is identified based the username that is being authenticated. The S2S Interface whose name matches the authenticated username is activated. If there is no matching S2S Interface name, the incoming connection is treated as remote access (Dial-in) VPN connection.

If the authentication method is certificates, then S2S interface is identified based on the subject name of the certificate being authenticated. The S2S interface whose name matches the certificate subject name is activated.

If authentication method is PSK, S2S interface is identified based on source IP address of the incoming IKE packets and is matched with the destination IP address of configured S2S interfaces.

Since authentication in IKE happens after initial IPsec negotiation is complete, all incoming connections are governed by server level policies except for PSK based authentication. In case of PSK based authentication, since identification of tunnel is based on IP address, only one interface can be enabled per destination and the initiator and responder policies are governed by S2S interface level policies. Also if authentication method is PSK, only IPv4/IPv6 addresses can be specified as destination. For other authentication methods, any resolvable name can be specified as destination of S2S interface.

Routing packets over S2S VPN interface

For connectivity to be established between applications in customer site and virtual network in service provider network, it is important to have proper routes configured across S2S VPN tunnel(s). The following options are available:

  • Static Routes
  • Route exchange via BGP

Static Routes

Static routes can be configured on S2S interface so that when packets arrive in the compartment of the tenant, they are routed over S2S interface. In Woodgrove example above, routes 10.1.0.0/24 and 10.2.0.0/24 are added on S2S interface IfWe1 & IfWe2 respectively with metric 100. These routes also trigger dialing of S2S connection (if it is disconnected) when packets that match the routes on S2S interface end up in Woodgrove compartment. There is also a provision to add routes on S2S interface only after the connection is established without triggering the connection using below cmdlet.

PS C:\>Set-VpnS2SInterface –Name IfC1  -PostConnectionIPv4Subnet 10.1.0.0/24:100 

For more details on static routes for S2S VPN interface,  please refer to this article.

Route exchange via BGP

Routes can be exchanged between customer site gateway(s) and customer virtual network in cloud service provider over S2S VPN via BGP. Details of BGP configuration and various topologies will be covered in next blog. For BGP connection establishment, host specific (/32 in case of IPv4 & /128 in case of IPv6) routes of BGP peer are added as trigger routes on S2S VPN interface. So that when BGP connection establishment is tried, S2S connection is dialed and routes of BGP peers are added on S2S interface. After routes for BGP peer are added on S2S interface, BGP packets are tunneled over S2S interface. Once BGP session is established routes are exchanged by BGP peers running at either end.

In Woodgrove example, BGP can be configured on interface IfWe1 or IfWe2 or IfWi1. Since interface IfWi1 is always present and has a fixed IP address, it is recommended to configure BGP on IfWi1 that connects to the virtual network. Once BGP on IfW1 establishes connection with BGP peers in NY & SFO sites, routes 10.1.0.0/24 and 10.2.0.0/24 are learnt and plumbed on S2S interface IfWe2 & IfWi1 respectively. Thus no manual configuration of routes on S2S interfaces is required.

Rate limiting of traffic on S2S VPN Interface

S2S Interfaces can be configured to limit rate at which data can be transmitted or received. Below cmdlet is an example where bandwidth is limited to 1024 Kilobits per second (kbps) in each direction.

PS C:\> Set-VpnS2SInterface –Name If2  -TxBandwidthKbps 1024  -RxBandwidthKbps 1024

Since the GW-VM terminates S2S VPNs from multiple tenants, by default all connections are created with 5120 kbps limit in each direction so that no single tunnel hogs the CPU.

Static IP Filtering on S2S VPN Interface

On each of the S2S Interfaces, static IP filters can be configured to allow/block packets in each direction based on 5-tuple: source IP address, destination IP address, source port, destination port and protocol. One can set packet filters per interface and configure them to do one of the following:

  • Pass through all traffic except packets prohibited by filters.
  • Discard all traffic except packets allowed by filters.

Contoso can limit traffic going out of the virtual network by adding outbound filters.

PS C:\> Add-RemoteAccessIpFilter -InterfaceAlias IfCi1 -Action Deny -Direction Outbound -AddressFamily IPv4 -List @("10.0.0.0/24:10.1.0.0/24:any”,”any:any:ICMP:0:0”)

The above cmdlet drops all outgoing packets through interface IfCi1 except those that match the following condition:

  • Source IP subnet is 10.0.0.0/24 and destination IP subnet is 10.1.0.0/24
  • Any ICMP traffic

Contoso can restrict incoming traffic into the virtual subnet ( 10.0.0.0/16) to SSL (TCP port 443) & Ping (IMCP request code 8) with below cmdlet

PS C:\> Add-RemoteAccessIpFilter -InterfaceAlias IfCi1 -Action Deny -Direction Inbound -AddressFamily IPv4 -List @("any:10.0.0.0/16:TCP:0:443”,” any:any:ICMP:8:0”)

The below cmdlet allows all outgoing traffic through interface IfCi1 except TCP traffic from IP subnet 10.0.0.0/16 to 10.2.0.0/16 with source port 1234 and destination port 4321.

PS C:\> Add-RemoteAccessIpFilter -InterfaceAlias IfCi1 -Action Allow -Direction Outbound -AddressFamily IPv4 -List @("10.0.0.0/16:10.2.0.0/16:TCP:1234:4321")

Summary

With windows server 2012 R2 a  cloud service provider will be able to deploy a single gateway for providing site to site VPN connectivity to multiple tenant virtual networks.   Tenants will be able to connect from their enterprise sites securely using IPsec /IKEv2  and route traffic between virtual networks and enterprise sites. Tenants will also able to filter ingress and egress traffic through the site to site VPN. Cloud service providers will be able to rate-limit ingress and egress bandwidth for each  connection. Since a cloud service provider will be able to  share a gateway across multiple tenants they will be able to reduce the total cost of offering site to site VPN connectivity service.

If you have any comments we would love to hear them!

Your comment has been posted.   Close
Thank you, your comment requires moderation so it may take a while to appear.   Close
Leave a Comment
  • If one GW is used for S2S VPN and NAT, does the NAT also have some default bandwidth?

  • Hi Stanislav,

      You can configure rate-limit for S2S VPN bandwidth, there is no such option for NAT bandwidth

  • Thank you Uma!

  • Does it have the capability to create static routes between compartments?

    The use case for this is a multi-tiered virtual data center, where the first compartment will be "DMZ", the second compartment will be "Internal" and the third would be "Protected". I want traffic to flow from DMZ to Internal to Protected, with no direct routes from DMZ to Protected. This is possible on Cisco routers using VRF's (pretty much the same as a compartment) and route leaking. Can I achieve the same thing using the HNV Gateway?