[Last updated: 13th January 2014]

Hi,

In this blog entry, I wanted to talk about some changes made in Syn attack protection on Windows Vista onwards systems.

Syn attack protection has been in place since Windows 2000 and is enabled by default since Windows 2003/SP1. In the earlier implementation (Windows 2000/Windows 2003), syn attack protection mechanism was configurable via various registry keys (like SynAttackProtect, TcpMaxHalfOpen, TcpMaxHalfOpenRetried, TcpMaxPortsExhausted). With this previous version of syn attack protection, TCPIP stack starts dropping new connection requests when the threshold values are met regardless of how much system memory or CPU power available to the system. As of Windows Vista and onwards (Vista/2008/Win 7/2008 R2/Windows 8/Windows 2012/Windows 2012 R2), syn attack protection algorithm has been changed in the following ways:

1) SynAttack protection is enabled by default and cannot be disabled!
 
2) SynAttack protection dynamically calculates the thresholds (of when it considers an attack has started) based on the number of CPU cores and memory available and hence it doesn’t expose any configurable parameters via registry, netsh etc.
 
3) Since TCPIP driver goes into attack state based on the number of CPU cores and the amount of memory available, systems with more resources will start dropping new connection attempts later compared to systems with less resources. That was hard-coded (as per the configured registry settings) on pre-Vista systems where the system was moved to attack state regardless of how much resources were available to the system. The new algorithm eliminates the need of any fine tuning and TCPIP stack will self-tune to best values possible depending on the available resources.

One of the questions asked most about TCP Syn attack protection is how an administrator could identify if a server has moved into attack state. Currently there's no event logged whether or not the system has entered into attack state and started dropping TCP Syn packets on Vista and later systems. The only way of understanding that syn attack protection has kicked in is to collect an ETL trace (and you need start it before the attack starts so that you can see the relevant TCPIP ETL entry).

The command that you need to run is the following from an elevated command prompt (Note: "netsh trace" command only works on Windows 7/Windows 2008 R2 and later systems)

netsh trace start capture=yes provider=Microsoft-Windows-TCPIP level=0x05 tracefile=TCPIP.etl

Once Syn attack starts, the ETL trace could be stopped with the below command:

netsh trace stop

Then you can open it up with Network Monitor 3.4. The ETL entry that you should be looking for is the below one:

Hope this helps

Thanks,
Murat

Hi,

I had to deal with a number of support cases where IPReassemblytimeout reg key was set but didn't take effect on Windows 2003 or a later system and I thought I should be sharing more information about this here. Here are some details:

IP fragmentation is needed when an upper layer packet whose payload is bigger than the IP MTU needs to be sent to a destination. This could happen when the packet initially leaves the host or could happen when a router needs to forward a packet that it received through one interface (with a bigger MTU) to another interface (Smaller MTU). That also happens when packets need to traverse VPN links where there's an additional VPN related overhead causing the original packet to be fragmented.

The final receiver of the fragmented IP packets re-assembles those fragments and forms the original packet before passing to upper layer protocols. Receiver waits for a period called "re-assembly timeout" for all the fragments that belong to the original packet to be received. If any one of the fragments is dropped on the way, receiver drops the other fragments belonging to the original packet.

In NT 3.1, there was a registry key called "IPReassemblytimeout" (which is referred to by KB 102973). But that registry key doesn't apply to Windows 2000, XP, 2003, Vista, 2008, Windows 7 or 2008 R2!

Some more facts:

1) IP Re-assembly timeout is hardcoded on Windows 2000, Windows 2003, Windows XP, Windows Vista, Windows 2008, Windows 7 and Windows 2008 R2 and cannot be changed by any means (registry, netsh etc)

2) For Windows Vista, Windows 2008, Windows 7 and Windows 2008 R2, it's hardcoded to 60 seconds as per section 4.5 of RFC 2460:

"If insufficient fragments are received to complete reassembly of a packet within 60 seconds of the reception of the first-arriving fragment of that packet, reassembly of that packet must be abandoned and all the fragments that have been received for that packet must be discarded."

For more information, please see RFC2460 (http://www.ietf.org/rfc/rfc2460.txt)

3) For Windows 2000, Windows XP and Windows 2003, it's at least 60 seconds but it may be higher depending on the value of TTL in the IP header and it may go up to 120 seconds.

Hope this helps..

Thanks,
Murat

Hi,

In this blog post, I would like to talk about a misconfiguration which is still in place on many customer installations. I dealt with many network performance issues where the problem was stemming from using a small MTU size (576 bytes) when communicating with off the subnet hosts.

PMTU discovery option helps communicating endpoints find the most optimum MTU in a TCP session. If this feature is turned off, MTU is set to 576 bytes for all communication with off the subnet hosts. This might badly impact the performance while communicating with the remote hosts.

By default PMTU Discovery is enabled (EnablePMTUDiscovery is set to 1) but due to some older security recommendations, it is set to to 0 as part of server hardening. The reason behind setting that registry key to 0 was to prevent an attacker from forcing Windows to use very small MTU values to decrease the performance.

But that security recommendation is not a valid recommendation anymore as of MS05-019. After that security update, it’s not a security concern anymore because an attacker cannot set MTU size lower than 576 even if PMTU Discovery is enabled. So it shouldn't be set to 0 for security reasons as part of server hardening. This causes performance loss where there's no security concern in terms of small MTU usage.

You can find more information about the changed behavior at the below article:

http://www.microsoft.com/technet/security/bulletin/MS05-019.mspx
Vulnerabilities in TCP/IP Could Allow Remote Code Execution and Denial of Service (893066)

(From General Information > Vulnerability Details > ICMP Path MTU Vulnerability > Faq for ICMP Path MTU Vulnerability at the above link)

What is Path MTU Discovery?
Path maximum transmission unit (PMTU) discovery is the process of discovering the maximum size of packet that can be sent across the network between two hosts without fragmentation (that is, without the packet being broken into multiple frames during transmission). It is described in RFC 1191. For more information, see RFC 1191. For additional information, see the following MSDN Web site.

What is wrong with the Path MTU Discovery process?
Path maximum transmission unit (PMTU) discovery allows an attacker to specify a value that can degrade network performance for other connections. On unsecured networks, allowing PMTU discovery carries the risk that an attacker might force the MTU to a very small value and overwork the local system's TCP/IP stack. Normally this behavior would be restricted to the single connection that an attacker could establish. However, this vulnerability allows an attacker to modify the MTU value on other connections beyond their own connection to the affected system.

What does the update do?
The update removes the vulnerability by restricting the minimum value of the MTU to 576 bytes. This update also modifies the way that the affected operating systems validate ICMP requests.

Thanks,
Murat

Hi there,

In one of the past cases, one of our customers wanted to know if we supported policy based routing on Windows 2003 or later OSes. First of all, it might be useful to clarify what "policy based routing" means in this context. Let's take the following as an example:

"A server is running as a router and have 3 network interfaces. When the server receives a packet from a specific host (let’s say running at a certain IP address) from one of its interfaces (say interface1), we would like the server to always route that host’s packets through interface2 without consulting the routing table. (The criteria might be different for different scenarios such as "all packets with a destination of TCP port 80 to be sent out from interface3 etc)"

This kind of advanced routing decisions are generally supported by network hardware vendors like Cisco. For example, by using route-map configuration in Cisco IOS, you can affect the conventional routing decisions made by looking up the routing table. You can find more information on that at the following link:

http://www.cisco.com/en/US/docs/ios/12_0/qos/configuration/guide/qcpolicy.html Configuring Policy-Based Routing

And the answer to the original question is: No, we don't support policy based routing on Windows server OSes since this is generally a feature that would be needed on hardware routers whose main purpose is to do packet routing.

Hope this helps

Thanks,
Murat

Hi,

In this blog entry, I will be discussing TCP keepalive mechanism and will also provide some information about configuration options on Windows systems.

a) Definition

Let's first understand the mechanism. A TCP keep-alive packet is simply an ACK with the sequence number set to one less than the current sequence number for the connection. A host receiving one of these ACKs responds with an ACK for the current sequence number. Keep-alives can be used to verify that the computer at the remote end of a connection is still available. TCP keep-alives can be sent once every KeepAliveTime (defaults to 7,200,000 milliseconds or two hours) if no other data or higher-level keep-alives have been carried over the TCP connection. If there is no response to a keep-alive, it is repeated once every KeepAliveInterval seconds. KeepAliveInterval defaults to 1 second. NetBT connections, such as those used by other Microsoft networking components, send NetBIOS keep-alives more frequently, so normally no TCP keep-alives are sent on a NetBIOS connection. TCP keep-alives are disabled by default, but Windows Sockets applications can use the setsockopt() function to enable them.

b) Configuration

Now let's talk a little bit about configuration options. There're 3 registry keys where you can affect TCP Keepalive mechanism on Windows systems:

KeepAliveInterval
Key: Tcpip\Parameters
Value Type: REG_DWORD-time in milliseconds
Valid Range: 1-0xFFFFFFFF
Default: 1000 (one second)
Description: This parameter determines the interval between TCP keep-alive retransmissions until a response is received. Once a response is received, the delay until the next keep-alive transmission is again controlled by the value of KeepAliveTime. The connection is aborted after the number of retransmissions specified by TcpMaxDataRetransmissions have gone unanswered.

Notes:
TCPIP driver waits for a TCP Keepalive ACK for the duration of time specified in this registry entry.

KeepAliveTime
Key: Tcpip\Parameters
Value Type: REG_DWORD-time in milliseconds
Valid Range: 1-0xFFFFFFFF
Default: 7,200,000 (two hours)
Description: The parameter controls how often TCP attempts to verify that an idle connection is still intact by sending a keep-alive packet. If the remote system is still reachable and functioning, it acknowledges the keep-alive transmission. Keep-alive packets are not sent by default. This feature may be enabled on a connection by an application

Notes:
In order for a TCP session to stay idle, there should be no data sent or received.

c) If OS is Windows XP/2003 the following registry entry applies:

TcpMaxDataRetransmissions
Key: Tcpip\Parameters
Value Type: REG_DWORD-number
Valid Range: 0-0xFFFFFFFF
Default: 5
Description: This parameter controls the number of times that TCP retransmits an individual data segment (not connection request segments) before aborting the connection. The retransmission time-out is doubled with each successive retransmission on a connection. It is reset when responses resume. The Retransmission Timeout (RTO) value is dynamically adjusted, using the historical measured round-trip time (Smoothed Round Trip Time) on each connection. The starting RTO on a new connection is controlled by the TcpInitialRtt registry value.

Notes:
This registry entry determines the number of TCP retransmissions for an individual TCP segment. There's no special registry entry to determine the retransmission behavior of TCP Keepalives and this registry entry is also used for the TCP keepalive scenario.

Important note: If OS is Windows Vista/2008, the number of TCP Keepalive attempts are hardcoded to 10 and could not be adjusted via the registry.

d) Some special considerations

=> Even if TCP KeepaliveTime and TCPKeepAliveInterval registry keys are set to a specific value (TCPIP driver uses the deafult values even if we don't set these registry keys from the registry), TCPIP driver won't start sending TCP Keepalives until Keepalives are enabled via various methods at upper layers (layers above TCPIP driver).

=> Native Socket applications can enable TCP keepalives by using anyone of the following methods:

- setsockopt() with SO_KEEPALIVE option
- WSAIoctl() with SIO_KEEPALIVE_VALS option (it's also possible to change Keepalive timers with this API call dynamically on a per-socket basis)

=> Managed applications (.NET), can use one of the following methods:

- SetSocketOption method from Socket Class in System.Net.Sockets namespace
- GetSocketOption method from Socket Class in System.Net.Sockets namespace

=> Effect of using Keepalives on bandwidth usage

Since TCP Keepalives are TCP segments without data (and the SEQ number set to one less than the current SEQ number), Keepalive usage bandwidth usage can simply be neglected. There's an example below to give an idea about how big a TCP Keepalive packet could be:

- 14 bytes (L2 header - Assuming that Ethernet protocol is used. This could be even lower for other WAN protocols like PPP/HDLC/etc)
- 20 bytes (IP header - assuming no IP options are used)
- 20 bytes (TCP header - assuming no TCP options are used)

Total: ~54 bytes

Even if TCP Keepalive interval is set to 5 minutes or so (default is 2 hours), given that TCP connection goes idle, TCPIP driver will send a ~54 TCP Keepalive message every 5 minutes and as can be seen it could simply be neglected.

You may also find some references below:

References
=============================
http://www.microsoft.com/downloads/details.aspx?FamilyID=06c60bfe-4d37-4f50-8587-8b68d32fa6ee&displaylang=en
Microsoft Windows Server 2003 TCP/IP Implementation Details

http://www.microsoft.com/downloads/details.aspx?FamilyId=12AC9780-17B5-480C-AEF7-5C0BDE9060B0&displaylang=en
TCP/IP Registry Values for Microsoft Windows Vista and Windows Server 2008

http://msdn.microsoft.com/en-us/library/ms740476(VS.85).aspx setsockopt Function
http://msdn.microsoft.com/en-us/library/ms741621(VS.85).aspx WSAIoctl Function
http://msdn.microsoft.com/en-us/library/system.net.sockets.socket.setsocketoption.aspx Socket..::.SetSocketOption Method
http://msdn.microsoft.com/en-us/library/system.net.sockets.socket.getsocketoption.aspx Socket..::.GetSocketOption Method

Hope this helps.

Thanks,
Murat

Hi there,

Recently I dealt with a connectivity issue that occurs when deploying OS images with Windows 2008 R2 WDS server to PXE clients running in different subnets. As you may already know, we have a workaround for router MTU size incompatibilities seen when deploying OS images to remote subnets in Windows 2008:

http://support.microsoft.com/kb/975710 Operating system deployment over a network by using WDS fails in Windows Server 2008

The same router packet drop issue was present in my case. Even we configured the MaximumBlockSize reg key on the Windows 2008 R2 WDS server, the issue was still in place and the TFTP server (part of WDS server) was still sending data in chunks requested by the client rather than the size we tried to limit wit the MaximumBlockSize reg key:

Note:
10.1.1.1 is PXE client
10.2.2.2 is TFTP server

=> Client is asking for a TFTP block size of 1456 bytes and TFTP server (that’s running as a part of WDS server) is honoring that request:

10.1.1.1          10.2.2.2            TFTP        Read Request, File: boot\x86\wdsnbp.com\000, Transfer type: octet\000, blksize\000=1456\000
10.2.2.2          10.1.1.1            TFTP        Option Acknowledgement, blksize\000=1456\000

=> Then TFTP server begins sending the first 1456 bytes block of wdsnbp.com file:

10.2.2.2            10.1.1.1          TFTP        Data Packet, Block: 1

=> Most likely because the router drops that packet, TFTP client doesn’t receive it and hence it re-sends the Read file request once more:

10.1.1.1          10.2.2.2            TFTP        Read Request, File: boot\x86\wdsnbp.com\000, Transfer type: octet\000, blksize\000=1456\000

=> And TFTP server sends the same block once more and tries a number of times:

10.2.2.2         10.1.1.1      TFTP      Data Packet, Block: 1
10.2.2.2         10.1.1.1      TFTP      Data Packet, Block: 1
10.2.2.2         10.1.1.1      TFTP      Data Packet, Block: 1
10.2.2.2         10.1.1.1      TFTP      Data Packet, Block: 1
...

So PXE boot fails at the end.

The analysis showed us that the reg key didn't take effect on Windows 2008 R2 WDS server. After a source code review, I saw that the feature wasn't integrated into Windows 2008 R2 RTM code due to release timing of Windows 2008 R2 and the fix for Windows 2008. I verified with internal resources and can say that the feature will be a part of Windows 2008 R2 SP1. The SP1 public beta is being made available by the end of July:

http://blogs.technet.com/b/itproaustralia/archive/2010/06/08/windows-7-and-windows-server-2008-r2-sp1-beta-available-end-of-july.aspx  Windows 7 and Windows Server 2008 R2 SP1 Beta available end of July

Hope this helps

Thanks,
Murat

Hi there,

In a number of support cases, I was requested to take a look at "access denied" network traces. In some cases the reason of why we get the access denied is related to Kerberos authentication and Local system account so I wanted to touch this subject in today's blog.

A service or application might be configured to run under "Local System" security context. Normally when such a service or application (like a scheduled task) needs to access network resources, it uses "domain\computername$" account to authenticate itself to remote parties. In one of the scenarios I was involved in, a scheduled task (which runs under Local system security context) ran a script which in turn accessed a server share (\\server\share). When the script was run that way, it got an access denied error from the destination server and failed to run properly.

In the network trace captured when that failure occured, it was seen that the source client tried to authenticate against the remote server with NULL credentials. Since anonymous authentication was not enabled at the destination server, it didn't authenticate the client and returned ACCESS DENIED:

Source            Destination    Protocol Info
======            ========    =======  ==== 
10.1.1.1          10.1.1.2          SMB   Session Setup AndX Request, NTLMSSP_NEGOTIATE
10.1.1.2          10.1.1.1          SMB   Session Setup AndX Response, NTLMSSP_CHALLENGE, Error: STATUS_MORE_PROCESSING_REQUIRED
10.1.1.1          10.1.1.2          SMB   Session Setup AndX Request, NTLMSSP_AUTH, User: \
10.1.1.2          10.1.1.1          SMB   Session Setup AndX Response
10.1.1.1          10.1.1.2          SMB   Tree Connect AndX Request, Path: \\Server\Share
10.1.1.2          10.1.1.1          SMB   Tree Connect AndX Response, Error: STATUS_ACCESS_DENIED
10.1.1.1          10.1.1.2          SMB   Logoff AndX Request
10.1.1.2          10.1.1.1          SMB   Logoff AndX Response

Note: It can be seen in the 3rd frame that the client is sending NULL credentials

The above problem was stemming from a Kerberos authentication failure. Normally "domain\computername$" account is used when the service/application runs under Local System security context but it's done only if the authentication protocol is Kerberos. So if Kerberos works fine, than the given account is used. If Kerberos authentication fails for a reason (like name resolution issues etc) and the client needs to fall back to NTLM, NULL credentials are used instead of "domain\computername$".

If the related service/application is configured to run under "Network service" account context, domain\computername$ account is used when authenticating with either Kerberos or NTLM protocol.

In the above scenario, after addressing the Kerberos issue, the NULL credentials problem was resolved.

As an additional information for Windows 7 & Windows 2008 R2, an application/service running under "Local System account" security context can use domain\computername$ account when authenticating against remote servers with either Kerberos or NTLM authentication. This could be configured via group policies for those systems

http://technet.microsoft.com/en-us/library/dd566199(WS.10).aspx Changes in NTLM Authentication

Hope this helps

Thanks,
Murat