Blog du Tristank : ISA Server

ISA Server 2006 TCP Retransmits

tristank — Wed, 14 Oct 2009 04:24:00 GMT

Health Checks

I perform ISA Server Health Checks for Premier Support (via Premier Field Engineering) as part of my role.

I’ve seen something a few times recently that I thought it might be helpful to call out, while poking around in the Performance Monitor TCPv4 counter area.

The Problem

In short: Lots of TCP retransmissions per second.

Like, lots. More than 1% is annoying; any more than 5% and you pretty surely have a problem.

Recently, I’ve been seeing 20%.

That’s right, kids, according to Perfmon’s statistics, one in five TCP packets requires retransmission. But! That doesn't necessarily jive with what's seen on the wire, suggesting it might be an internal driver or hardware problem.

If your ISA Server seems like it might be a bit slow, and you haven’t looked yet, go look. I’ll wait. You’re interested in the TCPv4 object, specifically the Segments/sec and Segments Retransmitted/sec counters.

What I’ve seen looks like this:

The green area is TCPv4\Segments/sec. The red area is TCPv4\Segments Retransmitted/sec. They’re using the same scale.

Notice that the retransmission figures track with the overall volume.

This 20% figure has been seen across HP and Broadcom (and possibly Intel) server NICs, so I don’t think it’s specific to either vendor.

Fixing It

In at least one of the places I found this, a simple driver upgrade to the latest version available looked like it fixed the problem.

Otherwise, it could indicate a NIC issue, or a hardware issue with the switch.

If you find yourself in this situation, and do resolve it, please do post details in the comments section below.

PL15W2SP.DLL vs Firewall Client

tristank — Wed, 19 Aug 2009 14:37:00 GMT

As I possibly misspelled or misremembered it, the PL15ws2p.dll (possible sic) file was installed as a Winsock Layered Service Provider on a couple of boxes at a customer site.

Coincidentally, these machines were Windows Server 2008 machines where we couldn’t get the Firewall Client to work properly.

We found that there was a third party LSP using:

NETSH WINSOCK SH CA > catalog.txt

And then opening catalog.txt in notepad. The properties of the Pl15ws2p.dll indicated that it was a signed DLL from American Power Corporation or similar (APC or ACP; one of those no-notes half-hours), and that it was used in some sort of management capacity.

But only one of the machines had this APC software installed on it, and the other didn’t… perhaps it got left behind when it was being uninstalled? The search engines didn’t seem to know much about it.

Either way, next step was clear:

NETSH WINSOCK RESET

To return the Windows Sockets provider list to its shiny defaults, and reboot the computer.

After that, the Firewall Client wasn’t working (which we expected).

A Repair from Not-Called-Add-Remove-Programs-Any-More-Now-It’s-Programs-And-Features-Silly fixed that up.

Cool, huh? Remember: when nothing makes sense and the configuration looks good, perhaps LSPs are to blame?

Now if only I could get my stupid Huawei 3G modem working on my Win7 laptop again (“Device attached to the system is not functioning”… thaaanks).

ISA Server 2006 on Windows Server 2008: Nup

tristank — Thu, 04 Jun 2009 18:52:46 GMT

Yuri’s blog explains some of the detail. But there’s slightly more subtlety to it, which I’ll try to snake-oil in front of you here:

Can I install ISA 2006 on 32-bit Windows Server 2008 ?

No, it only runs on Windows Server 2003. Okay, so technically, it also runs on Windows 2000, but if you’re installing it like that now, you should check the calendar. Windows 2000 is old, man.

Why not ISA Server 2006 on Windows 2008?

Whenever I asked that, people mumbled about TCP/IP stack changes. Sounds plausible, so I let it slide.

Well can I install ISA 2006 on 64-bit Windows Server 2008 ?

No. Wait – sort of, not really. Do you count virtualization?

What do you mean?

Hyper-V or an SVVP-validated platform. (Details on security. And the inimitable “Jim Harris” apparently pretending to be Jim Harrison. Giggle.)

Er, if I do count virtual machines?

Yes. You run it in a 32-bit Windows Server 2003 guest.

Isn’t that cheating?!

No. Well, maybe. Sorry, did you have a point there?

What about Windows Server 2003, x64 Edition?

Installing ISA on it? No. It’s 32-bit only and uses kernel-mode software; you can’t mix and match 32-bit with 64-bit k-mode drivers. Hint: I just helped you study for 070-351.

What about Service Pack 2?

X64 Edition?

Yes!

No.

You’re not being helpful.

Oh really? Your eyes are the wrong shape.

The next version of ISA Server, called Forefront Threat Management Gateway (TMG, or, I guess, Timmy to his friends (yep, I’m betting the G ends up semi-silent)), is available in its initial release in the Windows 2008 Essential Business Server thingo, which is 64-bit only.

The next standalone (i.e. non-EBS-integrated) release is currently available in Beta form, and runs exclusively on Windows Server 2008, x64 edition.

That was more helpful.

You still look funny.

Hey, why don’t your links open in new windows?

Because I think it’s nice for the reader to be able to choose whether an informational link should appear in the current frame or a new tab (or a new window).

Sometimes (probably quite often on this blog), you’ll be done with the content at the current page you’re reading, and just want to replace it with something else. Forcing a new window isn’t polite in the age of tabbed browsing.

Let the user choose.

I agree, that’s so wise. You’re like, amazing.

I know.

On the ISA Server Security Update

tristank — Wed, 15 Apr 2009 03:57:59 GMT

Rambling my way to a point

One of my most favourite “Favorites” (read: “he snarled”) in recent weeks has been the ISA Server Product Team’s Build Numbers post.

They helpfully list the version numbers of each ISA Server, um, version, along with a link to the most recent hotfix for that version. That’s so helpful.

But: In most cases, you had to use the self-service hotfix feature to get that hotfix. Which is better than calling someone, but still not quite one-click conweenyence.

And there was some useful stuff fixed in each – you can do the research (hint: research is typically along the lines of “isa server hotfix site:support.microsoft.com” in whatever search engine you use).

Back to the security update: if you look at the file list for the security updates, they look a lot like the file lists for the recent hotfixes.

(Aside from a little while ago: nice that we’re again using KB articles for file information and not just “you should read the bulletin” placeholders. Makes it easier to reliably find file version information in the one place. No idea who changed it in the first place, but my blunt message to you: that was suboptimal.)

I know you love short versions, Glenda

So, long story short, by applying the security update, you’re getting the most recent build of those binaries for your ISA Server.

Just one caveat: remember that with this patch, you’ll need to reapply it if you make any significant installation-level changes to ISA later (see the bulletin for that).

Antivirus software on ISA Server

tristank — Thu, 09 Apr 2009 02:44:00 GMT

There are two major classes of Anti Virus software (yes, I know I used one word above, it’s called SEO, okay?) that can be used on an ISA Server computer:

ISA-integrated antivirus scanning products
Regular desktop/server antivirus products

The first category is the cooler of the two, and typically involves a Web Filter and/or an Application Filter. It’s been designed to work with ISA Server, and will likely scan HTTP streams while ISA is processing them.

The second category is more common – a desktop or server antivirus product is installed on the ISA Server. That’s probably a good idea from a Defense In Depth perspective.

But if you’re using the second category (or it’s just part of your server build), did you know that there are a set of exclusions we recommend you should use?

The ISA Server product team did some great work in pulling together a set of recommendations for when Antivirus is used on ISA Server. Have a read, have a think, and then check whether yours is implemented correctly. If it isn’t, outages, poor performance and other issues might arise.

And (sorta getting into the ramble here) have you ever noticed that Support people tend to make uncomfortable noises about Antivirus products when you mention they’re installed (if not outright suggesting that you disable and/or uninstall them straight-off)? Well, that’s because when they’re not configured in a way that doesn’t interfere with the operation of other software, they really have, statistically, experientially, and commonly, been known to cause problems.

It’s almost a cliche to be asked to remove AV software while troubleshooting a problem – but the cliche came from somewhere to begin with. Configuring the AV as recommended is an excellent way of minimizing that risk.

IAG – now available for Hyper-V

tristank — Thu, 29 Jan 2009 12:47:03 GMT

Of all the things I could be doing right now, blogging is the one that won. Feel special? Procrastination, but with a helpful bent.

IAG SP2 is now a VHD for Hyper-V

Your mission, Jim, is to make that into a song.

The most interesting “wow” moment I had today was reading that IAG (Intelligent Application Gateway - that’s that Whale SSL thingo) is now available without accompanying hardware.

Previously (as I understand it) IAG 2007 was only available on a hardware appliance of sorts.

Now, at least as far as the Technet Deity is concerned, IAG 2007 SP2 is licensable as a Hyper-V Virtual Machine, if you don’t want to go for the hardware.

The VHD includes IAG 2007 SP2 (I’m downloading the trial now, to get up to Mischief) and ISA Server 2006 (for the firewalling capabilities), running on Windows Server 2003.

I’m something of a noob to IAG, so, um, if you want to ask something, go hit them up instead.

But yay, can’t wait to try it out.

The Cat's Out Of The Bag: ISA Server will become ForeFront TMG

tristank — Wed, 09 Apr 2008 09:21:00 GMT

So, we all know that ISA 2006 doesn't work on Windows Server 2008. Massive architectural changes to the IP stack, blah blah, etc, etc.

People (uh, yeah, just "people") have been asking about what's to become of ISA Server for a while:

"There's no ISA 2008 announced!" they'd scream.

"This surely means the end of one of the best product lines Microsoft has produced!" might have also been heard (in a somewhat muffled way).

"Won't Tristan be out of a job?" one person wailed, unconvincingly.

Well, that's right - the plan at this point is that there is no ISA Server 2008.

(pause for effect, teeth-gnashing, gasping, horror to subside)

As of the next version, Internet Security and Acceleration Server is ForeFront Threat Management Gateway!

(Now you're going to tell me that ISA was a perfectly good name and not at all unwieldy...)

See our well-formed Press Release for details!

Microsoft also today announced the name of its next-generation network edge security product, Forefront Threat Management Gateway. Forefront Threat Management Gateway is the future version of Microsoft ISA Server and will extend the capabilities of ISA Server 2006 with new features and security technologies, designed to help provide multiple-threat protection, simplified management and secure connectivity, and will be built on Windows Server 2008. More details about Forefront Threat Management Gateway will be available later this year.

Excellent! So, key takeaway: we are working on a successor. The product isn't going away!

There's an early beta available from here, though it's downloading at a slow trickle for me right now (got excitement?).

So, go forth and, you know, Manage Threats! In the future!

"Stacking" NTLM Authentication

tristank — Tue, 11 Mar 2008 09:30:49 GMT

This question came up today (well, actually, it was about four weeks ago I started typing this, but bear with me), and it's been a little while since I've rambled about authentication protocols, so let's enjoy a nice, calm discussion on a ~~Monday~~ Tuesday arvo.

The request was something like:
In a Web Publishing scenario, can I do NTLM at the ISA Server and NTLM at the Exchange server too?

No

And the answer is - well, no.

There's no way for the client browser to distinguish between the ISA Server (first) saying 401 WWW-Authenticate: NTLM , and then the IIS Server saying 401 WWW-Authenticate: NTLM.

Because it appears to be a repeated authentication sequence when the connection is already authenticated from IE's perspective (and IE doesn't think it's talking to a different server), IE assumes there's been an auth failure (why else would the server challenge again?).

So, lots of authentication prompts are going to happen. The solution (as described) is not workable.

But

With ISA 2006 and its amazingly-useful-how-did-we-ever-live-without-them Authentication features:

What you could do is Integrated Windows Authentication at the Exchange server (i.e. allow Kerberos), and use protocol transition at the ISA Server, from whatever form of authentication you can accept from a client to Kerberos Credential Delegation (or even another protocol, depending on the auth method used by the listener).

So

The question itself was a "no", but the question almost always isn't actually the question. That one's for free.

Special note: I worked really hard on the headings for this post. I hope it was appreciated.

MaxUserPort - what it is, what it does, when it's important

tristank — Tue, 11 Mar 2008 09:17:57 GMT

What can we say about MaxUserPort that hasn't already been said? Not a lot, it would seem. He's a beautiful dancer, perhaps? Ahh, such gentle humour, and nary a kitten drowned anywhere.

But TCP port shenanigans are fairly frequently misunderstood, so let's talk about the very basics of MaxUserPort.

NB: This is all pre-Vista behaviour - applicable from NT4 through to Windows Server 2003, including all the little NT-flavoured stops on the way.

MaxUserPort controls "outbound" TCP connections

MaxUserPort is used to limit the number of dynamic ports available to TCP/IP applications.

It's never going to be an issue affecting inbound connections. MaxUserPort is not the right answer if you think you have an inbound connection problem.

(I don't know why, I just know it is. Probably something to do with constraining resource use on 16MB machines, or something.)

To further simplify: it's typically going to limit the number of outbound sockets that can be created. Note: that's really a big fat generalization, but it's one that works in 99% of cases.

If an application asks for the next available socket (a socket is a combination of an IP address and a port number), it'll come from the ephemeral port range allowed by MaxUserPort. Typically, these "next available" sockets are used for outbound connections.

The default range for MaxUserPort is from 1024-5000, but the possible range is up to 65534.

When You Fiddle MaxUserPort

So, why would you change MaxUserPort?

In the web server context (equally applicable to other application servers), you'd usually need to look at MaxUserPort when:

- your server process is communicating with some type of other system (like a back-end database, or any TCP-based application server - quite often http web servers)

And:

- you are not using socket pooling, and/or

- your request model is something like one request = one outbound TCP connection (or more!)

In this type of scenario, you can run out of ephemeral ports (between 1024 and MaxUserPort) very quickly, and the problem will scale with the load applied to the system, particularly if a socket is acquired and abandoned with every request.

When a socket is abandoned, it'll take two minutes to fall back into the pool.

Discussions about how the design could scale better if it reused sockets rather than pooling tend to be unwelcome when the users are screaming that the app is slow, or hung, or whatever, so at this point, you'd have established that new request threads are hung waiting on an available socket, and just turn up MaxUserPort to 65534.

What Next? TcpTimedWaitDelay, natch

Once MaxUserPort is at 65534, it's still possible for the rate of port use to exceed the rate at which they're being returned to the pool! You've bought yourself some headroom, though.

So how do you return connections to the pool faster?

Glad you asked: you start tweaking TcpTimedWaitDelay.

By default, a connection can't be reused for 2 times the Maximum Segment Lifetime (MSL), which works out to 4 minutes, or so the docs claim, but according to The Lore O' The Group here, we reckon it's actually just the TcpTimedWaitDelay value, no doubling of anything.

TcpTimedWaitDelay lets you set a value for the Time_Wait timeout manually.

As a quick aside: the value you specify has to take retransmissions into account - a client could still be transferring data from a server when a FIN is sent by the server, and the client then gets TcpTimedWaitDelay seconds to get all the bits it wants. This could be sucky in, for example, a flaky dial-up networking scenario, or, say, New Zealand, if the client needs to retransmit a whole lot... and it's sloooow. (and this is a global option, as far as I remember).

30 seconds is a nice, round number that either quarters or eighths (depending on who you ask - we say quarter for now) the time before a socket is reusable (without the programmer doing anything special (say, SO_REUSEADDR)).

If you've had to do this, at this point, you should be thinking seriously about the architecture - will this scale to whatever load requirements you have?

The maths is straightforward:

If each connection is reusable after a minimum of N (TcpTimedWaitDelay) seconds
and you are creating more than X (MaxUserPort) connections in an N second period...

Your app is going to spend time "waiting" on socket availability...

Which is what techy types call "blocking" or "hanging". Nice*!

Fun* KB Articles:
http://support.microsoft.com/kb/319502/
http://support.microsoft.com/kb/328476

SetSPN improvements in Windows Server 2008! W00t!

tristank — Fri, 21 Dec 2007 10:21:00 GMT

All this stuff is based on a prerelease (RC1) version of Windows Server 2008 and may change before final release. Cheques may not be honoured.

I had a happy moment one night in India when the trainer for our IIS 7.0 TTT course discussed some of the Kerberos-related improvements in IIS7.

... SetSPN got revamped.

We all know (or knew, before my wiki collapsed) that duplicate Kerberos SPNs are bad. (The Wiki is still down, by the way, sorry).

We know that it's been a little bit iffy configuring said SPNs and that the chance of getting it wrong was quite high - there was no control that prevented the registration of the same SPN twice, against different accounts.

Worse: SetSPN was focused on the account (security principal) only - if you thought you had a duplicate, you needed to use a customized LDIFDE command to track it down based on the SPN, as SetSPN wouldn't search by SPN, only by account.

Buuut: Some wonderful SDE that should really be on my Christmas card list decided that SetSPN could become an all-singing all-dancing SPN troubleshooting tool!

Yes folks, SetSPN now has SANITY CHECK (-S) switches and FIND THE PROBLEM (-X) switches! HOW COOL IS THAT!?

In order to not break backwards compatibility (I infer; I didn't actually participate in the conversation or decision making process), these are implemented as new switches, not old ones: Existing scripts that rely on creating duplicates (and then presumably resolving that situation shortly afterwards) won't (er, shouldn't) suddenly break.

C:\Users\Administrator>setspn
Usage: setspn [modifiers switches data] computername
Where 'computername' can be the name or domain\name

Modifiers:
-F = perform the duplicate checking on forestwide level
-P = do not show progress (useful for redirecting output to file)

Switches:
-R = reset HOST ServicePrincipalName
Usage:   setspn -R computername
-A = add arbitrary SPN
Usage:   setspn -A SPN computername
-S = add arbitrary SPN after verifying no duplicates exist
Usage:   setspn -S SPN computername
-D = delete arbitrary SPN
Usage:   setspn -D SPN computername
-L = list registered SPNs
Usage:   setspn [-L] computername
-Q = query for existence of SPN
Usage:   setspn -Q SPN
-X = search for duplicate SPNs
Usage:   setspn -X

Examples:
setspn -R daserver1
It will register SPN 'HOST/daserver1' and 'HOST/{DNS of daserver1}'
setspn -A http/daserver daserver1
It will register SPN 'http/daserver' for computer 'daserver1'
setspn -D http/daserver daserver1
It will delete SPN 'http/daserver' for computer 'daserver1'
setspn -F -S http/daserver daserver1
It will register SPN 'http/daserver' for computer 'daserver1' if no such SPN exists in the forest

So, any instructions out there that currently use the positively archaic SETSPN -A can now be updated to use the shiny new SETSPN -S.

Again, I ask you: How cool is that!?

I should add that I haven't actually tried this yet, just gurgled at the wonderful new options and imagined their effect. If it throws a "NotYetImplementedException", please forgive my enthusiasm :)

Tip o' the Week: WEVTUTIL for EVTX/EVT file conversion

tristank — Fri, 05 Oct 2007 09:19:19 GMT

This week, a pointer to a solution to a problem I occasionally hit.

Windows Vista (and by extension Windows Server 2008, I assume) utilizes a new EVTX log format for event log exports. It's XML-based, natch.

Problem: Everyone's Favourite Log Digestion Tool Log Parser uses system APIs to read event log exports, and the old .EVT event log format isn't "native" any more. Long story short, it chokes on them.

This, to put it mildly, was annoying, as most customers haven't moved to Windows Server 2008 yet (I mean, it's only five months from release - is there ever a better time?) and so supply event logs in the old format when asked.

Anyway - you can convert the old-school event logs into shiny new event logs through the user interface (just double-click the EVT, wait for it to open and display in chronological order; then do a Save As, pick a location and filename and answer an obscure question about language formatting; then find and open the newly-resaved log file), but bluntly, the GUI process leaves a bit to be desired if you have the slightest inkling towards type-A behaviour, and all I really want is something that'll work in Log Parser, really.

WEVTUTIL (and NeilCar) to the rescue. It's included out of the box, and it'll convert those dusty old event logs from the command line, with nary a GUI or common dialog in sight, ready for consumption by Logparser, or any other EVTX-friendly file muncher.

Neil's example (for the click-inhibited):

wevtutil epl application.evt application.evtx /lf:true

Bewdiful.

Netmon vs Chimney

tristank — Mon, 23 Jul 2007 06:22:00 GMT

I recently encountered TCP Chimney for the first time in the wild.

Short version: Chimney is an offload technology that allows the NIC to deal with up to X TCP connections, with any overflow being handled by Windows. All good: get the NIC dealing with more networky stuff, and reduce CPU use. Excellent!

The reason it came up:

I was staring at a small network monitor capture (should have been much bigger) that should have had a few tens of megabytes of FTP but was mostly comprised of SYN, SYN-ACK, ACK s to port 21.

A lot! It'd look like

SYN -> 21, source port X
SYN-ACK
ACK
SYN -> 21, source port X+1
SYN-ACK
ACK

...Hundreds and hundreds of TCP 3-way handshakes, but next to no actual data sent. The server didn't even appear to be sending its connection banner!

Very, very rarely, I'd actually see a frame or two of FTP traffic, but I thought the symptom I was looking at was indicative of resource starvation on the FTP server.

Perfmon didn't confirm the diagnosis, and the FTP server logs showed it was transferring loads and loads of data; I just couldn't see it in the capture.

After ruling out network adapter teaming (The Old Enemy), I wondered if something from the Scalable Networking Pack might have been involved, and a quick internal search later, whop! A symptom match! Because the NIC handles the heavy lifting of all TCP work with Chimney enabled, after the TCP session is established, Netmon doesn't get to see the traffic!

To disable Chimney so you're able to gather captures for troubleshooting purposes, you can use the following netsh command:

Netsh int ip set chimney DISABLED

Once that's done, Netmon (and presumably other NDIS capture drivers, like WinPCap (ethereal/wireshark) should be able to capture all traffic, not just non-TCP stuff!

Kerbi Wiki

tristank — Thu, 12 Apr 2007 09:35:55 GMT

Back from another holiday (I call it the Arnold Rimmer technique) I've been trying to collect, rework and refurbish my thoughts on web-based Kerberos stuff in a personal Wiki.

You're welcome to peruse it with the usual disclaimers that I'm frequently wrong; unlike most people-friendly wikis, this one's read-only unless you're me.

So, feel free to poke around; there might be something useful in there somewhere.
http://sharepoint.tristank.com/Pub/

I think there's a definite market for a kerberos configurator - a drag-and-drop layout tool that lets you draw your network, and have it fill in the SPN registrations you need to use. When I get some time (ah, hahaha! I am so dry!), I might take a look at it.

I want to publish a website to the Internet. How do I enable Kerberos?

tristank — Fri, 02 Mar 2007 04:44:00 GMT

Ya don't.

You can't win. But there are alternatives to fighting.

Why Not?

Windows Kerberos doesn't work in an Internet scenario, it's intranet-only.

the client machine must be a member of the same Active Directory forest as the target site. You just can't guarantee (or even reasonably require) this for random Internet clients.
the client machine has to be able to contact a Domain Controller in that site to get a kerberos ticket. Most folk are understandably cautious about exposing their DCs directly to the Internet.

So Are You Just Telling Me I Can't Do It? You're A Bad Person.

Yes, I am, and no, I'm not telling you that - I'm saying it doesn't work on its own.

It's not a zero-setup-cost solution, but it is a block-level solution that lots of smaller companies don't know about or consider - they just switch to Basic authentication, forego the benefits of pre-authentication, and plonk the server in the DMZ.

That "alternative to fighting" I mentioned earlier is ISA Server 2006. You plonk ISA Server into the domain, either as or behind your outer firewall (depending on number of NICs). Then, you use ISA to publish the website. (ISA's publishing capabilities are laid out in gory detail here).

The website itself doesn't get put anywhere near the Internet, it gets to stay inside the safer part of the network.

How does that help?

ISA has the capability of authenticating a client connection using Basic (Internet friendly!) or Forms authentication (also Internet friendly!) then performing Kerberos Constrained Delegation inside the firewall. It converts one form of authentication into another. There's a big document on ISA 06 authentication options here.

Once ISA has converted the protocol to Kerberos, you're free to do whatever you'd normally do in an Intranet scenario, but only with the nominated website - the website can then use your pre-existing Kerberos delegation setup to do Native Authentication to a SQL Server, or talk to Active Directory, or, well, whatever.

You also gain the not-insubstantial added benefit of ISA being able to pre-authenticate and authorize clients - so that by the time the client even touches your actual website, you know who they are (or at the very least, who they've successfully been able to impersonate), and can potentially even restrict the users allowed to hit it to certain groups (keep in mind that this is before they've even seen a "real" web page on your web server - you have had to write zero code for this).

ISA's the same solution we recommend to help secure our premier applications - Exchange and Sharepoint - so why not use it for yours?

NLB Ain't Application-Aware

tristank — Thu, 01 Mar 2007 09:04:48 GMT

It's been ages since I touched on anything wibbles-related, but I realized I'd neglected a very common query:

If one of my applications is under load, will Network Load Balancing route/move/transfer all the additional load to the other server?

No. As long as the box still lives (or more specifically, the NLB driver is able to send heartbeats and receive incoming IP traffic), NLB will keep on allowing connections.

The load rules are used to govern the rough percentages of connections, but any web developer will tell you that connections don't necessarily map to load.

From NLB's perspective, it doesn't even matter if your application isn't running any more. It's simply there to filter out all the traffic you don't want to hit that machine. (Recall that getting NLB working basically means fire-hosing all incoming traffic at all members of the NLB cluster, and relying on each node to know which bits of traffic to ignore, and which they "own").

For Terminal Servers, this means that if one TS is overloaded and can't accept any more connections, NLB doesn't know or care. IIS is similar - if one Web app is chewing 100% CPU, don't expect connections to be balanced to another server based on that fact alone.

This leads to the existence of health-monitoring utilities that will pull a box from an NLB cluster (i.e. DRAINSTOP it) if they detect a problem with a key app (much as ISA Server 2006 and 2004 do when they detect a problem with an array member).

Technet describes this in more detail here.

(Thanks to 'softie Daniel Taylor for digging up the relevant links and mailing them to me.)