Today I wanted to give you a list of common things to try when troubleshooting an issue on Internet Security and Acceleration Server (ISA) or Forefront Threat Management Gateway (TMG). The next time you are facing an issue with either product, I encourage you to walk down through the list and see if any of them apply to your situation. If they do, it may be worth it to you to try before calling Microsoft for support. It is quite possible that it will save you both time and money and we can all use more of both.
1.) Antivirus. Disable (and preferably remove) any Antivirus from your ISA/TMG server machines. I can’t begin to count how many times that one of our products was blamed for something and it actually turned out to be the 3rd party antivirus software on the machine. Whether it was memory leaks that lead to performance issues or some other bizarre behavior being exhibited. It’s my personal opinion that if you take the proper steps to insure that no one is using ISA/TMG as their personal workstation, then you do not need antivirus on the server. My colleague Tom Shinder is of the same opinion and has an excellent blog about this very subject here.
2.) URL Filtering. Disable, and then reboot, any 3rd party URL Filtering service that you are using such as Websense, Surfcontrol, etc. This applies more to ISA Server than TMG because Forefront TMG has provides a feature for URL Filtering. Anytime that you put a filtering device on ISA Server it can have unexpected and unintended results. I have seen all types of behavior caused by these 3rd party services. I am not telling you NOT to use them, I am simply saying that you can save yourself a lot of time and money if you eliminate them as the cause before calling support. To completely take them out of the picture it is usually best to disable the appropriate services for them, then reboot the server. Does your issue still occur? If so, it is not likely that your 3rd party URL filtering service is causing the issue.
3.) Network Adapters. Update your Network Interface Card (NIC) drivers on your ISA or TMG server to the latest ones you can find from the OEM vendor. I have seen a ton of problems caused by drivers that were 2, 3, and even 4 years behind.
4.) Teaming. Disable NIC teaming on your ISA/TMG servers. To my knowledge there is no official documentation on not doing NIC teaming but it has been known to cause problems.
5.) Default Gateway. Default gateway should only be set on the NIC facing the Internet and only one default gateway should ever be set on your ISA/TMG servers. This is a fairly common mistake and can cause multiple problems. See this KB for more information.
6.) DNS. DNS Server settings on your ISA/TMG adapter properties should be set on the Internal facing NIC only and the DNS servers should be servers that your organization controls. If ISA/TMG is a member of a domain, these DNS servers should be Domain Controllers. I saw an issue recently where web proxy users through ISA Server were getting prompted for authentication credentials intermittently when browsing the Internet. This would happen sporadically throughout the day and would often resolve itself after 5 or 10 minutes of pain. The issue turned out to be that the first 2 DNS servers were Domain Controllers but the 3rd one on the list was not.
7.) 3rd Party Networking Devices. Take any 3rd party devices out of the mix whenever possible. Whatever the problem may be, if you can simplify your environment it will make troubleshooting it a whole lot easier. Trust me when I tell you this. Is there a 3rd party hardware load balancer sitting in front of your ISA/TMG server? Do you have the ability to bypass it and test? Is the issue still present? Is there a hardware firewall between your ISA/TMG server and your Domain Controllers? If you bypass it or create an ACL that allows everything through does it change the behavior? You get the idea. The more you can simplify your environment the less time you are going to spend on the phone with support. I cannot even begin to tell you how many times our product has gotten the blame for something that another device was doing.
8.) Service Packs, Hotfixes, and Rollups Oh My. When possible, upgrade to the latest service pack, hotfix, or rollup for ISA or TMG. Our engineers and developers are constantly working to find and fix code defects and to make them available to the public. Chances are we have already fixed the issue you are experiencing and have included it in a recent hotfix/rollup. I put together a blog for the version numbers and location of TMG hotfixes here.
9.) Alerts. In your ISA/TMG management console, under Monitoring, check your Alerts tabs. Is there anything listed with a recent date? Many of the Alerts are self explanatory and can point you in the right direction. Search for them on Bing if they are somewhat unclear.
10.) Disable IIS. If you have the World Wide Web Publishing service installed and running on the same machine as your ISA/TMG it can (and usually does) cause problems. Remove it or, at the very least, disable it and then restart the Firewall Service.
I hope these suggestions are helpful to you and I will add or update them from time to time. If any of these helped you avoid a support incident please feel free to leave me a comment and let me know. Follow me on Twitter @keithabluton
Keith, regarding DNS tip, I usally use 2 NIC's with DNS IP's on both. DNS set on Internal NIC will resolve internal requests and the same will occur for External NIC regarding to external requests. Is it wrong? Can it cause some issue? I have set up several environments like this and never had any trouble...
Great question Uilson. Although it can work, it is not recommended and can lead to problems. Here is a great document that explains it.
This is the most common setup. Multi-homed ISA Server computers that are members of the domain must point a network card only to internal DNS servers because it has to participate in the domain. The internal DNS servers need to forward to an ISP or use the root servers. This allows internal clients to resolve both internal names and Internet names.
Q: Why not point the external ISA NIC to the ISP for DNS?
A: The problem here is that ISA doesn’t know what is internal or external when trying to resolve names. This means ISA can end up trying to resolve internal names to the external ISP. Once it receives “name not found”, the ISA Server computer won’t look for the internal name again and you will fail to participate in the domain.