1. Another error 64?
After posting one of the reasons why ISA Server 2006 can come up with the generic error 64 in one of my posts, some readers asked me if this is the ultimate reason for this error. The answer is: it is not! Since the error 64 is generic it needs to be carefully interpreted, my previous post about this error mentions the “error 64” with the message: “host not available”.
This post will explain in more details why the error message below showed in the ISA Server 2006 Logging could occur while you are browsing Internet.
Figure 1 – Another error 64.
The error above was caught while the user was trying to browse www.fabrikam.com and download the Windows XP SP2 file. To simulate this problem I used the following lab:
Figure 2 – Lab used to simulate this problem.
2. Understanding the nature of this error
The 64: "The specified network name is no longer available" is a win32 error originally called ERROR_NETNAME_DELETED, this error is mapped in the winerror.h as:
//
// MessageId: ERROR_NETNAME_DELETED
// MessageText:
// The specified network name is no longer available.
In the network level, this problem could be cause by:
Network connectivity problems have various causes, but they typically occur because of incorrect network adapters, incorrect switch settings, faulty hardware, or driver issues. Some connectivity symptoms are intermittent and do not clearly point to any one of these causes.
Per KB325487.
Which means that is more under the TCP/IP level, which is controlled by the Windows OS rather than ISA Server itself.
3. Simulating the Problem
To simulate this problem I used a tool called Network Emulator for Windows and added high latency and random packet loss. Besides I also used the Web Application Stress Tool to add more load to my web server and really simulate a situation where server is busy. Now let’s take a look in the netmon trace got from the external interface of the ISA Server:
ISA Server sends the HTTP GET for the destination server:
12:39:13.355 192.168.1.113 192.168.1.95 HTTP HTTP:Request, GET /
- Http: Request, GET /
Command: GET
+ URI: /
ProtocolVersion: HTTP/1.1
Via: 1.1 ISACONTN1
If-None-Match: "304054985f13c91:4b2"
UserAgent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; SV1; .NET CLR 1.1.4322)
Host: www.fabrikam.com
If-Modified-Since: Wed, 10 Sep 2008 16:09:25 GMT
Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */*
Accept-Language: en-us
UA-CPU: x86
Connection: Keep-Alive
HeaderEnd: CRLF
Destination server sends the answer:
12:39:13.745 192.168.1.95 192.168.1.113 HTTP HTTP:Response, HTTP/1.1, Status Code = 200, URL: /
A HTTP GET is sent to get the XP SP2 file:
12:39:31.751 192.168.1.113 192.168.1.95 HTTP HTTP:Request, GET /XPSP2.zip
Destination server answers:
12:39:32.142 192.168.1.95 192.168.1.113 HTTP HTTP:Response, HTTP/1.1, Status Code = 200, URL: /XPSP2.zip
The file starts to be transferred:
12:39:32.242 192.168.1.95 192.168.1.113 TCP TCP:Flags=...A...., SrcPort=HTTP(80), DstPort=2050
12:39:32.242 192.168.1.113 192.168.1.95 TCP TCP:Flags=...A...., SrcPort=2050, DstPort=HTTP(80)
Suddenly the destination server resets the connection:
12:39:32.424 192.168.1.95 192.168.1.113 TCP TCP:Flags=.....R.., SrcPort=HTTP(80), DstPort=2050
12:39:32.584 192.168.1.95 192.168.1.113 TCP TCP:Flags=.....R.., SrcPort=HTTP(80), DstPort=2050
At this point the session was lost and the error showed in figure 1 appeared in the log.
4. Conclusion
What it is important for you after reading this post is to really understand that ISA Server for scenarios like this only externalize the problem. You need to focus on the real problem and start that by verifying:
· Which device is in between ISA and Internet?
o Don’t think that just because you have only a router in front of ISA Server that you will be “free of errors”. Routers do have updates and potential problems also.
· Can you sniffer the outside traffic to have the real picture of what comes into your network before hits the external interface of ISA Server?
o If you get the netmon trace only on the external interface of ISA and you have more devices in front of it you could be masquerading the real issue since you can’t see the clear traffic.
· If ISA is really the edge device, make sure that network interface card is update, the switch where ISA is connected is working properly, etc.
o Many administrators are only concern with updates on the OS level and forgot to address key updates do the drivers and active network devices.
Almost of the time the investigation of those errors occurs around ISA Server rather than in ISA Server itself. Keep your mind open to a broader set of possibilities instead of focus all our time and efforts in troubleshoot only ISA Server.
Consider a scenario where you have a client workstation behind Forefront TMG 2010 and you are trying to download files from a FTP Server. You are successfully able to logon on the FTP but after type the command “dir” you get the error message below:
The message is pretty clear about what is going on, isn’t it? Well, it is but where do I enable this option? I don’t remember having this on ISA!! To address this issue you just need to enable a new option that we have on TMG, this option is located on the FTP Filter properties as shown below:
After enabling this option and apply the changes you should be able to list your files just fine. It is important to mention that this setting has nothing to do with the FTP Read Only option, that you had in ISA 2004/2006 and still have it on TMG. The FTP Filter when running in read only mode (see figure below) will blocks all commands in the control channel except the following ones:
“ABOR, ACCT, CDUP, CWD /0, FEAT, HELP, LANG, LIST, MODE, NLST, NOOP, PASS, PASV, PORT, PWD /0, QUIT, REIN, REST, RETR, SITE, STRU, SYST, TYPE, USER, XDUP, XCWD, XPWD, SMNT”
You can customize this list by using the sample script below (from Configuring Add-ins MSDN article),in this example the script configures FTP Access Filter to allow only the USER and PASS commands:
Note: don’t change the default Read Only commands unless you have a real business need for that.
As new folks are starting to install Forefront TMG 2010 they are finding out that right after install it they already have an alert on Forefront TMG console similar to the one below:
This behavior is documented in the Forefront TMG 2010 Release Notes and says:
Windows Filtering Platform error message following a computer or Forefront TMG services restart
After you restart the Forefront TMG computer or services, the following error message might be displayed:“Forefront TMG detected Windows Filtering Platform filters that may cause policy conflicts on the server. The following providers may define filters that conflict with Forefront TMG firewall policy: Microsoft Corporation.”If this message is displayed, disable the alert from appearing again, since it does not indicate a real conflict.
From: http://technet.microsoft.com/en-us/library/dd440976.aspx#BKMK_WindowsFilteringPlatformerrormessageafteracomputerorservicesrestart
As the release notes says this is an expected error message, it happens because Forefront TMG firewall engine detects filters on Windows Filtering Platform and it can be safely ignored. If you want to confirm that Forefront TMG is handling the core WFP categories you can use the netsh command below:
Microsoft Windows [Version 6.1.7600]
Copyright (c) 2009 Microsoft Corporation. All rights reserved.
C:\Users\administrator.CONTOSO>netsh advfirewall monitor show firewall
Global Settings:
----------------------------------------------------------------------
IPsec:
StrongCRLCheck 0:Disabled
SAIdleTimeMin 5min
DefaultExemptions NeighborDiscovery,DHCP
IPsecThroughNAT Never
AuthzUserGrp None
AuthzComputerGrp None
StatefulFTP Disable
StatefulPPTP Enable
Main Mode:
KeyLifetime 480min,0sess
SecMethods DHGroup2-AES128-SHA1,DHGroup2-3DES-SHA1
ForceDH No
Categories:
BootTimeRuleCategory Microsoft Forefront Threat Management Gateway
FirewallRuleCategory Microsoft Forefront Threat Management Gateway
StealthRuleCategory Microsoft Forefront Threat Management Gateway
ConSecRuleRuleCategory Windows Firewall
For more information on TMG integration with WFP read Chapter 1 (page 7) of the Microsoft Press Forefront TMG Administrator’s Companion book.
1. Introduction
As we now have Exchange 2010 RC available for download, many of you that are testing Forefront TMG 2010 RC are asking if you can test the Email Protection feature using Exchange 2010. If you read the paper Understanding E-Mail Protection on Forefront TMG published at Tales from the Edge, you will see that one of the questions in the Q&A is:
Question 12) Which versions of Exchange do you support?
Answer: We support Exchange Edge 2007 SP2 and Exchange Edge 2010.
With that you know that it is supported, but the opening question is: how to install Exchange 2010 Edge role and Forefront Protection 2010 on top of an existing Forefront TMG 2010 RC installation? This is exactly the goal of this post; guide you through the steps to perform this installation. This post is assuming that TMG 2010 RC is running on Windows Server 2008 R2.
2. Preparing the Environment for Exchange 2010
Before install Exchange 2010 RC you should install a series of prerequisites and the best way to do this is by following the guidelines from Exchange 2010 Prerequisites document, under the section Install the Windows Server 2008 R2 operating system prerequisites. After complete this process, than you can run the Exchange 2010 setup and choose the following options:
1. Select Install Microsoft Exchange as shown below:
Figure 1 – Selecting Exchange setup option.
2. Click Next in the Introduction page. Read the license agreement, select I accept the terms in the license agreement and click Next to continue.
3. Select Yes in the Error Reporting page and click Next.
4. Select Custom Exchange Server Installation as shown below and click Next to proceed:
Figure 2 – Selecting Custom installation.
5. Select Edge Transport Role in the Server Role selection as shown below and click Next to continue:
Figure 3 – Selecting Edge Transport Role.
6. Chose the appropriated option for the CEIP and click Next to continue.
7. Wait until the readiness check finishes and when your window appears as shown below click Install to proceed:
Figure 4 – Click Install to proceed.
8. When the setup finishes as shown in the figure below, uncheck the option Finalize Installation using the Microsoft Exchange Console and click Finish button to conclude the process.
Figure 5 – Setup finished.
9. On Exchange Setup window, click step 5 – Get Critical update for Microsoft Exchange.
10. Install any critical update that it might have and close the Exchange Setup window.
At this point you already have Exchange 2010 Edge Role installed on your system; next step is to install Forefront Security 2010 for Exchange on TMG.
3. Running Exchange Installation via TMG 2010 Setup
Follow the steps below to install Forefront Security 2010 for Exchange from the TMG setup:
1. Execute the autorun.hta file and choose the option to Install Microsoft Forefront Protection 2010 for Exchange Server:
Figure 6 – Choose the option to install Forefront Security 2010 for Exchange.
2. Select to the terms of the license agreement and privacy statement and click Next.
3. You should receive a notification saying that Exchange Transport service will be restarted. Click Next to proceed.
4. Confirm the installation folders (or change according to your preference) and click Next.
5. Click Next on the Proxy configuration.
6. Leave the Enable antispam now option selected as shown in Figure below and click Next to proceed:
Figure 7 – Enabling Antispam.
7. Leave the Enable antispam now option selected as shown in Figure below and click Next to proceed:
8. Chose the appropriated option for the CEIP and click Next to continue.
9. Review all your selections in the Confirm Settings page as shown below and click Next to continue:
Figure 8 – Reviewing installation settings.
10. While the installing is happening you will also see the window below saying that the setup is configuring the product and services:
Figure 9 – Configuring product and services setup window.
11. After that you should see the last setup window saying that the installation finished successfully as shown below:
Figure 10 – Reviewing installation results.
12. Click Finish button to finish the setup.
13. Click Exit to close the TMG Setup window.
Now you have both consoles available: Exchange and Forefront Protection 2010 for Exchange as shown below:
Figure 11 – FSE and Exchange console available after finishing this procedure.
Note: Something to keep in mind: changes that you perform on TMG 2010 regarding Email Protection will be applied to Exchange Edge and FSE according to the option that you choose. Read the paper Understanding E-Mail Protection on Forefront TMG published at Tales from the Edge for more information on what feature each product owns.
In this post you learned how to install Exchage 2010 Edge Role and Forefront Protection 2010 Beta for Exchange on top of an existing Forefront TMG 2010 RC installation. Now that the setup is done, use the Configuring protection from e-mail-based threats article to configure this feature.
Error 64 can happen due many situations and I documented one of those situations last year and as you could see sometimes it is not easy to find out why this error happens. The issue that I’m about to describe here was identified while I was troubleshooting a third party application that uses TCP Port 80 to transmit files, but not using HTTP. What?? Yeah, I know. Although IANA has established port 80 for HTTP, anyone can create an application that uses port 80 to send whatever they want. This is fine, as long as you don’t try to use this application behind a Firewall that does application layer inspection and look to that traffic and say: what is that? This is not HTTP Protocol and it is using TCP Port 80…I shall block this traffic!
The firewall administrator was smart to understand that and what he did was, he created a custom protocol using port 80 and didn’t bind Web Proxy filtering to it. Fair enough, but doesn’t fully resolved this issue.
2. The Error
When the client (which had the 3rd party application installed on his computer) started to transmit the file to the destination it received an error and didn’t proceed. Using Logging feature the Firewall Administrator saw the error below:
Figure 1 – Error 64
On the netmon trace we could see that the TCP Handshake was established fine, but after the first HTTP Payload has being sent ISA Server 2006 didn’t like what he saw and the connection was reset.
Figure 2 – Connection reset right after first attempt to use TCP Port 80 (with a non compliance HTTP Protocol).
3. Resolution
To resolve this problem what you need to do is not only create a custom protocol and an access rule to use this protocol, but also a deny rule right after this access rule to block the regular HTTP Protocol that has the Web Proxy Filter bind to it. The access rules will look like this:
Figure 3 – Access rule with a Deny to HTTP (with filter) Protocol.
Why do I have to do this? Read this post here and you will know the reason:
Why do I need a deny rule to make an allow rule for a custom protocol work correctly?
http://blogs.technet.com/isablog/archive/2006/09/25/why-do-i-need-a-deny-rule-to-make-an-allow-rule-for-a-custom-protocol-work-correctly.aspx
One of the main aspects of security is the maintenance and to do that correctly the administrator needs to be able to track changes that are done in the environment. There are a lot of challenges on this area and one of the biggest challenge is to log what needs to be logged without overwhelm the server.
When I was working in the platforms team I remember that I received a call from a customer saying that he wants to know who deleted a record on his DNS Zone. First question was: do you have an audit policy for DNS enable? He was like: what is that? After review his environment I saw that the auditing was enabled, but not for the Active Directory Objects (his DNS Zone was integrated to AD).
This post will walk through the Auditing configuration of a DNS Zone (AD Integrated) on Windows Server 2003.
2. Preparing the Environment
There are three steps to prepare the environment:
· Verify if the Audit Policy called Audit Directory Service Access is enabled and what is the setting.
· Enabled the Auditing on the DNZ Zone that you want to audit.
· Use the Event Viewer to find out which object of modified (in this case the example will be an object deletion).
3. Configuring the Audit Policy
Open the Default Domain Controllers Policy, and check if the policy highlighted below is selected just like that:
Figure 1 – Configuring Upload and Download Policy.
In my case I changed to audit Success and Failure, but the final configuration will be according to your needs.
4. Configuring the DNS Zone
Now that we have enabled the Audit Policy to all Domain Controller in the domain, we need to change the DNS Zone. To do that follow the steps below:
1) Open ADSIEdit (Start / Run / ADSIEDIT.msc)
2) Right click in the ADSI Edit and click in Connect To…
3) In the Connection Settings window, configure just like show below:
Figure 2 – Connection Setting.
Note: Change the dc= to reflect your domain name.
4) After that click OK.
5) Now expand the container until you get to the same node as show below:
Figure 3 – Configuring the Zone.
5) Right click in the name of the zone located under CN=MicrosoftDNS and click in Properties.
6) Click in Security and then Advanced.
7) Click in Auditing and click in Add.
8) Type Everyone and click OK. Add the following type of access:
· Write All Properties
· Delete
· Delete Subtree
9) Click OK on all three windows.
Now we are ready to log !!
5. Testing
For testing purpose I delete the record called work01 and here what you should see on the security event log:
Event Type: Success Audit
Event Source: Security
Event Category: Directory Service Access
Event ID: 566
Date: 3/5/2008
Time: 7:33:51 PM
User: CONTOSO\Administrator
Computer: DCCONT
Description:
Object Operation:
Object Server: DS
Operation Type: Object Access
Object Type: dnsNode
Object Name: DC=work01,DC=contoso.msft,CN=MicrosoftDNS,DC=DomainDnsZones,DC=contoso,DC=msft
Handle ID: -
Primary User Name: DCCONT$
Primary Domain: CONTOSO
Primary Logon ID: (0x0,0x3E7)
Client User Name: Administrator
Client Domain: CONTOSO
Client Logon ID: (0x0,0x19062D)
Accesses: Write Property
Properties:
Write Property
Default property set
dnsRecord
dNSTombstoned
dnsNode
Additional Info:
Additional Info2:
Access Mask: 0x20
Note the following points in red (from top to down):
· The event type: the user was able to successfully perform this operation.
· Category: the object was categorized as a DS Object.
· User: the name of the user that performed this operation.
· Object Name: the complete path from where the object was located.
· dNSTombstoned: this is probably the only one that is not friendly. This attribute is logged whenever an object is deleted. For more information review the DNS-Tombstoned Attribute at MSDN.
5. Conclusion
This simple action can help you to track changes on your DNS Zone and prevent security compliance issues when auditors approach to review your environment.
Disclaimer: This article was originally posted in Portuguese on 09/08/2006 at Microsoft Latam Team Blog.
Lately we had received some calls where ISA Server was not using the latest updates, which is fine although is not recommended. However when the subject is Service Pack then it might be a supportability blocker if ISA Server is not running within the supported Service Pack level. ISABPA does a great job in warning an ISA administrator that his ISA Server 2004 is not running with SP3 as shown below:
But today the issue is not only having the system with the latest update, is really a supportability matter. ISA Server 2004 SP2 is not supported since January 13th 2009 as shown in the table below:
Products Released
General Availability Date
Mainstream Support Retired
Extended Support Retired
Service Pack Retired
Notes
Internet Security and Acceleration Server 2004 Enterprise Edition
3/1/2005
4/13/2010
4/14/2015
3/11/2006
Internet Security and Acceleration Server 2004 Enterprise Edition Service Pack 2
1/31/2006
Not Applicable
1/13/2009
Internet Security and Acceleration Server 2004 Service Pack 1
3/11/2005
4/10/2007
Internet Security and Acceleration Server 2004 Service Pack 3
10/29/2007
Review Note
Support ends 12 months after the next service pack releases or at the end of the product's support lifecycle, whichever comes first. For more information, please see the service pack policy at http://support.microsoft.com/lifecycle/#ServicePackSupport .
Internet Security and Acceleration Server 2004 Standard Edition
9/8/2004
10/13/2009
10/14/2014
Internet Security and Acceleration Server 2004 Standard Edition Service Pack 2
From: http://support.microsoft.com/lifecycle/?p1=2108
Same applies to ISA Server 2006 RTM (without SP1), which the support will end July 14th 2009 as shown in the table below:
Internet Security and Acceleration Server 2006 Enterprise Edition
10/17/2006
1/10/2012
1/10/2017
7/14/2009
Internet Security and Acceleration Server 2006 Service Pack 1
7/2/2008
Internet Security and Acceleration Server 2006 Standard Edition
From: http://support.microsoft.com/lifecycle/?p1=11928
So if you are in an unsupported scenario (or about to get into this stage) make sure to plan your update as soon as possible to avoid supportability concerns when opening an incident with Microsoft CSS.
Slow browsing experience is a behavior that can happen for so many reasons that cover everything in one single article is just not feasible, mainly when the list still growing. Here are some posts/articles that I wrote on this matter:
The list is growing because recently, troubleshooting another issue of this nature today I learned another cool thing: disabled adapters matters. What I mean with this is that if you have multiple adapters on your ISA/TMG and if some of the adapters are not in use and are disabled, this can still affect the performance of the system. How? According to the binding order. Remember that I talked about binding order here some time ago? The DNS best practice analyzer pointed me to the right direction on this, here what he said about disabled adapters on the top of the binding order:
Issue
Valid network interfaces should precede invalid interfaces in the binding order. A disabled or invalid adapter precedes a valid adapter in the network interface binding order list.
Impact
The binding order determines when network interfaces will be used to make network connections by the computer. A disabled adapter high in the binding order can degrade performance.
Resolution
Move all disabled and invalid interfaces to the bottom of the binding order list.
Keep this caveat in mind during your ISA/TMG performance health check analyzes.
This post is about a specific condition that can triggers the error 502 while browsing some web sites through TMG 2010 RC. The error message that the end users receives is similar to the one shown below:
The TMG Logging will not say much beyond as shown below:
This scenario was interesting because it was working sometimes but it didn’t work most of the time. Looking closely to the data I notice that when it works is because it hits one specific server in the destination web farm and when doesn’t work it hits another web server.
2. Understanding the Behavior
Using network monitor was possible to better understand why this happen:
1) The HTTP Header when it works:
- Http: Response, HTTP/1.1, Status Code = 200, URL: /
StatusCode: 200, Ok
Reason: OK
Date: Tue, 13 Oct 2009 15:57:06 GMT
Server: WEBSRV
Expires: Mon, 26 Jul 1997 05:00:00 GMT
Cache-Control: private, no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
P3P: CP="SRV"
Set-Cookie: reg_fb_gate=http%3A%2F%2Fwww.fabrikam.com%2F; path=/; domain=.fabrikam.com
Set-Cookie: reg_fb_ref=http%3A%2F%2Fwww.fabrikam.com%2F; path=/; domain=.fabrikam.com
Set-Cookie: test_cookie=1; path=/; domain=.fabrikam.com
Set-Cookie: lsd=zCI0G; path=/; domain=.fabrikam.com
Connection: close
TransferEncoding: chunked
+ ContentType: text/html; charset=utf-8
ContentEncoding: gzip
+ chunkSize: 10
- ChunkPayload: HttpContentType = text/html; charset=utf-8
HtmlElement: ヒ
FooterEnd: CRLF
+ chunkSize: 8192
We can see that the HTTP response says that the following content will be chucked and after that response, destination server sends the other HTTP chucks:
WEBSRV TMG HTTP HTTP:HTTP Payload, URL: /
2) HTTP Header when doesn’t work
P3P: CP="WEBSRV2"
Set-Cookie: lsd=PQ6kd; path=/; domain=.fabrikam.com
X-Cnection: close
Date: Tue, 13 Oct 2009 15:38:13 GMT
ContentLength: 9970
+ payload: HttpContentType = text/html; charset=utf-8
Notice that the failing server don’t say that the content is chucked, however it is still sending more chucks after that:
WEBSRV2 TMG HTTP HTTP:HTTP Payload, URL: /
Since Chunked Transfer Encoding is a mechanism that allows HTTP messages to be split in several parts the first server is answering correctly, while the second server it is not. According to RFC if a server is using chunked encoding it must set the Transfer-Encoding header to "chunked". In order to compress the content we need to accumulate all the chucks and then compress. When it works, TMG knows that all that content is part of the same HTTP request since it says in the HTTP Response Header; therefore it waits for the entire content, compress and send it back to the client. On the failing server we receive the first answer that doesn’t say that the content is chucked and right after that we receive other chucks, since HTTP Compression is enabled it fails to reassemble all the content since it doesn’t know that they belong to the same content.
3. What can I do to fix it?
The best and most correct thing to do is to contact the administrator for the destination web server and report the problem; they should fix the issue since TMG is acting correctly. However if you want a workaround, that will be disabling the HTTP compression filter.
There are many instances where you need to move a server to a new hardware. What you will carry from the old server to the new one will vary; you need to plan according to the server role in order to create your own checklist. Recently I had to move my own TMG Server to a new hardware (in this case a new VM) and I decided to install all over again and just import the rules. In a scenario with TMG, besides the core configuration (XML Backup) you also need to consider the certificates that are in use. As I only have a couple of certificates, there was no big deal, I just opened MMC and exported those certs. However, there are scenarios where you have a great amount of certificates and the process of exporting one by one can get quiet tedious.
But, you can use certutil to automate that. The first step in this procedure is to identity the certificate’s thumbprint (or serial number – depends on the approach in use). To do that you can use the PowerShell commands below:
PS C:\Users\Administrator> cd cert:
PS cert:\> dir
Location : CurrentUser
StoreNames : {SmartCardRoot, UserDS, AuthRoot, CA...}
Location : LocalMachine
StoreNames : {SmartCardRoot, AuthRoot, CA, Trust...}
PS cert:\> cd LocalMachine
PS cert:\LocalMachine> dir
Name : SmartCardRoot
Name : AuthRoot
Name : CA
Name : Trust
Name : Disallowed
Name : My
Name : Root
Name : TrustedPeople
Name : TrustedDevices
Name : Remote Desktop
Name : TrustedPublisher
Name : REQUEST
PS cert:\LocalMachine> dir My
Directory: Microsoft.PowerShell.Security\Certificate::LocalMachine\My
Thumbprint Subject
---------- -------
C571112B20BE45D10AD185FAA6A022ADB08F1693 CN=TMGFW
089D8C1FD45893D7BF76F3788D3B1ED7E0974100 CN=mail.contoso.com, OU=Security, O=Contoso, L=Dallas, S=Texas
Or you can also use the CertUtil command to list all the certificate in your local store:
C:\Users\Administrator>certutil -store my
my
================ Certificate 0 ================
Serial Number: 67137d4819445f8947dfe5975c2dcda4
Issuer: CN=TMGFW
NotBefore: 1/8/2010 6:19 PM
NotAfter: 1/8/2015 6:19 PM
Subject: CN=TMGFW
Signature matches Public Key
Root Certificate: Subject matches Issuer
Template:
Cert Hash(sha1): c5 71 11 2b 20 be 45 d1 0a d1 85 fa a6 a0 22 ad b0 8f 16 93
Key Container = cb450661-cef0-423d-8c8d-f35770442ef9
Unique container name: 1c5d966ed267ef52208611a02c81673c_5d23994c-569c-4045-8627-97f7be02ff89
Provider = (null)
Private key is NOT exportable
Encryption test passed
================ Certificate 1 ================
Serial Number: 610df5bb000000000002
Issuer: CN=Contoso CA, DC=contoso, DC=com
NotBefore: 1/29/2010 4:21 PM
NotAfter: 1/29/2012 4:31 PM
Subject: CN=mail.contoso.com, OU=Security, O=Contoso, L=Dallas, S=Texas
Non-root Certificate
Cert Hash(sha1): 08 9d 8c 1f d4 58 93 d7 bf 76 f3 78 8d 3b 1e d7 e0 97 41 00
Key Container = a5195f11b98a20b15d90c79844b9eeb8_b616de3c-8c7e-4aa6-bb33-aae84203823f
Unique container name: 45ec83860a1e2a33a77a819ede5c1d80_5d23994c-569c-4045-8627-97f7be02ff89
Provider = Microsoft Enhanced Cryptographic Provider v1.0
CertUtil: -store command completed successfully.
Write it down the certificate’s serial number and assuming that the key is exportable, you now just need to run the command below:
certutil -exportPFX -p "Password" my 610df5bb000000000002 contoso.pfx
See the article Manual Key Archival for more information about CertUtil tool with -ExportPFX parameter.
The Microsoft Forefront Threat Management Gateway promises to be the milestone that all ISA Server admins were waiting for. I heard all the time people saying that the difference between ISA Server 2004 and ISA Server 2006 were not that big and that we pretty much have the same product for 4 years already. Well, that isn’t really true; there are indeed many differences between 2004 and 2006. Maybe some people were waiting for a huge upgrade like it was from ISA 2000 to 2004 and this didn’t happen. After two years since ISA Server 2006 was released, we have now (without a doubt) a big change, maybe will not be noticeable now but it will in the final version.
You can download the beta version from here and use the installation guide article that my friend Tom Shinder wrote. This beta version available for download has only a limited set of features. However, before install read the release notes to see what you can and what you cannot do.
There are many things that you will notice and see that it is different from ISA Server 2006. As far as installation is concern there are some things that you need to remember:
· IIS will be installed: that’s correct; IIS now will be installed by TMG. You might be thinking: “I remember that we have issues with IIS and ISA in the same box…”. You are right for ISA Server, but for TMG we need IIS because TMG needs SQL Reporting Services 2005 and SQL Reporting Services 2005 needs IIS. It is important to emphasize that IIS is not removed if you uninstall TMG.
· 64 bits System: although the final version of TMG requires a 64-bit processor and Windows Server 2008 64-bit, this beta version can be installed in a 32-bit system with Windows Server 2008.
· WEBS: the TMG beta version that we have available for download it will be part of the Windows Essential Business Server. TMG will be available through WEBS Standard and Premium Edition.
Note: The official TMG documentation is available at Microsoft TechNet Library web site.
2. Just installed, now what?
Now that you installed let’s create a Web Access Policy, to do that click in Configure Web Access Policy in the screen below:
Figure 1 – Web Access Policy.
Now follow the steps below to use this new wizard:
1) In the welcome screen click in Next.
2) In the Web Protection page, click in Yes, enable malware inspection feature, and click in Next:
Figure 2 – Web Protection page.
3) In the Web Access Policy Type page, click in Create customized Web access policies for users, groups and computers and click in Next:
Figure 3 – Web Access Policy Type window.
4) In the Access Policy Groups page you can select the option to allow users, groups, computers by name or IP and also subnets. For the purpose of this demo, select Users and user groups only. Click in Next to continue.
Figure 4 – Configuring the Access Policy Groups
5) In the Default Web Access Policy Page click in Allow the Web requests and click in Next.
Figure 5 – Default Web Access Policy Page.
6) In the Authenticated Web Access Policies page, click in Add.
Figure 6 – Selecting the users that will have access.
7) In the Add Access Policy window, type the policy name, click in Add button to add the group that will have access. Notice that this window is similar to the ISA Server 2006 window; you can use the same functionalities to add: windows groups, LDAP, RADIUS or SecurID. For the purpose of this demo, select All Authenticated Users. Click Add and click Close. Select Allow access to the destinations below and click in Add button to add the External network. After finish, click OK and Next to proceed.
Figure 7 – Access Policy configuration.
8) In the Malware Inspection Setting page, leave the default options selected and click in Next.
Figure 8 – Malware Inspection Setting.
Note: for more information in the Malware Inspection feature read the TechNet Article about that.
9) Since the cache driver was not configured yet, the next window will allow you to configure that:
Figure 9 – Web Cache Configuration.
10) For the purpose of this example, I’m going to disable the option Enable Web Caching. Click in Next to continue and then click in Finish.
After finished creating the rule, click in Apply to commit it. Now check it out this nice improvement in the interface:
Figure 10 – New TMG Interface with enhanced items.
3. Try to Browse now
If you try to browse after creating this rule one thing that you will notice in the live logging is the presence of new fields that can identify “on the fly” if the web site has any threat or not:
Figure 11 – New TMG Logging Interface fields.
This is a just a little overview of what is the TMG that is part of the WEBS, but as I said before, much more will be available in the future. Keep watching the news and playing with this beta version to get used to.
This post is about a scenario where users were not able to authenticate on a FBA page published by ISA Server using LDAP as authentication repository. The error message that was showing up there was:
Figure 1 – Unable to authenticate.
Although it says to double check the domain name or password to see if it is wrong, this is a generic logon error and this may not be the case. We recently wrote an article at Tales from the Edge that has a troubleshoot framework for LDAPs authentication on ISA. More info check the article at http://technet.microsoft.com/en-us/library/dd316279.aspx.
2. Logging is your Friend
The ISA Server realtime logging can be very helpful in scenarios like this. In this case the error message was the one below:
Figure 2 – Error 58.
As you can see in the figure above the error message says that it was not possible to perform the requested operation. This can be a good start, but you can see even more information if you copy the whole logging to the clipboard using the option below in the task pane:
Figure 3 – Using the Clipboard option.
After copy, paste in a notepad file and save as TXT. Best thing to do is to open this file in Excel to see all the fields and be able to filter. After opening the file in Excel, I was able to see a key error in there:
Figure 4 – Using Excel to filter the logs.
Notice that in the Authentication Server field it says: dccont\No server available. This is it!! Now we can conclude that:
· ISA cannot reach the DC for some reason:
o Networking issue?
o Name resolution issue?
o DC not answering?
Before go crazy and start to investigate this deeper, what about just try to ping the server that are in the LDAP Server Set? This is what I did and the result was:
Figure 5 – unable to resolve the name.
Bingo, unable to resolve the name. After fix the name resolution problem the issue was gone and the authentication worked.
I recently worked in a very interesting case where customer’s Exchange Server got in the SPAM Block list although the environment was clear of malware and no SPAM was originated from that server at all. We ended up identifying why the server got blocked and it was because an external servers was using reverse DNS lookup to verify if the MX record for that email server matches with the source IP address from where the SMTP traffic was coming from. To make it easier to understand, let’s take a look on following diagram for contoso.com network:
Notice that the primary IP bound to ISA’s external interface is using IP 192.168.1.113. The SMTP Publishing rule correctly maps the internal Exchange Server IP but the outbound traffic always will leave with the primary IP of the ISA Server. This means that when the external Exchange Server performs the reverse lookup for the MX record (for example: mail.contoso.com) it will resolve for 192.168.1.60 which doesn’t match with the source IP received in the IP header of the SMTP packet.
The fast resolution here is to change the primary IP to be 192.168.1.60, but sometimes this cannot be done so fast due other policies for example. But….that’s the way it is on ISA Server, not much you can do other than plan to use the primary IP for scenarios like this.
The good thing here is: TMG resolves this problem! How? With a feature called Enhanced NAT (ENAT). Now you can create a network rule to specify which IP address you want to use for outbound traffic as shown below:
Isn’t that nice? It’s amazing for sure!!
Well, we are already in September and TMG is coming very soon…but while is not RTM yet, you still have a chance to download Beta 3 and play with it.
With Windows Server 2008 bring so many cool features such as SSL VPN, many customers are asking some questions about this integration. Here are some common questions and answers about this:
1) Can I install ISA Server 2006 in a Windows Server 2008?
No. TMG will be the first Microsoft Firewall that you can install on Windows Server 2008 system.
2) Can I install ISA Server 2006 in a 64bits System?
No. TMG will allow that.
3) Can I join and ISA Server 2006 to a Windows Server 2008 Domain?
Yes you can. We will update the articles below with that info:
http://technet.microsoft.com/en-us/library/bb794821.aspx
http://technet.microsoft.com/en-us/library/bb794807.aspx
4) Does ISA Server 2006 support SSL VPN?
No, but you can publish SSL VPN through ISA Server. Here it is a great article from Tom Shinder that explains how to do that:
http://www.isaserver.org/tutorials/Publishing-Windows-Server-2008-SSL-VPN-Server-Using-ISA-2006-Firewalls-Part1.html
5) Can I publish Secure FTP using II7 through ISA Server 2006?
Not in a supported manner. FTPS is not supported on ISA, for more information check the official article here: http://technet.microsoft.com/en-us/library/bb794745.aspx
Many customers are thinking in going to a 64 bits system for the amount of advantages that this type of system brings, such as performance improvement. You are not wasting your time waiting for TMG, the amount of features, improvements and robustness that this new Firewall will provide is just AMAZING. Do you want to know a little bit more about those advantages? Read this article from Tom Shinder and you will understand why TMG is the Cornerstone of EBS J
http://blogs.isaserver.org/shinder/2008/09/02/why-the-forefront-tmg-is-a-cornerstone-of-essential-business-server-ebs-network-security/
One of the most painful issues to resolve on ISA Server is when the Firewall Service stops and doesn’t come up again. Many times this happens without a previous warning and most of the times is because ISA is failing to load something or to commit some kind of configuration that was made. In this particular scenario, Firewall administrator claims that he didn’t change anything and believe or not he didn’t. ISA Server was untouchable for months and one day, after installing a security patch on Windows and restart the server, Firewall Service didn’t start.
In situations like that is easy to blame the patch, because the first thing that comes in people’s mind is: well, if it was working and after installing a patch stopped working, it got be the patch. Although this makes sense (logically speaking) it might not be true (technically speaking). This particular case confirmed that: after firewall administrator uninstalled the patch (not really a good security recommendation) the issue persisted.
Let’s see the approach to fix this issue.
2. Starting from the Basics
Start from the simplest thing which is: review the event viewer. In this case here the sequence of events that I found it:
Event Type: Error
Event Source: Microsoft ISA Server Web Proxy
Event Category: None
Event ID: 14127
Date: 8/2/2009
Time: 9:43:36 AM
User: N/A
Computer: ISACONTN1
The Web Proxy filter could not initialize (error code 501.3357.5.0.5723.493).
Time: 9:43:38 AM
The Web Proxy filter could not initialize (error code 505.78.5.0.5723.493).
Event Source: Microsoft Firewall
Event ID: 14060
ISA Server could not load the application filter Web Proxy Filter ({4CB7513E-220E-4C20-815A-B67BAA295FF4}). FilterInit failed with the error code 0x80070006. To attempt to activate this application filter again, stop and restart the Firewall service.
Event ID: 14001
Firewall Service failed to initialize. Previous event log entries might help determine the proper action.
In this case these events are very generic and really don’t say much, but it gives us an idea of the sequence of failures that we have.
3. Going Further
On issues related to Firewall Service not starting, one thing that is very handy is understand what is happening during the time that Firewall Service is starting. Which files is he loading? To better see what is happening I used WinDBG to attach to Firewall Service. I did that on a working system to see the sequence that I have and repeated the same in the system that was broken. Here it is the steps that I used on my working system:
0. On the system that is working I stopped Firewall Service.
1. Open WinDBG (if you don’t have, download it here).
2. Started Firewall Service, open WinDBG, click on File / Attach to a Process, choose the wspsrv.exe process as shown below and click OK.
Figure 1 – Attaching WiDBG to Firewall Service Process.
3. In the command window type g and press ENTER. The g command will starts executing this process and wait for a manual Break or will break for an external cause (if the process quits for example).
4. On my working system the following sequence appear:
(e94.1e8): Unknown exception - code 000006d9 (first chance)
ModLoad: 0c8e0000 0c909000 C:\Program Files\Microsoft ISA Server\authdflt.dll
ModLoad: 60290000 602f5000 C:\Program Files\Microsoft ISA Server\CookieAuthFilter.dll
ModLoad: 0c9b0000 0c9ef000 C:\Program Files\Microsoft ISA Server\ACECLNT.dll
ModLoad: 67de0000 67e05000 C:\Program Files\Microsoft ISA Server\sdmsg.dll
ModLoad: 71ca0000 71cf8000 C:\WINDOWS\system32\kerberos.dll
ModLoad: 766e0000 766ec000 C:\WINDOWS\system32\cryptdll.dll
ModLoad: 635e0000 635f7000 C:\Program Files\Microsoft ISA Server\radiusauth.dll
ModLoad: 0ea10000 0ea2d000 C:\Program Files\Microsoft ISA Server\ldapfilter.dll
ModLoad: 61470000 614b1000 C:\Program Files\Microsoft ISA Server\LinkTranslation.dll
ModLoad: 60fe0000 61008000 C:\Program Files\Microsoft ISA Server\HttpFilter.dll
ModLoad: 72e50000 72f6a000 C:\WINDOWS\system32\msxml3.dll
ModLoad: 0f480000 0f493000 C:\Program Files\Microsoft ISA Server\complp.dll
ModLoad: 68100000 68124000 C:\WINDOWS\system32\dssenh.dll
(e94.998): Unknown exception - code 000006d9 (first chance)
ModLoad: 633b0000 633c2000 C:\Program Files\Microsoft ISA Server\pptpfltr.dll
ModLoad: 60780000 60795000 C:\Program Files\Microsoft ISA Server\ftpfltr.dll
ModLoad: 641c0000 641de000 C:\Program Files\Microsoft ISA Server\StrmFltr.dll
ModLoad: 61350000 61363000 C:\Program Files\Microsoft ISA Server\issfltr.dll
ModLoad: 60ae0000 60b16000 C:\Program Files\Microsoft ISA Server\h323fltr.dll
ModLoad: 609b0000 609e5000 C:\Program Files\Microsoft ISA Server\h323asn1.dll
I repeated the same sequence on the non working system and WinDBG stopped in the following module:
(c38.1b8): Unknown exception - code 000006d9 (first chance)
ModLoad: 71bd0000 71be1000 C:\WINDOWS\system32\mpr.dll
ModLoad: 0eb90000 0eb9f000 C:\Program Files\Common Files\System\Ole DB\SQLOLEDB.RLL
eax=00000000 ebx=00000000 ecx=0006fdcc edx=00000000 esi=7c822028 edi=00000000
eip=7c82ed54 esp=0006fe18 ebp=0006ff0c iopl=0 nv up ei pl zr na pe nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000246
ntdll!KiFastSystemCallRet:
7c82ed54 c3 ret
Missing image name, possible paged-out or corrupt data. ç This happened because the wspsrv.exe process quit since it was not able to start.
Notice that on my working system I do not load this SQLOLEDB.RLL module, which immediately makes me think: what component ISA uses to communicate with SQL (if need)? Answer: Logging. Bingo !!! That was it; my system was using file text for logging while the non working system was using SQL.
4. Wait a minute, how this was working before?
Good question!! After identify that the issue was on the connectivity with SQL we engaged the database administrator that revealed his fault. He performed a migration of the hardware where SQL was located to a new hardware, restored the configuration but failed to give appropriate permissions to the ISA Server computer account. He fixed the issue using KB 838710, in particular the section called “How to set up SQL Server to accept the Open Database Connectivity (ODBC) from the ISA Server or from Microsoft Forefront Threat Management Gateway, Medium Business Edition”, step 7.
The new URL Filtering option on Forefront TMG 2010 allows you to manually add web sites to a specific category; such feature is called URL Category Override. This can be a good approach when you want to force a specific site to be categorized in such way that it fits into a category that currently you have on your block rule. This post will describe a scenario where the TMG Administrator added a web site to the “dubious” category as shown below:
The goal was to block access to this web site due company security policy. To test if this configuration was working fine, TMG administrator used the Category Query feature and there it was possible to see that the new categorization was working fine as shown below:
When the client was trying to access this URL from his workstation he was getting the expected error message.
2. The Problem
The problem on this case is that users figured out a way to bypass this by typing https:// in front of the URL, in order words if they type https://www.facebook.com they were able to access the web site. You might be thinking: how is that possible? Well, that was my question when I first heard the TMG Administrator explaining his problem to me, but then after reviewing the environment and client configuration it was possible to understand why such behavior was happening.
The problem is that client workstation was configured as Secure NAT client, no web proxy configuration. You need to remember that URL Filtering doesn’t do HTTPS categorization for Secure NAT requests, therefore such behavior was expected. On the article that me, Jim and Mohit wrote for TechNet Magazine (March 2010 issue) we say:
“…the ability of URL filtering to evaluate the request is dependent on two criteria:
- Is the connection directed to the default HTTP port? If so, the Web proxy may be able to intercept this request and pass it to URL filtering for comparison. If not, the request will not be seen by URL filtering and thus cannot be compared to the database.
- If the connection is directed at the default HTTPS port, is HTTPS inspection enabled? If so, HTTPS inspection can bridge the connection, and URL filtering will have an opportunity to compare the request to the database.”
Based on that you can imagine how to fix this problem, correct? Let’s take a look on the options that we have here.
3. The Solution
In scenarios like this there are a couple of solutions:
- Enable HTTPS Inspection: with HTTPS Inspection enabled, it will be possible to enforce the URL Filtering for requests that use HTTPS and are coming from SecureNAT clients.- Use Web Proxy Client: by using web proxy client, URL Filtering will work regardless of the protocol.
For this particular scenario the administrator preferred to use Web Proxy Client and deploy a GPO to force all IE users to go out to the Internet using this particular TMG. For that the following AD policies were used:
Policy 1 – Used to force the proxy server setting:
Policy 2 – Used to disallow users to change their proxy settings
It is always important to analyze all the possible options and which one will best fit on your environment. Sometimes concentrate all the policy enforcement on the edge it is good, however there are many times on which you will need to make sure that your infrastructure as a whole is enforcing the company security policy. By leveraging Windows security capabilities to enforce policies you can facilitate the overall administration overhead and have multiple layers of policy enforcement in place.
Sometimes I receive questions like: I don’t want that user’s use the application XYZ to grab content on the Internet. How can TMG block this application on my Web Proxy Client? This is a classical question and it can be done on TMG if you have TMG Client installed, but if this is just a web proxy client, then the approach should be different. It comes back to the subject of enforcing company’s security policy end to end. Ask yourself the questions below and you will realize that there are much more to be concern about:
- Why this client is running a non approved application on company’s desktop in first place?- Why not use software restriction policy via GPO for the company workstations? - Even if you block on the edge, who guarantees that this non approved application is not trying to harm other internal clients?
As you can see there are many questions that need to be answered on this area before try to fix a particular non compliance concern by solely use a fix on the edge.
The years working on Platforms and Networking were essential for me to build the foundation prior to migrate to the Security area. Regardless of the technology that I worked, most of the time I was dealing with situations where the customer wants to be secure while allowing access to the core network resources.
The scenario that I’m going to describe on this post was related to a really trick situation where only six users were having authentication problem on the 802.1x Wireless Network. Here the topology that was used on that case:
Figure 1 – Wireless Network.
As you can see, we have an unsupported operating system (Windows NT) on the environment which shows how heterogeneous the scenario was. On the infrastructure side, customer was using a Cisco and the servers were using Cisco Secure ACS for Windows. Those six users belong to the Windows Server 2003 Domain and all of them were located on the same OU. On that same OU we also had users that were not having problem at all.
To enhance the security on the network customer was using the 802.1x technology for wireless network. This technology comes originally from the 802.1x for wired network. The IEEE 802.1x is used to guarantee authentication on the link level (layer 2). This way the switch port where the computer is connected will stay in blocking state until the authentication is successfully completed.
For more information on 802.1x network review the article Understanding 802.1X authentication for wireless networks.
2. Collecting Data
Since we just have six users that were having problem, the first step was to use the LDIFDE command to dump the user’s properties. The idea behind this is to dump the properties of a user that works and the one that doesn’t work. Then compare the attributes and values. To do that the following command was used:
ldifde -f C:\User.ldf -d "DN" -p base
The DN (Distinguished Name) is the location of the user on the Active Directory. To find out this attribute you can use the ADSIEdit Tool.
After compare both accounts I couldn’t find any problem on the attributes and values that could lead to an authentication issue. The next step was to start a netmon capture and see what was going in the wire. The following steps were done:
· Installed Netmon on the Windows XP (Client), on the Windows Server 2003 DC and on the Windows 2000 Member Server.
· Enabled the RRAS Logging using the command netsh ras set tracing * enable. The logs will be added to the %systemroot%\tracing folder.
· Configured the switch that was connected to the access point to do a port mirror for one workstation.
· The CTEST\Bob user performed the logon on the Windows XP workstation.
Note: The same steps were done for a user that could successfully authenticate.
3. Analyzing
Here the traffic for a user that could successfully logon:
1. CiscoAP WindowsXPMAC EAP Request, Identity
2. WindowsXPMAC CiscoAP EAP Response, Identity
3. CiscoAP WindowsXPMAC EAP Request, PEAP
4. WindowsXPMAC CiscoAP TLS Client Hello
5. CiscoAP WindowsXPMAC TLS Server Hello
6. WindowsXPMAC CiscoAP TLS Change Cipher Spec
7. CiscoAP WindowsXPMAC EAP Success
8. CiscoAP WindowsXPMAC TLS Application Data
Opening the EAP header (frame 2) it is possible to verify that the user sends the credential:
802.1X Authentication
Version: 1
Type: EAP Packet (0)
Length: 22
Extensible Authentication Protocol
Code: Response (2)
Id: 1
Type: Identity [RFC3748] (1)
Identity (17 bytes): CTEST\Will
On the EAP header (frame 7) we have the successfully negotiation message:
Length: 4
Code: Success (3)
Id: 233
Now, let’s see the traffic for a user that was not working:
7. CiscoAP WindowsXPMAC EAP Request, Identity
8. WindowsXPMAC CiscoAP EAP Response, Identity
9. CiscoAP WindowsXPMAC EAP Request, PEAP
10. WindowsXPMAC CiscoAP TLS Client Hello
11. CiscoAP WindowsXPMAC TLS Server Hello
Clearly we can see that on this process we have multiple logon attempts without success. The interesting part of that was the customer’s revelation when he told me: if I don’t cancel this process the user account gets block on AD due the multiple bad logon attempts.
That was key information, this pretty much means that our package was going all the way up to the DC and trying to authenticate. However, for some reason that we didn’t know yet, it was failing.
4. Going further
Without a doubt netmon trace is something that helps a lot to understand the traffic. But, on this case we need something else to help us understand why was failing. Since we enabled the debug logs on the Windows XP we had the data that we need to figure that out. Looking to the files Wzctrace.log, Eapol.log, Netman.log and RASTLS.LOG located on the folder %systemroot%\tracing it was possible to determine that.
Let’s see the difference for a user that could logon for the one that could not logon in the file RASTLS.LOG:
- Successful logon:
[1244] 17:04:57:801: EapTlsBegin(CTEST\Will)
[1244] 17:04:57:801: State change to Initial
[1244] 17:04:57:801: EapTlsBegin: Detected 8021X authentication
[1244] 17:04:57:801: EapTlsBegin: Detected PEAP authentication
[1244] 17:04:57:801: MaxTLSMessageLength is now 16384
[1244] 17:04:57:801: EapPeapBegin done
[1244] 17:04:57:801: EapPeapMakeMessage
[1244] 17:04:57:801: EapPeapCMakeMessage
[1244] 17:04:57:801: PEAP:PEAP_STATE_INITIAL
[1244] 17:04:57:801: EapTlsCMakeMessage
[1244] 17:04:57:801: EapTlsReset
[1244] 17:04:57:801: GetCredentials
[1244] 17:04:57:801: Flag is Client and Store is Current User
[1244] 17:04:57:801: GetCachedCredentials
[1244] 17:04:57:801: PEAP GetCachedCredentials: Using cached credentials.
[1244] 17:04:57:801: MakeReplyMessage
[1244] 17:04:57:801: SecurityContextFunction
[1244] 17:04:57:801: InitializeSecurityContext returned 0x90312
[1244] 17:04:57:801: State change to SentHello
- Unsuccessful logon:
[2688] 16:58:02:568: EapTlsBegin(CTEST\Bob)
[2688] 16:58:02:568: State change to Initial
[2688] 16:58:02:568: EapTlsBegin: Detected 8021X authentication
[2688] 16:58:02:568: EapTlsBegin: Detected PEAP authentication
[2688] 16:58:02:568: MaxTLSMessageLength is now 16384
[2688] 16:58:02:568: EapPeapBegin done
[2688] 16:58:02:568: EapPeapMakeMessage
[2688] 16:58:02:568: EapPeapCMakeMessage
[2688] 16:58:02:568: PEAP:PEAP_STATE_INITIAL
[2688] 16:58:02:568: EapTlsCMakeMessage
[2688] 16:58:02:568: EapTlsReset
[2688] 16:58:02:568: GetCredentials
[2688] 16:58:02:568: Flag is Client and Store is Current User
[2688] 16:58:02:568: GetCachedCredentials
[2688] 16:58:02:568: FreeCachedCredentials
[2688] 16:58:02:568: No Cert Store. Guest Access requested
[2688] 16:58:02:568: No Cert Name. Guest access requested
[2688] 16:58:02:568: Will NOT validate server cert
[2688] 16:58:02:568: MakeReplyMessage
[2688] 16:58:02:568: SecurityContextFunction
[2688] 16:58:02:568: InitializeSecurityContext returned 0x90312
[2688] 16:58:02:568: State change to SentHello
[2688] 16:58:02:568: BuildPacket
Notice that the user is authenticating as Guest since there was no certificate available.
Since customer was not using user certificate to gain access to the system and using computer certificate only we could change the original behavior via registry change. What we did was to force each laptop that those users were using to use computer authentication only. The following registry key was changed:
HKEY_LOCAL_MACHINE\Software\Microsoft\EAPOL\Parameters\General\Global
Tipo: REGDWORD
Nome: AuthMode
Valor "2"
The number 2 means: Computer authentication is performed when the wireless client computer is started. User authentication is never performed. For more information on that registry key review the article Wireless LAN Support in Windows: Frequently Asked Questions.
On those laptops this key was configured to 1, therefore the user authentication was happening and the user’s certificate on those laptops was corrupted.
TMG installation problems can be a bit trick to troubleshooting if you don’t know which components are involved, however if you know then things start make more sense. Most of the setup problems that I faced up to now on TMG 2010 (since RTM) were related to ADLDS or some kind of domain connectivity problem. The most two recent examples are described on two articles that I tech reviewed from my friends Bala Natarajan and Niladri Dasgupta wrote:
Last week I worked on an issue where TMG admin was not able to install this brand new TMG to be used as Edge Firewall. The error message that he was receiving was:
After this error the setup process rolled back and finish without completing the installation. As recommended on the previous two articles mentioned in this post, the first step is to review the setup logs and look for more information in order to move the troubleshooting further. In the ADAM Log file we can see the following entry:
When you see an error where trust relationship between client and domain is failing, be sure to do your homework, in other words, check:
When I hit the third test I found out the problem:
This was the problem, because Windows (where I was trying to install TMG) was sending the traffic to the wrong interface. Once we moved the Internal to the top, flushdns (with ipconfig) and ran the setup again the issue went away and the installation finished successfully.
Note: same recommendation to have Internal on the top applies to UAG, check it out a great reference on that written by Jason Jones at http://blog.msedge.org.uk/2010/04/recommended-network-card-configuration_14.html
Happy New Year everybody!
I hope you enjoyed your new years eve because now you might want to take a look on this worm that is causing lots of headaches to all IT Admins. MMPC (Microsoft Malware Protection Center) has a report about this malware and how to proceed to avoid infestation:
http://www.microsoft.com/security/portal/Entry.aspx?Name=Worm%3aWin32%2fConficker.B
Good news is that ISA Server and TMG can block outbound requests for this worm and yesterday night (before midnight) our IR (Incident Response) Team in partnership with ISA Server Team brought together an action plan to allow ISA/TMG to block that. Jim Harrison automated this process by creating a script that you can use to create policies to block conficker and you can download from here:
http://jim.isatools.org/tools/block_conficker.vbs
Enjoy your day off and be sure to implement those actions ASAP.
Today I was assisting a friend of mine here from TMG team that was facing this issue, same issue that was also mentioned on this thread. The problem was happening when using Cryptography Next Generation (CNG) or also called V3, TMG was not recognizing the private key and was showing up this error message. This is a known issue because TMG (and ISA) don’t support CNG (V3 Certificates). This is well documented under the unsupported documentation here:
Issue: Forefront TMG does not support the use of certificates created using CNG (Certificate New Generation) based templates for Web listeners or as client certificate authentication in Web publishing or Web chaining rules.
Cause: CNG certificates are not usable by Forefront TMG.
Workaround: Create certificates using Windows 2000 or Windows 2003 templates.
From: http://technet.microsoft.com/en-us/library/ee796231.aspx#dfg9o9i8uuy6tre
Again, make sure to read this unsupported document before deploy TMG, there you will find the official statement from TMG Product Team about what it is supposed to work and what it is not.
Note: Important to emphasize that CNG V3 is not X.509 V3. CNG V3 refers to the new V3 Certificate Template on 2008 while X.509 V3 is the current certificate standard in which TMG is fully compatible.
This week I’m delivering an ISA Server 2006 Workshop to Microsoft Premier Customers. While the training is going really well one thing that I notice was the following common question from students in the class: why sometimes I can’t access a website while I’m behind ISA and if I bypass ISA it works?
This is a really broad question and we need data to better understand what is causing such behavior. I’m assuming that all firewall policies are correctly configured and that you have no issues on that side. If this assumption is correct then the next question is: what it is the error message do you receive when tries to access the page behind ISA?
Situations like that needs precision on those answers and data gathering is essential for the success of the case. To give them an example that sometimes ISA just does what is suppose to do, I showed them the following scenario:
Client is trying to access a website through ISA Sever 2006 and logging shows the error below:
Error Code 64: Host not available
Error 64 is generic and I agree that doesn’t help at all. For this reason you need to dig in to find out what is going on otherwise it will be hard to determine the root cause. For this scenario a simple netmon trace helped us to determine the root cause:
- ISA Sends the request to the destination server:
Via: 1.1 SRV
UserAgent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.04506.30; .NET CLR 3.0.04506.648; MS-RTC LM 8)
Host: www.contoso.com
Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/x-shockwave-flash, application/xaml+xml, application/vnd.ms-xpsdocument, application/x-ms-xbap, application/x-ms-application, */*
The destination server responds:
- Http: Response WebSite:
_BuildHTTPConversation:
- Response: 0x1
ProtocolVersion: HTTP/1.0
StatusCode: 302, Moved temporarily
Server: XXXXX
Location: https://srv.contoso.com
Looking closely to the binary details in netmon hex details pane we have the root cause for that:
48 54 54 50 2F 31 2E 30 20 33 30 32 20 4F 4B 20 28 43 6C 75 73 74 65 72 20 42 6F 6D 62 20 62 79 20 53 68 69 66 74 34 29 0D 0A 53 65 72 76 65 72 3A 20 41 70 61 63 68 65 2F 31 2E 33 2E 32 36 20 28 55 6E 69 78 29 0D 0A 4C 6F 63 61 74 69 6F 6E 3A 20 68 74 74 70 73 3A 2F 2F 73 65 72 76 65 72 31 31 2E 64 6F 6C 6C 61 72 73 6F 6E 74 68 65 6E 65 74 2E 6E 65 74 2F
The destination server does not terminate the response in accordance with RFC-2616. This RFC says:
“.. HTTP/1.1 defines the sequence CR LF as the end-of-line marker for all protocol elements except the entity-body”
From: http://www.ietf.org/rfc/rfc2616.txt
Therefore ISA correctly rejects it as malformed. What we should see in a normal HTTP Response is:
Conclusion: bypassing ISA as an attempt to proof that the issue is on ISA sometimes doesn’t prove the real point. ISA Server inspects the packet and act according to RFC for that protocol. If the destination server is not in accordance with that protocol ISA will correctly drop the packet as malformed.
They were surprise with the result and confident that from now on, they will research before blame ISA J.
One of the most challenges for the ISA Admin is to determine the culprit for an intermittent issue. This gets worse when the issue is related with performance. While there are many elements that can impact ISA Server’s performance, this post will describe an interesting case where the client was having problems to browse Internet through ISA Server. The web sites were coming up really slow and regardless of the browser (IE6 or IE7) the issue was happening.
2. Start from the Basics
Do not over estimate the basics, missing a basic check can cost you hours of deep troubleshooting while the monkey is right there looking for you. Here it is a checklist of things that should be reviewed:
· DNS Configuration: This can dramatically impact the performance if it is not correctly configured. Tom Shinder wrote a good post about an issue that he got because of the DNS that was not answering in time manner. Besides that, remember to review Microsoft recommendation on DNS configuration for ISA Server.
· RSS, Chimney and TCPCA: if you have Windows Server 2003 SP2 installed make sure to use KB936594 to address the issues that those keys can cause.
· Network Configuration: on ISA Server, make sure that the Network range is correctly defined. Review the article Troubleshooting Network Configuration in ISA Server.
· NIC Drivers: are the NIC drivers updated for the latest version?
· Speed and Duplex: autosense configuration between the Server and the Switch can also cause network intermittence behavior. Better force the speed (100 Mbps or 1Gbps) and duplex (full) in the switch port and in the NIC driver.
These are things that you can start looking in first hand. If all those elements look good than is time to move on and get more data.
3. The Output
After reviewing the netmon trace that was get it while the issue was happening the result was clear: did not have delay in the name resolution as well as the initial TCP handshake. The network communication was clear and there were not big gaps in between frames. The only thing that we notice was that it was taking too much time to transfer data to build the page. Something in the application level was not really going as good as it should.
The output from perfmon (using the counters from the article Monitoring and Troubleshooting Performance also did not show any suspicious activity. There was no leak in the server itself; processor utilization was good as well as memory and disk.
3. User Mode Dump
The next troubleshooting level was really to get a dump from wspsrv.exe and understand what that guy was doing during the page request. To do that we used the command below while repro the issue:
Cscript adplus.vbs -quiet -hang -pn wspsrv.exe
Note: for more information on how to use ADPLUS use the KB286350.
General Considerations:
· Install WinDbg that is part of the debugging tools.
· Configure the Symbol path using KB311503.
After load the dump in WinDBG we ran the command runaway to show the time that each thread is consuming:
0:000> !runaway
User Mode Time
Thread Time
21:f80 0 days 0:00:20.437
8:eb4 0 days 0:00:11.375
19:f78 0 days 0:00:02.953
25:fb8 0 days 0:00:02.656
42:10e8 0 days 0:00:01.859
45:152c 0 days 0:00:01.796
43:1318 0 days 0:00:01.718
35:1664 0 days 0:00:01.703
44:1534 0 days 0:00:01.562
…
The thread 21 is the one that seems to be using more resources. To see what this thread is doing run ~21kb, for this case the result was:
0:000> ~21kv
ChildEBP RetAddr Args to Child
0220f3a8 7c827d0b 7c83d236 000079e8 00000000 ntdll!KiFastSystemCallRet (FPO: [0,0,0])
0220f3ac 7c83d236 000079e8 00000000 00000000 ntdll!NtWaitForSingleObject+0xc (FPO: [3,0,0])
0220f3e8 7c83d281 000079e8 00000004 646f30a0 ntdll!RtlpWaitOnCriticalSection+0x1a3 (FPO: [Non-Fpo])
*** ERROR: Symbol file could not be found. Defaulted to export symbols for W3Filter.dll -
0220f408 64709542 1adf1eec 1adf1eec 6470bead ntdll!RtlEnterCriticalSection+0xa8 (FPO: [Non-Fpo])
*** ERROR: Symbol file could not be found. Defaulted to export symbols for msfpc.dll -
WARNING: Stack unwind information not available. Following frames may be wrong.
0220f44c 615e7407 00000000 00000000 0220f470 W3Filter!CNetAddr::operator=+0x15e
0220f47c 615eb41d 202f3c28 0020f4a8 646f3080 msfpc!HmacMD5CreateKey+0x3944
0220f4ac 6470a167 1ad91218 202f3c28 0220f4d0 msfpc!TsLogInformationA+0x1b7
0220f4d4 6471bd0b 202f3c28 202f3558 202f3508 W3Filter!CNetAddr::operator=+0xd83
0220f508 647670c4 04e74008 00000000 00002faa W3Filter!DllUnregisterServer+0x1b4e
0220f520 6472676c 00002faa 64702748 00000002 W3Filter!DllUnregisterServer+0x4cf07
0220fdb0 647773ad 00000000 00000000 0000000a W3Filter!DllUnregisterServer+0xc5af
*** ERROR: Symbol file could not be found. Defaulted to export symbols for wspsrv.exe
0220fdf4 004c37d4 18b21558 00000001 00000000 W3Filter!DllUnregisterServer+0x5d1f0
0220fe3c 0046982d 00000001 00000001 00000000 wspsrv!IsNameInRwsConfigsLdt+0xb2af
0220fe7c 0046a570 00000001 00000000 20320c78 wspsrv+0x6982d
0220fee0 0046a8e0 00000109 00000000 00000001 wspsrv+0x6a570
0220ff04 0046a9e7 1d6ac2a8 0046a856 0220ff30 wspsrv+0x6a8e0
0220ff14 0046ab8b 00000109 00000000 00000001 wspsrv+0x6a9e7
0220ff30 00469939 00000109 00000000 00000001 wspsrv+0x6ab8b
0220ff50 00452a27 1d6ac34c 00000109 00000000 wspsrv+0x69939
0220ff7c 004536a4 1d6ac34c 00000109 00000000 wspsrv+0x52a27
Note: I’m using the public symbol server (SRV*c:\symbols*http://msdl.microsoft.com/download/symbols) which doesn’t have too much info about ISA Server symbols. For this reason I’m receiving the error showed above.
Let’s look at the critical section information:
0:000> !cs -s -l -o
-----------------------------------------
DebugInfo = 0x000cc228
Critical section = 0x000d0e74 (+0xD0E74)
LOCKED
LockCount = 0x6
WaiterWoken = No
OwningThread = 0x00000f74
RecursionCount = 0x1
LockSemaphore = 0x7398
SpinCount = 0x00000000
OwningThread DbgId = ~18s
OwningThread Stack =
0214fd94 7c827d0b 77e61d1e 00000d38 00000000 ntdll!KiFastSystemCallRet
0214fd98 77e61d1e 00000d38 00000000 00000000 ntdll!NtWaitForSingleObject+0xc
0214fe08 77e61c8d 00000d38 ffffffff 00000000 kernel32!WaitForSingleObjectEx+0xac
0214fe1c 0043b9fd 00000d38 ffffffff 000d0e08 kernel32!WaitForSingleObject+0x12
0214fe6c 615e7407 00000000 00000000 0214fe90 wspsrv+0x3b9fd
0214fe9c 615eb41d 1dbea130 0014fec8 0041e4f8 msfpc!HmacMD5CreateKey+0x3944
0214fecc 004d3686 000abb98 1dbea130 0214fef4 msfpc!TsLogInformationA+0x1b7
0214ff04 0042beec 1dbea130 1dbea130 1dbea108 wspsrv!GetNetworkVIPAddress+0x66e5
0214ff20 00435549 00000000 00000001 1dbeaa94 wspsrv+0x2beec
0214ff50 0042b850 1dbeab3c 0214ff74 00430f1c wspsrv+0x35549
0214ff5c 00430f1c 00000000 6370a830 1dbeaa94 wspsrv+0x2b850
0214ff74 63705056 00000000 00000000 00409960 wspsrv+0x30f1c
0214ff94 00453616 00000001 00000000 00000000 ratlib!RatPollTimer+0x1a9
0214ffb8 77e64829 00000001 00000000 00000000 wspsrv+0x53616
0214ffec 00000000 00453576 00000001 00000000 kernel32!BaseThreadStart+0x34
DebugInfo = 0x000a8890
Critical section = 0x000d0e38 (+0xD0E38)
LockCount = 0x0
OwningThread = 0x000006dc
LockSemaphore = 0x0
OwningThread DbgId = ~37s
2294f96c 7c827d0b 71b21af5 0000890c 00000001 ntdll!KiFastSystemCallRet
2294f970 71b21af5 0000890c 00000001 2294f998 ntdll!NtWaitForSingleObject+0xc
2294f9ac 71b2c517 0000890c 00008fc4 00000000 mswsock!SockWaitForSingleObject+0x19d
2294fa24 71c094e5 00008fc4 2294fa84 00000001 mswsock!WSPRecv+0x203 (FPO: [Non-Fpo])
2294fa60 71bb1151 00008fc4 2294fa84 00000001 ws2_32!WSARecv+0x77 (FPO: [Non-Fpo])
2294fa8c 6d561686 00008fc4 00000000 00001000 wsock32!recv+0x31 (FPO: [Non-Fpo])
2294fadc 4e2597ce 00007530 012bdd40 012bdd40 dbnetlib!ConnectionRead+0x3b6 (FPO: [Non-Fpo])
2294fb10 4e25982d 012b0450 012bdd40 00000009 sqloledb!CDataSource::ConnectionRead+0x35
2294fb5c 4e252358 01710cc6 00000001 00000000 sqloledb!CDBConnection::GetBytes+0x269
2294fba8 4e2555c4 01158560 00000088 0000001e sqloledb!CDBConnection::ProcessTDSStream+0x157
2294fc64 4e255691 01155e78 00000049 01157688 sqloledb!CStmt::ExecDirect+0x786
2294fc7c 4e254d32 01155e78 00000049 00000000 sqloledb!CStmt::SQLExecDirect+0x28
2294fcac 4e25517d 00000000 4e25321c 00000049 sqloledb!CCommand::ExecuteHelper+0x157
2294fd30 4e254c4b 01157688 00000000 615d30b0 sqloledb!CCommand::Execute+0x76b
2294fd68 6160de22 0114dd78 00000000 615d30b0 sqloledb!CImpICommandText::Execute+0xdd
2294fd98 6160e2a1 00000000 00000000 2294fdfc msfpc!CFastSession::Insert+0xe2
2294fdb4 6160e776 201f8bf0 2294fe14 2294fdfc msfpc!SessionInfo::~SessionInfo+0x4f
2294fee0 6160e9f5 201f8bf0 0115a278 615d1a14 msfpc!OpenFastLoadRowset+0x4cb
2294ff08 6160eaa4 00000000 000d0010 6160ecd1 msfpc!CFastSession::Commit+0x29
2294ff20 0050a069 00000001 004d78e6 000d0010 msfpc!CFastSession::~CFastSession+0x18
The red line in the second stack shows that the machine is submitting a SQL statement using the SQLExecDirect function. Now let’s see what SQL command is being executed:
0:035> du 01155e78
01155e78 "EXEC sp_batch_insert [##Firewall"
01155eb8 "Log000000391SDCTSI00FW02], [Fir"
01155ef8 "ewallLog]"
Ok, now things start to make sense. The problem was happening during that time and the ISA Server Firewall Service was writing to the log file (Firewall Log) which was located in a SQL Server database.
4. Logging was the Problem
The browsing performance issue for this case was caused by a performance problem in the SQL Server. The SQL Server that ISA Server was using for logging had disk I/O problems and therefore it was very slow to answer to network requests. The workaround for that while customer fixed his SQL issue was to change the logging type for TXT in the local machine.
The browsing experience improved tremendous and this is actually expected. You can review the ISA Server Logging Best Practices and you will see that TXT log format has the best performance in comparison with the other log types.
5. Additional Reading - Learning More about Debugging
Here are good references for learning more about debugging:
Books
· Advanced Windows Debugging by Hewardt and Pravat
o This book has two great authors from Microsoft and it also foreword by Mark Russinovich, so you know that it is really worth to have it.
· Memory Dump Analysis Anthology, Volume 1 by Dmitry Vostokov
o This book is very good to startup on debugging. It really starts from the basics and it keeps going more in depth. Easy to read and very didactic.
· Windows Internals by Mark Russinovich (Author), David A. Solomon (Author)
o This should be a book that every IT professional should have on his desk if he really wants to know how Windows Architecture works. Just amazing.
Online Resources
· One of the best sites that I found about dump analyses: http://www.dumpanalysis.org/blog/ maintained by Dmitry Vostokov
· There are two really good blogs from two personal friends from Microsoft:
o http://blogs.msdn.com/debuggingtoolbox/ from Roberto Farah (Sharepoint PFE).
o http://blogs.technet.com/marcelofartura/ from Marcelo Fartura (IIS PFE).
· Very good in depth online resource: http://blogs.msdn.com/tess
· Microsoft Advanced Windows Debugging and Troubleshooting: http://blogs.msdn.com/ntdebugging
Once again, ISA Server was only a victim of an environmental issue J !!
Last Tuesday night I was helping out a friend from my team that was handling a case where customer was unable to access Outlook Anywhere from outside network. As usual, everything works inside, so who’s to blame? Of course ISA Server, it is the only thing different, right? Will see…
To better isolate the problem we eliminated tests using Outlook Client and just tried to access the RPC URL using IE (example: https://mail.contoso.com:443/rpc/rpcproxy.dll) and the result was the error below:
Figure 1 – host not available error.
We used Fiddler and we got an interesting result, see below:
Figure 2 – Fiddler result.
Since this is a real traffic I’m hiding some of the legitimate URLs, but the point in the different colors are:
Color
Meaning
Expected traffic using the external URL (for example mail.contoso.com)
Non expected traffic using internal URL (for example mail.contoso.local)
What this means? This means that ISA is for same reason losing the host name during this conversation, which is exactly what error 64 means: "The specified network name is no longer available", which is a win32 error originally called ERROR_NETNAME_DELETED.
At that time the question was: who is changing this name and sending it to ISA? Since the answer was not on our side (we saw on netmon trace that CAS was doing that) we collaborated with an Engineer from the Exchange Team that after some other troubleshooting steps fixed the issue by using the following article:
http://blogs.technet.com/asksbs/archive/2008/12/10/intermittent-outlook-anywhere-connectivity-in-sbs-2008.aspx
Note: the error 400 mentioned in the above article is the same as the one that we received from the CAS server (by looking the netmon trace).
Very interesting case where “again” everything works internally but doesn’t work externally. But again we proved the point that ISA was not causing this issue with a very useful help (as usual) from the Exchange folks.
Last September 16th David Cross (Microsoft Product Unit Manager) announced in the ISA Server Team blog the new Microsoft Firewall called Forefront Threat Management Gateway (TMG) and the first version release to the market called TMG Medium Business Edition (MBE).
There is already a good amount of documentation about this new product in the Forefront TMG TechNet Library. One documentation that it is really important there is about the new integration that you have with TMG and NAP for VPN Client Access. If you are planning to have TMG and also planning secure VPN access, take a look at this documentation here:
http://technet.microsoft.com/en-us/library/cc441515.aspx
This week we launched ISA BPA V7 and this is a great opportunity to continue the explanation on how ISABPA can be useful for proactive and reactive work. Last session I explained how you can use ISA BPA for proactive work with ISA Server. This session will explain the benefits of using ISABPA while troubleshooting an issue.
2. ISA Data Packager
Besides the ISABPA itself, when you install this tool a group of programs is created within Microsoft ISA Server group as you can see in Figure 1:
Figure 1 – Tools that are installed by ISA BPA.
ISA Data Packager is a data gathering tool that can assist you to collect a set of data in one single shot. Let’s use as an example a scenario where user can’t access certain web sites. You can launch the ISA Data Packager and the first screen will present you the templates that are available:
Figure 2 – ISA Templates.
The template that you will choose will depend on the scenario that you are dealing with; here are some examples of usability of the main templates:
Scenario
Template
Unable to access Internet
Web Proxy and Web Publishing
Unable to access some parts of the web site
Get prompt for authentication when accessing a web site
OWA Exchange Publishing rule not working
Sharepoint Publishing rule no working
Unable to establish a VPN Site to Site
VPN
Unable to connect from a client to ISA using PPTP or L2TP
When I open my ISA Console I receive a 0x800 Error and nothing shows up
ISA Administration
ISA Console is crashing
Firewall Policy doesn’t show the rules
Monitoring / Configuration shows one node is not out of Sync
Configuration Storage Server
When accessing Internet Firewall Client turns red.
Firewall Client (NEW in ISABPA7)
Collect data from ISA Server to review the configuration later (no issue to reproduce)
Basic Repro and Static Configuration
Next step is to choose the template according to the scenario, for this example I’m going to chose Web Proxy and Web Publishing. After select and click Next you will see the following screen:
Figure 3 – Summary of the default selections
A set of options are selected by default when you choose the template, those options will vary according to the template that was previously selected. Notice that ISAInfo Report is not selected, which is something that is very useful since with this information you will be able to review all the details from this particular ISA box. In case you want to add that in your data collection you just need to click Modify Options and the following screen will appear:
Figure 4 – Changing default Options.
Here are some other guidelines about this screen:
· If you are having issues such as prompt for authentication when browsing internet, or ISA Server losing the secure channel with the DC, make sure to enable the option Netlogon Logging.
· If you are not dealing with Performance issue, disable the option Performance Monitor Snapshot.
· If you are using MSDE Database for logging and you want to collect data from it, select MSDE Error Logs.
· Change Tracking is NEW in ISABPA7.
After making the selection, click Start Data Collection and wait until the option press space bar to continue appears as shown in Figure 5:
Figure 5 – Starting capture.
At this point you should go to the workstation that is facing the problem and reproduce the issue that you are having. After reproduce the issue press space bar again in the collecting data window and wait until the CAB is generated.
3. Now What?
ISA Data Packager Creates a file called ISAPackage.CAB, by default located in the desktop. This file contains the following folder / files:
Folder
File(s)
Description
BpaDataPackagerLogFiles
BpaDataPackagerLogFile.txt
This is the ISA Data Packager Log that has information about the moment of the data collection. You will use this file to troubleshoot issues where the Data Packager failed to run for example.
IDP.2009-5-7.9-8-29.trace.log
Verbose logging for ISA Data Packager, also used to troubleshoot ISA Data Packager itself.
BpaReportFiles
BPAReport_ISACONTN1_200905070911.xml
This is the ISA BPA Health Check report that you can load using the ISA BPA Tool.
BPAReport_ISACONTN1_200905070911.xml.log
Log for data collection of the ISA BPA, used to troubleshoot ISA Health Check itself.
IsaConfigExport.200905070911.xml
This is the export configuration of the ISA Server. Consider this your backup if you never made one, here it goes.
EventViewerEvents
Application.evt
NEW on BPA7 – this is the export of the Application log in EVT format.
EventViewer_ErrorEvents6.csv
Only errors events (windows event viewer events) in CSV format.
EventViewer_IsaEvents6.csv
Only ISA errors events logged in windows event viewer.
System.evt
NEW on BPA7 – this is the export of the System log in EVT format.
IsaInfoFiles
ISAInfo_isacontn1.log
Log for data collection of the ISA Info, used to troubleshoot ISA Info itself.
ISAInfo_isacontn1.xml
ISAInfo file that you can open using ISA Info tool from ISA Tools .org.
ISALogs
IsaLogs_Firewall_TextEXT_200905070911.csv
Firewall Logging in CSV format.
IsaLogs_WebProxy_TextEXT_200905070911.csv
Web Proxy Logging in CSV format.
IsaTraces
isalog.bin
Files used by Microsoft CSS Engineers only since they require internal symbols to parse it.
manifest.txt
NetworkCaptures
External_20090507090839.cap
Network Capture from the external interface. Correct, you don’t need to start netmon separately when using ISA Data Packager.
LocalCorp_20090507090839.cap
Network Capture from the Internal interface.
Note1: amount of files and folders will vary according to the template that you choose.
Note 2: file name will vary according to the date of the day.
With this set of data you have enough data to start troubleshooting the issue that you are facing with ISA Server. You have logs, network captures and capability to read ISA Server configuration. My recommendation is that you install this tool in your lab, and start to test simple scenarios so you get used to read those logs. Try to simulate simple issues in your lab and look the logs to see what you can do to fix the issue.