Yuri Diogenes's Blog

Error 502 “The data is invalid” while browsing Internet through TMG 2010 RC

yuridio — Sat, 07 Nov 2009 15:35:00 GMT

1. Introduction

This post is about a specific condition that can triggers the error 502 while browsing some web sites through TMG 2010 RC. The error message that the end users receives is similar to the one shown below:

The TMG Logging will not say much beyond as shown below:

This scenario was interesting because it was working sometimes but it didn’t work most of the time. Looking closely to the data I notice that when it works is because it hits one specific server in the destination web farm and when doesn’t work it hits another web server.

2. Understanding the Behavior

Using network monitor was possible to better understand why this happen:

1) The HTTP Header when it works:

- Http: Response, HTTP/1.1, Status Code = 200, URL: /

ProtocolVersion: HTTP/1.1

StatusCode: 200, Ok

Reason: OK

Date: Tue, 13 Oct 2009 15:57:06 GMT

Server: WEBSRV

Expires: Mon, 26 Jul 1997 05:00:00 GMT

Cache-Control: private, no-store, no-cache, must-revalidate, post-check=0, pre-check=0

Pragma: no-cache

P3P: CP="SRV"

Set-Cookie: reg_fb_gate=http%3A%2F%2Fwww.fabrikam.com%2F; path=/; domain=.fabrikam.com

Set-Cookie: reg_fb_ref=http%3A%2F%2Fwww.fabrikam.com%2F; path=/; domain=.fabrikam.com

Set-Cookie: test_cookie=1; path=/; domain=.fabrikam.com

Set-Cookie: lsd=zCI0G; path=/; domain=.fabrikam.com

Connection: close

TransferEncoding: chunked

+ ContentType: text/html; charset=utf-8

ContentEncoding: gzip

HeaderEnd: CRLF

+ chunkSize: 10

- ChunkPayload: HttpContentType = text/html; charset=utf-8

HtmlElement: ヒ

FooterEnd: CRLF

+ chunkSize: 8192

- ChunkPayload: HttpContentType = text/html; charset=utf-8

We can see that the HTTP response says that the following content will be chucked and after that response, destination server sends the other HTTP chucks:

WEBSRV TMG HTTP HTTP:HTTP Payload, URL: /

2) HTTP Header when doesn’t work

- Http: Response, HTTP/1.1, Status Code = 200, URL: /

ProtocolVersion: HTTP/1.1

StatusCode: 200, Ok

Reason: OK

Cache-Control: private, no-store, no-cache, must-revalidate, post-check=0, pre-check=0

Expires: Mon, 26 Jul 1997 05:00:00 GMT

P3P: CP="WEBSRV2"

Pragma: no-cache

Set-Cookie: lsd=PQ6kd; path=/; domain=.fabrikam.com

Set-Cookie: reg_fb_gate=http%3A%2F%2Fwww.fabrikam.com%2F; path=/; domain=.fabrikam.com

Set-Cookie: reg_fb_ref=http%3A%2F%2Fwww.fabrikam.com%2F; path=/; domain=.fabrikam.com

Set-Cookie: test_cookie=1; path=/; domain=.fabrikam.com

ContentEncoding: gzip

+ ContentType: text/html; charset=utf-8

X-Cnection: close

Date: Tue, 13 Oct 2009 15:38:13 GMT

ContentLength: 9970

HeaderEnd: CRLF

+ payload: HttpContentType = text/html; charset=utf-8

Notice that the failing server don’t say that the content is chucked, however it is still sending more chucks after that:

WEBSRV2 TMG HTTP HTTP:HTTP Payload, URL: /

Since Chunked Transfer Encoding is a mechanism that allows HTTP messages to be split in several parts the first server is answering correctly, while the second server it is not. According to RFC if a server is using chunked encoding it must set the Transfer-Encoding header to "chunked". In order to compress the content we need to accumulate all the chucks and then compress. When it works, TMG knows that all that content is part of the same HTTP request since it says in the HTTP Response Header; therefore it waits for the entire content, compress and send it back to the client. On the failing server we receive the first answer that doesn’t say that the content is chucked and right after that we receive other chucks, since HTTP Compression is enabled it fails to reassemble all the content since it doesn’t know that they belong to the same content.

3. What can I do to fix it?

The best and most correct thing to do is to contact the administrator for the destination web server and report the problem; they should fix the issue since TMG is acting correctly. However if you want a workaround, that will be disabling the HTTP compression filter.

Enhanced Mitigation Evaluation Toolkit

yuridio — Thu, 29 Oct 2009 12:21:00 GMT

Developers, developers, developers, developers (I bet you that you know this song). As you build your new application you should start thinking about security from the source (inside out approach). However even when you try to mitigate all scenarios that you can imagine there is always a concern before ship the application about any potential flaw that you forgot to cover. Microsoft can help you on that with the new Enhanced Mitigation Evaluation Toolkit. Read it more about what this tool can do for you here:

http://blogs.technet.com/srd/archive/2009/10/27/announcing-the-release-of-the-enhanced-mitigation-evaluation-toolkit.aspx

….and download it from here:

http://www.microsoft.com/downloads/details.aspx?displaylang=en&FamilyID=4a2346ac-b772-4d40-a750-9046542f343d

To read a non Microsoft perspective about this tool, read the article below:

http://www.theregister.co.uk/2009/10/27/microsoft_security_tool/

Forefront TMG 2010 Book now Available for Pre Order

yuridio — Wed, 28 Oct 2009 01:06:00 GMT

I’m really happy to announce that our Forefront TMG 2010 book from MSPress is now available for Pre Order at Amazon.com and they really have a great deal waiting for you…got get yours J

MBSA 2.1.1 for Windows 7 released

yuridio — Wed, 28 Oct 2009 01:03:00 GMT

If you are using Windows 7 or Windows Server 2008 R2, make sure to test the new version of the Microsoft Baseline Security Analyzer 2.1.1. Read more about it here http://technet.microsoft.com/en-us/security/cc184923.aspx and download it from MS Download Center.

The Firewall Madness

yuridio — Thu, 22 Oct 2009 15:09:00 GMT

This week I worked in an issue where ISA Server 2006 was stopping answering request and NLB on ISA Server was constantly appearing with the status for “Unavailable”. The odd thing about this scenario was that every time that the firewall admin changed a rule in one node and forces a synchronization the status of the NLB changed to “Unavailable”. I have to admit that I saw this couple of other times, but I didn’t want to jump into conclusions without gathering data and analyze the result. The issue was inconsistent, sometimes it happened performing the same operation, and sometimes it didn’t. Since I had already a felling of what was going on, I went to the NIC properties of the ISA Server and found that there was a third-party firewall bound to the interface:

This is not good..not good at all. ISA Server is already a firewall and FWENG is the filter driver that runs in Kernel Mode intercepting the traffic and inspecting it. If you add another firewall (that also runs in Kernel mode) on the same box, you should expect inconsistent results like this, because both will dispute the incoming traffic to analyze and inspect it.

Is like….

Anyway…remember the built in Windows Firewall that comes with Windows Server 2003? Here it is what Microsoft says about it in a scenario of multiple host firewalls on the same box:

“Microsoft recommends that you disable Windows Firewall if you are already using a third-party host firewall product.”

(From: http://technet.microsoft.com/en-us/library/cc875816.aspx)

The logic is quiet simple now: this means that on the ISA Server scenario, if you install a third-party host firewall product you probably don’t want to disable ISA Server firewall and leave the third-party one enabled, right? J

In this particular situation the firewall admin didn’t even know that this product that he installed was a Firewall, he thought it was only an Antivirus. This raises another flag: if you are going to install an antivirus on ISA, first use the recommendations from http://technet.microsoft.com/en-us/library/cc707727.aspx and secondly, make sure that this product doesn’t install a firewall module on top of that; otherwise you might experience those weird behaviors.To fix this specific issue we uninstalled the third-party firewall and left only the AV installed by this product.

BTW, have a great Windows 7 day !!

Walkthrough Exchange 2010 RC Edge Role and Forefront Protection 2010 Beta for Exchange installation on Forefront TMG 2010 RC

yuridio — Mon, 19 Oct 2009 11:57:00 GMT

1. Introduction

As we now have Exchange 2010 RC available for download, many of you that are testing Forefront TMG 2010 RC are asking if you can test the Email Protection feature using Exchange 2010. If you read the paper Understanding E-Mail Protection on Forefront TMG published at Tales from the Edge, you will see that one of the questions in the Q&A is:

Question 12) Which versions of Exchange do you support?

Answer: We support Exchange Edge 2007 SP2 and Exchange Edge 2010.

With that you know that it is supported, but the opening question is: how to install Exchange 2010 Edge role and Forefront Protection 2010 on top of an existing Forefront TMG 2010 RC installation? This is exactly the goal of this post; guide you through the steps to perform this installation. This post is assuming that TMG 2010 RC is running on Windows Server 2008 R2.

2. Preparing the Environment for Exchange 2010

Before install Exchange 2010 RC you should install a series of prerequisites and the best way to do this is by following the guidelines from Exchange 2010 Prerequisites document, under the section Install the Windows Server 2008 R2 operating system prerequisites. After complete this process, than you can run the Exchange 2010 setup and choose the following options:

1. Select Install Microsoft Exchange as shown below:

Figure 1 – Selecting Exchange setup option.

2. Click Next in the Introduction page. Read the license agreement, select I accept the terms in the license agreement and click Next to continue.

3. Select Yes in the Error Reporting page and click Next.

4. Select Custom Exchange Server Installation as shown below and click Next to proceed:

Figure 2 – Selecting Custom installation.

5. Select Edge Transport Role in the Server Role selection as shown below and click Next to continue:

Figure 3 – Selecting Edge Transport Role.

6. Chose the appropriated option for the CEIP and click Next to continue.

7. Wait until the readiness check finishes and when your window appears as shown below click Install to proceed:

Figure 4 – Click Install to proceed.

8. When the setup finishes as shown in the figure below, uncheck the option Finalize Installation using the Microsoft Exchange Console and click Finish button to conclude the process.

Figure 5 – Setup finished.

9. On Exchange Setup window, click step 5 – Get Critical update for Microsoft Exchange.

10. Install any critical update that it might have and close the Exchange Setup window.

At this point you already have Exchange 2010 Edge Role installed on your system; next step is to install Forefront Security 2010 for Exchange on TMG.

3. Running Exchange Installation via TMG 2010 Setup

Follow the steps below to install Forefront Security 2010 for Exchange from the TMG setup:

1. Execute the autorun.hta file and choose the option to Install Microsoft Forefront Protection 2010 for Exchange Server:

Figure 6 – Choose the option to install Forefront Security 2010 for Exchange.

2. Select to the terms of the license agreement and privacy statement and click Next.

3. You should receive a notification saying that Exchange Transport service will be restarted. Click Next to proceed.

4. Confirm the installation folders (or change according to your preference) and click Next.

5. Click Next on the Proxy configuration.

6. Leave the Enable antispam now option selected as shown in Figure below and click Next to proceed:

Figure 7 – Enabling Antispam.

7. Leave the Enable antispam now option selected as shown in Figure below and click Next to proceed:

8. Chose the appropriated option for the CEIP and click Next to continue.

9. Review all your selections in the Confirm Settings page as shown below and click Next to continue:

Figure 8 – Reviewing installation settings.

10. While the installing is happening you will also see the window below saying that the setup is configuring the product and services:

Figure 9 – Configuring product and services setup window.

11. After that you should see the last setup window saying that the installation finished successfully as shown below:

Figure 10 – Reviewing installation results.

12. Click Finish button to finish the setup.

13. Click Exit to close the TMG Setup window.

Now you have both consoles available: Exchange and Forefront Protection 2010 for Exchange as shown below:

Figure 11 – FSE and Exchange console available after finishing this procedure.

Note: Something to keep in mind: changes that you perform on TMG 2010 regarding Email Protection will be applied to Exchange Edge and FSE according to the option that you choose. Read the paper Understanding E-Mail Protection on Forefront TMG published at Tales from the Edge for more information on what feature each product owns.

4. Conclusion

In this post you learned how to install Exchage 2010 Edge Role and Forefront Protection 2010 Beta for Exchange on top of an existing Forefront TMG 2010 RC installation. Now that the setup is done, use the Configuring protection from e-mail-based threats article to configure this feature.

TMG and UAG supportability boundaries

yuridio — Mon, 12 Oct 2009 14:19:00 GMT

I remember in the past (IAG and ISA) many people asking what it was and what it was not supported doing on ISA that comes with IAG. As TMG and UAG approaches to release date we do have now an official supportability statement about using TMG that comes with UAG installation. So, before exercise your creativity using TMG that comes with UAG, make sure to read the article below that cover the main scenarios:

http://technet.microsoft.com/en-us/library/ee522953.aspx

Forefront TMG 2010 RC is now Available

yuridio — Sun, 11 Oct 2009 15:21:00 GMT

If you were playing with TMG Beta 3 for awhile, it is time to taste the RC version of Forefront TMG 2010. Yes, it is available and you can get it here: http://www.microsoft.com/DOWNLOADS/details.aspx?FamilyID=e05aecbc-d0eb-4e0f-a5db-8f236995bccd&displaylang=en

You will notice a series of changes right in the begging, starting with a whole new setup experience, that was actually explained in this post on ISA/TMG Team blog. Due that amount of changes we (authors of Forefront TMG Administrator Companion) are real busy updating the book before we release. But, I’m sure it will be worth it to wait…you will see. Now go get your TMG 2010 RC and start playing with it.

Unwanted prompt for authentication while browsing a web site that uses SQL Reporting Services 2008 published by ISA Server 206

yuridio — Thu, 08 Oct 2009 01:36:00 GMT

Introduction

This post is about an interesting case where the final goal of the system administrator was to allow “all” public users to access some reports that were located/generated by SQL Reporting Services 2008. The regular web site was published by ISA Server and it was working perfectly. ISA Server was not doing any type of authentication since the real goal was to allow anonymous. The relevant parameters from the rule and listener were configured like this:

· Rule:

o Authentication: No Authentication and client may authenticate directly.

o Users: All Users

o Paths: /*

· Web Listener:

o Authentication: No Authentication

§ Advanced: Allow client authentication over HTTP

This means that ISA was allowing the traffic all the way from Internet to the published server. When the rule has those parameters you can be certain that ISA is not asking for authentication, therefore if you are receiving prompts for authentication in a publishing rule you should start investigating the server that you are trying to publish. Interesting enough, in this case the IIS Server that has the web site was indeed allowing anonymous and we were able to browse the whole web site but the link that generates the report. During that time, we were able to see that the internal NIC of ISA was receiving the following answer from the web server:

- Http: Response, HTTP/1.1, Status Code = 401, URL: /reportserver/Pages/ReportViewer.aspx, Using NTLM

Date: Authentication

ProtocolVersion: HTTP/1.1

StatusCode: 401, Unauthorized

Reason: Unauthorized

ContentLength: 0

+ WWWAuthenticate: Negotiate

WWW-Authenticate:

+ WWWAuthenticate: NTLM

Date:

Date: Mon, 28 Sep 2009 17:56:59 GMT

Connection: close

HeaderEnd: CRLF

Weird, since IIS was configured to allow anonymous, right? Wrong…because on top of IIS we have Reporting Services virtual web site and there was the catch. According to “Authentication Types in Reporting Services” paper, anonymous authentication is not supported, it says:

Authentication method

Explanation

Anonymous

The report server will not accept unauthenticated requests from an anonymous user, except for those deployments that include a custom authentication extension.

Report Builder will accept unauthenticated requests if you enable Report Builder access on a report server that is configured for Basic authentication.

For all other cases, anonymous requests are rejected with an HTTP Status 401 Access Denied error before the request reaches ASP.NET. Clients receiving 401 Access Denied must reformulate the request with a valid authentication type.

There we go. This explains why ISA’s internal NIC was receiving the 401….and all that started because system admin was saying that it was an ISA issue since it worked perfectly inside of his network. Well, it make sense working internally since it will negotiate NTLM for domain users and it will be transparent that something is going on behind the scenes. However, the SQL guy that worked with me on this issue did find a workaround which is documented in here: http://blogs.msdn.com/jameswu/archive/2008/07/15/anonymous-access-in-sql-rs-2008.aspx

Again…not an ISA issue J.

IIS on ISA, just one more reason to avoid this type of setup

yuridio — Wed, 07 Oct 2009 04:39:00 GMT

There are many reasons for me to not enjoy the idea of having IIS on the same box as ISA Server. First and foremost you need to remember that ISA is a Firewall so you shouldn’t use as web server anyway (put your web server behind ISA). Another reason is the resource allocation failure that happens when ISA tries to grab a port that is being use by IIS. Thanks to ISA product team for showing a red alert on ISA Console when it detects that (see below).

This week I got another good reason for not using this setup and here a brief story about this:

“Once upon a time, there was a system admin that just got hire to work for Fabrikam. He was happy with his new job when he found out that he inherited an environment that was about to explode on his hand. One day he restarted his ISA Server as part of a maintenance window and when it came back ISA Server services were not starting. No internet access, no inbound or outbound emails, no OWA…chaos”

Troubleshooting

Reviewing the event viewer it was possible to see the following sequence on system log from 10/4:

Event Type: Error

Event Source: Service Control Manager

Event Category: None

Event ID: 7023

Date: 10/4/2009

Time: 7:36:10 PM

User: N/A

Computer: ISASRVSTD

Description:

The IIS Admin Service service terminated with the following error:

Access is denied.

Event Type: Error

Event Source: Service Control Manager

Event Category: None

Event ID: 7001

Date: 10/4/2009

Time: 7:36:10 PM

User: N/A

Computer: ISASRVSTD

Description:

The HTTP SSL service depends on the IIS Admin Service service which failed to start because of the following error:

Access is denied.

Event Type: Error

Event Source: Service Control Manager

Event Category: None

Event ID: 7001

Date: 10/4/2009

Time: 7:36:10 PM

User: N/A

Computer: ISASRVSTD

Description:

The World Wide Web Publishing Service service depends on the IIS Admin Service service which failed to start because of the following error:

Access is denied.

Clearly we have IIS on this box although system admin didn’t know why this box has IIS and why it was broken before and now is working just fine. For day 10/06 we have the following failure from ISA and all was clear from IIS side:

Event Type: Error

Event Source: Service Control Manager

Event Category: None

Event ID: 7024

Date: 10/6/2009

Time: 7:36:10 PM

User: N/A

Computer: ISASRVSTD

Description:

The Microsoft Firewall service terminated with service-specific error 2148073494 (0x80090016).

The application log also has entries about ISA Services failure to start:

Event Type: Error

Event Source: Microsoft Firewall

Event Category: None

Event ID: 14060

Date: 10/6/2009

Time: 7:35:38 PM

User: N/A

Computer: ISASRVSTD

Description:

ISA Server could not load the application filter Web Proxy Filter ({4CB7513E-220E-4C20-815A-B67BAA295FF4}). FilterInit failed with the error code 0x80090016. To attempt to activate this application filter again, stop and restart the Firewall service.

Event Type: Error

Event Source: Microsoft Firewall

Event Category: None

Event ID: 14001

Date: 10/6/2009

Time: 7:35:38 PM

User: N/A

Computer: ISASRVSTD

Description:

Firewall Service failed to initialize. Previous event log entries might help determine the proper action.

The current status now is:

· IIS up and running.

· ISA down and crying out loud.

The event 7024 from Microsoft Firewall was triggering the error 0x80090016, which means: object already exists. Using this error code I found the article below:

You receive error message 0x80090016 or error message 0x8009000f when you try to schedule a task

That’s a very interesting point, because ISA does read the Documents and Settings\All Users\Application Data\Microsoft\Crypto\RSA\MachineKeys folder as explained in the troubleshooting setup article:

“Before removing ISA Server, be sure to close ISA Server Management and ISA Server Performance Monitor.

If the storage is corrupted, as part of the procedure, you will also have to specify the array membership and Internal network configuration. When you install a certificate, a private key container is created on the Configuration Storage server in the Documents and Settings\All Users\Application Data\Microsoft\Crypto\RSA\MachineKeys folder. The account running ISASTGCTRL service (by default, the Network Service account) must have appropriate permissions to the private key container. Certificates typically have an expiration period, usually no more than one year. ISA Server cannot use an expired certificate. Be sure to renew your certificates before they expire, so that ISA Server can continue to function.”

I went to this folder to see what we have there and found something weird:

Very odd! However after many questions and investigations, myself and the system admin found out that the previous admin used the KB884872 to fix an IIS issue and ended up breaking ISA. The content of MachineKeys folder just had 1 file on it while the MachineKeys.old has all the other keys and by default MachineKeys folder stores certificate pair keys for both the computer and users.

Note: This is a very sensitive folder and before playing around with it beware that you understand the consequences of changing anything on it. Read KB278381 for more information about usability and permissions needed for this folder.

The solution was quiet simple: rename the MachineKeys folder to MachineKeys.tmp and rename the original MachineKeys.old to MachineKeys, after that Firewall Service started just fine. Now you are probably asking: so you broke IIS again by doing this procedure? Yes, I did (on purpose). The bottom line here was that this ISA didn’t need IIS anyway so we ended up uninstalling IIS from this box (YAY).

Conclusion

From now on, instead of thinking twice before install IIS on ISA, think ten times before doing that. Much better having another Windows Server 2003 box dedicated to be Web Server.

Error 500 while trying to access some parts of the CRM 4 Web Site Published through ISA Server 2006

yuridio — Thu, 01 Oct 2009 22:01:00 GMT

This short post is about an issue that I worked a couple of weeks ago about publishing CRM through ISA Server. The admin had followed the post that I wrote on how to publish CRM through ISA Server 2006 and most of the things within CRM web sites were working fine. The problem is was externalized when he was trying to do a certain operation and got the following error on IE: Error Code: 500 Internal Server Error.

By using diagnostic logging it was possible to see that when ISA HTTP Filter evaluates the request, it triggers the error. The request (HTTP GET) was:

/activities/email/edit.aspx?pId={80BB5777-ED93-DE11-8C2A-0003FFCE4329}&pType=112&pName=Fluxo%20de%20Caixa%20-%20CIT&partyid={FD5E1DA2-9E85-DD11-AD2C-0003FFCE4329}&partytype=2&partyname=M%C3%A1rio%20Falcheti%20SenorSolicitante&partyaddressused=&contactInfo=', Context:216E1B6C

ISA Server diagnostic logging shows: The request was rejected by the HTTP Security filter.

The solution was disabling Block high-bit characters from the HTTP Filter. As KB837865 says:

“When you configure HTTP filtering to block high-bit characters, URLs that contain characters from a double-byte character set (DBCS) or URLs that contain Latin 1 characters are blocked. This configuration may affect scenarios such as OWA publishing or Microsoft SharePoint Portal Server publishing. Additionally, this configuration may affect any scenario where a GET request passes a parameter that includes a character from a double-byte character set.”

Although this is a known issue, sometimes administrators out there think that this only applies to OWA or SharePoint Publishing rule, which is not true. This setting can affect any web site publishing rule that has such condition.

MS Reputation Services Web Site

yuridio — Thu, 01 Oct 2009 06:56:00 GMT

Did you know that Microsoft has created a portal to enable you to check the classification by the Microsoft Reputation Service? You don’t? So check this out at:

https://www.microsoft.com/security/portal/mrs/default.aspx?CustomerID=%7b0900B81A-6C5F-470D-B7FC-A1A335E56A0D%7d&CustomerVersion=TMG-V1+7.0.7731.100&DisputedCategoryID=29360149

Unable to Install ISA BPA

yuridio — Thu, 01 Oct 2009 03:49:00 GMT

Today while troubleshooting in ISA Server issue I had to install ISABPA in order to get an ISA Data Packager in repro mode. When the admin was trying to install ISABPA we got the following error:

The user that was trying to execute this setup was the local admin of the ISA Box, no software restriction policy was applied to this box…arghh…now I have to troubleshoot why BPA doesn’t install to them troubleshoot the real issue, nice !!

With a little help of Process Monitor I found that when launching this file, the windows shell process (explorer.exe) reads HKKU\Software\Microsoft\Windows\ CurrentVersion\Internet Settings to determine the zone as shown below:

From that I found KB182569 that explains in details how this registry key works and what the options are. The problem was a setting on Internet Explorer 7 called “Launching applications and unsafe files” , after changing setting that for prompt, the issue was fixed we were able to install ISABPA.

Error 0x800706fd Installing TMG Beta 3

yuridio — Sat, 26 Sep 2009 16:34:00 GMT

As TMG RTM is very close, the amount of downloads for TMG Beta 3 is growing and with that, some users are facing some issues while trying to install TMG Beta 3. This one in particular refers to an error trying to check domain membership while installing ADLS and if you open the file %windir%\temp\ ISAADAM_INSTALL_XXX.log you might see the following error:

adamsetup 7E4.BEC 0163 10:05:53.536 Enter CheckServiceSecurity

adamsetup 7E4.BEC 0164 10:05:53.536 Enter InitSecWinntAuthIdentity

adamsetup 7E4.BEC 0165 10:05:53.536 Enter State::GetOperation UNIQUE

adamsetup 7E4.BEC 0166 10:05:53.957 Enter State::GetOperation UNIQUE

adamsetup 7E4.BEC 0167 10:05:53.957 NtdsAdamValidateServiceAccount() => 1789

adamsetup 7E4.BEC 0168 10:05:53.957 info.eValidationResult = 0

adamsetup 7E4.BEC 0169 10:05:53.957 Enter GetErrorMessage 800706FD

adamsetup 7E4.BEC 016A 10:05:53.957 ADAMERR_OK

adamsetup 7E4.BEC 016B 10:05:53.957 AD LDS Setup was unable to validate the selected service account.

Error code: 0x800706fd

The trust relationship between this workstation and the primary domain failed.

adamsetup 7E4.BEC 016C 10:05:53.957 ADAMERR_OK

adamsetup 7E4.BEC 016D 10:05:53.957 Enter Feedback::ShowMessage AD LDS Setup was unable to validate the selected service account.

Error code: 0x800706fd

The trust relationship between this workstation and the primary domain failed.

Also, keep in mind that if you are trying to install TMG Beta 3 on Windows Server 2008 R2 it might not install or it might install and do not work properly, the reason why is because TMG Beta 3 was not projected to run on Windows Server 2008 R2 therefore is not supported (for this release – Beta 3). Don’t worry, by TMG RTM version we will be fine with that, however if you want to play with TMG Beta 3 now make sure to install Windows Server 2008 version (no R2).If you are indeed using Windows Server 2008 (only) and you are receive this error on the log, than go ahead and start troubleshooting domain membership problems, secure channel, RPC, etc…you know the drill J.

If you need to discuss technical issues about TMG Beta 3, Microsoft has an open forum for that on the link below:

http://social.technet.microsoft.com/Forums/en-US/FTMGNext/threads

Troubleshooting Tips for VPN Client Access on ISA Server 2006

yuridio — Fri, 25 Sep 2009 20:46:00 GMT

1. Introduction

I wrote some time back about an issue on ISA Server related to VPN and on that post I emphasized the following:

“…ISA Server VPN component uses Windows RRAS component and that when the ISA is isolated from the main problem, the troubleshooting line needs to follow RRAS tools and techniques.”

This is what boils down to on the most case that I work where ISA Server is being used as VPN Server. Troubleshooting on the ISA Server side most of the time is minimum, most of the time on the ISA Server side is just a matter of configuration, or should I say misconfiguration. The big and deep troubleshooting involve other elements around:

· RRAS

· Networking in general

· Connectivity with the DC (or RADIUS Server)

This week I got a couple of VPN cases that were really interesting and I would like to share with you the troubleshooting steps and solution for those.

2. Scenario 1 – VPN Client was unable to authenticate

In this scenario, all VPN settings were correct on ISA and on client; on ISA Server logging we just see error 0x80074E20, which means “A connection was gracefully closed in an orderly shutdown process with a three-way FIN-initiated handshake” (see error codes for more information). Basic connectivity with the DC was also good…but, basic means only a ping with a reply. No other device in between internal NIC of ISA and the DC, full communication was allowed.

Investigating further it was possible to see the following:

· Event Viewer

Event Type: Error

Event Source: Microsoft Firewall

Event Category: None

Event ID: 21137

Date: 9/24/2009

Time: 5:51:18 PM

User: N/A

Computer: ISACONTN1

Description:

The connectivity verifier "Domain Controller" reported an error when trying to connect to DCCONT.contoso.msft.

Reason: The request has timed out.

Event Type: Error

Event Source: NETLOGON

Event Category: None

Event ID: 5719

Date: 9/24/2009

Time: 5:54:27 PM

User: N/A

Computer: ISACONTN1

Description:

This computer was not able to set up a secure session with a domain controller in domain CONTOSO due to the following:

There are currently no logon servers available to service the logon request.

This may lead to authentication problems. Make sure that this computer is connected to the network. If the problem persists, please contact your domain administrator.

ADDITIONAL INFO

If this computer is a domain controller for the specified domain, it sets up the secure session to the primary domain controller emulator in the specified domain. Otherwise, this computer sets up the secure session to any domain controller in the specified domain.

· RRAS Tracing (need to run “netsh ras set tracing * enable” in order to have those files under %systemroot%\tracing)

Content of the file IASSAM.LOG:

[924] 09-24 17:53:39:429: NT-SAM Names handler received request with user identity CONTOSO\administrator.

[924] 09-24 17:53:39:429: Username is already an NT4 account name.

[924] 09-24 17:53:39:429: SAM-Account-Name is "CONTOSO\administrator".

[924] 09-24 17:53:39:429: NT-SAM Authentication handler received request for CONTOSO\administrator.

[924] 09-24 17:53:39:429: Processing MS-CHAP v2 authentication.

[924] 09-24 17:54:27:198: LogonUser succeeded.

[924] 09-24 17:54:27:198: NT-SAM User Authorization handler received request for CONTOSO\administrator.

[924] 09-24 17:54:27:198: Using native-mode dial-in parameters.

[924] 09-24 17:54:27:198: Sending LDAP search to DCCONT.contoso.msft.

[924] 09-24 17:54:27:198: LDAP ERROR in ldap_search_ext_sW. Code = 81

[924] 09-24 17:54:27:218: Extended error string: (null)

[924] 09-24 17:54:27:218: Retrying LDAP search.

[924] 09-24 17:54:27:618: Failed to connect to the cached DC, try DC locator ...

[828] 09-24 17:54:31:284: NT-SAM Names handler received request with user identity CONTOSO\administrator.

[828] 09-24 17:54:31:284: Username is already an NT4 account name.

[828] 09-24 17:54:31:284: SAM-Account-Name is "CONTOSO\administrator".

[828] 09-24 17:54:31:284: NT-SAM Authentication handler received request for CONTOSO\administrator.

[828] 09-24 17:54:31:284: Processing MS-CHAP v2 authentication.

[828] 09-24 17:54:31:284: LogonUser succeeded.

[828] 09-24 17:54:31:284: NT-SAM User Authorization handler received request for CONTOSO\administrator.

[924] 09-24 17:54:50:922: Failed to connect to the DC discovered by DC locator, try DC enumerator ...

[828] 09-24 17:55:02:979: Using native-mode dial-in parameters.

[828] 09-24 17:55:02:979: Could not open an LDAP connection to domain CONTOSO.

[924] 09-24 17:55:02:979: Could not open an LDAP connection to domain CONTOSO.

[828] 09-24 17:55:03:089: NTDomain::getConnection failed: This operation returned because the timeout period expired.

[828] 09-24 17:55:03:089: Retrying LDAP search.

[828] 09-24 17:55:03:089: Could not open an LDAP connection to domain CONTOSO.

[828] 09-24 17:55:03:089: NTDomain::getConnection failed: This operation returned because the timeout period expired.

[924] 09-24 17:55:03:089: NTDomain::getConnection failed: This operation returned because the timeout period expired.

By this time you can be thinking: how we have basic connectivity and it doesn’t reach the DC? That was the key question. The next step was a simple test: try to change the groups that had permission to access VPN and user a local group, from ISA itself. In order to do that I created a local group called VPNUsers, added the local administrator on it (for testing purpose) and try to add the CONTOSO\Domain Users when….got the following error: The RPC server is unavailable. Ok, now it makes sense and to start troubleshooting this I used:

Troubleshooting RPC Endpoint Mapper errors using the Windows Server 2003 Support Tools from the product CD

It turns out that the issue was related to the binding order of the network adapter. On ISA Server we had the following binding order:

Figure 1

It is strongly recommended that the internal network (in this case LocalCorp) is on the top. This can avoid many issues such as this one. Here are some references on that:

· Delay in NetBIOS connections from a multi-homed computer

· How to properly configure network binding order on a BackOffice Small Business Server system

· DNS: Valid network interfaces should precede invalid interfaces in the binding order

After changing the binding order it worked like a charm!

3. Scenario 2 – Unable to Authenticate using RSA VPN Client

First and foremost, even before start touching on ISA Server, it is very important to review the article below from RSA about how to configure ISA Server 2006 to use RSA for VPN authentication purpose:

http://www.rsa.com/rsasecured/guides/imp_pdfs/Microsoft_ISA2006_AM7.1_VPN.pdf

By following the above article you can pretty much configure all you need in order to make the VPN work with RSA. However in this case, although everything was done, it was not authenticating. The key here it was that in this case the authentication was done using RADIUS.

By looking the Netmon trace while authentication was happening (internal NIC of ISA) we could see that ISA (10.20.20.1) keep sending requests to the RADIUS Server (10.20.20.20) without answer:

Figure 2

According to the system administrator this RADIUS is being used for other purposes and it only fails with ISA. Going to the RADIUS Server it was possible to see the following event on Event Viewer:

Figure 3

That was simple, right? What happens is that if the RADIUS Server doesn’t have the RADIUS Client configured in there, it will fail to answer the request. To fix that we just need to add ISA Server as RADIUS Client and make sure that the secret between them is the same. Here are some references:

Event ID 13 — RADIUS Client Configuration

Error 930; The authentication server did not respond to authentication requests in a timely fashion

4. Conclusion

These are more examples about how troubleshoot VPN on ISA Server 2006. Try to not focus so much on ISA itself because many times ISA is not the culprit.