Marcus Hass' [MS] Blog : Windows

Microsoft Online and SBS 2003

mhass — Wed, 03 Jun 2009 05:16:00 GMT

I have been working with the Microsoft BPOS aka Microsoft Online guys in Enterprise accounts for a while to help big companies migrate to BPOS dedicated. Don’t know what that is? Check out www.microsoftonline.com. To sum it up, it is hosted Exchange, SharePoint, OCS, LiveMeeting, and a few other offerings. For bigger businesses, Microsoft sets up dedicated hosting servers, for small it is multitenant.

I help out a small company from time to time because they have 15 employees and a Small Business Server 2003 environment. They are constantly running out of space on their 5 year old server because mail boxes keep growing because of attachment sizes. These guys are the perfect scenario to migrate to Microsoft Online!

So, I setup a free trial and started loading some of the coexistence tools like email sync and dirsync onto the Small Business server. Well, that was the plan. Turns out, dirsync can’t be run on a domain controller and will only run on Windows Server 2003. I think the BPOS guys missed the Small Businesses aren’t going to have an extra server lying around. How can you miss this scenario when building your tools, especially a segment of the market so perfect for BPOS?

So, they will have to forgo the coexistence and migrate mailboxes in one fell swoop over a weekend, which won’t be pretty over the small network connection they have.

I am sure there are technical reasons, and that's what will be used as an excuse. It just disappoints me when we have really smart guys that miss such a big opportunity to help small businesses.

Windows XP/XPe and Remote Desktop Services Single Sign On

mhass — Thu, 16 Apr 2009 18:03:00 GMT

This week I was working with a retail customer that has plans to place HP Windows XP Embedded devices at their many retail stores. Applications will be served up either locally on the XPe device, through a remote desktop, or through Remote Applications.

There is a slight challenge with this setup because technically Microsoft supports this configuration, but doesn’t give you great tools to setup Single Sign On (SSO). When Vista was first introduced, Microsoft created a new credential manager that could handle SSO for Terminal Server as well as products such as HyperV. Fortunately, the product team also back-ported the credential manager (CredSSP) functionality to Windows XP. While Vista has an easy enough local Group Policy you can edit, Windows XP never got the same treatment. In order to get it to work in XP and XPe, you have to make a bunch or registry edits, which are also not provided in an easy to copy .REG format.

Well, as a service to the public, I have included a text copy of my .REG file below. The information below is provided as-is, no warranty, no support, please don’t cry to me. But, I have tested it pretty thoroughly and it seems to work.

A couple caveats:

If you use a smartcard to authenticate to Windows, no matter how hard you try you won’t be able to get an RDP session to honor your Windows credentials, you will always be prompted for credentials when running MSTSC or a .RDP file. This is counter-intuitive as you would think 2 factor authentication would be more trusted than simple username/password, but it is a known limitation in Windows XP. Citrix does provide their own credential manager that can add functionality here.
One of the registry entries is in hex so you can’t see what it is. It is one of two entries that require you to APPEND the necessary settings for CredSSP to work. If you have other entries for GINA’s or other credential providers, please be careful as this will overwrite them with the default+CredSSP entries

Many thanks to Olga and Sergey on the product team as well as Kevin Martin from HP for their help this week.

References:

http://support.microsoft.com/default.aspx/kb/951608

http://blogs.msdn.com/rds/archive/2007/04/19/how-to-enable-single-sign-on-for-my-terminal-server-connections.aspx

Here is my .REG file, I hope to create an ADM file at some point that I can share. You can go ahead and cut/paste the rest of this blog entry into a text file and rename it to a .REG file.

Windows Registry Editor Version 5.00

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders]
"SecurityProviders"="msapsspc.dll, schannel.dll, digest.dll, msnsspc.dll, credssp.dll"

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Lsa]
"Security Packages"=hex(7):6b,00,65,00,72,00,62,00,65,00,72,00,6f,00,73,00,00,\
00,6d,00,73,00,76,00,31,00,5f,00,30,00,00,00,73,00,63,00,68,00,61,00,6e,00,\
6e,00,65,00,6c,00,00,00,77,00,64,00,69,00,67,00,65,00,73,00,74,00,00,00,74,\
00,73,00,70,00,6b,00,67,00,00,00,00,00

[HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows\CredentialsDelegation]
"AllowDefaultCredentials"=dword:00000001
"ConcatenateDefaults_AllowDefault"=dword:00000001
"AllowDefCredentialsWhenNTLMOnly"=dword:00000001
"ConcatenateDefaults_AllowDefNTLMOnly"=dword:00000001
"AllowFreshCredentials"=dword:00000001
"ConcatenateDefaults_AllowFresh"=dword:00000000
"AllowFreshCredentialsWhenNTLMOnly"=dword:00000001
"ConcatenateDefaults_AllowFreshNTLMOnly"=dword:00000000
"AllowSavedCredentials"=dword:00000000
"ConcatenateDefaults_AllowSaved"=dword:00000000
"AllowSavedCredentialsWhenNTLMOnly"=dword:00000000
"ConcatenateDefaults_AllowSavedNTLMOnly"=dword:00000000
"DenyDefaultCredentials"=dword:00000000
"ConcatenateDefaults_DenyDefault"=dword:00000000
"DenyFreshCredentials"=dword:00000000
"ConcatenateDefaults_DenyFresh"=dword:00000000
"DenySavedCredentials"=dword:00000000
"ConcatenateDefaults_DenySaved"=dword:00000000

[HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows\CredentialsDelegation\AllowDefaultCredentials]
"1"="TERMSRV/*"

[HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows\CredentialsDelegation\AllowDefCredentialsWhenNTLMOnly]
"1"="TERMSRV/*"

[HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows\CredentialsDelegation\AllowFreshCredentials]
"1"="TERMSRV/*"

[HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows\CredentialsDelegation\AllowFreshCredentialsWhenNTLMOnly]
"1"="TERMSRV/*"

[HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows\CredentialsDelegation\AllowSavedCredentials]

[HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows\CredentialsDelegation\AllowSavedCredentialsWhenNTLMOnly]

[HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows\CredentialsDelegation\DenyDefaultCredentials]

[HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows\CredentialsDelegation\DenyFreshCredentials]

[HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows\CredentialsDelegation\DenySavedCredentials]

Exchange 2007 – Snags during my upgrade

mhass — Fri, 26 Jan 2007 20:34:00 GMT

This week I had some time to spend in my lab at home, so I thought I would catch up on some overdue projects, My biggest project was to get my lab up to Exchange 2007 from 2003. The complication here is that although I have a rack of “real” servers, I don’t have any spare capacity.

Virtual PC to the rescue! I grabbed a spare laptop from our inventory at the office, and snagged a copy of my sysprep’d Windows Server 2003 R2 image and installed Exchange 2007.

I decided to write about my experience so that the search engines catch it, and hopefully get you on your way quicker.

Mailbox Migration

After updating my AD schema and making reasonably sure that the 2007 box could talk to the 2003 box, I moved my mailbox. I checked, and I could still access my mailbox through OWA, RPC/HTTP, Local MAPI, and EAS (still accessed through the EX2003 box via publishing rules on my ISA 2006 box). Since all of this worked, I migrated over the 20 or so mailboxes that I host for friends.

Since I didn’t have any spare boxes, I would have to pave my old EX2003 box, install Exchange 2007, and move the mailboxes back off the VPC Exchange server. I decided to take an outage and didn’t change the ISA publishing rules to the new EX2007 box, so I don’t know if Exchange out of the box worked for me (something that in retrospect might have helped me). Uninstalling EX2003 was uneventful. I had to turn off NNTP and SMTP to allow EX2007 to install on the box, as well as apply a .NET hotfix that the installer guided me to install.

The “real hardware” EX2007 box was up and running, and was part of the Org. I moved the mailboxes back, and did a quick check with a local Outlook client to ensure I could still get to mailboxes.

Decommissioning the EX2007 VPC

This is where I hit my first real snag. I took care to move mailboxes and Public Folders over to the “Real EX2007” server. I wanted to ensure that everything was moved over by deleting the Mailbox and Public Folder database before I did the uninstall. When I tried to delete the Public Folders database using the GUI I kept getting this error:

--------------------------------------------------------
Microsoft Exchange Error
--------------------------------------------------------
The public folder database 'Public Folder Database' cannot be deleted.
Public Folder Database Failed
Error:
The public folder database specified contains folder replicas. Before deleting the public folder database, remove the folders or move the replicas to another public folder database.

--------------------------------------------------------
OK
--------------------------------------------------------

With the help of some really smart Exchange product team guys, they pointed me to a couple TechNet articles:

How to Remove a Public Folder Database

How to Remove the Last Public Folder Database in the Organization

For those with link impairment, and for the sake of search engines, I ran the following commands to resolve this issue:

Get-PublicFolder -Server <server with public folder database> "\" -Recurse -ResultSize:Unlimited | Remove-PublicFolder -Server <server with public folder database> -Recurse -ErrorAction:SilentlyContinue

Get-PublicFolder -Server <server with public folder database> "\Non_Ipm_Subtree" -Recurse -ResultSize:Unlimited | Remove-PublicFolder -Server <server with public folder database> -Recurse -ErrorAction:SilentlyContinue

Remove-PublicFolderDatabase -Identity "<server>\<storage group>\<public folder database>"

NOTE: This is where I had my first Eureka! moment. The GUI sucks, you can’t do much more than very basic management from the new Exchange System Management console. The Shell is where it’s at, the more you use it, the more you like it.

So all is good in the world: no more mailboxes or public folders on the EX2007 VPC. When I try and remove Exchange 2007, I started getting the error:

“this computer is configured as a bridgehead server for 1 routing group connector(s) in the organization. These must be moved or deleted before setup can continue”

Again, product team guys easily direct me to the good Exchange 2007 documentation regarding the cmdlets in the shell.

In this case, the GUI didn’t show any routing group connectors (please see my note above about how much the GUI is a waste of time). So, I had to use a command to first find out the names of the routing group connector and then delete it. I ran the following commands:

Get-RoutingGroupConnector [-Identity <RoutingGroupConnectorIdParameter>] [-DomainController <Fqdn>]

Remove-RoutingGroupConnector -Identity <RoutingGroupConnectorIdParameter> [-DomainController <Fqdn>]

Phew, after deleting the server-to-server routing connector I was able to uninstall EX2007 from the VPC.

Can’t send or receive email

After numerous attempts to send and receive email from internal and external clients, I wasn’t able to send or receive internal or external email. I tried using the queue viewer tool in the GUI, and it didn’t give me any clues. I figured I was missing an external send connector, and a quick glance at the GUI verified my assumption (reminder to self: must stop using the GUI). To polish my mad Shell skillz further, I decided to create an external connector for all external domains (*) using the following command:

New-SendConnector -Name <String> -AddressSpaces <MultiValuedProperty> [-AuthenticationCredential <PSCredential>] [-Comment <String>] [-ConnectionInactivityTimeOut <EnhancedTimeSpan>] [-DNSRoutingEnabled <$true | $false>] [-DomainController <Fqdn>] [-DomainSecureEnabled <$true | $false>] [-Enabled <$true | $false>] [-Force <SwitchParameter>] [-ForceHELO <$true | $false>] [-Fqdn <Fqdn>] [-IgnoreSTARTTLS <$true | $false>] [-MaxMessageSize <Unlimited>] [-Port <Int32>] [-ProtocolLoggingLevel <None | Verbose>] [-RequireTLS <$true | $false>] [-SmartHostAuthMechanism <None | BasicAuth | BasicAuthRequireTLS | ExchangeServer | ExternalAuthoritative>] [-SmartHosts <MultiValuedProperty>] [-SourceIPAddress <IPAddress>] [-SourceTransportServers <MultiValuedProperty>] [-TemplateInstance <PSObject>] [-Usage <Custom | Internal | Internet | Partner>] [-UseExternalDNSServersEnabled <$true | $false>]

After creating the send connector, I thought my troubles were over. Wrong! Nothing was working. I did a quick telnet to my server on port 25 and got the error:

“452 4.3.1. Insufficient system resources

Connection to host lost.

Press any key to continue…”

Well, it just so happens that this machine had two partitions, one for OS and one for the stores. By default the SMTP queue is located on the C drive, which only had about 1GB left. Exchange 2007 has a “Back Pressure” feature that disables the SMTP queue when there is low disk space. Unfortunately, there is no handy-dandy shell command to move the queue location. There is a pretty good article up on Technet that tells you how to Change the location of the Queue Database, It involves moving some files, granting “Full Control” to the network service on the new directory, and editing an XML file that contains the location of the queue. I also spotted a way to disable the Back Pressure feature. Just sucks that this isn’t in the Shell….

Immediately after this adjustment I got a shotgun of test emails, and the queue monitor lit up light a Christmas tree.

Certificates, OWA, and ISA

Because I want my buddies to be able to use OWA, EAS and RPC/HTTP securely, I have a public SSL certificate (really cheap from GoDaddy.com). Exchange actually generates its own certificates which is great, but doesn’t really work for my purposes. I also wanted to have ISA do the forms authentication, so I had to have the SSL cert on both the ISA server and Exchange box. It was pretty routine to export the cert with public key and install it on ISA using certificate manager.

I used the ISA publishing wizard for Exchange 2007 for OWA and it made it pretty brainless. I also published IMAP and POP3 for those friends I have that aren’t quite on the RPC/HTTP bandwagon. Additionally, I had already published the SMTP server and created a rule to allow outgoing SMTP from the Exchange server.

When I tried OWA, I kept getting the following Error:

500 Internal Server Error – The target principal name is incorrect”

Turns out that ISA’s interface in 2006 has changed a bit, and was misleading for me. I had created an HTTPS listener with the SSL cert, and everything looked good. ISA allows you to “bridge” the names by allowing you to have an “outside” name and route it to an “internal” name. Turns out, that on the “To” tab of the OWA publishing rule, I had mistakenly specified the “outside” DNS name of instead of my internal server. To set this up correctly, it needs to be:

This rule applies to this published site:

Mail.mydomain.com (external certificate name)

Computer name or IP address (required if the internal site name is different or not resolvable):

10.1.1.1

Summary

When I setup numerous Exchange 2003 servers for customers, I have a set way of doing it. And between having done it a bunch of times, and most of the tweaking in the GUI, 2003 seems easier. That said, I think that if you set the expectation that the Shell is your new config tool, it isn’t much harder. I really like the flexibility of the Shell, and I assume we took the Shell approach because the GUI would be impossibly complex to design for effective management especially with the Unified Communication components.

On your side, Microsoft has provided great documentation this time and it is already published on Technet and other resources.

Is it worth the hassle? Heck ya. Can you run setup.exe and be ready to go in 20 minutes? Nope. This is a complex, powerful product with lots of options. But, most admins familiar with Exchange should not have many issues getting it up and going.

Vista RTM + 1 week

mhass — Mon, 20 Nov 2006 23:17:00 GMT

If you have read some of my previous posts around Vista, you know that it was one of few Microsoft products during this “wave” of products that I wasn’t getting excited about. In fact, I had so many issues with B2->RC1 builds that I stopped using it on my tablet back in September.

Well, I loaded RTM of Business on my tablet the day it RTM’d and it has been surprisingly pleasant. Why didn’t I load Ultimate? I wanted to experience corporate user experience, I loaded Ultimate on my MCE at home (Ultimate is fantastic BTW, but is starting to tax my old desktop machine). After some initial struggles with Toshiba drivers (Toshiba pulled their drivers from Windows Update to fix a couple things, should be posted again this week), I have been really happy.

Here are some of the tips and highlights and lowlights after one week of real usage:

· Install Vista, once complete go to Windows Update and get all new drivers/software. 3^rd parties as well as Microsoft will be providing more and more drivers and software through Windows Update this time.

· I was able to join my computer to the domain over a VPN connection and get all required certs (IPSec, Smart card, etc). This was a huge problem with pre-RTM builds

· I never had to make my user account a local administrator (and still isn’t). This is huge for security guys out there.

· I like the Gadgets, specifically the weather, time and performance Gadgets.

· Performance has been outstanding

· The new display drivers accurately detect if I am using a second monitor now (neither Toshiba or Nvidia drivers ever did this correctly in Windows XP)

· Search is fantastic, all items on my HDD as well as Outlook are indexed and immediately available

· BIGGEST HIGHLIGHT: Life goes on. No major problems, but also no major revolutions. I can still find everything I need.

· BAD: I don’t like the implementation of the network GUI’s. It seems that to enable my wireless connection I have 2 more menus to navigate. This is great for new Windows users, but who really is a new Windows user these days. I am sure there is a better way to shortcut me to what I want, but I haven’t found it yet.

· BAD: When prompted for administrator credentials, I use the local administrator. Vista never helps me select that user like it did in pre-RTM builds. It assumes that I want to use domain user credentials or smart card. Which means that I have to type in a long machinename\localadmin credential. Security good, having to remember my machine name bad.

· REALLY BAD: Can’t figure out how to defrag a specific drive. It only has a “Defrag Now” button which won’t let you select a drive and doesn’t give you “percentage complete” feedback. I use a lot of external USB/Firewire drives and can’t wait for it to defrag on its own.

As I play with Vista more, and find shortcuts for some of my frustrations, I plan on posting them here as usability nuggets. If you find solutions or frustrations, please feel free to comment on them.

Update: Thanks to Tom Beerley for pointing out that the core defrag is still \windows\system32\defrag.exe which supports command line. Just make sure you run the command line as administrator. Below are the command line switches.

V:\Windows\system32>defrag
Windows Disk Defragmenter
Copyright (c) 2006 Microsoft Corp.
Description: Locates and consolidates fragmented files on local volumes to
improve system performance.

Syntax: defrag <volume> -a [-v]
defrag <volume> [{-r | -w}] [-f] [-v]
defrag -c [{-r | -w}] [-f] [-v]

Parameters:

Value Description

<volume> Specifies the drive letter or mount point path of the volume to
be defragmented or analyzed.

-c Defragments all volumes on this computer.

-a Performs fragmentation analysis only.

-r Performs partial defragmentation (default). Attempts to
consolidate only fragments smaller than 64 megabytes (MB).

-w Performs full defragmentation. Attempts to consolidate all file
fragments, regardless of their size.

-f Forces defragmentation of the volume when free space is low.

-v Specifies verbose mode. The defragmentation and analysis output
is more detailed.

-? Displays this help information.

Examples:

defrag d:
defrag d:\vol\mountpoint -w -f
defrag d: -a -v
defrag -c -v

Why I need Office 2007

mhass — Fri, 14 Jul 2006 00:53:00 GMT

I am back to running XP because Vista 5456 just didn’t like my wireless device in my laptop and kept toggling to connection on and off. But, I can’t seem to live without Office 2007.

I like the ribbon, and quickly adjusted to it. I really like the contextual stuff like when you have highlighted text and want to change the font or size, it “previews” it by actually changing the text as you hover above the font or size buttons. But there are 2 killer features for me.

1) RSS Aggregator in Outlook 2007. Most of you are reading this through some type of aggregator such as SharpReader or Windows Live. Outlook has an aggregator that behaves just like reading email, it does read/unread and preview pane. It also lets you import and export your feeds via OPML. About the only thing I seem to be missing with the built in aggregator is a way to force it to go out and grab my feeds, it seems to have a mysterious timer that can’t be forced into submission. Below is a picture of feeds in Outlook 2007

2) Publish to Blog (also document management server, SharePoint) in Word 2007. I can write this post in Word with nice spell checkers and all it’s glory, and then click Publish to Blog and presto.

Like I said before, I am way more excited about Office than I am about Vista. Sure there are great features in Vista, but so far they haven’t helped me be nearly as productive as the changes in Office.

Windows 2000 to XP Upgrade Nuances

mhass — Mon, 24 Apr 2006 19:20:00 GMT

In the process of helping a customer upgrade their remaining 5,000 desktops from Windows 2000 to XP, I have found a couple entertaining and frustrating issues with the upgrade process.

First of all, this only applies to an unattended upgrade of Windows 2000 Professional but it might also affect servers as our internal bugs indicate. Second, Microsoft does not recommend that corporate desktop us the Upgrade process, rather we recommend “wipe and load”. But, in a customer scenario where you don’t have applications packaged and easily deployable to 5,000 users, upgrades are the way to go.

In our specific scenario, we are using various Dell desktops and laptops. Through testing, we discovered that two of the desktop models completely loose their NIC’s during the upgrade. So, we tried to upgrade the drivers for the NIC’s before the upgrade. No dice, once the OS is Windows XP the system has an error with the Network Adapter in Device Manager.

We then tried to use the Microsoft provided means to upgrade the drivers during Windows Installation in the unattended file (below).

[Unattended]

Oempreinstall=yes

OEMSkipWelcome=yes

This tells setup to look in the SourceFiles\i386\$oem$\ folder for drivers and copy them over during the install process.

The only problem is that we are using the UPGRADE flag in the unattended file that negates much of the settings. When you set NTUpgrade=Yes, Windows ignores most of the Unattend file. In fact, Microsoft officially only supports and tests the following flags when NTUpgrade=Yes:

Productkey =

AutoActivate =

DuDisable =

DuShare=

DuStopOnError=

This means that we can’t use any of the cool $oem$ functionality like upgrading the drivers or even running the $oem$\cmdlines.txt which would allow us to kick off a script before Windows logs in. The idea being that we could upgrade the drivers with a batch file before the user logs in. What a nightmare!

I did find that although Microsoft doesn’t support the entries, there are more settings that the unattended process will honor, these include:

[GuiRunOnce]

Command0="C:\upgrade\apf\CmdLines.cmd"

Command1="C:\upgrade\apf\PostOS.cmd 1"

[WindowsFirewall]

Profiles=WindowsFirewall.TurnOffFirewall

[WindowsFirewall.TurnOffFirewall]

Mode=0

So, how did I get around these challenges? I did a bunch of scripting before the upgrade and after the upgrade. Basically, here are the steps:

1) Download all the Windows XP source files and drivers to the workstation hard drive

2) Run a script that prepares the box for an upgrade

a. Disable Wired/Wireless NICS (see my blog on devcon on how to automate this)

b. Removes LegalNoticeText to ensure AutoAdmin logons

c. Resets Local administrator password

d. Sets AutoAdmin logon settings

3) Starts Windows Unattended Setup and uses local source files

4) After the OS is upgraded, it does a GUIRunOnce that I specified
Command0="C:\upgrade\apf\CmdLines.cmd"
Command1="C:\upgrade\apf\PostOS.cmd 1"
This kicks off a PostOS scripting engine call APF (automated purposing framework) that can be found as part of MSA 2.0.

5) The APF engine does the following

a. Phase 1 – HotFixes

b. Phase 2 – Dell Drivers

c. Phase 3 – Enable Dell NICS

d. Phase 4 – Update GPO’s

e. Phase 5 – Finish

i. Set RunOnce (sets user profile settings)

ii. Copies “Default User” profile

iii. Clears AutoAdminLogon Settings

iv. Deletes Source bits

f. Phase 6 – Reboot

6) Once a user logs in for the first time, we fire the RunOnce script that sets some corporate standard user experience settings

I guess the overall advice I can lay down for upgrades are, “This isn’t as easy as running winnt32.exe and then upgrading drivers”. This can be a very complex process to get right, especially when you want to do it for thousands of machines. Wipe and load is still the preferred mechanism, and there is good justification for getting your application packaging strategy working first before you tackle upgrading OS’s,

Updating drivers from the command line

mhass — Sat, 08 Apr 2006 07:53:00 GMT

So, I am off doing a desktop migration for a customer of approximately 5,000 desktops. Vista you ask? No, not that sexy. Just trying to help one of our core telco customers get everyone off Windows 2000 onto XP. And by the way, get in infrastructure in place like BDD using OSD with SMS (it's cool to say all those acronyms really quickly) so they can be ready for Vista someday.

Unfortunately there are forces at work that will not allow us to do wipe-and-loads of desktops, like Microsoft recommends. So, me and Mike Vrabel from INS are having to work through an OS upgrade scenario. This causes us to upgrade a few corporate applications before we can upgrade the OS, because they just won’t work after the upgrade. Mike is primarily of storming the beach head of application remediation, while I work on unattend files and drivers.

I ran into some serious issues trying to upgrade drivers so that the NIC, Sound and Display would still work after an upgrade. Here were my approaches:

Upgrade Drivers while still in Windows 2000 to the XP version. Since most drivers are compatible, this should work. WRONG – the closest I got was the “Windows has found new hardware” once the box was upgraded to XP. Since this needs to be completely scripted, this isn’t an option
Use the Windows Unattend file to specify new drivers. I have used unattend files a million times before when building out big datacenters, so I have a really high level of confidence when it comes to unattend files. But, the upgrade scenario is different than a bare metal install or sysprep. It appears that Windows doesn’t care about the drivers I specify in the Unattend, and won’t even copy the ones I place in the \i386\$oem$\$1\drivers\yadayada folders. From what I can tell, these drivers are only copied during text mode (which does not occur in an upgrade), and it never looks in the unattend file for the drivers during an upgrade.
Upgrade the drivers once the upgrade is done. Ya, but how do you do it from the command line without user interaction? Well, DEVCON is your friend. Devcon.exe is a command line version of Device Manager, and the latest version is included in the Windows Server 2003 Service Pack 1 Support Tools.

The thing I learned about devcon, thanks to Scott McArthur from Microsoft Support, is that when you specify a device to update, use the “base” id. If you run:

Devcon driverdetails * > drivers.txt

You will get a list of all devices with their associate drivers. The devices are very specific like:

PCI\VEN_8086&DEV_100E&SUBSYS_01511028&REV_02\4&1C660DD6&0&60F0

There are other devices associated with this device, so if you try and update the specific device, it usually comes back with a “devcon failed”. You need to broaden the device path, and take the “base” device like this:

PCI\VEN_8086&DEV_100E

So the command that I end up running to update the drivers:

devcon updateni "C:\DRIVERS\Intel10.3\PRO1000\WS03XP2K\e1000325.inf" PCI\VEN_8086&DEV_100E

So hopefully this great little devcon tool will help you out, and make sure to be generic when specifying device drivers.

Changes to SharePoint 2003 Installs

mhass — Thu, 26 Jan 2006 00:54:00 GMT

I recently worked on another SharePoint project where we did unusual things with SharePoint. But, this time we actually used some of the core SPS functionality like search to index about 5,000 documents for a call center.

We built out Production last week and were doing some testing and discovered that search was broken. We could see in the Search Settings page that we were getting an error: “An error occurred attempting to connect to the index server”. What made this even stranger was that I had just done a full SPS Restore, including the index, which meant the topology was correct and communicating well.

Turns out that after Windows Service Pack 1, security changed and there are new requirements for the service account used for Application Pools. Here are the two changes:

1) The SPS service administrator account was not added in the admin group on the servers in farm. Originally, the SPS documentation stated this user needed to be a local “Power User”. This is documented in Q555309.

2) The SPS website application pool account was not added to the DCOM users group on all the servers in farm. Apparently this is not documented in a Q article (not sure why), but this is required for search.

Lastly, our search is still broken because I specified the name of the web site: https://myapp.mycorp.com. We hadn’t generated the SSL cert yet, so when the crawler tried to index the site, it doesn’t exist.

These provided to you free of charge as a public service. 8)

Watch out for sober attack on Jan 6, 2006

mhass — Thu, 05 Jan 2006 22:04:00 GMT

Stephen shares information about a virus attack timed for Jan 6, 2006

Get a third admin RDP session for FREE!!

mhass — Thu, 03 Nov 2005 21:16:00 GMT

Many administrators have missed that there is a "hidden" third connection that can be made to Windows Server 2003 servers. I often work with system administrators that are swearing in their cubes because the two admin connections are used up and they can't connect. I ask them why they don't just connect to the console session and boot them? Console session? Huh?

It's pretty easy, and I actually modify my RDP shortcuts to always connect to the console session so that I never conflict with anyone. Here's how it works:

MSTSC /v:<name or IP of the server> /console

This basically uses session 0 or the console session. So, if someone is standing in fron of the monitor connected to the server, it will say locked as you work away (much like Remote Desktop in Windows XP).

Small Business Server 2003 Upgrade from Hell

mhass — Thu, 03 Nov 2005 06:32:00 GMT

Last week, I decided to take a few vacation days and fly out to Ogden, Utah to help my wife’s old company upgrade from Small Business Server (SBS) 2000 to SBS 2003. They are a small, 10 person operation that manufactures high end ski and board apparel (www.descente.net or www.ridedna.com). The have a main office in downtown Ogden, a warehouse about 5 blocks away and a Canadian office in Vancouver.

The primary reason for the upgrade is that the president now resides in Canada, and his mailbox is back in Ogden. OWA is great, but it times out and the Exchange 2000 version was not the best and fastest interface. So, RPC/HTTP aka Outlook over the Internet is the perfect solution! BTW, his laptop runs Windows XP Japanese as well as Office 2000/Outlook 2003 Japanese but when I sit down it almost looks like I can read Japanese because I have almost everything memorized, I was often asked by other employees if I spoke Japanese.

Anyway, SBS has always meant in my mind “super tight integration of Microsoft Infrastructure products and a super easy GUI for non-computer type people to manage their business”. Of course another way of saying this is “I am going to hate using the SBS tools, please god give me normal MMC consoles”. I also thought, “what a simple upgrade this is going to be, should I fly out or can I do it over VPN if someone sites there on the phone with me”. Flying turned out to be a godsend.

First of all, support calls for Microsoft employees are not free. We either have to pay, or we get 3 Quick Assist calls that we can give to people. These are mainly meant to give to the guys that stop you and say, “Hey you work for Microsoft? I have Windows 98’ and I can’t print to this HP LaserJet II, can you help?” In this case, I needed all three Quick Assists and didn’t have any with me so I bummed a couple from coworkers.

Here are the highlights:

Support Call 1:

SBS upgrade halted, keeps insisting that “All domain controllers could not be contacted”. Some braniac when the system was first installed decided to implement a second DC on some old hardware. The hardware failed shortly after installation and AD was never cleaned up.

I made sure that all the roles were seized by their primary DC (they were). And I tried to delete the DC out of the domain, no luck. I used NTDSUTIL, ADSI Edit, DNS srv records, everything was gone, but it still insisted that “All domain controllers could not be contacted”.

Support ended up finding a way around this little check in the upgrade process and we were able to continue with the upgrade.

Support Call 2:

ISA 2004 is included in on the Technologies disk of SBS 2004 Premium Edition. They don’t’ advertise that, but I feel it is critical because the ISA 2004 GUI is worlds better than ISA 2000/Proxy Server 2.0.

During the install, ISA would bomb out with a .Net runtime error. It appeared that ISA completed installing itself and the MSDE for ISA, but it never installed the rules for SBS.

Turns out that the SBS wrapper around ISA 2004 forces it to utilize some of the SBS Admin tools that get installed. The Admin tools were never installed during the upgrade, and I never unselected them. To me, there must be a bug in the upgrade process or they purposely defaulted them not to be installed.

After installing SBS admin tools, I reran ISA setup and it went through fine.

Support Call 3:

After a long debate with SSL certs because for some reason their old SSL cert didn’t correctly move over to the Windows 2003 certificate store, I had to have the cert authority reissue it.

After reissue, I imported it into both IIS and used it for the web listener in ISA. It is a cert from www.xramp.com that has a public cert authority at very reasonable prices.

After OWA, OMA, and EAS were working, I decided to tackle RPC/HTTP for the president and their warehouse. By this time, I had flown back home and I built a Windows XP Virtual Server image and joined it to their domain to test RPC/HTTP. I VPN’d in from my Virtual Server image, joined the domain and got standard MAPI over TCP/IP working, cool! I disconnected the VPN, and setup the RPC/HTTP proxy settings on the client, and I new that the Outlook settings were correct and the certs were good, but it wouldn’t connect. It kept prompting me for login credentials.

Support traced the problem to the “Proxy Authentication Settings” being set to NTLM Authentication, for SBS apparently it must use Basic Authentication. The support tech also claimed that you can’t hit the “Check Name” button when you use RPC/HTTP, which I knew for a fact not to be an issue when you initially create the profile with TCP/IP. I tested this, and there isn’t an issue if you create a profile when you have the full MAPI TCP/IP connection, and later add RPC/HTTP.

Summary:

I am disappointed that this wasn’t as smooth as an update as I expected. Again, SBS is targeted at business of 100 or less people that probably don’t have a full time IT person, or have access to $295 per incident support from Microsoft.

In dealing with the SBS products, there seems to be a GUI that has simplified administrative tasks, but the underlying technology seems to still be hobbled together. Many of the products are wrapped or functionality is hidden/taken away, and don’t appear to be engineered from the beginning to work together on a single server. Overall, I highly recommend SBS 2003 especially since the premium edition includes ISA 2004. But, I think that we need to have the SBS teams sit in early on Windows, Exchange/Office, ISA, and SQL engineering design sessions and architect those products to operate better together on a single box to give SBS the reliability and ease of updates it deserves.

Microsoft Support from India

mhass — Fri, 28 Oct 2005 04:41:00 GMT

I admit it, I am not the all-knowing Microsoft consultant I should be. The real truth is, our support folks have the best tools to diagnose problems a lot faster than I can muddle through things.

I have had the occasion over the last few weeks to call Microsoft Support (PSS) several times with issues around SQL, AD, SharePoint and Small Business Server. Every single time, I was routed to someone in India. The support was phenomenal (as always), but what was more impressive was the language skills and how hip the support guys were on the other end of the phone, especially for being so far from the US. I think my expectation was that I would still receive good support, but a bit of a language barrier as well as some bias to culture.

Next time you call in, I bet you won’t be able where the person is located that is helping you fix your issue.

LCS and UPN's

mhass — Tue, 18 Oct 2005 17:03:00 GMT

I was messing around with LCS last night here in my lab. I had installed LCS a couple times and helped customers get up and running, but I am not an LCS expert. I was trying to duplicate a customer issue where I wanted to limit how certain users could talk to and add to their various chat client. Instead, I ran into an issue where I couldn't get any clients to sign in.

After messing around with SRV records, "LCS enabling" users in AD, and making sure the service was starting, I still couldn't log in with ANY users. I then started looking at what the SIP URI was defaulting to. I discovered that it was defaulting to one of the valid UPN names that I have in my domain. But, this wasn't the UPN for the domain the users were in.

Example:

Domain name extended for LCS: contoso.com

SIP URI: myuser@vanitydomain.com

Since the SIP URI doesn't match the domain extended for LCS, the user can't login. After adjusting the LCS user properties for my users to reflect the the "real" domain name (myuser@contoso.com) everything worked fine.

I am not sure why any valid UPN wouldn't work since the users are still part of the domain, but I assume it is just how LCS validates users to allow them to use the server.

IPSEC: Member to DC isn’t supported, but…

mhass — Sun, 25 Sep 2005 16:28:00 GMT

I stumbled across this issue a couple weeks ago, and Steve Riley clarified some of it for me. We have a really, really large project going with some new technology where securing the networks is a priority. We have run into the normal RPC challenges of limited port ranges because the firewall guys don’t want to open thousands of ports.

We also brought up the notion of running IPSec everywhere. I guess I had never realized that member server to DC IPSec is not supported by Microsoft. It is because of an issue with Kerberos: How can you use Kerberos to authenticate for IPsec if the computers haven't yet logged onto the domain? I never knew the official stance on this because I know customers that have implemented this with no problems. I have heard that Vista and the Longhorn Servers might have a fix for this so that it is officially supported, but just something to think about.

Don’t you just hate it when you have those circumstances that Microsoft doesn’t officially support something, but you know it works, and works well? We run into it all the time, and either have to back off into what is supported or sign a custom support agreement with Premier support, which can be a politically charged event.

Oh, and Steve made a good suggestion of forcing RPC authentication since most RPC based attacks are anonymous. Good suggestions, I believe this can be done rather easily with GPO, so we will take a look at it.

Zotob, a hard and unfortunate lesson (again) for Enterprises

mhass — Thu, 18 Aug 2005 01:00:00 GMT

Unless you have been on vacation on the moon for the last few days, you know that there is new exploit in the wild that mainly affect Windows 2000 and pre SP2 Windows XP OS’s called Zotob (and variants). This worm is hitting some of the same big name companies that were crippled by CodeRed, Sasser and Blaster.

In the years since those nightmares, there have been a lot of great tools that have come out to help avoid these nightmares. I can’t believe that some of the big enterprises (some of them are my customers) don’t have a better strategy or mandatory policy to update desktops and servers with security updates quickly. Microsoft and others have been preaching for at least the last 3 years that it is only a matter of time for a 0 day exploit. While Zotob is not a zero day exploit, it is close.

Everyone knows about “patch Tuesday”, including malware authors. You absolutely have to have a patch strategy these days to push security updates immediately or within 24 hours. This means 1000’s of boxes in some cases.

If you are an enterprise, don’t just stand there, get something! Windows/Microsoft Update, SMS, SUS, WSUS, ONiPatch, Hercules, etc it doesn’t matter!! Some you pay for, some are free, but just do it. I guarantee this isn’t the last time, and the next one might be even bigger.

I don’t want to get into a debate about how many security flaws Microsoft has over distributions of Linux or IIS vs Apache. You need a strategy for all of these systems.

If you have a single PC, home network with a couple computers or if you have an enterprise with 10,000 of pc/servers get a plan on how you get and apply updates.

BTW, don't get me started on the Media. Every single time I watch the "tech" portion of our local news channels I want to call them up and make at least 3 corrections (most of the time non-Microsoft stuff). They can't even get mobile phones right.

A great blog to subscribe to is the Microsoft Security Response Center Blog!.