Today's tip comes from Michael Sadoff, one of the top Senior Support Escalation Engineers in our North Carolina office. If you're seeing mutual authentication fail and notice that a node in an RMS cluster registers the ServicePrincipalName (SPN) for the Health Service with the physical node's computer account then this one's for you:
Issue: In System Center Operations Manager 2007, a node in an RMS cluster registers the servicePrincipalName (SPN) for the Health Service with the physical node's computer account. This is a problem because the SPN must be registered with the account of the RMS cluster computer. When the SPN registration is duplicated or the registration is with the wrong computer account, mutual authentication fails. This causes the RMS and agents to go into gray state.
Additionally, when the RMS cluster group is active on the affected node, the service state folders for the Config Service, SDK Service and Health Service are on the node's local drive instead of the shared cluster drive.
Cause: This can occur when the ManagementServerConfigTool.exe has not been run successfully on an RMS cluster node.
When you run ManagementServerConfigTool.exe with the InstallCluster or AddRMSNode argument, the tool creates a registry value named HealthServiceVirtualHost in the following registry sub-key:
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft Operations Manager\3.0
When the Health Service starts, it tries to read the HealthServiceVirtualHost registry value. If that value exists, Health Service skips SPN registration and logs the following Information event in the Native trace log:
0 00000000 7592.8548::10/15/2008-15:32:47.019 [MOMConnector] Information CConnectorSolutionSharedState::TryRegisterServiceSpn(ConnectorSolutionSharedState_cpp2788)Received value for GetHealthServiceVirtualHostName. Assuming that this is clustered RHS so not trying to register SPN.
ManagementServerConfigTool.exe creates the HealthServiceVirtualHost registry value using the name specified in the /vs argument. It also changes the service state path for the OpsMgr services to the drive specified in the /Disk argument. That way, the service state folders will be on the shared cluster drive instead of the local installation directory.
For more information about installing an RMS cluster and using ManagementServerConfigTool.exe, refer to the following topic in the OpsMgr 2007 Deployment Guide:
Deploying a Root Management Server on a Windows Cluster in Operations Manager 2007 http://technet.microsoft.com/en-us/library/bb432140.aspx
Resolution: Run ManagementServerConfigTool.exe on the affected cluster node with the AddRMSNode argument. This will configure the HealthServiceVirtualHost registry value and prevent the incorrect SPN registration. It will also configure the clustered OpsMgr services to use the shared cluster drive for storage state instead of a local drive. To do this, use the following steps:
1. Using Cluster Administrator, move the RMS cluster group to the affected node. 2. Take the Config Service, SDK Service and Health Service resources offline. 3. Copy the latest version of ManagementServerConfigTool.exe from the OpsMgr source media (in the SupportTools folder) to the System Center Operations Manager 2007 installation folder. 4. From a command prompt, run the following command:
ManagementServerConfigTool.exe AddRMSNode /vs:<VirtualServerNetbiosName> /Disk:<VirtualServer Disk Resource>
In the above command, VirtualServerNetbiosName is the Network Name resource in the RMS cluster group. The value you enter for VirtualServerNetbiosName must be the value that appears in the Name text box located on the Parameters tab of the Properties dialog box for the Network Name cluster resource. VirtualServerDiskResource is the disk resource allocated to the RMS cluster group . The Disk location can be found by on the Parameters tab of the Properties dialog box for the Disk Resource.
5. After this command completes, verify that the HealthServiceVirtualHost registry value exists in the following registry sub-key:
6. Use adsiedit.msc or setspn.exe to remove the Health Service SPN from the computer account for the physical node. Make sure that the SPN is registered correctly with the account of the RMS cluster computer.
7. Bring the OpsMgr services online and verify that the invalid Health Service SPN registration does not recur.
J.C. Hornbeck | Manageability Knowledge Engineer