Kevin Holman's OpsMgr Blog : active directory

AD Replication monitoring – getting lots of slow replication errors?

kevinhol — Fri, 21 Aug 2009 04:38:52 GMT

I recently hit this with a customer – and feel it needs a bit more exposure.

The guide that ships with the current ADMP version 6.0.6452.0 leaves out a LOT of information on how to properly set this up…. specifically – how to make any overrides to the replication values, which is extremely common.

Even after adjusting the overrides as one would think to be correct, you may still continue to get alerts about no or slow replication.

I recommend checking out Jimmy Harper’s blog post on this topic, it really covers the issue and how to correctly override the rules. It turns out there are FOURTEEN rules that you should modify, if you want to change this – for EACH OS version that you run domain controllers on:

Check it out:

http://blogs.technet.com/jimmyharper/archive/2009/05/20/configuring-or-disabling-replication-monitoring-in-the-active-directory-management-pack.aspx

AD Replication is occurring slowly (there are three rules with this name)
One or more domain controllers may not be replicating (there are three rules with this name)
DC has failed to synchronize naming context with its replication partner (there are three rules with this name)
All of the replication partners failed to replicate.
AD Replication Performance Collection - Metric Replication Latency
AD Replication Performance Collection - Metric Replication Latency:Minimum
AD Replication Performance Collection - Metric Replication Latency:Maximum
AD Replication Performance Collection - Metric Replication Latency:Average

DNS MP – Noisy resolution time alerts, and how to deal with them

kevinhol — Tue, 24 Feb 2009 23:52:55 GMT

This is a problem in the 6.0.6480.0 version of the DNS MP.

You will likely see a lot of DNS Resolution Time alerts popping into your console – then disappearing.

This is because these alerts are generated by a monitor, which is frequently changing state. These alerts get auto-resolved when the monitor flips back to healthy status, by design.

The root cause of the problem, if often that the server is busy when we run our script to check the local DNS resolution response… and the default threshold is set to 1 second.

Even in some of the best DNS environments, with good hardware… we will find DNS servers on Domain Controllers can get busy… and this is compounded by SCOM running multiple scripts at the same time – from the ADMP and DNS MP… sometimes we cannot return results in less than 1 second.

The best thing to do – is to chart out your current environment, using the provided performance views in the MP…. and adjust this moniotr for your servers:

What I can see – is that my Server 2003 DC/DNS server, with only 1 zone, but running on a PIII 933 mhz CPU, with 512mb of RAM…. is taking a baseline of 2-3 seconds. I will override this monitor for this SERVER, or for ALL 2003 DNS servers… to be 5 seconds.

Granted – our expectation is that our DNS servers can respond to a DNS query faster than 5 seconds – but this number is relative… due to how OpsMgr is collecting it. So the goal here, is to look at what is normal when the server is functioning well, establish that as our baseline, and set the threshold just above it.

Now – my Server 2008 DC/DNS server, which has 1GB of ram, and is a VM on very fast disk, and has a better CPU available, has a baseline of .2 seconds… so I will leave this monitor alone, since it is obviously not changing state so frequently.

When a real problem arises, load increases, or DNS is performaing poorly, we will be alerted – because we will breach our *baseline*.

DNS MP External Resolution Monitor always in a critical state?

kevinhol — Tue, 24 Feb 2009 23:03:49 GMT

This is a problem in the 6.0.6480.0 version of the DNS MP.

This MP includes a monitor to inspect the DNS server’s ability to resolve external domains.

The problem is – that the moniotr defaults to inspecting “www.microsoft.com” when it should use a base domain, like “microsoft.com”.

You will find all of your DNS servers in this state – if you dont override it. This is true for 2003 and 2008 servers:

To resolve this – simply override the monitor, for ALL OBJECTS OF TYPE, and change the Host string from “www.microsoft.com” to “microsoft.com”. If your DNS servers cannot resolve external addresses by design – disable this monitor.

Getting and keeping the SCOM agent on a Domain Controller – how do YOU do it?

kevinhol — Fri, 20 Feb 2009 21:27:57 GMT

I’d like to hear some community feedback on this….

In OpsMgr – deploying a SCOM agent to a DC often presents companies with a bit of a challenge. The reason is – in order to install software to a DC and manage it – we need rights on the DC to accomplish this. These rights are needed, anytime we are going to deploy an agent, hotfix an agent, or run a repair on a broken agent to keep the agent healthy.

When we push agents from the console, the default account used to perform the push is the Management Server Action Account. If this account does not have Domain Admin rights – the push will fail to a DC, with an Access Denied. We do allow the option to type in temporary (encrypted) credentials, which are used to deploy the agent, one time, and then are discarded. See the image below:

Here is a list of the most common options I have observed, in place at customer sites… and potential custom options that can be developed. I’d be interested in any community feedback on any options you are using, that I dont cover or haven't seen before.

1. Grant the Management Server Action account Domain Admin or Builtin\Administrators.

a. Not recommended as a best practice, this gives rights to the MSAA that are not required for day to day activities.

b. Con - SCOM Admins now control a domain admin account.

2. Grant a SCOM Administrator a special domain account, for this purpose, that is a domain admin.

a. This allows us to track the actions of that SCOM admin, when he/she uses that special privileged account.

b. That SCOM admin will be able to do repairs, hotfixes, and deployments for DC’s.

c. Con – Domain Admin teams often wont delegate these rights as they are tightly controlled.

3. The SCOM admin team delegates console based agent management to a Domain Administrator for DC agent health.

a. The domain admin must become a SCOM Admin, and therefore could potentially hurt the SCOM environment.

b. Pro – the admins in charge of the DC’s now have full responsibility to keep the agents healthy.

c. Con – the Domain Admins might not understand components of SCOM, and create something that impacts the monitoring environment.

4. The SCOM admin team must partner with the Domain Admin team, and have the Domain Administrator type in his credentials any time the SCOM administrator needs to deploy/hotfix/repair an agent on a domain controller.

a. This is a bit more labor intensive… because the SCOM admin must wait for a domain admin to be available to work on DC agents, but tight security boundaries are maintained.

5. All DC based agents will be manually installed/updated/repaired.

a. This is very common, when the two teams do not trust each other. The Domain Admin team is now required to manually deploy agents to domain controllers, and keep them up to date, and healthy.

6. Use a software deployment tool already in place to deploy/update/repair agents.

a. If a software deployment tool is already in place on DC’s, like SMS/SCCM, you can create packages to deploy, hotfix, and repair agents, similar to your patching of the OS today.

7. Customized solution: Create a Run-As account that is a domain admin, one time, for use in agent deployment/repair.

a. This involves the domain admin typing in credentials ONCE, into a RUN-AS account, which is stored securely and encrypted in the SCOM database.

b. This run-as account can be associated with a run-as profile, which is used by a custom task, which will remotely deploy the agent to the domain controller. This task will execute under the security context of the privileged run-as account.

c. The benefit is that the domain admin gets to control the password for this account, the SCOM admin does not need to know the account credentials.

d. The downside, is that this run-as account could potentially be leveraged by some other workflow, if a SCOM admin intentionally misused it…. Similar to solution #2 above.

e. This is just an idea I had – curious if anyone has already developed a solution like this?

Helper Objects are not copied to gateways

kevinhol — Fri, 11 Jul 2008 23:47:00 GMT

Something I noticed today..... when you deploy a gateway server - the helper object oomads.msi was not copied to the local \AgementManagement directory for agent push.

This means, that if you have a DC in the untrusted forest, managed by a gateway, that Oomads will not get copied or installed automatically. You will need to manually copy and install Oomands on any DC's you will monitor in the untrusted forest.

Active Directory Integration - How it works

kevinhol — Thu, 27 Mar 2008 00:22:00 GMT

Steve Rachui wrote a great post on this - which goes a little deeper than some of the other documents and blogs presently out there:

http://blogs.msdn.com/steverac/archive/2008/03/20/opsmgr-ad-integration-how-it-works.aspx

I want to add one comment:

Q: "How often does the agent poll active directory if it doesn't find policy when the machine first joins the domain?"

A: The agent will poll AD to look at the SCP's referenced above, when the Healthservice first starts up. Then - it will poll, by default, every hour from that point forward, looking in AD to see if it has information about management groups to join.

So - the RMS runs the AD assignment rules once per hour to update AD containers.... and the agent checks those containers once per hour. Theoretically - the maximum time from when you add an agent assignment rule, to the time the agent picks this up - should be 2 hours. Sometimes it can take a little longer, due to a modification of an assignment rule on the MS is really a delete action, then a write action.

The time interval that an agent inspects AD for policy is configurable as well:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\HealthService\Parameters\ConnectorManager

Create a DWORD value named “ADPollIntervalMinutes” to the period you wish for the healthservice to check AD for new config. Without setting this key yourself it defaults to 60 (minutes).

Creating a Group based on OU (Organizational Unit) in Active Directory

kevinhol — Tue, 05 Feb 2008 19:02:08 GMT

Here is a really cool feature of Opsmgr: the ability to create groups easily based on any discovered attribute of an object.

OU is something that is part of the Windows Computer object discovery. If you examine a state view – you will see in the details pane discovered information… and OU is there. Typically this means we can likely use that object (Windows Computer) and OU will be a discovered attribute of that.

This also means is we personalize a Windows Computer based state view – we can add OU:

To use a grouping, create a group, add Windows Computer object, and then a rule based on OU:

A right click – view group members reveals:

System Center Operations Manager SDK service failed to register an SPN

kevinhol — Fri, 14 Dec 2007 01:52:00 GMT

System Center Operations Manager SDK service failed to register an SPN

Have you seen this event in your RMS OpsMgr event logs?

Event Type: Warning

Event Source: OpsMgr SDK Service

Event Category: None

Event ID: 26371

Date: 12/13/2007

Time: 2:58:24 PM

User: N/A

Computer: RMSCOMPUTER

Description:

The System Center Operations Manager SDK service failed to register an SPN. A domain admin needs to add MSOMSdkSvc/rmscomputer and MSOMSdkSvc/rmscomputer.domain.com to the servicePrincipalName of DOMAIN\sdkaccount

This seems to appear in the RC1-SP1 build of OpsMgr.

Every time the SDK service starts, it tries to update the SPN’s on the AD account that the SDK service runs under. It fails, because by default, a user cannot update its own SPNs. Therefore we see this error logged.

If the SDK account is a domain admin – it does not fail – because a domain admin would have the necessary rights. Obviously – we don’t want the SDK account being a domain admin…. That isn’t required nor is it a best practice.

Therefore – to resolve this error, we need to allow the SDK service account rights to update the SPN. The easiest way, is to go to the user account object for the SDK account in AD – and grant SELF to have full control.

A better, more granular way – is to only grant SELF the right of modifying the SPN:

Run ADSIEdit as a domain admin.
Find the SDK domain account, right click, properties.
Select the Security tab, click Advanced.
Click Add. Type “SELF” in the object box. Click OK.
Select the Properties Tab.
Scroll down and check the “Allow” box for “Read servicePrincipalName” and “Write servicePrincipalName”
Click OK. Click OK. Click OK.
Restart your SDK service – if AD has replicated from where you made the change – all should be resolved.

To check SPN's:

The following command will show all the HealthService SPN's in the domain:

Ldifde -f c:\ldifde.txt -t 3268 -d DC=DOMAIN,DC=COM -r "(serviceprincipalname=MSOMHSvc/*)" -l serviceprincipalname -p subtree

To view SPN's for a specific server:

"setspn -L servername"