System Center: Operations Manager Engineering Team Blog

MVP Cameron Fuller Presents “Operations Manager Evolution: Taking Your Operations Manager to the Next Level”

2013-05-24T01:23:25Z

Earlier this week MVP Cameron Fuller presented in the Windows Server/System Center MVP Pro Speaker Series webcast on the topic of “Operations Manager Evolution: Taking Your Operations Manager to the Next Level”. If you didn't catch it live, please check out the recorded version below. You can access the PowerPoint file from here. You can also add the remaining sessions in the series to your calendar from here. We’ll be announcing the new series of topics soon!

New Video: System Center 2012 Operations Manager Dashboards: Visualizing Alert Data

2013-05-21T19:01:15Z

Richard Pesenko, Microsoft Senior Support Escalation Engineer on our Operations Manager support team, has a new video out that walks you all through visualizing alert data in OpsMgr 2012. It’s only 10 minutes long so it’s a great way to quickly become more efficient in how you handle your alerts.

J.C. Hornbeck | Knowledge Engineer | Microsoft GBS Management and Security Division

Get the latest System Center news on Facebook and Twitter:

System Center All Up: http://blogs.technet.com/b/systemcenter/
System Center – Configuration Manager Support Team blog: http://blogs.technet.com/configurationmgr/
System Center – Data Protection Manager Team blog: http://blogs.technet.com/dpm/
System Center – Orchestrator Support Team blog: http://blogs.technet.com/b/orchestrator/
System Center – Operations Manager Team blog: http://blogs.technet.com/momteam/
System Center – Service Manager Team blog: http://blogs.technet.com/b/servicemanager
System Center – Virtual Machine Manager Team blog: http://blogs.technet.com/scvmm

Windows Intune: http://blogs.technet.com/b/windowsintune/
WSUS Support Team blog: http://blogs.technet.com/sus/
The AD RMS blog: http://blogs.technet.com/b/rmssupp/

App-V Team blog: http://blogs.technet.com/appv/
MED-V Team blog: http://blogs.technet.com/medv/
Server App-V Team blog: http://blogs.technet.com/b/serverappv

The Forefront Endpoint Protection blog : http://blogs.technet.com/b/clientsecurity/
The Forefront Identity Manager blog : http://blogs.msdn.com/b/ms-identity-support/
The Forefront TMG blog: http://blogs.technet.com/b/isablog/
The Forefront UAG blog: http://blogs.technet.com/b/edgeaccessblog/

Local Resolution Time monitor in the OpsMgr DNS Management Pack may show unexpected results

2013-05-15T16:02:16Z

Arun Kumar | Senior Support Engineer

When viewing the Local Resolution Time monitor in the DNS Management Pack for Operations Manager, you may notice that the response time reported does not reflect your own real world results. For example, I recently worked with a customer where the Local Resolution Time view under Performance showed around 12 seconds as the response time for the DNS servers even though the response times actually observed were almost instantaneous.

In the above screen shot you can see the response time was shown to be around 12 seconds except for a little portion at the end where is it showed a response time of essentially 0. However, when we run nslookup and resolve a name, the DNS server in question would respond immediately to all queries. So where are we getting this 12 seconds response time from? Let me give you a little background on how this works and later I will talk about the cause of this behavior.

Collect local resolution time is the rule that collects this information from the DNS servers and by default this rule runs once every 15 minutes. There isn’t much detail concerning how exactly it’s trying to get the data in the rule itself but if you go through the DNS management pack guide it mentions the scripts that are used. The script that is used for this rule is Nslookupalltest.js.

To see how this script is used, I took diagnostic tracing (see KB942864 - How to use diagnostic tracing in System Center Operations Manager 2007 and in System Center Essentials) and went through the TracingGuidsScript.log file that was created. By doing so we found that the script was being run with the following parameters:

cscript.exe with command line '"C:\Windows\system32\cscript.exe" /nologo "NslookupAllTests.js" "dc1.contoso.com" "192.168.100.101,fe80::1452:6556:3ba0:6465" "a" "false" "none" "360" "2" "false"

I then ran the same command manually on the DNS server in question and found the following information:

<DataItem type="System.PropertyBagData" time="2013-04-29T11:47:11.5453960-04:00" sourceHealthServiceId="2BF76ID7-7810-89E7-59D5-000EAT0C66FF">
< Property Name="SuccessCount" VariantType="3">2</Property>
< Property Name="NonAuthoritativeCount" VariantType="3">2</Property>
< Property Name="FailureCount" VariantType="3">0</Property>
< Property Name="BestHost" VariantType="8">dc1.contoso.com</Property>
< Property Name="BestServer" VariantType="8">192.168.100.101</Property >> Here we query the IPv4 address of the DNS
< Property Name="BestTime" VariantType="5">0.125</Property >> Notice that the best time discovered is 0.125 seconds
< Property Name="WorstHost" VariantType="8">dc1.contoso.com</Property>
< Property Name="WorstServer" VariantType="8">fe80::1452:6556:3ba0:6465</Property >> Now we query the IPv 6 address
< Property Name="WorstTime" VariantType="5">11.715</Property >> Notice here that the time discovered is 11.715 seconds
< Property Name="FailingPairs" VariantType="8"></Property></DataItem>

So the issue appears to be only when we are trying to resolve the name using the IPv6 address of the DNS server. So what is the delay when the name is resolved via the IPv6 address?

What happens is that the script queried for both the A record and the PTR record for the give name (e.g. dc1.contoso.com). This also goes through DNS client side devolution before the actual name is resolved. So when the A record and the PTR record of dc1.contoso.com is queried using the IPv4 address of the DNS server, we get a successful response for both the queries because the DNS server is configured with the respective forward and reverse lookup zones and both the zones have the necessary records.

On the contrary, when the script queried for the A and PTR records for the name dc1.contoso.com against the IPv6 address of the DNS server, the server was able to find the A record but was not able to find the IPv6 PTR record for the same because a reverse lookup zone had not been configured. Since the use of IPv6 is not very widespread yet, having no IPv6 reverse lookup zone is not uncommon.

And if you are wondering how the script knows to append the IPv6 address of the server in the command, go to the Server State view in the DNS management pack and select the server in question and see the detail view as shown in the dialog below. You should be able to see the listening IP addresses of the DNS server, and if you see the IPv6 address there then it will be used.

So how do we fix this so that our graph shows accurate response time? There are two ways of getting around this situation.

1. Configure the DNS service so that it listens only on the IPv4 address. To do this select “Only the following IP addresses:” in the dialog above and choose only the IPv4 address from the list.

Note that disabling IPv6 at the OS level is not recommended as there is no way for us to predict how this may effect your environment. Also, if IPv6 is disabled on Windows Vista, Windows Server 2008 and later versions, some components will not function as expected. See Support for IPv6 in Windows Server 2008 R2 and Windows 7 for more information.

2. The preferred fix for this is to create a reverse lookup zone as well as the associated PTR records on the DNS.

One last note: You can run into the same issue If you don’t have a reverse look up zone for the IPv4 address of the DNS server as well. This is not isolated to IPv6 but rather is a function of the lack of a proper reverse lookup zone on the DNS.

Arun Kumar | Senior Support Engineer | Microsoft GBS Management and Security Division

Get the latest System Center news on Facebook and Twitter:

Windows Intune: http://blogs.technet.com/b/windowsintune/
WSUS Support Team blog: http://blogs.technet.com/sus/
The AD RMS blog: http://blogs.technet.com/b/rmssupp/

App-V Team blog: http://blogs.technet.com/appv/
MED-V Team blog: http://blogs.technet.com/medv/
Server App-V Team blog: http://blogs.technet.com/b/serverappv

MP Blog: Exchange 2013 Management Pack released

2013-05-14T17:10:00Z

We are excited to announce that the Exchange team has released the Exchange 2013 Management Pack. You can find the Management Pack here and the guide here. The guide walks through the details of deploying, configuring and using the Management Pack.

In this blog, I am focusing on describing the benefits of the Management Pack, as well as some of the internals of how it works, and how it is different from previous Exchange Management Packs.

My target audience for the blog post is the Operations Manager administrator.

Management Pack Scope and Benefits

The Exchange team takes monitoring seriously, and they have created a pretty comprehensive Management Pack in terms of monitoring coverage, with the primary focus being MTTR, i.e. reduced downtime for your Exchange environment.

The Management Pack contains about 75 monitors that cover Exchange component health (such as Hub Transport health), customer touch point health (such as “is OWA working”), clustered scenarios, as well as dependencies monitoring (“is Active Directory healthy”). Monitoring covers primarily availability and performance scenarios.

With this Management Pack, the Exchange team is sharing knowledge and experience from managing the Office 365 environment, with its extreme demands on availability and performance. This means a lot has been done for you already, i.e. the relevance of alerts is high. Basically Exchange developers get notified 24x7 on this same set of monitors, and (similar to you) they are not interested in being awakened in the middle of the night for some irrelevant or non-actionable alerts.

The Exchange team is also sharing knowledge online for the monitors, so each monitor has a link to online knowledge.

The Exchange 2013 product comes with monitoring “built in”, more on that below. This means that Exchange itself has means to detect and try to automatically recover from availability and performance issues, before an operator is notified. This means reduced alert noise, as well as reduced administrative overhead for the Exchange product itself. In concrete terms, an Exchange server may detect an issue, then try for some time to fix it automatically. Only after the automatic recovery attempts have failed is the operator notified via an alert.

What’s in the Management Pack?

The Management Pack is very simple. It contains a handful of classes, 3 views, and about 75 monitors described here. There are also some dependency monitors.

All the monitors are simple event-based monitors using events in the Microsoft-Exchange-ManagedAvailability/Monitoring event log, logged by each Exchange server. So, each Exchange server is responsible for monitoring itself and its health.

In terms of scalability, this means this Management Pack will have a low impact to your Operations Manager environment. You will not require a separate Management Group for this MP for scalability purposes.

Wait, where are the monitors?

If you look at the Management Pack in some tool like MP Viewer, as I am sure some of you will, you will discover that there are not 75 monitors in the Management Pack. This is because the Management Pack has logic to dynamically determine the set of monitors by communicating with the built-in monitoring features of Exchange. More on that below. The way to see the monitors is by installing the Management Pack in an Operations Manager environment with monitored Exchange servers and using Health Explorer (you can also see them listed in the Management Pack Guide).

What’s not in the Management Pack?

If you are familiar with the Exchange 2010 Management Pack, you know that it had a service called the correlation engine that ran on the Root Management Server. It basically correlated health data from all monitored Exchange components. In the Exchange 2013 Management Pack, the correlation engine is no longer used. Each monitored Exchange server is responsible for monitoring its own health, and simply reports this via the Operations Manager agent. There is a little bit of roll-up going on, from Exchange server to Organization health. There are no special components running on the Operations Manager Management Servers.

The Exchange 2010 Management Pack had tens of classes, leading to a pretty complex health model and many object instances being created. The Exchange 2013 Management Pack is very simple. It has only a few classes and should have a very low impact on Operations Manager in terms of instance space.

There are no performance counters collected by this Management Pack (as mentioned above, the monitoring does cover performance scenarios). However, all Exchange PerfMon counters are still available. It should be simple to create your own performance collection rules, if you do require them.

This means that by default there is way less pressure on the Management Group compared to the Exchange 2010 Management Pack, since there are no performance counters to store. But, what about reporting? There are also no reports in this Management Pack. The primary focus of this version of the Management Pack is reduced MTTR. However, you can use some of the built-in Operations Manager reports (such as the Health report) to track organization availability, or define SLAs against the Organization. Exchange is also looking for your feedback on this topic, feel free to comment on this post and I will pass that information along.

How does the Management Pack work?

Exchange 2013 has a built in monitoring engine called Managed Availability. It runs on every Exchange server. It contains logic for how to determine Exchange health. It detects issues, automatically performs recoveries (Exchange calls these “responders”) and ultimately notifies operators of issues, if the recoveries were not successful. The purpose of this of course, is high availability. Managed Availability is explained in more detail here.

Notification/Alerting to Operations Manager is handled via events, so the Management Pack has a set of simple event monitors that trigger based on these events. Events are logged to the Microsoft-Exchange-ManagedAvailability/Monitoring event log.

As an Operations Manager admin, how should I work with this Management Pack?

First of all, this Management Pack should be simple to implement from an Operations Manager perspective. There are no special components to install on Management Servers, you do not need to worry about the impact to the Management Group in terms of database size, instance space, Management Server workload etc. You should be fine just importing this MP into your existing environment also for large Exchange deployments (as usual, we do recommend gradual deployment just in case). The Management Pack Guide includes a chapter on deployment, as usual you will need to enable Agent Proxy when you install the Operations Manager agents on Exchange 2013 servers.

Also, the Exchange team has been using the same monitoring logic in the Office365 environment. Normally, few changes to the Management Pack should be required.

However, if you should require some adjustments to the Management Pack, you will need to work together with your Exchange administrators.

If you want to disable some monitor, you can just create an override in Operations Manager as usual. This is straightforward.

If you want to change a threshold for some monitor, this is done in the Exchange Managed Availability engine via PowerShell cmdlets. This does not involve Operations Manager at all. The Exchange Management Pack Guide walks through this scenario in some detail here. Since this kind of override is a modification of Exchange behavior, this kind of override is most commonly done by the Exchange administrator.

Also, Exchange Cumulative Updates may contain new or updated monitoring logic. These should be reviewed together with your Exchange administrators to determine the impact of that updated logic.

In terms of interoperability, this Management Pack does not upgrade the Exchange 2010 Management Pack, this is a completely new MP. It is possible to run these Management Packs side-by-side as you upgrade your Exchange environment from 2010 to 2013.

What’s the update mechanism for this Management Pack?

The Managed Availability feature is built in to Exchange 2013. It contains the logic for how to detect issues and how to recover from them. The results of that is then reported to the event log for Operations Manager to pick up via event monitors. This means that updates in terms of Managed Availability monitors are shipped with Exchange Cumulative Updates.

As mentioned previously, the Exchange 2013 Management Pack has functionality to automatically detect these changes, so typically no Management Pack update is required.

Questions?

Let me know if you have any questions, I will answer the ones I know about and pass other questions back to the Exchange team.

OpsMgr Support Tip: Email notification may not work as expected if a time range is configured only with an exclusion

2013-05-14T14:30:00Z

Vasanth Kumar | Support Engineer

Hey there again! This is Vasanth from the Operations Manager team and today I have a quick tip for you all that I found while working on a recent issue.

When using Systems Center Operations Manager (any version of OpsMgr 2007 or OpsMgr 2012), if you configure Notification Recipients (so called up to OpsMgr 2007 SP1) or Subscribers (as called in OpsMgr 2007 R2 and OpsMgr 2012) with the option of "Only send notification during the specified times" and you configure it with only an exclude range (like "Except from" with a time range of, say 1:00am to 7:00am for example), email notifications fail altogether.

In the Operations Manager event log you will also see the event below:

Log Name:      Operations Manager
Source:        Health Service Modules
Date:          <date> <time>
Event ID:      31501
Task Category: Notification
Level:         Warning
Keywords:      Classic
User:          N/A
Computer:      <RMS Server Name>
Description:
No primary recipients were specified for the notification or all recipients are unavailable according to the schedules. Notification will not be sent

One or more workflows were affected by this.

Workflow name: Subscription431bd31a_37fc_4375_9cd7_29bb8a11c600
Instance name: Alert Notification Subscription Server
Instance ID: {E07E3FAB-53BC-BC14-1634-5A6E949F9230}
Management group: <Mgmt Group Name>

So what’s going on? Since we configured it to send only during specified times and then only included an excluded range, the system does not recognize any valid time that emails can be sent so it never sends any.

So to keep this from happening, or to fix it if you’ve already found yourself in this situation, you need to configure the Recipient or Subscriber first with an included time range, then configure your excluded time range.

For example, first configure you include Range like Date | Range = Always | Time Range = All Day | Weekdays = Week days. then once you have that, configure your Exclude Range like Data Range = Always | Time Range = Excluding 1:00am to 7:00am | Weekdays = Week Days. The screen shot below shows this configuration.

I hope this helps!

Vasanth Kumar | Support Engineer | Microsoft GBS Management and Security Division

Get the latest System Center news on Facebook and Twitter:

Windows Intune: http://blogs.technet.com/b/windowsintune/
WSUS Support Team blog: http://blogs.technet.com/sus/
The AD RMS blog: http://blogs.technet.com/b/rmssupp/

App-V Team blog: http://blogs.technet.com/appv/
MED-V Team blog: http://blogs.technet.com/medv/
Server App-V Team blog: http://blogs.technet.com/b/serverappv

KB: Operations Manager agents with teamed network adapters fail to be discovered and monitored

2013-05-13T16:47:45Z

Just wanted to let you know about a new Knowledge Base article we published today, This one talks about an issue where System Center Operations Manager 2007 R2 agent computers that utilize NIC teaming may not be discovered and log the following errors:

Log Name:       Operations Manager
Source:            Health Service Script
Date:               <date>
Event ID:          4001
Task Category: None
Level:              Error
Keywords:       Classic
User:              N/A
Computer:      <computerName>
Description: Microsoft.Windows.Server.NetwokAdapter.BandwidthUsed.ModuleType.vbs : The class name 'Win32_PerfFormattedData_Tcpip_NetworkInterface Where Name ='VLAN100:HP Network Team _1'' did not return any valid instances. Please check to see if this is a valid WMI class name.. Invalid class

=====

The process started at 09:44:43 failed to create System.PropertyBagData, no errors detected in the output. The process exited with 0
Command executed: "C:\Windows\system32\cscript.exe" /nologo "Microsoft.Windows.Server.NetwokAdapter.BandwidthUsed.ModuleType.vbs"
Working Directory: C:\Program Files\System Center Operations Manager 2007\Health Service State\Monitoring Host Temporary Files 23\\
One or more workflows were affected by this.
Workflow name: Microsoft.Windows.Server.2008.NetworkAdapter.PercentBandwidthUsedTotal.Collection
Instance name: Local Area Connection
Instance ID: {}

You can find the complete article and get all the details here:

KB2847767 - Operations Manager agents with teamed network adapters fail to be discovered and monitored (http://support.microsoft.com/kb/2847767)

J.C. Hornbeck | Knowledge Engineer | Microsoft GBS Management and Security Division

Get the latest System Center news on Facebook and Twitter:

Windows Intune: http://blogs.technet.com/b/windowsintune/
WSUS Support Team blog: http://blogs.technet.com/sus/
The AD RMS blog: http://blogs.technet.com/b/rmssupp/

App-V Team blog: http://blogs.technet.com/appv/
MED-V Team blog: http://blogs.technet.com/medv/
Server App-V Team blog: http://blogs.technet.com/b/serverappv

The Launch of the System Center Operations Manager (SCOM) Management Pack for Amazon Web Services (AWS)!

2013-05-09T22:57:55Z

Solution – The AWS Management Pack for Operations Manager

Whether they are on-premises or in the AWS cloud, this SCOM Management Pack (MP) allows organizations running their Microsoft-based workloads at AWS to monitor their AWS resources directly in the Operations Manager console, essentially from a “single pane of glass”. This project was driven by demand from the Microsoft EPG sales field and most importantly joint AWS and Microsoft very large enterprise customers. These companies were also our key beta customers for the AWS Management Pack. We enlisted the fantastic development team at VIAcode, a key SI partner to develop the MP for AWS.

A couple of things make this Management Pack unique. First, it is the first MP of its kind to be able to separate the computer (Operating System) from the AWS Instance (Virtual Machine). Secondly, this creates a logical monitoring and reporting mechanism that can intelligently identify where a problem or error state exists, either in the AWS cloud or the server OS/application running within AWS.

What does the Management Pack do?

Microsoft customers who have existing Microsoft and Linux based workloads can view and monitor their on-premises and AWS resources together in a SCOM management console. The management pack can monitor EC2 instances (Windows and Linux), Elastic Block Store (EBS) volumes, Elastic Load Balancing, CloudFormation stacks, Auto Scaling groups, and Elastic Beanstalk applications. Through a public .NET API at AWS and the SCOM “Watcher Node,” the MP can reach into the AWS infrastructure and capture AWS “CloudWatch” metrics to gain insight into the health of the managed AWS resource.

Why did Microsoft engage in this joint development project with AWS?

This Management Pack (MP) for AWS is not simply a product release for AWS. Rather, it also provides the proof that System Center can extend the rich monitoring, alerting and reporting foundation to manage any application, from any location in any cloud. This “proof point” can enable other opportunities for development and deployment scenarios (provisioning, orchestration, management) for System Center at other Service Provider clouds.

Check out Amazon’s blog post about this on their blog.

VP Brad Anderson's Blog Article on OpsMgr 2012 Dashboards

2013-05-02T21:00:00Z

Our Corporate Vice President Brad Anderson recently started blogging on TechNet. He covers various topics such as Private Cloud and Product Features released in Windows Server and the System Center suite of products.

His most recent blog post talks about the value the OpsMgr 2012 SP1 Dashboards bring to customers. He also has a plug from our customer T.Rowe Price on how they are leveraging the OpsMgr 2012 Dashboarding capabilities.

Definitely worth a read and something to share with our customers and the rest of the community.

Link to the article: http://blogs.technet.com/b/in_the_cloud/archive/2013/05/02/data-visualization-for-it-system-center-ops-manager-dashboards.aspx

- Satya Vel

Support Tip: OpsMgr 2012 SP1 upgrade fails with error "Failed to set DW upgrade scenario"

2013-04-30T17:34:18Z

Satish Phatge | Senior Support Engineer

Here’s a quick tip that might help you out when upgrading to System Center 2012 Operations Manager (OpsMgr 2012) Service Pack 1 (SP1).

When performing an upgrade to OpsMgr 2012 SP1, the upgrade process fails with the following error:

Failed to set DW upgrade scenario: Threw Exception.Type: System.Data.SqlClient.SqlException, Exception Error Code: 0x80131904, Exception.Message: Cannot open database "OperationsManagerDW" requested by the login. The login failed.

This can occur if there is an invalid Data Warehouse server name or SRS database instance name in the registry on the Management Server.

To resolve this, verify that the correct Data Warehouse server name and SRS database instance name exist in the registry on the Management Server:

Data Warehouse server name:

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft Operations Manager\3.0\Setup\DatawarehouseDBServerName

NOTE: This is a DWORD value and is the NetBIOS name or FQDN of the server.

SRS database instance name:

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft Operations Manager\3.0\Reporting\SRSInstance

NOTE: The default SQL instance name is MSSQLSERVER but may be different in your environment if using a named instance.

Once this is done try the upgrade again. If this was our issue then the upgrade should complete as expected.

Satish Phatge | Senior Support Engineer | Microsoft GBS Management and Security Division

Get the latest System Center news on Facebook and Twitter:

Windows Intune: http://blogs.technet.com/b/windowsintune/
WSUS Support Team blog: http://blogs.technet.com/sus/
The AD RMS blog: http://blogs.technet.com/b/rmssupp/

App-V Team blog: http://blogs.technet.com/appv/
MED-V Team blog: http://blogs.technet.com/medv/
Server App-V Team blog: http://blogs.technet.com/b/serverappv

Windows Server/System Center MVP Speaker Series Tomorrow: Using Visual Studio Authoring Extensions

2013-04-30T14:59:13Z

Just a reminder that we rescheduled the Windows Server/System Center MVP Speaker Series session on Using Visual Studio Authoring Extensions to tomorrow at 8:00 Pacific time. Please join us for this 300 level session where MVP David Allen will wow us with his authoring skills using the Visual Studio Authoring Extensions (VSAE).

You can find the meeting invite for this particular meeting or any of the other meetings in the speaker series on our SkyDrive.

See you tomorrow!