Kevin Holman's System Center Blog

Posts in this blog are provided "AS IS" with no warranties, and confers no rights. Use of included script samples are subject to the terms specified in the Terms of UseAre you interested in having a dedicated engineer that will be your Mic

OpsMgr 2012: Cumulative Update Rollup 1 (UR1) ships–and my experience installing it

OpsMgr 2012: Cumulative Update Rollup 1 (UR1) ships–and my experience installing it

  • Comments 13
  • Likes

 

image

 

 

Cumulative Update Rollup 1 (UR1) for OpsMgr 2012 has shipped.

Download:  http://www.microsoft.com/en-us/download/details.aspx?id=29697

KB link:  http://support.microsoft.com/kb/2686249

 

Here is a list of the fixes:

  • Environment crashes in Operations Manager due to RoleInstanceStatusProbe module in AzureMP.
  • When multiple (2-3) consoles are running on the same computer under the same user account the consoles may crash.
  • Cannot start or stop tracing for Reporting and Web Console if they were installed to a standalone IIS server.
  • Connected Group alert viewing is not working but no error is given in console.
  • Task result - CompletedWithInfo not supported with the SDK2007 assemblies.
  • SeriesFactory and Microsoft.SystemCenter.Visualization.DatatoSeriesController need to be public to allow the controls extensibility and reuse.
  • WebConsole is not FIPS compliant out of the box.
  • Network Dashboard should overlay Availability when displaying health state.
  • Dashboards: Group picker does not show all groups in large environment.
  • IIS Discovery: prevent GetAdminSection from failing when framework version was detected incorrectly by IIS API.
  • Performance Counters do not show up in Application list view of AppDiagnostics.
  • Console crashes when state view with self-contained object class is opened.
  • PerformanceWidget displays stale 'last value' in the legend due to core data access DataModel.
  • Availability Report and "Computer Not Reachable" Monitor show incorrect data.
  • Agent install fails on Win8 Core due to dependency on .Net framework 2.0.
  • Web Services Availability Monitoring Wizard - Console crashes if wizard finishes before test has finished.
  • Several Powershell changes needed:
    • Changed License parameter in Get-SCOMAccessLicense to ShowLicense
    • Changed SCOMConnectorForTier cmdlets to SCOMTierConnector
    • Some formatting changes
  • Update Rollup 1 for System Center 2012 – Operations Manager resolves the following issues for UNIX and Linux monitoring:
    • Schannel error events are logged to the System Event Log on Operations Manager Management Servers and Gateways that manage UNIX/Linux agents.
    • On HP-UX, Operations Manager cannot discover and monitor a logical volume composed of more than 127 physical volumes
    • Upgrade of UNIX and Linux Agents fails when using Run As credentials in the Agent Upgrade Wizard or Update-SCXAgent PowerShell Cmdlet
  • Update Rollup 1 for System Center 2012 – Operations Manager adds the following feature:
    • Support for Oracle Solaris 11 (x86 and SPARC)

 

 

Let’s Roll:

So – first – I download it. The hotfix is ONLY about 76 megabytes!!!  This is a HUGE improvement over the 1.2GB CU’s for SCOM 2007 – lets hope it stays this way!

The KB instructions show me that CU1 for SCOM 2012 has changed in a very fundamental way.  No longer is there a bootstrapper program for the CU.  Now – the package simply extracts the files to a directory.  We will then run each MSP file independently based on whatever server roles are installed. 

 

Next step – READ the documentation… understand all the steps required, and formulate the plan.

I build my deployment plan based on the release notes in the KB article. My high level plan looks something like this:

  1. Backup the Operations and Warehouse databases, and all unsealed MP’s.
  2. Apply the hotfix to all the Management Servers
  3. Apply the hotfix to my Gateway Servers.
  4. Apply the hotfix to the OpsMgr Reporting server.
  5. Apply the hotfix to my Web Console server
  6. Apply the hotfix my dedicated consoles (Terminal servers, desktop admin machines, etc…)
  7. Import the management pack updates
  8. Agents:  Apply the hotfix to my agents by approving them from pending.
  9. Agents:  Update manually installed agents…. well, manually.
  10. Unix/Linux:  Download and import the updated MP’s for Unix/Linux monitoring.
  11. Unix/Linux:  Upgrade the Unix/Linux agent providers.

 

Ok – looks like 11 easy steps. This order is not set in stone – it is a recommendation based on logical order, from the release notes.

 

****Requirement – as a required practice for a major update/hotfix, you should log on to your OpsMgr role servers using a domain user account that meets the following requirements:

  • OpsMgr administrator role
  • Member of the Local Administrators group on all OpsMgr role servers (RMS, MS, GW, Reporting)
  • SA (SysAdmin) privileges on the SQL server instances hosting the Operations DB and the Warehouse DB.

These rights (especially the user account having SA priv on the DB instances) are often overlooked. These are the same rights required to install OpsMgr, and must be granted to apply major hotfixes and upgrades (like RTM>SP1, SP1>R2, etc…) Most of the time the issue I run into is that the OpsMgr admin logs on with his account which is an OpsMgr Administrator role on the OpsMgr servers, but his DBA’s do not allow him to have SA priv over the DB instances. This must be granted temporarily to his user account while performing the updates, then can be removed, just like for the initial installation of OpsMgr as documented HERE. At NO time do your service accounts for MSAA or SDK need SA (SysAdmin) priv to the DB instances…. unless you decide to log in as those accounts to perform an update (which I do not recommend).

 

Ok, Lets get started.

1.   Backups. I run a fresh backup on my OpsDB and Warehouse DB’s – just in case something goes really wrong.

I also will take a backup of all my unsealed MP’s. You can do the backup in PowerShell, here is an example which will backup all unsealed MP’s to a folder C:\mpbackup:

Get-SCOMManagementPack | where {$_.Sealed -eq $false} | export-SCOMmanagementpack -path C:\MPBackup

We need to do this just in case we require restoring the environment for any reason.

 

2. Apply the hotfix to the Management Servers.

Pro Tip #1:  Here is a tip that I have seen increase the success rate: Reboot your Management Server nodes before starting the update. This will free up any locked processes or WMI processes that are no longer working, and reduce the chances of a timeout for a service stopping, file getting updated, etc.  This is not a requirement, just something to consider if you have had issues applying such a fix.

Pro Tip #2:   If you are running any SDK based connectors – it is a good idea to stop these first. Things like a Remedy product connector service, Alert Update Connector, Exchange Correlation Engine, etc… This will keep them from throwing errors like crazy when setup bounces the SDK services.

I start by running the download, SystemCenter2012OperationsManager-CU1-KB2674695-X86-X64-IA64-ENU.exe.  This is simply an extractor.  You can run this anywhere, and provide a location for the update.  No longer do we need to run this extractor on each server role.  We can now simply extract these files to a network share, and update all server roles from there.

Here are the files after extraction:

image

 

 

I start on my first management server, OMMS1.  This has the Server role, Web Console Role, and Console installed.  So I will need to run 3 MSP’s. 

I will start by running KB2674695-AMD64-Server.msp.  Right click the file and choose “apply” or call it from an elevated CMD.

Pro Tip: Open an ELEVATED COMMAND PROMPT and run these MSP' files from the command prompt.  User Access Control (UAC) will still block a successful install in some cases, so you will experience greater success if you always run updates from an elevated CMD.

 

You will see a dialogue box like below:

image

image

The server update goes pretty quickly.  It bounced the DAS, Config, and SC Mgmt services during the update.

 

Next, I will update the web console by running KB2674695-AMD64-WebConsole.msp.  This only takes a few seconds.

 

Next, I will update the Console files.  I need to make sure that I close any open consoles on this server, from my session or any other logged in sessions via RDP.  I will apply KB2674695-AMD64-Console.msp.  Again – this only takes a few seconds to run.

Now – I am done updating all three installed server roles on this server.  I can spot check to make sure the files got updated:

Server role update:  From \Program Files\System Center 2012\Operations Manager\Server

image

Web Console role update:  From \Program Files\System Center 2012\Operations Manager\WebConsole\WebHost\bin

image

Console role update:  From \Program Files\System Center 2012\Operations Manager\Console

image

 

Next – I am ready to update the rest of my management servers.  I only have one other MS – OMMS2.  This only runs the console and server roles, so only two MSP’s to apply.

I apply each MSP, takes a couple minutes.

Another spot-check to make, is to each that the Agent binaries get updated on each Management Server role, for Windows agents.  These are located at: \Program Files\System Center 2012\Operations Manager\Server\AgentManagement\ in the AMD64 or i386 directory:

Check to ensure that the CU dropped the appropriate agent update file, such as KB2674695-amd64-Agent.msp.

image

 

 

 

3.  Apply the hotfix to my Gateway Servers.

I don’t have any gateways in my lab right now – but this would be a very simple execution of the KB2674695-AMD64-Gateway.msp file.

 

4.  Apply the hotfix to the OpsMgr Reporting server.

I have SCOM reporting and SSRS installed on a SQL server, DB01. 

I log on and apply KB2674695-AMD64-Reporting.msp.  This runs in a few seconds.

Spot check:  From \Program Files\System Center 2012\Operations Manager\Reporting\Tools

image

 

5.  Apply the hotfix to my Web Console server

I already did this in step 2 – but I could have waited and done this now, or patched any dedicated Web Console servers.

 

6.  Apply the hotfix my dedicated consoles (Terminal servers, desktop machines, etc…)

I need to apply this update anywhere the console is installed – including consoles installed on management servers.  I have already updated the console patch on my management servers OMMS1 and OMMS2.  Now I have a Terminal Server where I run the console – so I will patch this server now, which only runs the console.  Make sure you close the console and close any other user sessions out – or you might require a reboot to finish the update for the console files which will be locked by an open console.  This also includes any open Powershell windows connected to the SDK.  You will get a warning if setup detects any.  You can run the same spotcheck for the file version update as above.  (Note – help/about will not show the updated version in the console – this stays the same major version, so don’t go looking there.)

 

7.  Import the management pack updates

Open a console – and import the files previously extracted:

Microsoft.SystemCenter.DataWarehouse.Library.mp

Microsoft.SystemCenter.Visualization.Library.mpb

Microsoft.SystemCenter.WebApplicationSolutions.Library.mpb

image

These all import in a few minutes without a hitch.

 

8.  Agents: Apply the hotfix to my agents by approving them from pending

I open the console – Administration > Device Management > Pending Management, and see all my agents in pending that are not manually installed (Remotely Manageable = Yes)

Let me stop and talk about how agents get into Pending.  This is a ONE TIME operation, which is created at the time that you run the CU on a management server.  What will happen, is that the CU runtime will look for all agents ASSIGNED to that management server, that are Remotely Manageable (not manually installed) and will put ONLY those agents into pending at that time.  We will not ever go back and re-inspect to put old agents into pending, because this is not a SCOM workflow that handles this.  It is done only by applying the update.  If you don’t have agents in pending – you either aren't running the MSP in an elevated fashion, or you aren't running the update as a user that is BOTH a SCOM admin, Local OS Admin, and SQL Systems Administrator role to the databases.

image

 

I right click all mine – provide an account that has rights to deploy/install the update remotely, and kick it off.

100% Success!

How can I tell?  Open the Console, Monitoring > Operations Manager > Agent Details > Agents by Version (state view):

image

 

9.  Agents: Update manually installed agents…. well, manually.

I simply run the file KB2674695-AMD64-Agent.msp or KB2674695-i386-Agent.msp depending on which OS version (64bit or 32bit) I am updating.  I can deploy these manually, or via software distribution packaging up each MSP and applying them to the correct OS by version/architecture.

 

10.  Unix/Linux: Download and import the updated MP’s for Unix/Linux monitoring

The Unix/Linux update files are located in a separate download.  This includes new versions of the management packs, and updated agent binaries.  I go to the link in the KB article and grab these update files.  Monitoring Pack for UNIX and Linux Operating Systems.msi.  It is the typical MP installer/extractor program.  Run setup and extract these files to your preferred location.  I generally just recommend installing/extracting these to your default location on any workstation/server, and then copying the extracted files to your OpsMgr software/MP/Updates network share.

Download link:  http://www.microsoft.com/en-us/download/details.aspx?id=29696

Next – import all the files for your versions f Unix/Linux Agents that you support.  The RTM version of the Unix/Linux MP’s was 7.3.2026.0.  The CU1 version is 7.3.2097.0.  This also includes a new MP to add support for Solaris 11.

***Note – several of these new MP files are actually in the new .MPB format.  This new format allows management packs to take advantage of the new schema extensions in System Center 2012, to be able to transfer binaries among other items.  Such as the agent binaries

 

image

 

11.  Unix/Linux: Upgrade the Unix/Linux agent providers

One you have imported the updated MP files for Unix/Linux agents, spot check that these files got updated on your management servers.  Look in the following folder:

\Program Files\System Center 2012\Operations Manager\Server\AgentManagement\UnixAgents\DownloadedKits

image

 

Notice the new update files are version 1.3.0-214.  Also, it may take some time for this folder/files to show up, after importing the MP’s, as your database and management servers will be experiencing high CPU utilization from SDK and Config activities, and this is normal.  So be patient.  You need to ensure that these files are updated on any members of your management server resource pool that monitors Unix/Linux agents before continuing.

Now – to update my Unix/Linux agents.  I can use the Update-SCXAgent cmdlet in powershell, or I can use the Administration wizard.  I right click my version .206 agent, and choose Upgrade Agent.

 

image

image

You can use an existing RunAs account profile if you have configured these with an agent upgrade account that has the necessary rights, or you can input new credentials using an account that has SSH privileges in order to remotely install software on the Unix/Linux agent machine.   If all goes well – you get a success:

image

 

image

 

 

Now – the update is complete.

image

 

 

 

 

The next step is to implement your test plan steps. You should build a test plan for any time you make a change to your OpsMgr environment. This might include scanning the event logs on the Management servers for critical and warning events… looking for anything new, or serious.  Testing reporting is working, test your web console, check the database for any unreasonable growth, run queries to see if anything looks bad from a most common alerts, events, perf, state perspective.  Run a perfmon – and ensure your baselines are steady – and nothing is different on the database, or management servers.  If you utilize any product connectors – make sure they are functioning.

The implementation of a solid test plan is very important to change management. Please don't overlook this step.

Comments
  • Good post.  Curious why MSFT names this CU1 and you parenthesized it as UR1?  Is MSFT moving to a new naming convention?

  • @Jonathan - looks that way.

  • I had an issue with running the main server update patch. UAC would pop up, but the updater would still report that it got an Access Denied trying to stop the SCOM services. That would be followed by an MSIEXEC exception.

    I then tried it with from an elevated CMD windows, and then it went perfectly.

  • Thanks TheFluffysysop - I havent experienced this but I will add this to my recommendations.

  • We have not upgraded to 2012 yet so we will be doing the 2012 upgrade and CU1 at the same time.

    All of our agents (9000) are manually installed so we need to ugrade them before the servers. This will be done by SCCM which means we usually have a few hundred to upgrade manually or through psexec etc. So the agent upgrade process may take a week which leaves us with 2012 agents and 2007 backend for a week.

    My understanding (please correct me if i am wrong) is that scenario is ok.

    My question is: Is it ok to apply CU1 to the agents at the same time as the 2012 upgrade? Or the real question; is a 2012 agent with CU1 backward compatible to a 2007 infrastructure?

  • @MedeBay -

    Yes - the agent upgrade will be the most painful part of the upgrade process.  We understand that it will take some time to get 100% of all agents updated.... this is quite a project.  So once you begin the upgrade, you will be in a mixed mode where some components are 2007 and some are 2012.  

    The order of agent upgrade depends on how the agents were deployed. If you installed the agents manually, you upgrade the agents before you upgrade the management servers and gateways. If you installed the agents by using the Computer and Device Management Wizard (the Discovery wizard), you upgrade the agents after you upgrade the management servers and gateways. These kinds of agents are known as push-installed agents.

    A 2012 agent can communicate with a 2012 and/or a 2007 management group.  We support multi-homing a 2012 agent to multiple management groups, and these can be 2012 or 2007.  This support is necessary for a side-by-side migration to complete an upgrade, and also to support an in-place upgrade where the agent gets upgraded FIRST.  

    That said - we recommend completing the upgrade of all components to 2012 as soon as possible.

    As to applying the CU1 at the same time as installing the agent - this should be OK.... but I am not 100% sure we tested this scenario - where a CU1 agent is talking to a 2007R2 or 2012 RTM management group infrastructure.  I can see the desire to knock this out at one time, so I'd recommend opening a case and getting a support statement from Microsoft support via your TAM for that one.

  • Just some thoughts -

    At first I thought it was a great idea to get rid of the bootstrapper. Now i can easily update a computer that only has a console.

    But as i started updating my 6 Dev servers (an invisioning updating my 20 production servers) I realized it is sort of a pain to have to figure out which components are on which server and run the individual KB's.

    And what seems even a larger pain is trying to verify that each of the required components actually got updated by finding their respective dll's etc.

    What I think would be nice is to get a "CU# Upgrade helper MP" (Yes I stole the idea from the 2012 upgrade). have it determine all your server roles and verify that each was updated appropriately, as well as agents.

  • re: SCOM 2012

    I have an issue during the CU1 update. I hope someone can help. Evey thing went fine and Jim Dandy until I got to the linux machines.

    First I could not push the agent updates so I ran them manually. I got the same error during the push as I do now trying to re-discover a new box.

    Data at the root level is invalid. Line 2 position 1.

    If anyone has any ideas, I would appreciate it.

  • Running the KB2674695-AMD64-WebConsole.msp - didn't work for me.  I get an error:

    error 26004 Failed while processing WebVirtualDirs. (-2147024894   )  

    And the update rolls back. After this my web-console dosen't work at all..

    Any ideas, please post or contact me.

    Yours Erling B. Kjeldsen

    University of Southern Denmark.

  • It turns out that i must have made some error editing the Web.config file, after reaplying the old web.config file mu web-console update worked like a charm.

    Sorry for bothering you.

    Erling B. Kjeldsen

  • Thanks for this Good post.   it works really fine !!!!

  • Hi Mr Kevin,

    We Have SCOM 2012 RTM.. we planning to upgrade to SP1.. next, can we upgrade into UR2 without installing UR1.

    Regard's

  • @Andy -

    You can go straight to UR2.  Update releases are cumulative.

Your comment has been posted.   Close
Thank you, your comment requires moderation so it may take a while to appear.   Close
Leave a Comment
Search Blogs