Kevin Holman's System Center Blog

Posts in this blog are provided "AS IS" with no warranties, and confers no rights. Use of included script samples are subject to the terms specified in the Terms of UseAre you interested in having a dedicated engineer that will be your Mic

OpsMgr 2007 R2 CU2 rollup hotfix ships – and my experience installing it

OpsMgr 2007 R2 CU2 rollup hotfix ships – and my experience installing it

  • Comments 24
  • Likes

 

****NOTE OpsMgr 2007 R2 CU5 is now shipped and this is an old article.

http://blogs.technet.com/b/kevinholman/archive/2011/08/03/opsmgr-2007-r2-cu5-rollup-hotfix-ships-and-my-experience-installing-it.aspx

 

 

image

 

The Cumulative Update 2 (CU2) for OpsMgr 2007 R2 has shipped

Get it from the download Center:

http://www.microsoft.com/downloads/details.aspx?displaylang=en&FamilyID=61714687-668a-46e4-b127-ad8519594351

The KB article describing the fixes, changes, and instructions:

http://support.microsoft.com/kb/979257

 

 

 

This is not a huge hotfix rollup…. It does contain some substantial fixes however, and if impacted, you should consider applying this update.  If you have not yet tested and applied CU1 to your R2 management group, then you absolutely should skip CU1 and apply CU2 as soon as possible.

The release notes in the KB article above cover the fixes that are included.  The main ones I have seen in the field are the SNMP device view issue, and the Console crashing when opening multiple views.  I have added it to my Recommended Hotfix page, but if you aren't impacted by any of these issues, you can certainly pass on a cumulative update, and pick up the next one that provides a fix you need.

 

So – first – I download it, and extract it.  The hotfix is about 100MB in size.  This is because it includes update packages for ACS, reporting, SQL, MP’s, agents, and servers.  It all adds up.  I always install to the default location when it comes to a hotfix like this.  Then I STOP – and go find in the installed path – the README file.  Open it up – and use it to formulate your deployment and testing plan.  Well, in THIS version of the CU hotfix – the release notes are simply a pointer to the online documentation.  I see this as a good thing, because it allows us to make updates to the instructions if enhancements are needed.  I am directed to the following link: http://go.microsoft.com/fwlink/?LinkId=186987

 

I build my deployment plan based on the release notes (KB article).  My high level plan looks something like this:

 

  1. Backup the Operations and Warehouse databases.
  2. Apply the hotfix to the RMS
  3. Run the SQL script update against the OpsDB.
  4. Import the updated Management pack provided.
  5. Apply the hotfix to all secondary Management Servers.
  6. Apply the hotfix to my Gateway Servers.
  7. Apply the hotfix to my agents by approving them from pending
  8. Apply the hotfix my dedicated consoles (Terminal servers, desktop machines, etc…)
  9. Apply the hotfix to my Web Console server
  10. Apply the hotfix to my Audit collection servers
  11. Update manually installed agents…. well, manually.

 

Ok – looks like 11 easy steps.  This order is not set in stone – it is a recommendation based on logical order, from the release notes.  For instance – if you wanted to update ALL your infrastructure before touching any agent updates – that probably makes more sense and should be fine.

 

****Added note – as a best practice, you should log on to your SCOM role servers using a domain user account that meets the following requirements:

  • SCOM administrator role
  • Member of the Local Administrators group on all SCOM role servers (RMS, MS, GW, Reporting)
  • SA privileges on the SQL server instances hosting the Operations DB and the Warehouse DB.

These rights (especially the user account having SA priv on the DB instances) are often overlooked.  These are the same rights required to install SCOM, and must be granted to apply major hotfixes and upgrades (like RTM>SP1, SP1>R2, etc…)  Most of the time the issue I run into is that the SCOM admin logs on with his account which is a SCOM Administrator role on the SCOM servers, but his DBA’s do not allow him to have SA priv over the DB instances.  This must be granted temporarily to his user account while performing the updates, then can be removed, just like for the initial installation of SCOM as documented HERE.  At NO time do your service accounts for MSAA or SDK need SA priv to the DB instances…. unless you decide to log in as those accounts to perform an update (which I do not recommend).

 

Ok, Lets get started.

 

1.  I run a fresh backup on my OpsDB and Warehouse DB’s – just in case something goes really wrong.  Since I haven’t grabbed my RMS encryption key in a long while – I go ahead and make a backup of that too.

2.  NOTE:  If applying this update to a cluster – FIRST see:  How to apply a SCOM hotfix to a clustered RMS

Since my RMS is running Server 2008 – I need to open an elevated command prompt to install any SCOM hotfixes. That is just how it is.  So I launch that – and call the MSI I downloaded (SystemCenterOperationsManager2007-R2CU2-KB979257-X86-X64-IA64-ENU.MSI).  This will install the Hotfix Utility to the default location.  Then – a splash screen comes up:

 

image

 

I choose Run Server Update, and rock and roll.

Setup finishes very quickly – and I am presented with the final setup dialogue:

 

image

 

Always MAKE SURE you READ this screen.  If there is an error, or if setup is interrupted prior to completing with success, it will often be on this same screen, which you might overlook, and assume the install completed ok.

Then I click finish, wait around 30 seconds, and click “Exit” on the splash screen.

IF you ever get prompted for a REBOOT – ALWAYS choose NO.

 

image

 

Now – it is time to validate the update applied correctly.  I can see the following files got updated on the RMS in the standard install path:  \Program Files\System Center Operations Manager 2007\

 

image

**note – this isn't all the files included in the hotfix package, just a spot check to make sure they are getting updated. 

 

Next I check my \Agentmanagement folder.  This is the folder that any agents will get updates from.  I check the \x86, \AMD64, and \ia64 directories:

image

 

It is good – that our 979257 CU2 agent MSI got copied over.

In this CU2, we did not remove the 974144 MSI file.  However, that CU1 file will be ignored on future agent updates, repairs, and push installs.  You don't need it.  The key thing to check here is that you DID get a KB979257 agent msp file.

 

3.  Time to run the SQL script against the OperationsManager database.  This is simple enough – the script is located on the RMS in the \Program Files\System Center 2007 R2 Hotfix Utility\KB979257\SQLUpdate\ folder, named DiscoveryEntitySprocs.sql.

**Note – this is the EXACT same patch file that was included in CU1 for R2.  If you are upgrading from CU1 > CU2, you can skip this step.  If you are upgrading from R2-RTM > CU2, then you need to run this step.  This is called out in the release notes.

I simply need to open this file with SQL management studio – or edit it with notepad – copy the contents – and paste it in a query window that is connected to my OpsDB.  I paste the contents of the file in my query window, it takes about 10 seconds to complete, and returns “Command(s) completed successfully”.

 

4.  Next up – import the MP update.  That's easy enough.  It is located at \Program Files\System Center 2007 R2 Hotfix Utility\KB979257\ManagementPacks\ and is named Microsoft.SystemCenter.DataWarehouse.Report.Library.mp.  It takes a few minutes to import.

 

5.  Time to apply the hotfix to my management servers.  I have 3 secondary MS servers, one is Windows 2008 and the other two are older, they are running Windows 2003.  So on the 2008 server I open an elevated command prompt to apply the hotfix utility MSI, and just run it directly on the older servers.  Once the splash screen comes up I “Run Server Update”  These all install without issue.  I spot check the \AgentManagement directories and the DLL versions, and all look great.  REMEMBER – you can sure patch all your management servers at the same time, however, your agents WILL fail over during this time because we stop the MS HealthService during the update.  Keep this in mind.  It is best to update management servers one at a time, synchronously, to keep your agents from failing over to the RMS and overloading it, or causing massive Heartbeat failures because they have nowhere to report to.

 

6.  I would patch my Gateway machines here.  I still don't have a GW in my lab, so I move on to the next step.  Remember to spot check your DLL and \AgentManagement directories.  Previous hotfixes did not copy the hotfix MSI to the \AgentManagement directories, and so you might have to do that manually.

7.  I check my pending actions view in the console – and sure enough – all the agents that are set to “Remotely Manageable = Yes” in the console show up here pending an agent update.  I approve all my agents (generally we recommend to patch no more than 200 agents at any given time.)

After the agents update – I need to do a quick spot check to see that they are patched and good – so I use the “Patchlist” column in the HealthService state view to see that.  For creating a “Patchlist” view – see LINK

 

image

 

The CU2 actually REPLACES any previous patches applied in the PatchList table – this is NICE.   Looks good.  (Note) I will have to formulate a plan to go and update my manually installed agents (Remotely Manageable = No)

 

8.  I have a few dedicated consoles which need updating.  One is a desktop machine and the other is my terminal server which multiple people use to connect to the management group.  So – I kick off the installer – and just choose “Run Server Update as well.  I do a spot check of the DLL files – and see the following was updated on the terminal server:

 

image

 

9.  Next up – Web Consoles.  I actually have two – and both are running on management servers, which I have already patched.  So – I will simply just go check their DLL files to ensure they got updated:

From:   \Program Files\System Center Operations Manager 2007\Web Console\bin

image

 

10.  I don't have ACS set up at the moment – but at this point if I did – I would go hit those Management servers that have already been patched – but this time run the update and choose to “Run ACS Server Update”

image

 

    

11.  Manually installed agents.  I have a fair bit of these… so I will do this manually, or set up a SCCM package to deploy them.  Most of the time you will have manually installed agents on servers behind firewalls, or when you use AD integration for agent assignment, or when you installed manually on DC’s, or as a troubleshooting step. 

 

 

Now – the update is complete.

image

 

 

 

 

 

The next step is to implement your test plan steps.  You should build a test plan for any time you make a change to your OpsMgr environment.  This might include scanning the event logs on the RMS and all MS for critical and warning events… looking for anything new, or serious.  Testing reporting is working, check the database for any unreasonable growth, run queries to see if anything looks bad from a most common alerts, events, perf, state perspective.  Run a perfmon – and ensure your baselines are steady – and nothing is different on the database, or RMS.  If you utilize any product connectors – make sure they are functioning.

The implementation of a solid test plan is very important to change management.  Please don't overlook this step.

Comments
  • Thx again mate. Excellent explanation.

    John Bradshaw

  • Any considerations for RTM agents reporting to server roles that have been upgraded to CU2?  Will non-CU2 agents behave normally while upgrades are in progress, and how long can old agents be left out there?  Indefinitely?

    Thanks for another great post.

  • We fully support SP1 agents, and R2 agents, reporting to a R2 management server, whether that MS is CU1, or CU2.

    No issue with your scenario.

  • Thanks Kevin,

    Stressing to everyone to check the agents folder and file versions..I had to run the server update a second time on 2 of my MS despite the wizard saying completed OK.

    A

  • Hi,

    Thanx for the great post. I have some problems when creating the Patch List. When i open the patchlist he says 'Page Cannot be displayed'.

    Do you know a solution for this?

  • Leon,

    To view the Agent patch Level select Monitoring, then Right-click and select New State View.

    Give it a name like Agents Patch Level

    Under Show Data related to: select Agent

    Click OK.

    This will show you the Agent patch Level for your monitored servers.

    HTH,

    JB

  • Thanks for the instructions.  I have some questions.  How can I role back to update 1?

    The reason I ask is that our change management team have requested a roll back plan.  I could ininstall the update manually from each client and the RMS, but what about the database.  Is there a better way?

  • Can I update manually installed agents by only copying down the KB979257-x86-agent.msp and running it on the agent server or do I need the entire 100 MB package on every server?

  • @Jeffrey -

    You can copy down only the KB979257-x86-agent.msp and manually run it for manually installed agents.  You dont need the whole package for agents.

  • Kevin,

    Thanks for posting these instructions.  In my environment, we have over 1700 manually installed agents (as part of our server deployment process) leveraging AD integration.  We are currently running SCOM R2 and would like to upgrade to CU2 but updating all the manually installed agents is a time consuming task and probably won't be done for several days or even weeks.  What will happen if I upgrade my SCOM environment to R2 CU2 but not the agents?  Will they continue to function properly?  I wish you could just push the updates from the management servers regardless of whether the agents were installed manually or not.

    Thanks,

    Huyh

  • Huyh -

    I completely agree - I dont know why we cannot push agents and agents updates from the console - and still leverage AD integration.  I think the thought process was at the time - that customers who leveraged AD integration already had a software distribution mechanism in place... which I think is pretty silly to assume.  This would be excellent feedback to file on connect to make sure the next version of SCOM support this.  There really isnt a good reason why this could not be supported.

    As to your question - we support SP1 and R2 agents connecting to a R2 CU2 management group infrastructure.  We always recommend to get all agents updated as soon as possible to the same level as the the management group servers, but this isnt required.  We understand this is a long process for large environments, and could take weeks or even months to complete, especially when we disable the ability to push this update out for manually installed agents.  :-(

  • Thanks for your article.  I have successfully deployed the CU in my environment following it!

    However my RMS and MS servers are not showing as having the new client, is this something I should worry about?

    Thanks again.

    Allen

  • If you are referring to the RMS/MS not having values in Patchlist view - this is normal.  This should be scoped to agent managed computer group.

  • I have a test and production managment group all reporting to the same SQl Server. If I upgrade my infastructure (RMS, MS, GW, RS etc..) and choose not to upgrade the agents initally will that be OK? I am afraid the agents servers will need reboots or I will have to do some manually.

  • Hi Bryan,

    Yes - you can upgrade the infratructure first, and delay on applying the update to agents.  We fully support an R2 agent reporting to a R2CU2 infrastructure, but we do recommending applying CU2 to all agents as soon as possible - since there are so many of the updates included in CU1 and CU2 that apply to agent based issues.

    If you are going to do this - I recommend rejecting the pending actions for agents, since it is not good to leave a large number of agents in pending actions for an extended period.  Better to reject them and compe back later and run "repair" on those that are remotely manageable.

    To update agents - you should not need a reboot - that is very rare.  only required in troubleshooting an agent that wont update from my experience.

Your comment has been posted.   Close
Thank you, your comment requires moderation so it may take a while to appear.   Close
Leave a Comment
Search Blogs