Jonathan's Virtual Blog

System Center Virtual Machine Manager and Orchestrator - Solutions and Guidance

 Jonathan's Virtual Blog

   Virtual Machine Manager - Orchestrator - Solutions and Guidance

  • Update Rollups for SC 2012 SP1 and 2012 R2

    downloadAnother quarterly Update Rollup (UR) has been released for System Center 2012 SP1 and SC 2012 R2. In this post we focus on SC VMM and SC Orchestrator updates. A note about Update Rollups: do not consider these optional. Do not consider them hotfixes. Install them always when they are released. Each UR contains up to 25 fixes per product based on past releases. Here’s what is new with UR’s:



    Description of the fixes in Update Rollup 5 for System Center Orchestrator 2012 SP1

    Resolved issues
    Update Rollup 5 for System Center Orchestrator 2012 SP1 resolves the following issues. 
    Issue 1
    The Monitor Service activity does not let the configuration be saved if Monitor Service cannot resolve the hostname that is entered in the Computer Input property. If you try to save the configuration, you receive the following error message:

    Connection to the selected computer has failed

    Issue 2
    The Compress Files activity does not include files that are in subfolders that match the filter when the Include files in sub-folders option is selected.
    Issue 3
    The Send SNMP Trap activity does not send the correct results for certain combinations of values.
    Issue 4

    The SNMP Trap Variables are not retained in the correct order after the Properties dialog box is closed and then opened again.


    Description of Update Rollup 1 for System Center 2012 R2 Virtual Machine Manager

    There are special install instructions for this update. Please read the article!

    Update Rollup 1 for System Center 2012 R2 Virtual Machine Manager resolves the following issues:
    • System Center 2012 R2 Virtual Machine Manager cannot deploy a new or imported VMWare template.
    • A virtual machine with that uses VHDX cannot be refreshed correctly in System Center 2012 R2 Virtual Machine Manager, and you receive the following error message:

      Refresh job failed with error 2912: The requested operation cannot be performed on the virtual disk as it is currently used in shared mode (0xC05CFF0A)

    • Database operations sometimes fail with "FailedToAcquireLockException."
    • A new virtual machine template from a template that specifies an operating system profile doesn't use credentials from the operating system profile.
    • Virtual machines in VMWare that connect by the using Cisco N1000V dvSwitch are unavailable for management from Virtual Machine Manager.
    • System Center Virtual Machine Manager service crashes if you disable one of the teamed network adapters.
    • The Get-Scstoragearray -host command should return storage arrays that are visible to a host that is using zoning.
    • During the discovery of a network-attached storage (NAS) provider, the credentials that are used do not include a domain name.
    • Some localized strings are not displayed correctly in the UI.
    • A query to find the certificate should match both the subject name and the friendly name because FindBySubjectName is a wildcard search.
    • Template deployment fails, and you receive the following error message:

      Error (2904)
      VMM could not find the specified path on the <Server name> server.
      The system cannot find the path specified (0x80070003)

    • Virtual Hard Disk (VHD) cannot be mounted on a host because VHD conflicts with other disks because of a stale entry that was left in the dictionary of Virtual Machine Manager memory.
    • Differencing disk based deployment may fail because the parent disk is being refreshed as noncached.




  • SC VMM: Moving or Creating a VM fails with errors (2912) 0x80041001

    BITSRecently I have seen a rush of VMM cases that involve moving a VM through migration, or otherwise moving data (VHD, VHDX, ISO, etc.) around. I have found that in many of these cases there are suspended Background Intelligent Transfer Service (BITS) jobs. As long as there is a suspended job in the BITS queue on the VMM server or the Host you are moving data to, the move will fail.



    Typical VMM trace showing the issue: In this case a VMM trace shows a copy job beginning. Locate the first step of an exception at the bottom of the stack and read upwards.The first line shows that BITS is starting a copy job: 'at Microsoft.VirtualManager.Engine.Deployment.BitDeployer.Copy().'

    20:46:41.271 01-22-2014,0x09C4,0x0994,4,BitsDeployer.cs,506,0x00000000, Caught Exception,{00000000-0000-0000-0000-000000000000},1,
    20:46:41.272 01-22-2014,0x09C4,0x0994,4,BitsDeployer.cs,506,0x00000000,"Microsoft.Carmine.WSManWrappers.WSManProviderException: An internal error has occurred trying to contact an agent on the server.
    17993 Ensure the agent is installed and running. Ensure the WS-Management service is installed and running; then restart the agent.
        at Microsoft.Carmine.WSManWrappers.ErrorContextParameterHelper.ThrowTranslatedCarmineException(ErrorInfo ei; Exception ex)
        at Microsoft.Carmine.WSManWrappers.WsmanAPIWrapper.RetrieveUnderlyingWMIErrorAndThrow(SessionCacheElement sessionElement; COMException ce)
        at Microsoft.Carmine.WSManWrappers.WsmanAPIWrapper.Enumerate(String url; String filter; Type type)
        at Microsoft.Carmine.WSManWrappers.WSManRequest`1.Enumerate(String url; String wqlQuery)
        at Microsoft.VirtualManager.Engine.Deployment.NativeDeploymentUtils.IsBitsRemoteApiAvailable(WSManConnectionParameters connectionParams; BitsRemoteApi remoteApi)
        at Microsoft.VirtualManager.Engine.Deployment.LANAcceleratorFactory.GetDeploymentClientJob(WSManConnectionParameters connParams; WSManConnectionParameters remotePeerConnParams; String sourceFileName; String targetFilename; UInt16 port; Boolean privacy; UInt32 flags; String sessionID; Boolean resetJob)
        at Microsoft.VirtualManager.Engine.Deployment.BITSDeployer.CreateClientJob(DeploymentFile file; CLIENT_JOB_TYPE clientJobType; WSManConnectionParameters clientConnection; WSManConnectionParameters serverConnection; UInt16 serverTcpPort; Boolean clientPrivacy; Boolean startAfresh)
        at Microsoft.VirtualManager.Engine.Deployment.BitDeployer.Copy()
    *** Carmine error was: HostAgentFail (2912); HR: 0x80041001


    Perform these steps on the VMM server and any Hosts where data is being moved. Ignore any messages referring to the command being deprecated

    1. Open an elevated command prompt
    2. Enter 'bitsadmin /list /allusers'
    3. You will see the number of jobs suspended. Each job begins with a GUID
    4. Cancel each suspended job by entering 'bitsadmin /cancel {GUID}', where GUID represents the BITS job suspended
    5. Repeat the previous step for all BITS jobs
    6. Enter 'bitsadmin /list /allusers' when done and no jobs should show suspended

    I’ve written a KB article that details this same process. It will be available online soon.




  • Orchestrator Functions and Regular Expressions (Regex)

    imageThere are a number of ways to manipulate data using the Orchestrator built-in Activities. Text manipulation is a common need with Runbooks. An Activity pulls a string of data that you need only a part of, or need it in a different format. Fortunately, Orchestrator has many built-in Functions that can be used to further manipulate strings. Use these tools in the same places you would subscribe to data and they will be processed.

    For example, the string below will be converted to all UPPERCASE.

    [Upper('this will be inserted in upper case')]

    You must enclose the entire string to be manipulated in square brackets, with the command and the string. Some other commands

    Here’s the link to Orchestrator Functions


    Function and Definition

    Upper - converts text to uppercase.

    Lower - converts text to lowercase.

    Field - returns text in a specific position.

    Sum - returns the sum of a set of numbers.

    Diff - returns the difference of two numbers.

    Mult - returns the product of a set of numbers.

    Div - returns the quotient of two numbers.

    Instr - returns the position of first occurrence of text within another text.

    Right - returns a subset of the text from the right side of the full text.

    Left - returns a subset of the text from the left side of the full text.

    Mid - returns a subset of the text from the middle of the full text.

    LTrim - trims leading spaces from text.

    RTrim - trims the trailing spaces from text.

    Trim - trims leading and trailing spaces from text.

    Len - returns the length of text.


    Regular Expressions

    Regular expressions let you match a string to a pattern. This is a bit more complex, but allows for pulling really anything you need from a string.




    Matches any character except a newline.


    Matches the preceding item 0 or more times. For example, the "a*" pattern matches any string of a's in a row "a", "aaa", "aaaaaaaaaaaa", and an empty string "". To match any string of any character, use a dot followed by an asterisk. For example "a.*" matches any text that begins with the letter "a" and ends with any string of characters such as "abbb", "abcdef", or "automatic restart".


    Matches the preceding item 1 or more times. This is like * but you must have a least 1 of the preceding item to make a match. For example, the "ab+" pattern matches "abbbbb", "ab", but does not match "a". To contrast, the "ab*" pattern matches "a".


    Matches the preceding item 0 or 1 time. For example, the "ab?" pattern matches "a" or "ab" but does not match "abbb".


    Matches either the preceding expression or the following expression. Logical OR operator.


    Matches the expression at the end of the input or line. For example, "ab$" matches "I took a cab" or "drab" but does not match "absolutely not".


    Matches the expression at the beginning of the input or line. For example, "^ab" matches "absolutely not" or "abacuses are great!" but does not match "I took a cab" or "drab".


    For characters that are usually treated as special. This indicates that the next character is literal and is not to be treated as a special character. For example, "\." means match the "." character and not just any character.

    [ ]

    A character set. Matches any one of the enclosed characters. You can specify a range of characters by using a hyphen. For example, [a-zA-Z] matches any letter of the alphabet.

    [^ ]

    An excluded character set. This is the opposite of []. If any of the characters inside the brackets exist, the regular expression match fails. You can specify a range of characters by using a hyphen. For example, [^a-zA-Z] ensures that none of the letters in the alphabet are present.

    ( )

    A group expression. This groups an expression into an item that you can apply special characters to. For example, "a*(ba)+" matches "ba" "aba" or "ababa" but does not match "abbba" or "abaa"





  • Orchestrator Web Console Fails after Upgrade to 2012 R2

    This is an issue we have been tracking for a while now. Please see the KB article on how to get around this error.


    After upgrading to System Center 2012 R2 Orchestrator, attempts to start the Orchestration Web Console fails and the following messages will pop up in the browser:
    Error Executing the current operation
    Arguments: NotFound

    Error Executing the Current Operation

    Starting the Orchestration Web Console fails after upgrade to System Center 2012 R2




  • What version of VMM am I running?!?

    A new KB has been released that makes it simple to know what build of VMM you are running. This is important as you should always be at the latest Update Rollup or Service Pack possible. By knowing your build number you can identify updates that need to be installed. Find your build number on the list, then look for a higher build number in the same release of VMM. Best of luck!

    Stay up to date on updates!


    How to determine the version of Virtual Machine Manager


    an excerpt:

    System Center 2012 SP1 RTM  KB Date Released  3.1.6011.0  
    System Center 2012 SP1 Update Rollup 1 2785682
    System Center 2012 SP1 Update Rollup 2 2802159

    System Center 2012 SP1 Update Rollup 3 2836751
    System Center 2012 SP1 Update Rollup 4 2879276
    System Center 2012 R2 RTM     3.2.7510.0


    Denis pointed out that there is a similar list here. Thanks!




  • Technical Documentation: Virtual Machine Manager 2012 R2

    This is the go to source for VMM. Can VMM do x? How do you make VMM do x? Both questions answered in this ~800 page document, with a multitude of links to related information. Keep this one in your VMM toolkit.

    … and to answer your first question, yes; SC2012_VMM does not reflect that there is updated R2 material. This is the right doc.

    Applies To

    System Center 2012 – Virtual Machine Manager

    System Center 2012 Service Pack 1 (SP1) – Virtual Machine Manager (VMM)

    System Center 2012 R2 Virtual Machine Manager – NICE!

    Word and PDF versions available.


    Technical Documentation Download for System Center 2012 – Virtual Machine Manager





  • Free ebook: Microsoft System Center: Cloud Management with App Controller

    cover for System Center Cloud Management ebookA great free book has been released for App Controller 2012 R2. This is not a product generally covered in this blog, but can be an integral part of your System Center installation.

    Briefly… an excerpt from the post below.


    Microsoft System Center 2012 R2 App Controller is uniquely positioned as both an enabler and a self-service vehicle for connecting clouds and implementing the hybrid computing model. In Microsoft’s cloud computing solutions, both System Center and Windows Azure play critical roles. System Center can be used to transform enterprise IT from a device-based infrastructure and deployment strategy to a service-based user-centric consumption model based on private cloud computing. Windows Azure on the other hand is a subscription-based public cloud platform that enables the development, deployment, and management of cloud solutions. App Controller is the glue that unifies these two platforms by providing a single interface that enables administrators to perform complex operations without overwhelming them with the underlying technical complexities involved.

    Click here to read the post and grab the book!






  • You may have problems with Virtual Machine Manager if some SPNs are missing

    As a support engineer at Microsoft, I see almost daily some issues with Service Principle Names (SPNs). SPNs are as important to VMM communication as DNS. They are attributes on DNS entries that allow other services to find VMM. This is not a new issue, and this article covers how SPNs should be set up on your system. If everything is working well, please do not make changes. To see if you have SPNs registered correctly, at an elevated command prompt type ‘setspn –L <computername>’. This will list what is registered. You DO  want to find SCVMM/Hostname and SCVMM/FQDN in the list. Below is only an example. Good luck with the article!


    You may have problems with Virtual Machine Manager if some SPNs are missing






  • System Center Update Rollups!


    Two new updates have been released for System Center 2012 and for System Center 2012 SP1. There are many fixes in these rollups, so treat them as mandatory, not optional. Links below. Happy troubleshooting!



    System Center 2012

    Description of Update Rollup 7 for System Center 2012


    System Center 2012 SP1

    Description of Update Rollup 4 for System Center 2012 Service Pack 1







  • Free ebook: Designing Orchestrator Runbooks

    5025_image_thumb_23B34BDBA great new book has been released from Microsoft Press that covers Orchestrator and the creation of Runbooks very well. As a support professional I found the book to be informative from an architectural perspective, and full of great examples to get started.

    From the posting on the Orchestrator TechNet blog site:

    We’re very excited to announce a free ebook offering from Microsoft Press: Microsoft System Center: Designing Orchestrator Runbooks (ISBN 9780735682986) by David Ziembicki, Aaron Cushner, Andreas Rynes and the System Center series editor, Mitch Tulloch. This is the first in a series of unique (and free) ebooks that bring System Center experts together to discuss designing, deploying, and troubleshooting some of the most complex and mission critical aspects of key System Center capabilities.”

    … and

    “Our objective with this book is to provide a framework for runbook design and IT process automation to help you get the most out of System Center Orchestrator 2012 and to help you utilize Orchestrator in concert with the rest of the System Center for an enterprise-wide and systematic approach to process automation.”

    Click here to get the book!





  • VMM Repro: Simple tracing for VMM


    VMM 2012 SP1
    VMM 2012 R2

    VMM Repro captures detailed information on systems involved in a reproducible VMM problem. This is a verbose data capture and should be run for a short period of time while the problem is reproduced.


    • It will test to see that the right version of VMM is installed, and will point you to my other tool for older systems as needed.
    • Displays explanation of what the tool is going to do.
    • Log files are renamed after the PC, plus the date and time.
    • Log files are automatically zipped up.
    • The folder that contains the zip file opens in Explorer for you.

    When to use?

    When you’ve made it through the items below and are still are no closer to resolution you should consider capturing a trace.

    • Review Application and System Event logs on the SCVMM server and Hosts involved in the problem
    • Install all update and verify using Windows Update
    • Verify required basics again:
      • DNS resolution in all directions
      • Account running VMMService is located in ‘Administrators’ and ‘Virtual Machine Manager Servers’ groups on SCVMM server and Hosts
      • Hyper-V, Cluster and so on are healthy

    How to use?

    Elevated PowerShell prompt required.

    • Change to the directory of the script.
    • Unblock the script as it was downloaded from the internet: unblock-file .\VMMReprov1.ps1
    • Kick off with ‘.\VMMReprov4.ps1’.
    • If you get an error about not being digitally signed:
      • Set-ExecutionPolicy Unrestricted
      • (When done with the script you can run ‘Set-ExecutionPolicy Restricted’ if you like)

    Troubleshooting – If ‘How to use?’ does not work

    If the script does not kick off it is for one of three reasons:

    1. PowerShell console is not being run as Administrator.
    2. Scripts are not allowed to run on your system in PowerShell. Type ‘set-executionpolicy unrestricted’.
    3. The script has been blocked by the OS as it was downloaded from the internet. See illustration below for instructions.

    If you get this error it means that the script has been blocked. To unblock scripts (especially ones you pull from the internet) see the next picture.


    Go to properties of the script and unblock it.





    Download here:

  • SCOtrace – Orchestrator Trace Tool

    What is SCOtrace?

    SCOtrace increases the Orchestrator log level, provides time for you to reproduce a problem, then sets all log levels back to their lowest (normal) level. Log files are then gathered for you in a central location.

    How does it work?

    Orchestrator activity is logged all the time, whether you know it or not. The log level is very low and does not impact your system. There are times, however, when it would be ideal to generate very detailed logs in order to better understand a problem. Following this article it is possible to increase the level of logging significantly (The article discuses Opalis, but it works the same for Orchestrator).

    Why would I need this?

    The idea of this tool is to automate the collection of verbose logs for you or a Microsoft Support Professional to review. You can perform these exact steps following the article mentioned above, but this way is simply quicker.

    Where should I use this?

    Run SCOtrace on the Management server, and if you think Runbook servers are involved run it there as well. Only the Orchestrator components you have installed on a system will be traced.

    How do I run SCOtrace?

    The latest version of SCOtrace can be downloaded here.

    Remove the .txt extension from the file and run from an elevated command prompt.

    Note: The latest file from each log directory is gathered. You may find some files that are old. Ignore these.








  • Update Rollup 2 for System Center 2012 SP1

    imageThis blog focuses on VMM and Orchestrator, and for VMM alone there are 20 fixes in this update. Many of these corrections relate to networking. This update has helped a number of customers I have worked with resolve their issues entirely. Even in a controlled classroom environment (I’m out delivering training this week. Go OKC!) the update has made a strong positive impact. As for Orchestrator, there are a few fixes as well, both of which stemmed from customer cases I worked involving Oracle results and SNMP Traps.


    This is one update that should be installed immediately, or as soon as your outage window allows. There is one caveat: ‘In order to install Update Rollup 2 package for System Center 2012 SP1-  Virtual Machine Manager, you will need to uninstall Update Rollup 1 for System Center SP1 - Virtual Machine Manager package from your system.’ See this blog post for a great deal of information on the subject. Also, if you need to uninstall SC VMM for some unrelated reason, you will need to remove this update first to do so.

    Grab the update here. Description of Update Rollup 2 for System Center 2012 Service Pack 1

    A list of fixes in the update…

    Virtual Machine Manager Server (KB2826405) and Administration Console (KB2826392)

    Issue 1

    The SUSE Linux Enterprise Server 11 operating system is missing from the Linux OS list.

    Issue 2

    A virtual machine cannot start after migration from Windows 7 to Windows 8 when the DiscardSavedState method is used.

    Issue 3

    A connection to the VMware virtual machine remote console session cannot be established.

    Issue 4

    Externally published VMNDs are filtered incorrectly.

    Issue 5

    When you remove a virtual switch extension property or edit a virtual switch extension manager connection string, a user-interface generated script also removes the HostGroups that are associated with VSEM.

    Issue 6

    UPPSet is not set on a physical network adapter when you add the network adapter to a team and when the network adapter is the first in the list of network adapters.

    Issue 7

    The default gateway is missing on a host virtual network adapter after you add a second physical network adapter to the logical switch.

    Issue 8

    Static IP pool that has the first address in a subnet fails for external network type.

    Issue 9

    VMM crashes during host refresher when VMM is unable to create a CimSession with the remote host.

    Issue 10

    Standard (legacy) virtual switch creation on Windows 8 hosts with management virtual network adapter does not preserve the IP properties of the physical network adapter.

    Issue 11

    The administration user interface crashes with a NullReferenceException error when you click Remediate on a host instead of a virtual network adapter.

    Issue 12

    The Virtual Machine Manager user interface displays a network adapter in a "Not Connected" state.

    Issue 13

    The Virtual Machine Manager stops responding with high CPU usage for five to ten minutes when you configure a VMND that has 2,000 network segments.

    Issue 14

    The host virtual network adapter property for a management adapter does not show port classification.

    Issue 15

    Live Migration fails at 26 percent when the network adapter is attached to an isolated virtual machine network.

    Issue 16

    The Virtual Machine Manager Service crashes when a virtual machine that does not have a port profile is migrated to a cluster by using a logical switch that has a default port profile set.

    Issue 17

    Running Dynamic Optimizer on a cluster with incompatible host CPUs causes a Virtual Machine Manager Service crash.

    Issue 18

    The Host refresher crashes for any host that has the RemoteFX role enabled.

    Issue 19

    The minimum memory for dynamic memory greater than 32GB is a security risk.

    Issue 20

    The status of the network adapter is displayed as Not Connected in Virtual Machine Manager.







  • Update-Help: VMM PowerShell

    PowerShell iconIf you've opened the VMM 2012 SP1 PowerShell console and performed a command to get help on cmdlets, you may be in for a surprise. Examples are not included out of the box, and there were tons of great examples in VMM 2008 R2. The syntax below should, but does not, output all help including example use.

    get-help <cmdlet-name> -full


    There is a simple way of resolving this issue. Open the ‘Virtual Machine Manager Command Shell’ located in Start. Open this elevated as administrator. Now type the following and updated help will be available.


    When this is done you can use a non-elevated VMM PowerShell and updated help will be available.






  • Poster: Networking in Virtual Machine Manager

    There is a new post on the Download Center of networking in Virtual Machine Manager. It’s a bit intimidating posting this official poster right after my own attempts (see my previous post), but they all offer different ways of depicting the same abstract ideas. Pull down a copy of the post and use it for reference when you just can’t seem to put the pieces together. Enjoy!








Jonathan's Virtual Blog

Jonathan's Virtual Blog