April, 2010

The Beta for Virtual Machine Servicing Tool 3.0 is now available. This next version of the tool builds on the version 2.1’s functionality, adding powerful new features designed to streamline the process of keeping your offline virtual machines, templates and VHDs up-to-date with the latest operating system and application updates, without introducing vulnerabilities into your IT infrastructure. VMST 3.0 also automates servicing a Windows Server 2008 R2 Hyper-V host clusters leveraging live migration.

Join the Beta for Virtual Machine Servicing Tool 3.0 and tell us what you think! Click here to download the Beta materials.

Customer Benefits 

Carmen Summers | SCVMM Community Lead PM

There are many updates available for SCVMM and the technologies it manages. Staying up to date can be difficult. There is a tool, the VMMCA, that will check the configuration and updates installed on the SCVMM Server and any Hosts specified and produce a simple report. This covers a great number of required updates.

Now, how about all the updates that have been released since the VMMCA was last released? Jonathan Jordan published a blog article that lists all of the updates the VMMCA already checks for as well as those that you will have to verify on your own.

Thanks, Jonathan!

Carmen Summers | SCVMM Community Lead PM

We should be rolling out a KB article on this pretty soon (if it's not already) but I figured a heads up here would be a good idea too.

Symptoms

The vmmservice may crash while attempting to add a VMware vCenter environment that uses distributed switches. 

Cause

System Center Virtual Machine Manager 2008 does not currently support the use of distributed switches.  This is a new feature from VMware introduced with VI4. 

The following output is from the vmm_mpsreport trace that shows a VMware vCenter environment being added under management by VMM.  In the following exception, note the "System.InvalidCastException: Unable to cast object of type 'Microsoft.VirtualManager.VMWareService.VirtualDeviceBackingInfo' to type 'Microsoft.VirtualManager.VMWareService.VirtualDeviceDeviceBackingInfo'."

85000 84661,20:32:16.616 03-31-2010,0x115C,0x12E8,4,WatsonExceptionReport.cs,755,0x00000000,Unhandled exception caught.,{00000000-0000-0000-0000-000000000000},1,
85001 84662,20:32:16.616 03-31-2010,0x115C,0x12E8,4,WatsonExceptionReport.cs,756,0x00000000,Unhandled exception.,{00000000-0000-0000-0000-000000000000},1,
85002 84663,20:32:16.617 03-31-2010,0x115C,0x12E8,4,WatsonExceptionReport.cs,756,0x00000000,"System.InvalidCastException: Unable to cast object of type 'Microsoft.VirtualManager.VMWareService.VirtualDeviceBackingInfo' to type 'Microsoft.VirtualManager.VMWareService.VirtualDeviceDeviceBackingInfo'.
85003    at Microsoft.Carmine.VMWareImplementation.VmwVMNetworkAdapter.get_ConnectedPort()
85004    at Microsoft.Carmine.VMWareImplementation.VmwVMNetworkAdapter.get_ConnectedSwitch()
85005    at Microsoft.VirtualManager.Engine.BitBos.VMRefresherBase.AddNewNICToVM(IVMNetworkAdapter vmAdapter)
85006    at Microsoft.VirtualManager.Engine.BitBos.VMRefresherBase.UpdateNICs(IVMComputerSystem vmComputer)
85007    at Microsoft.VirtualManager.Engine.BitBos.VMRefresherBase.UpdateFullVMObjectToCarmine(IVMComputerSystem vmComputer; VMData vmData; UpdateRequired updateRequired; Boolean vmObjectHasChanged)
85008    at Microsoft.VirtualManager.Engine.BitBos.VMRefresherBase.UpdateFullVMObject(IVMComputerSystem vmComputer)
85009    at Microsoft.VirtualManager.Engine.BitBos.VMRefresherBase.AddFullVMObjectToCarmine(IVMComputerSystem vmComputer)
85010    at Microsoft.VirtualManager.Engine.BitBos.VMRefresherBase.AddorUpdateNewlyFoundVM(IVMComputerSystemSummary vmComputer)
85011    at Microsoft.VirtualManager.Engine.BitBos.VMRefresherBase.AddNewVirtualMachines(List`1 hostVirtualMachines; List`1 toAdd)
85012    at Microsoft.VirtualManager.Engine.BitBos.VMRefresherBase.RunLightRefresher()
85013    at Microsoft.VirtualManager.Engine.BitBos.VMRefresherBase.UpdateHostandVMs(VMRefresherType refresherType; Guid vmObjectId; VM tempVm)
85014    at Microsoft.VirtualManager.Engine.BitBos.VmLightRefresher.RefreshData(HostReference hostRef)
85015    at Microsoft.VirtualManager.Engine.RefreshDriver`1.RefreshThreadFunction(Object obj)",{00000000-0000-0000-0000-000000000000},1,

The following events may also be logged in the VM Manager log:

Log Name:      VM Manager
Source:        Virtual Machine Manager
Date:         
Event ID:      19999
Task Category: None
Level:         Error
Keywords:      Classic
User:          N/A
Computer:     
Description:
Virtual Machine Manager (vmmservice:4676) has encountered an error and needed to exit the process. Windows generated an error report with the following parameters:
Event:VMM20
P1(appName):vmmservice
P2(appVersion):2.0.4271.0
P3(assemblyName):VMWareImplementation
P4(assemblyVer):2.0.4273.0
P5(methodName):M.C.V.VmwVMNetworkAdapter.get_ConnectedPort
P6(exceptionType):System.InvalidCastException
P7(callstackHash):f20b

Log Name:      VM Manager
Source:        Virtual Machine Manager
Date:         
Event ID:      1
Task Category: None
Level:         Error
Keywords:      Classic
User:          N/A
Computer:     
Description:
System.InvalidCastException: Unable to cast object of type 'Microsoft.VirtualManager.VMWareService.VirtualDeviceBackingInfo' to type 'Microsoft.VirtualManager.VMWareService.VirtualDeviceDeviceBackingInfo'.

Resolution

This is a known issue and will be resolved with the next rollup for System Center Virtual Machine Manager 2008 R2 which is scheduled to be released in Q2 2010.

Mike Briggs | Senior Support Escalation Engineer

If you're an administrator for System Center Virtual Machine Manager, at times you may come across cases where a host shows as "Needs Attention" or "Not Responding" or even as "Intermittent connectivity loss to Host".  So how do you go about troubleshooting this type of issue?  Here are a couple tips to get you started:

1. Install the Microsoft Baseline Configuration Analyzer (MBCA) and then scan the problem server with the System Center Virtual Machine Manager 2008 and 2008 R2 Configuration Analyzer (VMMCA).
    a. Run the VMMCA and select "Windows Server-based host" and enter the name of the server experiencing this problem
    b. Upon completion, provide screenshot of report to Microsoft
    c. Install any identified missing hotfixes as directed by the VMMCA.

2. Verify that the recommended updates are applied:

KB962941: Recommended hotfixes for System Center Virtual Machine Manager 2008

KB2397711: Recommended hotfixes for System Center Virtual Machine Manager 2008 R2

3. Identify the problem server (managed host).  Is the server currently experiencing the issue?  If so then on the problem server open an elevated command prompt and run the following command:

net stop winrm

Note the time it takes to stop this service.  It should only take a few seconds for the service to stop so anything longer could indicate a problem with winrm being backed up.  If this is the case then you may also notice that the problem can be resolved, albeit temporarily, by a host reboot.  If this is the case then the solution may be to configure the Windows Remote Management service to run in a separate Svchost.exe process. To do this, open an elevated command prompt and run the following commands:

net stop winrm

and then

sc config winrm type= own

Note: there is a space after the "=" in the line above.

If the command completes successfully you should see the following output:

[SC] ChangeServiceConfig SUCCESS

Note: There are occasions in Windows 2008 servers where the Network Location Awareness (NLA) service can become temporarily backed up.  Since NLA shares the same instance of SVCHOST with WinRM, if NLA becomes backed up then WinRM ends up in a waiting state and sometimes does not recover.  To workaround this issue with NLA we can run WinRM in its own instance of SVCHOST which is what the procedure above accomplishes.

4. Another possible cause of this issue can be if a "Restrictive Groups" group policy is removing the VMM server machine account from the local administrators group on the host computer.  This issue is discussed in further detail in KB969164.  If this is the case, move the VMM server and Host computers to a new OU that is blocking inheritance of all group policy objects.

5. Some additional causes of this problem include:
    a. The VMM Agent is not running.
    b. The anti-virus software is scanning ports or protocols

Mike Briggs | Senior Support Escalation Engineer

The App-V Team blog: http://blogs.technet.com/appv/
The WSUS Support Team blog: http://blogs.technet.com/sus/
The SCMDM Support Team blog: http://blogs.technet.com/mdm/
The ConfigMgr Support Team blog: http://blogs.technet.com/configurationmgr/
The OpsMgr Support Team blog: http://blogs.technet.com/operationsmgr/
The SCVMM Team blog: http://blogs.technet.com/scvmm/
The MED-V Team blog: http://blogs.technet.com/medv/
The DPM Team blog: http://blogs.technet.com/dpm/
The OOB Support Team blog: http://blogs.technet.com/oob/
The Opalis Team blog: http://blogs.technet.com/opalis

SCVMM can use an existing virtual machine as a source for future virtual machines. The source in this case can reside on a virtual machine host or on the library server. Creating a virtual machine from a virtual machine is referred to as cloning although the 'New VM' wizard experience does not refer to cloning directly.  It simply lists 'virtual machine' as a source type.

Cloning a virtual machine involves copying the virtual hard disks, any memory state, and associated configuration files to the host. The source is not deleted or modified. To clone a virtual machine, it must be in a powered-off state or saved state if it is located on a virtualization host. Virtual machines in the library are in a valid state for cloning. Cloning is also supported for virtual machines with snapshots.

Cloning Virtual Machine Identity

Note that a cloned virtual machine is an identical copy of the source, including Active Directory identity, SID, computer name, and MAC address statically assigned to the virtual network adapters. If both virtual machines are powered on and connected to the same network, you will have two identical computers potentially communicating on the same network.  Please plan and account for this accordingly.

Mike Briggs | Senior Support Escalation Engineer

You can use the new TechNet Wiki site to share your VMM expertise with the community, and to learn from others who have already shared their knowledge. Anyone who signs on to the site by using a Windows Live ID can contribute – so go ahead and sign up! You can share things like troubleshooting information, how-to information, best practices, overviews, and cross-platform advice, such as your experience managing Hyper-V hosts and virtual machines by using VMM. For information about what makes a good article, see Wiki: What Makes a Great Wiki Article.

You can find the TechNet Wiki at http://social.technet.microsoft.com/wiki/. You’ll also notice that if you go to the TechNet home page, there is now a Wiki tab at the top.

The TechNet Wiki already contains some great VMM content. You can find VMM-related Wiki content in several ways:

· From the TechNet Wiki site, enter a search term such as “VMM” in the search box at the top of the page.

· On the TechNet Wiki home page, under Portals, click Virtualization. Then look under the System Center Virtual Machine Manager heading for links to VMM-related articles. Keep in mind that this portal page is manually updated with links to new articles. If you find any other VMM articles that you would like in this list, go ahead and add them!

· From the Wiki home page, you can click the link to the tag cloud, or go directly to the tag cloud page to search by popular tags. VMM-related articles are typically tagged with at least “VMM,” “SCVMM” and “Virtual Machine Manager”, so you can click any one of these. If you create any new VMM-related Wiki articles, please try to add these tags.

You can also receive notifications about articles via e-mail or RSS feeds.

· To keep track of changes to a specific article, you can subscribe to the article. When you view an article, you’ll see the options to Subscribe to Article (RSS) and Share via Email in the right pane (shown below).

· You can subscribe via RSS to a specific search term or tag (like “VMM”). To do this, type the search term or tag in the search box. When the search results are returned, you’ll see a Subscribe via RSS link on the right side (shown below).

For more information about monitoring options, see Wiki: How to Monitor Content.

You may notice that some articles consist of previous SCVMM team blog posts that have been ported to the Wiki. Several of these include troubleshooting information or tips provided by Microsoft Customer Service and Support (CSS). The advantage to the Wiki is that you can add to these articles (or create new ones) based on your own personal experience. So go ahead – sign up and share your knowledge today!

Tessa Wooley |Technical Writer, Microsoft

I know I've mentioned this here before but in case you haven't seen the TechNet Wiki or had a chance to play with it, let me tell you, you should definitely check this out.  So what makes the TechNet Wiki so different?  Well for starters it's community driven, meaning that it's not just a forum for a bunch of MS people to post docs in a one-way fashion, but it's a way for you, the IT Pro, to post articles yourself.  You can even comment and edit docs written by others, including yours truly.  But don't take my word for it, our very own Keith Combs has a fantastic post covering just about everything you'd want to know.  You can check out his post here.

Enjoy!

J.C. Hornbeck | System Center Knowledge Engineer

Okay, maybe not, but whether you are new to Windows PowerShell scripting or consider yourself an expert, the System Center Virtual Machine Manager 2008 Scripting Guide is for you!

Because System Center Virtual Machine Manager (VMM) is built on Windows PowerShell, any action you that can do in the UI is an action you can also do using VMM cmdlets. The Scripting Guide provides examples of how to automate tasks and augment the functionality of the product. If you're new to using VMM cmdlets, start by visiting the Introduction to help get your bearings. There, you can discover how to:

· Add the VMM snap-in.

· List all VMM cmdlets.

· Use VMM cmdlets.

· Connect to the VMM server.

After you have explored the VMM cmdlets that are available to you, you can peruse the remaining sections, which contain examples for managing the various aspects of VMM such as hosts, virtual machines, and a VMware environment.

One of the goals of the Scripting Guide is to help administrators write scripts that can be used to perform repetitive tasks. For example, if you want to create a virtual machine, you can do this using the UI. However, you can only create one virtual machine at a time. If you want to create, say, 50 virtual machines, it would take quite a few trips through the New Virtual Machine wizard to accomplish this. However, we have provided a couple of scripts that you can use to create multiple virtual machines. If you want to use a template to create virtual machines, check out CreateNewVMsFromTemplate.ps1. After you specify the template you want to use, and the number of virtual machines to create, the script will take care of the rest. Or, if you want to create virtual machines from scratch, you can use CreateNewVMs.ps1. This script creates a hardware profile, a guest operating system profile, and a template using these profiles. It then creates the number of virtual machines you've specified, and deploys each of them to the host with the highest rating.

You may have heard about rapid provisioning. This is a feature of VMM 2008 R2 where you can create a virtual machine from an existing hard disk file stored locally on a destination host instead of copying it from the library, which saves time when creating the virtual machine. This feature is only available using the VMM cmdlets, not through the UI. The script entitled RapidProvisionVM.ps1 walks you through the process. If you have an answer file that you'd like to use during virtual machine creation, you can use the RapidProvisionVMwithAnswerFile.ps1 script.

We've also demonstrated how to use core Windows PowerShell cmdlets in a VMM environment. For example, CheckForPendingReboot.ps1 imports computer names from a CSV file, looks for the RebootPending registry key on those computers, and, if present, restarts the computers for you. This script contains no VMM-specific cmdlets; however, this task is very useful in the VMM environment.

Another script that combines core Windows PowerShell cmdlets with VMM cmdlets is SignScriptsInLibrary.ps1. You can store your scripts in the VMM library, but if your Windows PowerShell execution policy is set to run only signed scripts, you must sign all of the scripts in the VMM library before you can run them. The SignScriptsInLibrary.ps1 script looks for all scripts you have stored in your library and signs them for you. But, don't forget to sign SignScriptsInLibrary.ps1 prior to running it, or it will fail on you!

We've also built script-based suggestions from customers. For example, VMM does not automatically perform load balancing among hosts in a cluster. However, based on a request from a customer, we created the LoadBalanceVMsInCluster.ps1 script, which uses the available memory of hosts in a cluster in conjunction with the memory usage of the virtual machines to balance the load across the cluster. As a matter of fact, the CheckForPendingReboot.ps1 script mentioned earlier was also a suggestion from a customer.

If you have any comments on the VMM 2008 Scripting Guide, or suggestions for future scripts, please send e-mail to mailto:%20scvmfdbk@microsoft.com?subject=Scripting%20Guide. You can also download the VMM 2008 R2 Scripting Guide to read it offline.

Susan Hill, Senior Technical Writer, Microsoft

This article describes the performance metrics details that System Center Virtual Machine Manager (SCVMM) collects and how to make sense of performance information that is displayed in SCVMM. 

SCVMM uses a periodically running performance refresher that gathers performance data from hosts and virtual machines (VMs) and stores it in the VMM database to be used by the Intelligent Placement feature (this is the nifty star rating prioritization of hosts that you see when you try to migrate a VM or when you try to create a new VM). Besides Intelligent placement purposes, the UI (SCVMM Administrator Console) also displays this performance data. This refresh happens every 9 minutes by default and can also be triggered as a side effect of certain SCVMM actions (explained below).  With every refresh, SCVMM gets the average of every metric over the past 9 minutes. Here is how it’s done on different platforms.

Hyper-V – Hyper-V hosts** expose certain performance counters for a physical host and the virtual machines residing on it. SCVMM Server talks to each Hyper-V host over WSMAN to gather these performance metrics for the host and its VMs. SCVMM collects virtual machine performance metrics directly from its host because of some limitations of what a virtual machine understands about its true performance. Here a few interesting metrics explained.

1- Host CPU Average is exposed in SCVMM console as the “%Total Run Time” counter of “Hyper-V Hypervisor Logical Processors(_Total)” counter set*** averaged over past 9 minutes.  This counter captures processor utilization being caused on the host (parent partition) and by the VMs. This value and other host performance metrics could be refreshed on demand as part of Refresh-VMHost or Refresh task in the SCVMM Console. Note that this value is different than what the Host’s Task Manager or %Processor Time(_Total) counter would show, which only considers parent partition utilization.

2- VM CPU Average is exposed in SCVMM console as the “%Guest Run Time” counter of “Hyper-V Hypervisor Virtual Processor(_Total)” counter set . This is the percent of time spent by the virtual processor(s) that run the VM.  Since Hyper-V allows control of how many processors and what percent of their capacity can be assigned to a VM, the Guest Run Time counter value can never exceed the specified limit(see image below). Note that this is very different from the VM CPU% displayed in Hyper-V console, in which CPU utilization is displayed as VM Host Logical Processors utilization.

For example: The Hyper-V VM settings shown in the image below have been intentionally allotted 60% of 2 processors capacity. So, if this VM’s utilization is 100% from inside the guest, this VM is actually using 60% of the 2 virtual processors assigned to it from a Host perspective. The SCVMM console would show ~60% in this case, Hyper-V console will show ~30% for this VM.

The reason the Hyper-V console shows 30% is because the physical system has 4 logical processors. Thus 60% of 2 logical processors assigned to the VM is 30%. The 60% that SCVMM shows in the UI is averaged/tiered over the last 9 minutes and is not an instant value for a point-in-time CPU percentage of a VM.

*= default time interval of performance refresh, which is configurable through registry for troubleshooting purposes

**= Windows Server computer with HyperV role enabled

***= All counter sets on a computer can be seen using Performance Monitor (perfmon.exe).

VMware – SCVMM talks to Virtual Center server to get performance metrics for all its Hosts and their VMs.

Mike Briggs – Senior Support Escalation Engineer

A new KB is in the process of being published on an SCVMM update for Red Hat Linux running on Hyper-V. This information is currently available as a blog post, containing the same information. Those of you wanting the 'official' KB will have it soon enough.

Read the article here: http://blogs.technet.com/jonjor/archive/2010/04/02/scvmm-reports-error-10637-when-red-hat-linux-selected-as-the-operating-system.aspx

URL of KB when published: http://support.microsoft.com/kb/2022557

JonJor