Jonathan's Virtual Blog : Updates/Hotfixes

Are All of My Servers Really Up to Date?

Jonathan — Tue, 21 Apr 2009 20:47:00 GMT

With a single command you can pull a list of all installed updates. Use this to compare patch levels on all systems so you can easily target those not meeting the bar.

We start with simple commands, then build on these to create robust queries with optional CSV and HTML formatting. I think of these commands, which I use on a daily basis, as basics for my Toolkit. If you don’t have a Toolkit I suggest you create one now. Place your often used tools in a folder such as ‘C:\Toolkit’ so that they are always at hand. Grab the first five items under ‘Top 10 Downloads’ from Sysinternals.com to round out your basics.

TIP: Go one step further and sign up for a Live Mesh account with your Windows Live ID. With this you can keep your Toolkit directory, and other directories, synched up across all machines you designate. The folders will be available privately online as well. Further, remote control is built-in for all Mesh enabled systems. Sweet! Clearly remote capabilities are geared more towards workstations than servers, but I use it for both.

And yeah, it’s free. Get started here.

Basic Commands

Warm up by opening an elevated command prompt on your SCVMM server (or any server for that matter). If your command window does not show ‘Administrator:’ as below, you are not running an elevated console.

> Single command to pull updates.
wmic qfe

http://support.microsoft.com/?kbid=963660 VMM2008R2-03 Update                        KB963660
                       NT AUTHORITY\SYSTEM 2/26/2009
http://support.microsoft.com/?kbid=967062 VMM2008R2-03 Update                        KB967062
                       NT AUTHORITY\SYSTEM 2/26/2009

You should see a torrent of information scroll through the command window. This is a list of all QFEs (Hotfixes and Updates) installed, along with a good bit of extraneous info.

> Now let's work on formatting. This is my everyday method.
wmic qfe list brief

Update KB963660 NT AUTHORITY\SYSTEM 2/26/2009
Update KB967062 NT AUTHORITY\SYSTEM 2/26/2009

Now some of the noise is removed and it is easier to read the list of updates.

TIP: Is KBXXXXXX installed on this system?!? I can only remember the last three digits of the update anyway… Easy to check. Just pipe your output to ‘find.’

wmic qfe list brief | find /i "660"

Update KB963660 NT AUTHORITY\SYSTEM 2/26/2009

Apparently it is installed. Nice.

> This is good, but we can do better.
wmic qfe get csname,hotfixid,description

VMM2008R2-03 Update KB963660
VMM2008R2-03 Update KB967062

Now we're getting somewhere. We've selected the header for only those columns we are interested in and have returned a clean list of updates along with the system name.

This much you can keep in your head. You might want to create a batch file for items in the next section.

Advanced Queries

Now that you've waded into the deep end let's perform some real work. These are the commands to place in a batch file for your Toolkit. The steps below assume you have already created a ‘C:\Temp’ directory on your system. CSName is not required for the next two commands; it is implied.

> Create a CSV file from the output.
wmic /output:c:\temp\qfe.csv QFE GET HotFixID,Description /format:csv

The resulting CSV file lists these items, ready to be opened in Excel: Node,Description,HotFixID

> CSV files are not as interesting as HTML, so let's shift output formats.
wmic /output:c:\temp\qfe.html QFE GET HotFixID,Description /format:htable

Now you have a clean multicolored table with results, complete with column headers.

Remote Systems

The only thing left to do is perform this same work on remote machines. First a single machine, then a list of systems. Replace <remotesystem> with your remote system name. Note that we add CSName to pull the remote name.

> List remote system updates.
wmic /node:"<remotesystem>" /output:c:\temp\qfe_remote.html QFE GET CSName,HotFixID,Description /format:htable

If this did not work for you keep in mind that you must be an administrator on the remote machine, and Remote Management must be enabled.

> Finally, let's feed a list of server names and create a single HTML file. Create a file named 'serverlist.txt' with one system per line.
wmic /node:@serverlist.txt /output:c:\temp\qfe_serverlist.html QFE GET CSName,HotFixId,Description /format:htable

That's it. You should have a table similar to this.

Enjoy your new tools. Thanks for reading!

For further reference:

MSDN - WMI Command Line Tools
http://msdn.microsoft.com/en-us/library/aa394531.aspx

Hotfix Rollup Released for SCVMM - KB961983

Jonathan — Fri, 17 Apr 2009 19:12:00 GMT

Released April 14, 2009, a number of Hotfixes have been rolled up into a single download. I strongly recommend this update for all SCVMM Servers. This update may require a reboot. This update replaces ‘KB959596 - Description of the System Center Virtual Machine Manager 2008 update to address physical to virtual (P2V) issues.' You should still install updates from KB962941. Best of luck!

Description of the hotfix rollup package for System Center Virtual Machine Manager 2008: April 14th, 2009
http://support.microsoft.com/kb/961983

This update will be provided automatically by 'Microsoft Update,' or it can be manually downloaded here. If installing manually, open an elevated command prompt and execute (without the quotes): “msiexec /update vmmServer64Update.msp BOOTSTRAPPED=1”

Issues Addressed
VMware virtual machine migration
Hyper-V VMs showing “unsupported cluster configuration”
Differencing disk issues
Agent status on Cluster nodes
Replication software causing “unsupported cluster configuration"
Error 2951 with VMware Virtual Center

Summary

Install updates from KB962941 and KB961983
KB959596 is no longer necessary if KB961983 is installed.

Recommended hotfixes for System Center Virtual Machine Manager 2008
http://support.microsoft.com/kb/962941

Recommended hotfixes for System Center Virtual Machine Manager 2008

Jonathan — Wed, 11 Feb 2009 20:59:00 GMT

Here it is. We have a new KB that details updates and hotfixes required for SCVMM, Hyper-V and Clustering. Read the article here.

I've commented on the hotfixes inline below, providing a Support Engineer's perspective.

________________________________________________

Required hotfixes for Hyper-V servers

Install these. No question.

956589 (http://support.microsoft.com/kb/956589/ ) Description of the Hyper-V update for issues that may occur when you manage the Hyper-V role on the 64-bit editions of Windows Server 2008 by using SCVMM
956774 (http://support.microsoft.com/kb/956774/ ) A Background Intelligent Transfer Service (BITS) client cannot handle files that have paths that contain the volume GUID in Windows Server 2008 or in Windows Vista
950050 (http://support.microsoft.com/default.aspx?scid=kb;EN-US;950050/ ) Description of the update for the release version of the Hyper-V technology for Windows Server 2008

Note If these hotfixes are missing from a Hyper-V server, the System Center Virtual Machine Manager 2008 Admin Console lists the server status as Needs Attention. Hotfix 956774 should be installed on the Hyper-V servers and on System Center Virtual Machine Manager 2008 Server.

Back to the top

Recommended hotfixes for System Center Virtual Machine Manager 2008 Server

I would change this from 'Recommended' to 'Required' based on experience.

958124 (http://support.microsoft.com/kb/958124/ ) A wmiprvse.exe process may leak memory when a WMI notification query is used heavily on a Windows Server 2008-based or Windows Vista-based computer
954563 (http://support.microsoft.com/kb/954563/ ) Memory corruption may occur with the Windows Management Instrumentation (WMI) service on a computer that is running Windows Server 2008 or Windows Vista Service Pack 1
955805 (http://support.microsoft.com/kb/955805/ ) Certain applications become very slow on a Windows Server 2008-based or Windows Vista S955805-based computer when a certificate with SIA extension is installed
959596 (http://support.microsoft.com/kb/959596/ ) Description of the System Center Virtual Machine Manager 2008 update to address physical to virtual (P2V) issues

Back to the top

Recommended hotfixes for the Hyper-V servers

Install ALL hotfixes on ALL Hyper-V servers, whether SCVMM is involved or not. This is more than a recommendation based on cases I have worked.

957967 (http://support.microsoft.com/kb/957967/ ) Stop error message on a Windows Server 2008-based computer that has the Hyper-V role installed: "STOP 0x0000001A"
958124 (http://support.microsoft.com/kb/958124/ ) A wmiprvse.exe process may leak memory when a WMI notification query is used heavily on a Windows Server 2008-based or Windows Vista-based computer
954563 (http://support.microsoft.com/kb/954563/ ) Memory corruption may occur with the Windows Management Instrumentation (WMI) service on a computer that is running Windows Server 2008 or Windows Vista Service Pack 1
955805 (http://support.microsoft.com/kb/955805/ ) Certain applications become very slow on a Windows Server 2008-based or Windows Vista S955805-based computer when a certificate with SIA extension is installed

Back to the top

Recommended hotfixes for Hyper-V failover clusters

957311 (http://support.microsoft.com/kb/957311/ ) Recommended hotfixes for Windows Server 2008-based server clusters

SCVMM 2008 Releases Management Pack Update

Jonathan — Thu, 29 Jan 2009 03:04:00 GMT

An updated SCVMM management pack is now available for download. Read the announcement here!

Brief Description

The System Center Virtual Machine Manager (VMM) 2008 Management Pack for System Center Operations Manager 2007 monitors availability of all components of VMM 2008 and the availability, health, and performance of all virtual machines and virtual machine hosts that VMM manages.

Resolve Communication Issues in SCVMM - Part 1

Jonathan — Thu, 15 Jan 2009 22:51:00 GMT

Disable Task Offloading and Update WMI

Communications issues with SCVMM can take on the form of access denied, timeouts, and often as intermittent connectivity issues. The steps in this post are the same ones I use to resolve a good number of customer issues.

Feedback on this article is welcome. If you have technical questions please post them appropriately in the forums:

http://social.technet.microsoft.com/Forums/en-US/category/virtualmachinemanager/

Two types of issues are addressed here

1) Many new settings available to NIC drivers that do not always function as well as would be expected.

2) SCVMM relies heavily on WMI. The updates listed in this article resolve issues with WMI that will allow SCVMM to function properly.

This MDSN article lists all values available.

Using Registry Values to Enable and Disable Task Offloading
http://msdn.microsoft.com/en-us/library/aa938424.aspx

WMI

Install three WMI updates on all Windows 2008 server running Hyper-V (and therefore are SCVMM Hosts)

958124 A wmiprvse.exe process may leak memory when a WMI notification query is used heavily on a Windows Server 2008-based or Windows Vista-based computer

http://support.microsoft.com/default.aspx?scid=kb;EN-US;958124

954563 Memory corruption may occur with the Windows Management Instrumentation (WMI) service on a computer that is running Windows Server 2008 or Windows Vista Service Pack 1

http://support.microsoft.com/default.aspx?scid=kb;EN-US;954563

955805 Certain applications become very slow on a Windows Server 2008-based or Windows Vista SP1-based computer when a certificate with SIA extension is installed

http://support.microsoft.com/default.aspx?scid=kb;EN-US;955805

Network Settings

NOTE

- If working with a core server you may want to connect via remote registry and make these changes.
- All values should be set to '0' EXCEPT for the last one which is related to IPSEC, and thus Kerberos. Leave this one alone ---> 'HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\Ipsec\EnabledOffload'

1 - Locate all physical NICs in the registry under:
'HKLM\System\CurrentControlSet\Control\Class\{4D36E972-E325-11CE-BFC1-08002BE10318}'

2 - There will be folders with four digits, starting with '0000'. Locate the ones that show the physical NIC names on the right. Look for the 'DriverDesc' value on the right; this should have the name of a NIC such as 'HP NC360T PCIe Gigabit Server Adapter.' For each of these, make the changes below.

3 - Disable All Vendor specific Offloading
Set values for any entries below that include the word 'Offload' to '0' (Disabled)
*FlowControl

No description available

*IPChecksumOffloadIPv4
     Describes whether the device enabled or disabled the calculation of IPv4
checksums.
*TCPChecksumOffloadIPv4
     Describes whether the device enabled or disabled the calculation of TCP Checksum
over IPv4 packets.
*TCPChecksumOffloadIPv6
     Describes whether the device enabled or disabled the calculation of TCP checksum
over IPv6 packets.
*UDPChecksumOffloadIPv4
     Describes whether the device enabled or disabled the calculation of UDP Checksum
over IPv4 packets.
*UDPChecksumOffloadIPv6
     Describes whether the device enabled or disabled the calculation of UDP Checksum
over IPv6 packets.
*LsoV1IPv4
     Describes whether the device enabled or disabled the segmentation of large TCP
packets over IPv4 for large send offload version 1 (LSOv1).
*LsoV2IPv4
     Describes whether the device enabled or disabled the segmentation of large TCP
packets over IPv4 for large send offload version 2 (LSOv2).
*LsoV2IPv6
     Describes whether the device enabled or disabled the segmentation of large TCP
packets over IPv6 for large send offload version 2 (LSOv2).
*IPsecOffloadV1IPv4
     Describes whether the device enabled or disabled the calculation of IPsec headers
over IPv4.
*IPsecOffloadV2
     Describes whether the device enabled or disabled IPsec offload version 2
(IPsecOV2). IPsecOV2 provides support for additional crypto-algorithms, IPv6, and
co-existence with large send offload version 2 (LSOv2).
*IPsecOffloadV2IPv4
     Describes whether the device enabled or disabled IPsecOV2 for IPv4 only.
*RSS
     Receive side scaling
*TCPUDPChecksumOffloadIPv4
     Describes whether the device enabled or disabled the calculation of TCP or UDP
checksum over IPv4.
*TCPUDPChecksumOffloadIPv6
     Describes whether the device enabled or disabled the calculation of TCP or UDP
checksum over IPv6.

4 - Disable Offloading in Windows.
Use the following registry values to enable or disable task offloading for the TCP/IP protocol:
HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\TCPIP\Parameters\DisableTaskOffload

Setting this DWORD value to ‘1’ disables all of the task offloads from the TCP/IP transport. Setting this value to zero enables all of the task offloads.

Related Links
Network Issues with Windows Server 2008 RDP and VS/Hyper-V on Dell Servers
http://www.petri.co.il/network-issues-with-windows-server-2008-rdp-on-dell-servers.htm

Update for P2V with SCVMM 2008

Jonathan — Fri, 02 Jan 2009 17:48:00 GMT

Description of the System Center Virtual Machine Manager 2008 update to address physical to virtual (P2V) issues

http://support.microsoft.com/kb/959596

INTRODUCTION

This article describes the Microsoft Center Virtual Machine Manager 2008 issues that are fixed in the System Center Virtual Machine Manager 2008 update.

MORE INFORMATION

List of issues that are fixed

•

You use System Center Virtual Machine Manager 2008 to perform an offline physical to virtual (P2V) conversion. The computer that is running System Center Virtual Machine Manager 2008 is located on a different subnet from the source computer. However, both computers are configured to use an IP address range, such as 10.x.x.x. The offline P2V conversion fails with the following error message:

Error (3133) Virtual Machine Manager could not connect to source computer servername.domainname.com after it restarted into Windows PE (temporary computer name is p2v-ldjj). Automatic restart to the original operating system is scheduled to occur within 15 minutes. Recommended Action If servername.domainname.com does not restart in 15 minutes, manually restart it back into the original operating system using the boot menu

•

You convert a physical computer to a virtual machine by using VMWare VirtualCenter. One of the disks on the physical computer is larger than 256 gigabytes (GB). You add the VMWare VirtualCenter server to System Center Virtual Machine Manager 2008. When you try to create a virtual machine, the operation fails with the following error message:

VMM Cannot find VirtualHardDisk object. Ensure the library object is valid, then try the operation again. ID: 801

How to Troubleshoot Host Status - Quick Table

Jonathan — Tue, 30 Dec 2008 03:18:00 GMT

From the Microsoft TechNet SCVMM TechCenter

Article URL: http://technet.microsoft.com/en-us/library/bb740860.aspx

(Don't forget the VMMCA tool. This should always be top in your troubleshooting list. More info here.)

The table below works as a quick reference for why you may be experiencing connectivity issues. This will at least get you off and running in the right troubleshooting direction!

You can use Host Properties dialog box to check the overall status of a host, the status of the agent that is installed on the host, and the status of the virtualization software that is installed on the host. These are important indicators of the operational state of a host. Should a host status indicate a need for attention, see the Troubleshooting Guide for System Center Virtual Machine Manager (VMM)

(http://go.microsoft.com/fwlink/?LinkId=83589 [ http://go.microsoft.com/fwlink/?LinkId=83589 ] ).

Note

The host status values do not change in the VMM Administrator Console until the VMM server performs a refresh.

The following table describes the host status types and values.

Status Type

Status Value and Definition

Overall host status

· OK—no issues exist with any host status.

· Needs Attention—a problem exists with one or more host statuses.

Agent connection status

· Responding—the VMM server is able to communicate with the agent.

· Not Responding—the VMM server is unable to communicate with the agent.

· Access Denied—the agent is no longer associated with the VMM server.

Agent version status

· Current—the version of the agent is up to date.

· Needs update—the version of the agent must be upgraded to match the version of VMM server.

· Unsupported—the version of the agent is not supported for any VMM functions.

Virtual Server service status

· Running—the Virtual Server service is started.

· Stopped—the Virtual Server service is stopped.

Virtual Server version status

· Current—the version of Virtual Server is up to date.

· Needs update—the version of Virtual Server needs to be upgraded.

· Unsupported—the version of Virtual Server is not supported for any VMM functions.

To view the status values of a host

1. In Hosts view, navigate to the host group that contains the host, and then, in the results pane, double-click the host.

2. In the Host Properties dialog box, click the Status tab.

Technorati Tags: microsoft,scvmm,hyper-v,virtual,virtualization,windows

Updates! Updates! SCVMM, Hyper-V, Cluster

Jonathan — Mon, 29 Dec 2008 22:23:00 GMT

There are many updates for SCVMM and Hyper-V. Some are required just following product install, others are provided to offer greater functionality. Some updates/hotfixes address issues that may constitute a headache for you down the road. I'll list everything I know and provide recommendations. This post will change over time as new updates and issues arise. If you find a hotfix or update that you think should be posted here, please let me know. Include the environment conditions (Windows version, etc) and what the update addresses. Here's to being proactive!

Article Ratings

A – Required by product to operate correctly

B – Resolves known issues

C – Adds Functionality

JJ – Get it. This is on my short list when working with customers.

A – SCVMM – Hyper-V

956589 Description of the Hyper-V update for issues that may occur when you manage the Hyper-V role on the 64-bit editions of Windows Server 2008 by using SCVMM

http://support.microsoft.com/default.aspx?scid=kb;EN-US;956589

A – SCVMM | BITS

956774 A Background Intelligent Transfer Service (BITS) client cannot handle files that have paths that contain the volume GUID in Windows Server 2008 or in Windows Vista

http://support.microsoft.com/default.aspx?scid=kb;EN-US;956774

B – JJ – SCVMM | WMI

958124 A wmiprvse.exe process may leak memory when a WMI notification query is used heavily on a Windows Server 2008-based or Windows Vista-based computer

http://support.microsoft.com/default.aspx?scid=kb;EN-US;958124

B – JJ – SCVMM | WMI

954563 Memory corruption may occur with the Windows Management Instrumentation (WMI) service on a computer that is running Windows Server 2008 or Windows Vista Service Pack 1

http://support.microsoft.com/default.aspx?scid=kb;EN-US;954563

B – JJ – SCVMM - WMI

955805 Certain applications become very slow on a Windows Server 2008-based or Windows Vista SP1-based computer when a certificate with SIA extension is installed

http://support.microsoft.com/default.aspx?scid=kb;EN-US;955805

B – JJ – SCVMM | P2V

959596 Description of the System Center Virtual Machine Manager 2008 update to address physical to virtual (P2V) issues

http://support.microsoft.com/default.aspx?scid=kb;EN-US;959596

B – JJ – Hyper-V | VSS

956697 Windows Server 2008 Hyper-V VSS writer is not used during a backup job because of corrupted or invalid virtual machine configuration files

http://support.microsoft.com/default.aspx?scid=kb;EN-US;956697

B – Hyper-V | VSS

959978 Error message when you back up a Windows Server 2003-based virtual machine on a Windows Server 2008 Hyper-V-based computer: "GetWriterStatus FAILED for Selected writer [Microsoft Hyper-V VSS Writer], writer is in state [9] [VSS_WS_FAILED_AT_FREEZE]"

http://support.microsoft.com/default.aspx?scid=kb;EN-US;959978

B – Hyper-V |SCVMM

957967 Stop error message on a Windows Server 2008-based computer that has the Hyper-V role installed: "STOP 0x0000001A"

http://support.microsoft.com/default.aspx?scid=kb;EN-US;957967

B – Cluster | Hyper-V

957311 Recommended hotfixes for Windows Server 2008-based server clusters

http://support.microsoft.com/default.aspx?scid=kb;EN-US;957311

B – JJ - Hyper-V | Drive Letters

958184 Virtual machine backup operations fail in Windows Server 2008 when Hyper-V virtual machine files are saved on a volume that is mounted on a failover cluster by using a volume GUID

http://support.microsoft.com/default.aspx?scid=kb;EN-US;958184

B – Hyper-V | Cluster

958065 You cannot configure a Hyper-V virtual machine by using Windows Server 2008 Failover Clustering when the virtual machine uses a storage device that is managed by a third-party clustered file system or a third-party replication solution

http://support.microsoft.com/default.aspx?scid=kb;EN-US;958065

C – Hyper-V | Cluster

951308 Increased functionality and virtual machine control in the Windows Server 2008 Failover Cluster Management console for the Hyper-V role

http://support.microsoft.com/default.aspx?scid=kb;EN-US;951308

Technorati Tags: microsoft,scvmm,hyper-v,virtual,virtualization,windows

Ultimate Interactive How To for SCVMM

Jonathan — Mon, 29 Dec 2008 18:44:00 GMT

Need to add a Host? Click 'To add hosts'. Want to configure a SAN environment for VMM? There's a button for that too. This screen capture shows a few of the options available in the decision flow designed to take you to the right content for the job at hand.

Here's the link: http://technet.microsoft.com/en-us/scvmm/cc835256.aspx

Content below is the table version of the interactive decision flow located here:

http://technet.microsoft.com/en-us/scvmm/cc835256.aspx

Virtual Machine Manager 2008 Decision Flow

Updated: October 21, 2008

To use this high-level decision flow to help you implement System Center Virtual Machine Manager (VMM) 2008, follow the path from top to bottom. After each decision point, click the link to review content about that feature. To view a flowchart version of this table, see http://go.microsoft.com/fwlink/?LinkId=126290 [ http://go.microsoft.com/fwlink/?LinkId=126290 ] .

SCVMM Decision Flow Table

Step	Statement	Decision
1	Do you have VMM 2007 or VMM 2008 Beta installed?	If Yes, go to step 2. If No, go to step 3.
2	To Migrate to VMM 2008	See http://go.microsoft.com/fwlink/?LinkId=125913 [ http://go.microsoft.com/fwlink/?LinkId=125913 ] Go to step 4.
3	To install VMM 2008 components	See http://go.microsoft.com/fwlink/?LinkId=125914 [ http://go.microsoft.com/fwlink/?LinkId=125914 ] . Go to step 4.
4	Do you want to add hosts?	If Yes, go to step 5. If No, go to step 6.
5	To add hosts	See http://go.microsoft.com/fwlink/?LinkId=125921 [ http://go.microsoft.com/fwlink/?LinkId=125921 ] . Go to step 6.
6	Do you want to add a VMware Infrastructure 3 Environment?	If Yes, go to step 7. If No, go to step 8.
7	To add a VMware VirtualCenter Server	See http://go.microsoft.com/fwlink/?LinkID=128560 [ http://go.microsoft.com/fwlink/?LinkID=128560 ] . Go to step 8.
8	Do you want to configure a VMM library server?	If Yes, go to step 9. If No, go to step 10.
9	To configure a VMM library server	See http://go.microsoft.com/fwlink/?LinkId=125945 [ http://go.microsoft.com/fwlink/?LinkId=125945 ] . Go to step 10.
10	Do you want to integrate Operations Manager 2007 with VMM?	If Yes, go to step 11. If No, go to step 12.
11	To integrate Operations Manager 2007 with VMM	See http://go.microsoft.com/fwlink/?LinkId=125948 [ http://go.microsoft.com/fwlink/?LinkId=125948 ] . Go to step 12.
12	Do you want to configure delegated administration	If Yes, go to step 13. If No, go to step 14.
13	To configure delegated administration	See http://go.microsoft.com/fwlink/?LinkId=125951 [ http://go.microsoft.com/fwlink/?LinkId=125951 ] . Go to step 14.
14	Do you want to configure a SAN environment for use with VMM?	If Yes, go to step 15. If No, go to step 16.
15	To configure a SAN environment for VMM	See http://go.microsoft.com/fwlink/?LinkId=125953 [ http://go.microsoft.com/fwlink/?LinkId=125953 ] . Go to step 16.
16	Do you want to review optional post-installation tasks?	If Yes, go to step 17. If No, go to step 18.
17	To review optional post-installation tasks	See http://go.microsoft.com/fwlink/?LinkId=125954 [ http://go.microsoft.com/fwlink/?LinkId=125954 ] .
18	To review VMM Operations content	See http://go.microsoft.com/fwlink/?LinkId=125955 [ http://go.microsoft.com/fwlink/?LinkId=125955 ] .

Technorati Tags: microsoft,scvmm,hyper-v,virtual,virtualization,windows

How to Troubleshoot Slow BITS Performance, Hosts 'Not Responding' and 'Needs Attention' Communication Issues

Jonathan — Mon, 29 Dec 2008 15:37:00 GMT

You find yourself with one of the following situations

1. Deployment or migration of data utilizing LAN (not SAN Transfer) is very slow in general

2. BITS is slow in one direction only: Fast from Host01 to Host02, but slow from Host02 to Host01 for example

3. Host reports in SCVMM Admin Console "Host not responding" or "Needs attention"

4.     Under 'Status' of the Host the following may show "Not Responding"
    - Connection status
    - Agent Status

All of these issues can be related to networking components. Three common tasks can be performed to address this

1. Test for BITS errors

2. Verify GPO settings

3. Correct vendor specific and Windows system settings for ‘Task Offloading’ (This item will likely have the greatest effect)

Additional Items that just make sense

Verify all network settings on all involved SCVMM and Hyper-V systems. Set the network speed on the NICs and switches. Do not leave as auto negotiate. Also, remove all NIC teaming software. Network Card drivers currently do not always play nice with virtualization technologies. (Uninstall the teaming software. Disabling will not prevent drivers from loading at startup). If you cannot remove your teaming software at this time, at least make sure the namespace is available by typing ‘wmic nic’ at an elevated command prompt. If you get a screen full of data, you may be ok. If you get any kind of error, yank teaming software now and reboot.

1 - BITS Errors
Determine if BITS (Background Intelligent Transfer Service) is encountering errors. Attempt a P2V/Migration/Deployment transfer with SCVMM. While the transfer is occurring between servers, run the following commands on both servers:
Bitsadmin /list /allusers /verbose >bitsadmin.txt

Example BITS output below shows how long the transfer lasted (MODIFICATION TIME minus CREATED TIME). Retry delay shows that there were no errors. This is a clean run. If there are errors review the system and application event logs for errors that are BITS related. Involve your network team as well for a trace.

Example output…

BITSADMIN version 1.0 [ 5.1.2600.0 ]
BITS administration utility.
(C) Copyright 2000-2001 Microsoft Corp.

GUID: {BA568D81-B234-47F2-9BA3-686743F89794} DISPLAY:
https://cotoso.com:443/8272f226d8b8cd818406b28de9b6ad0f012b9a55/C:A5cProgramDataA5cM
icrosoftA5cWindowsA5cHyperVA5calbtest1A5cvmguest.iso/E:A5cVirtualMachinesA5calbtest
1A5cvmguest.iso
TYPE: UNKNOWN STATE: TRANSFERRED OWNER: NT AUTHORITY\SYSTEM
PRIORITY: FOREGROUND FILES: 1 / 1 BYTES: 42948608 / 42948608
CREATION TIME: 12/3/2008 9:04:48 AM MODIFICATION TIME: 12/3/2008 9:16:44 AM
COMPLETION TIME: 12/3/2008 9:16:44 AM
NOTIFY INTERFACE: UNREGISTERED NOTIFICATION FLAGS: 3
RETRY DELAY: 60 NO PROGRESS TIMEOUT: 0 ERROR COUNT: 0
PROXY USAGE: NO_PROXY PROXY LIST: NULL PROXY BYPASS LIST: NULL
DESCRIPTION: VMM LAN Transfer Job
JOB FILES:
42948608 / 42948608 WORKING
https://contoso.com:443/8272f226d8b8cd818406b28de9b6ad0f012b9a55/C:A5cProgramDataA5c
MicrosoftA5cWindowsA5cHyperVA5calbtest1A5cvmguest.iso/E:A5cVirtualMachinesA5calbtes
t1A5cvmguest.iso -> E:\VirtualMachines\albtest1\vmguest.iso

2 - Verify GPO settings
By default, BITS transfers are wide open, not throttled down. Follow the MDSN article and look for the following settings in case they have been changed from default values.
- Any setting with 'Peercache' in the name is not relevant
- The rest of the settings should be set to 'Not Configured' or 'Disabled' to ensure that BITS is not being throttled at all. 'Maximum network bandwidth for BITS background transfers' is most important to be set to 'Not Configured' or 'Disabled'

BITS uses the following Group Policies to configure BITS
http://msdn.microsoft.com/en-us/library/aa362844(VS.85).aspx

The group policies for BITS are located in the registry here…

HKLM\Software\Policies\Microsoft\Windows\BITS

(Note that only those policies that are configured are listed in the registry.)

REMEMBER: Run ‘gpupdate /force’ following any changes from an elevated command prompt

3 - Disable Task Offloading
This work can be done easily on a non-Core server by disabling all properties on the 'Advanced' tab of all NICs. You should then follow the manual directions below to verify the rest of the Offload related settings for the NICs, and to disable this feature in Windows.

If working with a core server you may want to connect via remote registry and make these changes.
This MDSN article lists all values available.

Using Registry Values to Enable and Disable Task Offloading
http://msdn.microsoft.com/en-us/library/aa938424.aspx

All values should be set to '0' EXCEPT for the last one which is related to IPSEC, and thus Kerberos. Leave this one alone. 'HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\Ipsec\EnabledOffload'

A. Locate all physical NICs in the registry under:
'HKLM\System\CurrentControlSet\Control\Class\{4D36E972-E325-11CE-BFC1-08002BE10318}'

B. There will be folders with four digits, starting with '0000'. Locate the ones that show the physical NIC names on the right. Look for the 'DriverDesc' value on the right; this should have the name of a NIC such as 'HP NC360T PCIe Gigabit Server Adapter.' For each of these, make the changes below.

C. Disable All Vendor specific Offloading
Set values for any entries below that include the word 'Offload' to '0' (Disabled)
*FlowControl

No description available

*IPChecksumOffloadIPv4
     Describes whether the device enabled or disabled the calculation of IPv4
checksums.
*TCPChecksumOffloadIPv4
     Describes whether the device enabled or disabled the calculation of TCP Checksum
over IPv4 packets.
*TCPChecksumOffloadIPv6
     Describes whether the device enabled or disabled the calculation of TCP checksum
over IPv6 packets.
*UDPChecksumOffloadIPv4
     Describes whether the device enabled or disabled the calculation of UDP Checksum
over IPv4 packets.
*UDPChecksumOffloadIPv6
     Describes whether the device enabled or disabled the calculation of UDP Checksum
over IPv6 packets.
*LsoV1IPv4
     Describes whether the device enabled or disabled the segmentation of large TCP
packets over IPv4 for large send offload version 1 (LSOv1).
*LsoV2IPv4
     Describes whether the device enabled or disabled the segmentation of large TCP
packets over IPv4 for large send offload version 2 (LSOv2).
*LsoV2IPv6
     Describes whether the device enabled or disabled the segmentation of large TCP
packets over IPv6 for large send offload version 2 (LSOv2).
*IPsecOffloadV1IPv4
     Describes whether the device enabled or disabled the calculation of IPsec headers
over IPv4.
*IPsecOffloadV2
     Describes whether the device enabled or disabled IPsec offload version 2
(IPsecOV2). IPsecOV2 provides support for additional crypto-algorithms, IPv6, and
co-existence with large send offload version 2 (LSOv2).
*IPsecOffloadV2IPv4
     Describes whether the device enabled or disabled IPsecOV2 for IPv4 only.
*RSS
     Receive side scaling
*TCPUDPChecksumOffloadIPv4
     Describes whether the device enabled or disabled the calculation of TCP or UDP
checksum over IPv4.
*TCPUDPChecksumOffloadIPv6
     Describes whether the device enabled or disabled the calculation of TCP or UDP
checksum over IPv6.

D. Disable Offloading in Windows.
Use the following registry values to enable or disable task offloading for the TCP/IP protocol:
HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\TCPIP\Parameters\DisableTaskOffload

Technorati Tags: microsoft,scvmm,hyper-v,virtual,virtualization,windows