This tool has been deprecated. Please see the new tool!

Click here to use VMMTrace

So you’ve run into a problem with VMM. Maybe your Hosts have lost connectivity with the SCVMM Server. Or, P2V fails for a certain machine every time. If you’ve searched many TechNet blogs or forums for answers you have probably come to one conclusion: you’re going to have to run a trace to collect the required information.

How do I capture a trace? Armed with DbgView (available from Sysinternals.com) and a number of registry modifications, followed by service restarts on each involved machine, you’re ready to begin. Problem is, configuring a server for tracing is time consuming, tedious, and prone to human error. I’ve worked to alleviate your pain.

Easy? Seriously?

True. You need two files on each server you wish to run the trace on. The process is automated, requires very little input, and works. I’ll skip how it works other than to say it performs the work in Cheng’s article for you. Manual steps are below. If you would like to download a zip file with everything ready click the button below.

Instructions

1. DOWNLOAD SCTrace from the 'Download' button above.

2. SAVE ‘SCTrace_x.x.zip to a directory near the root of C: such as ‘C:\Temp’. (Remaining instructions assume this is where you saved the file...)

3. EXTRACT 'sctrace.zip'. By default this will create a folder named 'SCTrace_x.x' in 'C:\Temp'

4. From an ELEVATED command prompt change to the directory where the files are extracted (C:\Temp\SCTrace_x.x) and type sctrace.cmd

5. FOLLOW the prompts in the COMMAND WINDOW at all times

Tips for Successful Tracing

Follow the instructions in the command prompt. Remember that the goal is to capture the issue while is is reproduced. If the problem is between a Host and the SCVMM Server, run this same script on both machines at the SAME TIME. We need both ends of the conversation to figure out what is going on. The script will provide an opportunity to get things kicked off on all servers then start all the traces at once.

Disclaimer

Every effort has been made to accommodate various environments, while at the same time keeping the length of the script to a minimum. This script may not work if you have a locked down system. This script is provided as is. That said, I would love to hear what you think of it. Corrections appreciated as well. Check back for version updates.

:: START SCRIPT COPY HERE

:: sctrace.cmd
:: v1.6

@echo off

if (%1)==(/?) goto :HELP
if (%1)==(-?) goto :HELP
if (%1)==(odsoff) goto :ODSOFF

:: Test Running Elevated
if not exist %windir%\SCVMM_TRACE\test (
mkdir %windir%\SCVMM_TRACE\test
)
If Not %ERRORLEVEL% EQU 0 (
echo Error: You must run this from an elevated command prompt
Goto End
)

:: Check for DbgView
if not exist dbgview.exe (
cls
echo DbgView is required to capture a trace.
echo Please download from http://live.sysinternals.com/dbgview.exe
echo and place in same directory as script.
echo.
echo.
Goto End
)

:: BIG NOTICE
cls
echo v1.6
echo.
echo    YOU MUST FOLLOW 3 STEPS TO CAPTURE A TRACE
echo.
echo  FOLLOW THE DIRECTIONS ON THIS SCREEN AT ALL TIMES
echo.
echo                READ CAREFULLY
echo.
pause

:: Verify DbgView Settings
cls
echo.
echo  STEP 1 of 3  -  CONFIGURE DEBUGVIEW  -  STEP 1 of 3
echo.
echo  - DebugView will open. Select the CAPTURE menu.
echo.
echo  - Verify 'Capture Win32' and 'Capture Global Win32' are checked.
echo.
echo  - Exit DebugView and return to this window.
echo.
echo.
CHOICE /C C /N /M "Press [C] to configure DbgView now."
IF errorlevel 1 dbgview.exe

:: Cleanup
rmdir %windir%\SCVMM_TRACE\test
Start taskkill /IM debugview.exe
Start taskkill /IM dbgview.exe

:Begin
:: Verify Environment
if not exist %windir%\SCVMM_TRACE\logs (
mkdir %windir%\SCVMM_TRACE\logs
)

:: Capture Exact Time to Prevent Overwrites
set hh=%time:~0,2%
if "%time:~0,1%"==" " set hh=0%hh:~1,1%
set yymmdd_hhmmss=%date:~12,2%%date:~4,2%%date:~7,2%_%hh%%time:~3,2%%time:~6,2%

:: Format and Remove Spaces
set Filepath=%windir%\SCVMM_TRACE\logs
set Filename=%Filepath%\%COMPUTERNAME%_%yymmdd_hhmmss%.log
set Filename=%filename: =%

::Enable Tracing
Reg Add "HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Tracing\Microsoft\Carmine" /v ODS /t REG_DWORD /d 00000001 /f
Reg Add "HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Tracing\Microsoft\Carmine" /v ODSFLAGS /t REG_DWORD /d 255 /f

:: Restart Services
cls
net stop vmmagent
net stop vmmservice

net start vmmagent
net start vmmservice

:: IISRESET will NOT occur unless /IIS switch is used
if (%1)==(iis) iisreset

:: Choose Systems
cls
echo.
echo  STEP 2 of 3  -  STAGE OTHER SERVERS  -  STEP 2 of 3
echo.
echo  MAKE A DECISION...  ONE SERVER OR MANY?
echo.
echo  Start this trace on other involved systems now if needed.
echo  When all systems are sitting at  STEP 2 of 3  proceed below.
echo.
CHOICE /C C /N /M "Press [C] to continue when all systems ready."
IF errorlevel 1 Start Dbgview.exe /t /f /l %Filename%

:: All Systems GO!
:STOPDBGVIEW
cls
echo.
echo  STEP 3 of 3  -  TRACING NOW  -  STEP 3 of 3
echo.
echo  Trace is capturing all activity on this system now.
echo  Make sure all systems involved show STEP 3 of 3 then...
echo  REPRODUCE YOUR ISSUE NOW.
echo.
echo  Press [F] to Finish on all systems AFTER reproducing issue
echo.
CHOICE /C F /N
IF errorlevel 1 (
Start taskkill /IM debugview.exe
Start taskkill /IM dbgview.exe
cls
echo Writing out log file... Please wait
timeout /t 5
)

:: Disable Tracing
Reg Add "HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Tracing\Microsoft\Carmine" /v ODS /t REG_DWORD /d 00000000 /f

:: Open Explorer Showing Trace File
start %Filepath%
cls
echo DONE!
echo.
echo Provide the latest file in %FilePath% to support
echo.
echo.

Goto End

:HELP
cls
@echo Trace Options
@echo.
@echo From an elevated command prompt type Trace.cmd
@echo Options: /?, -?    This Help Menu
@echo          /odsoff   Disable tracing
@echo          /iis      Perform iisreset
@echo.
@echo.
Goto End

:ODSOFF
Reg Add HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Tracing\Microsoft\Carmine /v ODS /t REG_DWORD /d 00000000 /f

:END

:: STOP SCRIPT COPY HERE

jonjor

With a single command you can pull a list of all installed updates. Use this to compare patch levels on all systems so you can easily target those not meeting the bar.

We start with simple commands, then build on these to create robust queries with optional CSV and HTML formatting. I think of these commands, which I use on a daily basis, as basics for my Toolkit. If you don’t have a Toolkit I suggest you create one now. Place your often used tools in a folder such as ‘C:\Toolkit’ so that they are always at hand. Grab the first five items under ‘Top 10 Downloads’ from Sysinternals.com to round out your basics.

TIP:  Go one step further and sign up for a Live Mesh account with your Windows Live ID. With this you can  keep your Toolkit directory, and other directories, synched up across all machines you designate. The folders will be available privately online as well. Further, remote control is built-in for all Mesh enabled systems. Sweet! Clearly remote capabilities are geared more towards workstations than servers, but I use it for both.

And yeah, it’s free. Get started here.

Basic Commands

Warm up by opening an elevated command prompt on your SCVMM server (or any server for that matter). If your command window does not show ‘Administrator:’ as below, you are not running an elevated console.

> Single command to pull updates.
wmic qfe

http://support.microsoft.com/?kbid=963660  VMM2008R2-03  Update                        KB963660
                       NT AUTHORITY\SYSTEM  2/26/2009
http://support.microsoft.com/?kbid=967062  VMM2008R2-03  Update                        KB967062
                       NT AUTHORITY\SYSTEM  2/26/2009

You should see a torrent of information scroll through the command window. This is a list of all QFEs (Hotfixes and Updates) installed, along with a good bit of extraneous info.

> Now let's work on formatting. This is my everyday method.
wmic qfe list brief

Update                        KB963660                                 NT AUTHORITY\SYSTEM  2/26/2009
Update                        KB967062                                 NT AUTHORITY\SYSTEM  2/26/2009

Now some of the noise is removed and it is easier to read the list of updates.

TIP:  Is KBXXXXXX installed on this system?!? I can only remember the last three digits of the update anyway… Easy to check. Just pipe your output to ‘find.’

wmic qfe list brief | find /i "660"

Update                        KB963660                                 NT AUTHORITY\SYSTEM  2/26/2009

Apparently it is installed. Nice.

> This is good, but we can do better.
wmic qfe get csname,hotfixid,description

VMM2008R2-03  Update           KB963660
VMM2008R2-03  Update           KB967062

Now we're getting somewhere. We've selected the header for only those columns we are interested in and have returned a clean list of updates along with the system name.

This much you can keep in your head. You might want to create a batch file for items in the next section.

Advanced Queries

Now that you've waded into the deep end let's perform some real work. These are the commands to place in a batch file for your Toolkit. The steps below assume you have already created a ‘C:\Temp’ directory on your system. CSName is not required for the next two commands; it is implied.

> Create a CSV file from the output.
wmic /output:c:\temp\qfe.csv QFE GET HotFixID,Description /format:csv

The resulting CSV file lists these items, ready to be opened in Excel: Node,Description,HotFixID

> CSV files are not as interesting as HTML, so let's shift output formats.
wmic /output:c:\temp\qfe.html QFE GET HotFixID,Description /format:htable

Now you have a clean multicolored table with results, complete with column headers.

Remote Systems

The only thing left to do is perform this same work on remote machines. First a single machine, then a list of systems. Replace <remotesystem> with your remote system name. Note that we add CSName to pull the remote name.

> List remote system updates.
wmic /node:"<remotesystem>" /output:c:\temp\qfe_remote.html QFE GET CSName,HotFixID,Description /format:htable

If this did not work for you keep in mind that you must be an administrator on the remote machine, and Remote Management must be enabled.

> Finally, let's feed a list of server names and create a single HTML file. Create a file named 'serverlist.txt' with one system per line.
wmic /node:@serverlist.txt /output:c:\temp\qfe_serverlist.html QFE GET CSName,HotFixId,Description /format:htable

That's it. You should have a table similar to this.

Enjoy your new tools. Thanks for reading!

For further reference:

MSDN - WMI Command Line Tools
http://msdn.microsoft.com/en-us/library/aa394531.aspx

Released April 14, 2009, a number of Hotfixes have been rolled up into a single download. I strongly recommend this update for all SCVMM Servers. This update may require a reboot. This update replaces ‘KB959596 - Description of the System Center Virtual Machine Manager 2008 update to address physical to virtual (P2V) issues.' You should still install updates from KB962941. Best of luck!

Description of the hotfix rollup package for System Center Virtual Machine Manager 2008: April 14th, 2009
http://support.microsoft.com/kb/961983

This update will be provided automatically by 'Microsoft Update,' or it can be manually downloaded here. If installing manually, open an elevated command prompt and execute (without the quotes): “msiexec /update vmmServer64Update.msp BOOTSTRAPPED=1”

Issues Addressed
  VMware virtual machine migration
  Hyper-V VMs showing “unsupported cluster configuration”
  Differencing disk issues
  Agent status on Cluster nodes
  Replication software causing “unsupported cluster configuration"
  Error 2951 with VMware Virtual Center 

Install updates from KB962941 and KB961983
KB959596 is no longer necessary if KB961983 is installed.

Recommended hotfixes for System Center Virtual Machine Manager 2008
http://support.microsoft.com/kb/962941