Brad Rutkowski's Blog : Metrics

Interacting with Data Collector Sets via Powershell

Brad Rutkowski — Thu, 19 Feb 2009 00:30:12 GMT

Background:

In an earlier post I talked about some new features for Windows 2008 and Vista. One of those new features that is often overlooked are the data collector sets (DCS). One particular role that leverages data collector sets is active directory. Active directory has put “hooks” into tracing that can really take a lot of the thinking out of the question “why is my domain controller sluggish”. For those of you still running Windows 2003 I go over a similar concept called Server Performance Advisor.

Anyways, you can play around with DCS by typing perfmon and then traversing to the section called Data Collector Sets (shocking). If you have performance issues, go here first as it’s like combining a netmon capture with a kernel trace and then handing you the smoking gun.

Challenge:

In my current role, we have a need to automate things quite a bit and so one of the actions I was looking at solving was collecting diagnostic information when a server is performing poorly. Usually when a high CPU alert comes in, someone would need to logon to the server and go to perfmon and start at DCS collection. More often is the case that by the time someone had been alerted and went to the server the sluggish behavior had subsided (the dreaded “close ticket, no problem found”).

My solution was to try and figure out a way to start a DCS collection remotely at the time of event so that the data was present when an actual human became engaged.

After some hard work, here is the code to do so! You can create your own XML file (your own DCS template) and pass it in, but more than likely you’ll be happy at just kicking off one of the built-in templates (AD/System Perf/System Diags).

Running it via powershell:

First, how to do it on the fly:

## PLA.dll lives under system32 on Vista and 2k8.  This will create a powershell com object.
$datacollectorset = new-object -COM Pla.DataCollectorSet
##This is the name of the predefined DCS collector.  It's read-only and will always be System\<something>
$name = "System\Active Directory Diagnostics"
##If you make the second param $null it will be the local machine.
$datacollectorset.Query($name,"serverA") 
$datacollectorset.start($false)
## Status ReturnCodes: 0=stopped 1=running 2=compiling 3=queued (legacy OS) 4=unknown (usually autologger)
$datacollectorset.status
##When you're ready to stop it call stop.
$datacollectorset.stop($false)
##If you call status here, it will probably be '2' for a while as the server compiles the report.
$datacollectorset.status

And like so, you started and stopped a collection for Active Directory on you’re local computer or a remote server! Like I said though, you can create you’re own templates too. You might want to do this if you want to setup a built-in template to be scheduled to run daily, or perhaps you want to send the data to a network location, run more tasks at completion, etc. If you do want to create a custom template then the code changes a bit:

$datacollectorset = new-object -COM Pla.DataCollectorSet
## If you're making you're own (shows up under user defined).  
$xml = get-content C:\custom.xml #You're custom exported XML file.
$datacollectorset.SetXml($xml)
##Commit codes: http://msdn.microsoft.com/en-us/library/aa371873(VS.85).aspx this is add or modify.  Can't do this on a system created PLA instances (read only).
$datacollectorset.Commit($DCSPath , $null , 0x0003)     
$datacollectorset.Query($DCSPath,$null)
$datacollectorset.start($false)
#Runs...
$datacollectorset.stop($false)

Scripting a solution:

Finally if you wanted to script this you could do something like what I’ve done below. This would collect for a desired interval (in seconds) and then when compilation completed display the path to the report. I wrote this in CTP3, but you can easily take the concepts and backport them. If the destination server is inaccessible, or you don't have permissions, then the script will blow up…

<#
    .SYNOPSIS
    This will fire up a PLA (Data Collector Set collection on a server and then copy it to the proper debug server
 
    .DESCRIPTION
    This is a proof of concept and only acceppts System defined collections.  No error handling so I hope you type well.

#>

##Inputs
[CmdletBinding()]
param(
   [Parameter(Mandatory = $true)]
   <#A system provided report to run like "System\System Performance", System\System Diagnostics, etc. #>
   [string]$DCSPath,
   [Parameter(Mandatory = $true)]
   <# This is how long you want the DCS collection to run in seconds#>
   [int32]$time,
   [Parameter(Mandatory = $false)]
   <#If you don't pass in a server name it will be $null and run on the local system#>
   [string]$serverName
    )

    $datacollectorset = new-object -COM Pla.DataCollectorSet  
    $datacollectorset.Query($DCSPath,$serverName)
    $datacollectorset.start($false)
    Start-Sleep $time
    $datacollectorset.stop($false)
    
    ##Now we'll loop while the report compiles.
    $retries = 0
    do 
        {sleep 30; $returnCode = $datacollectorset.Status ; $retries++} 
    while ($returnCode -eq 2 -and $retries -lt 60)
    
    if ($retries -eq 60)
    {
        Write-Warning "Compiling has been running on the server for 30 minutes!  You'll need to check the following location on the server later for the report:"
        Write-Warning $datacollectorset.OutputLocation
        break
    }
    
    ##Compiling has finished, now we can copy the folder to some location
    $path = $datacollectorset.OutputLocation
    if ($serverName)
    {
    $path = $path.Replace(":","$")
    Write-Host "`nReport complete and can be viewed at \\$serverName\$path\report.html on the server.`n" 
    }
    else
    {
    Write-Host "`nReport complete and can be viewed at $path\report.html`n"
    }

The result:

More info:

PLA reference: http://msdn.microsoft.com/en-us/library/aa372634(VS.85).aspx

Technorati Tags: Powershell,Windows 2008,Active Directory,Windows

Hey Admins! Taking some of the pain out of analyzing perfmon captures.

Brad Rutkowski — Thu, 14 Feb 2008 02:03:20 GMT

Performance Analysis of Logs (PAL) tool

Project Description:

Ever have a performance problem, but don't know what performance counters to collect or how to analyze them? The PAL (Performance Analysis of Logs) tool is a new and powerful tool that reads in a performance monitor counter log (any known format) and analyzes it using complex, but known thresholds (provided). The tool generates an HTML based report which graphically charts important performance counters and throws alerts when thresholds are exceeded. The thresholds are originally based on thresholds defined by the Microsoft product teams and members of Microsoft support, but continue to be expanded by this ongoing project. This tool is not a replacement of traditional performance analysis, but it automates the analysis of performance counter logs enough to save you time. This is a VBScript and requires Microsoft LogParser (free download).

My take on the tool:

For those of us out there that don't have to deal with performance data on a daily basis I see a few options to help troubleshoot performance issues on your servers. 1) If your using 2k3 use SPA. 2) If you're running 2k8/Vista use data collection sets. 3) Collect analyze your own perfmon captures.

Now you might want to look into this tool. I found the tool simple to use and it's really a four step process. The web page created for the analysis has a plethora of info and links to the codeplex site for more info. Sweet.

Really in the end it's just a time saver. After collecting performance data on a server you need to analyze that data. This entails opening the log file, adding the counters that you've collected and finding out if any of the counters are above any thresholds (deemed by you). This tool does that analysis for you. It comes out-of-the-box with some predefined thresholds defined as high according to the MSFT consulting/development but those can be adjusted to whatever suits your fancy.

Once you get everything installed its time to do some analysis. It comes with some threshold templates for AD, System Overview, IIS, SQL, Exchange, etc (see pic) You point the app at the performance log you've captured during your perf issue, choose a threshold template to your liking, answer some basic questions, add the form and execute:

Once it completes it generates a webpage with the analysis information you desire. The webpage shows you alerts for activity that it finds suspect and graphs for the different areas of interest. I can't paste all the pics/info in here as it is quite lengthy depending on the interval you provide. But this definitely seems like a tool that could be handy down the road. Looking at the web page it looks really similar to SPA, but with graphs provided via the Office Web Components add-in. For example here is how I could find out LDP was using too much CPU:

First I found the alert which said that something was being excessive and I clicked on the link (sorry for the blurriness):

Then I found LDP consuming the CPU:

Add it to your bag of tricks, hope it helps.

Technorati Tags: Windows 2003,Windows 2008,Perfmon,Performance

Some docs to help get you ready for Windows 2008.

Brad Rutkowski — Tue, 12 Feb 2008 01:23:40 GMT

Yes you are going to want Windows 2008 in your shop. We've been running 2k8 for over two year sin production and I'm very proud of the product we're shipping shortly. Some of you will want to start thinking about what's new in the OS, what has changed, performance, and deployment. Here are some docs/links that will help get you started.

Changes in Functionality from Windows Server 2003 with SP1 to Windows Server 2008

This document describes new features and technologies, which were not available in Windows Server 2003 with Service Pack 1 (SP1), that will help to increase the security of computers running Windows Server 2008, increase productivity, and reduce administrative overhead.
These topics apply to the next release of Windows Server 2008, based on the functionality expected to be included in the Beta releases in 2007. They do not describe all of the changes that are included in Windows Server 2008. Instead, they highlight changes that will potentially have the greatest impact on your use of Windows Server 2008 and provide references to additional information.

Performance Tuning Guidelines for Windows Server 2008

•Performance Tuning for Server Hardware

•Performance Tuning for Networking Subsystem

•Performance Tuning for Storage Subsystem

•Performance Tuning for Web Servers

•Performance Tuning for File Servers

•Performance Tuning for Active Directory Servers

•Performance Tuning for Terminal Server

•Performance Tuning for Terminal Server Gateway

•Performance Tuning for File Server Workload (NetBench)

•Performance Tuning for Network Workload (NTttcp)

•Performance Tuning for Terminal Server Knowledge Worker Workload

•Performance Tuning for SAP Sales and Distribution Two-Tier Workload

Windows Server 2008 Step-by-Step Guides (updated 02/08/2008)

All other inquiries head to the Technical Library for 2k8.

BTW when you install 2k8 and notice that in WinVER or System Properties it shows SP1, don't be alarmed this is by design. Vista SP1 and Win2k8 were developed in parallel, so fixes in the code were included in both versions. This isn't the first time, when we rolled Windows 2003 x64 edition out the door it went as SP1 too...

Technorati Tags: windows 2008

How long did it take that command to run?

Brad Rutkowski — Wed, 16 Jan 2008 03:08:51 GMT

When troubleshooting latency issues I've found it helpful to have firm numbers of how long it took for a command to run? For instance we were looking at an issue where net viewing a server took a long time, but we didn't have firm number of how long it took each time to compare with healthy servers. What are we going to do watch the clock?

Internally we use a tool called timer.exe which does what we want, so I went scavenging around the intertubes to try and find a similar tool that would useful externally. Here it is. It's an old tool but don't hold that against it.

http://www.microsoft.com/downloads/details.aspx?familyid=913795CD-7026-4143-AE85-1F5E096F9BE0&displaylang=en

Syntax:

C:\localbin>timethis

-----------------------------------
TIMETHIS : Command Timing Utility
-----------------------------------

Usage : TIMETHIS "command"

TimeThis executes the command specified by its arguments, then reports its
run time in HH:MM:SS.TTT format. Quotes around the command are required only
when the command involves redirection via <, >, >>, or |, etc. Quotes ensure
that the redirection is applied to the command being timed, rather than the
TimeThis command itself.

Example:

C:\localbin>timethis net view \\red-dc-03

TimeThis : Command Line : net view \\red-dc-03
TimeThis : Start Time : Tue Jan 15 16:04:52 2008

There are no entries in the list.

TimeThis : Command Line : net view \\red-dc-03
TimeThis : Start Time : Tue Jan 15 16:04:52 2008
TimeThis : End Time : Tue Jan 15 16:04:56 2008
TimeThis : Elapsed Time : 00:00:03.446

What do you say you DO here?

Brad Rutkowski — Fri, 21 Dec 2007 22:21:10 GMT

Just noticed the AD jigsaw poster has been updated for 2k8. So next time your boss asks you "What do say you do here?", don't reply with I'm a people person! Do the following:

1) Download one of the jigsaw posters from http://www.microsoft.com/downloads/details.aspx?FamilyID=c2b9e44e-0bbd-47cb-bc09-b3d48be7f867&DisplayLang=en

2) Print out on plotter

3) Give to boss

4) Ask for raise

We used to have one of these hanging outside our office, and as the rotating bosses would come by to see what we did, we'd just point them to the poster. Wonder where that thing is now...

Technorati tags: Active Directory, Windows 2008

Using TypePerf to get performance data on the command prompt.

Brad Rutkowski — Sat, 22 Sep 2007 23:29:36 GMT

I was tracking a high CPU issue this week and needed to know when one of my servers was pegged so I could investigate. I could of used perfmon I guess but I really like to do everything I can from the command prompt. I always like it when there is a tool that can do data collection from the command prompt as this gives you the ability to easily script it if warranted.

http://www.microsoft.com/resources/documentation/windows/xp/all/proddocs/en-us/nt_command_typeperf.mspx?mfr=true

C:\Windows\system32>typeperf "\\Server1\Processor(_Total)\% Processor Time"

"(PDH-CSV 4.0)","\\Server1\Processor(_Total)\% Processor Time"
"09/20/2007 15:42:42.926","18.097697"
"09/20/2007 15:42:43.928","21.217785"
"09/20/2007 15:42:44.929","15.757631"
"09/20/2007 15:42:45.931","16.537653"

The command completed successfully.

You can collect any counter that lives in the perfmon world, I'm sure some of you out there will find a use for this.

C:\Debuggers>typeperf "\\serverX\Server\Server Sessions" -sc 2

"(PDH-CSV 4.0)","\\serverX\Server\Server Sessions"
"09/22/2007 13:21:54.110","8.000000"
"09/22/2007 13:21:55.117","8.000000"

The command completed successfully.

Full Syntax below, you can adjust the collection in numerous ways.

C:\Debuggers>typeperf -?

Microsoft r TypePerf.exe (6.0.6001.16656)

Typeperf writes performance data to the command window or to a log file. To stop Typeperf, press CTRL+C.

Usage:
typeperf { <counter [counter ...]> | -cf <filename> | -q [object] | -qx [object] } [options]

Parameters:
<counter [counter ...]> Performance counters to monitor.

Options:
-?                            Displays context sensitive help.
-f <CSV|TSV|BIN|SQL>          Output file format. Default is CSV.
-cf <filename>                File containing performance counters to monitor, one per line.
-si <[[hh:]mm:]ss>            Time between samples. Default is 1 second.
-o <filename>                 Path of output file or SQL database. Default is STDOUT.
-q [object]                   List installed counters (no instances). To list counters for one object, include the object name, such as Processor.
-qx [object]                  List installed counters with instances. To list counters for one object, include the object name, such as Processor.
-sc <samples>                 Number of samples to collect. Default is to sample until CTRL+C.
-config <filename>            Settings file containing command options.
-s <computer_name>            Server to monitor if no server is specified in the counter path.
-y                            Answer yes to all questions without prompting.

Technorati tags: Windows 2008, Vista, XP, Admin

Hey Admins! Windows System State Analyzer (Beta)

Brad Rutkowski — Sat, 25 Aug 2007 06:56:00 GMT

Windows System state analyzer tool
Helps create snapshots of the computer—some of which include fixed drives, services, drivers and the registry. Users can create two snapshots at different points in time and compare them to view differences. A detailed report could be generated at the end of a compare operation.

Microsoft is starting to release some tools to validate system configurations and verify server application/driver compliance for the Windows Server 2008 logo and certification program.

http://www.innovateonwindowsserver.com/learnbuild.aspx

What does this mean for you? It means you now have a tool that you can use to take a snapshot of your server, save the output to a snapshot file, and then compare it to another snapshot from the same system, or another system all together and compare the differences.

Granted it can be quite verbose (detailed report), but in my test I flipped a few registry keys for diagnsotics, and started a couple of services and they showed up in the quick report as expected.

Why is this useful to me? You can configure your domain controller with all the registry keys you like, drivers you want on a certain platform (DL380 G2), services enabled/disabled and directories you want unchanged. Save that snapshot for a rainy day when you get an escalation on another DC. You know what your good DC looks like (You have the snapshot) now you can compare that to DC that is not working correctly and see if some of the settings are different.

When I ran the DCs in MSIT, variance was one of the things that was hard to keep under control. You'd turn something on for troubleshooting, or more likely turn something off and now you have differences in your service. I think this tool could be helpful in reducing some of the varianc.

Updated Link: http://microsoft.mrmpslc.com/InnovateOnWindowsServer/Download/WindowsSystemStateAnalyzer_x86.msi?bcsi_scan_412712405F8D0B5D=1

Great tool for Windows 2003: Server Performance Advisor (SPA)

Brad Rutkowski — Tue, 26 Jun 2007 18:47:07 GMT

First off you can download SPA 2.0 here. I'm going to explain how to quickly use SPA, and then what type of data is returned in this post.

What is SPA?

So what is SPA and how can you use it? Well the official overview is:

Microsoft ® Windows Server ™ 2003 Performance Advisor is the latest version of Server Performance Advisor, which is a simple but robust tool that helps you diagnose the root causes of performance problems in a Microsoft Windows Server 2003 deployment. Server Performance Advisor collects performance data and generates comprehensive diagnostic reports that give you the data to easily analyze problems and develop corrective actions
Microsoft ® Windows Server ™ 2003 Performance Advisor provides several specialized reports, including a System Overview (focusing on CPU usage, Memory usage, busy files, busy TCP clients, top CPU consumers) and reports for server roles such as Active Directory, Internet Information System (IIS), DNS, Terminal Services, SQL, print spooler, and others.

Really I think of it as network monitor and performance monitor wrapped into one package so that you can correlate which clients might be causing load on your system.

Some nifty things about SPA:

1) It's XML based so the reports that are collected get organized "automagically" by date and server so you can drill down on a particular server. You could have a thousand reports on your reporting server and its quite easy to navigate via IE to the server and date that you are looking for.

2) You can setup SPA on your servers in "Data" mode and then setup a member server as a SPA "reporting" server, then you can schedule your servers to collect at a certain time and send that data to the reporting server. You can also have SPA (with version 2.0) take the data from those servers and put it in a SQL database for trending purposes. This is what we do internally, we setup the jobs to run at 10 and 2 to get peak utilization trending on our domain controllers. There is a chm file with SPA with more details on this.

3) Doesn't require a reboot to install.

4) Was deemed so awesome it is built right into Vista and Windows Server 2008 (Data Collection Sets)

I'm not going to dabble into the trending and reporting server side of SPA as that would require a lot more typing but like I said if you install SPA, you can read the chm about scheduling tasks and trending. I just wanted to point it out because some people might not have a monitoring solution where you can do some rudimentary trending and this could be a free solution.

The install

Double click MSI, leave defaults.

How and when to use

We're going to be focusing on how to use SPA to troubleshoot, lets look at an example of that. SPA is useful at narrowing down resource issues on a system with regards to processor, memory, network, and disk.

Last week we had a WINS server that was throwing database errors and so our team was engaged. I installed SPA using the steps above,I then could have used the GUI to launch SPA and start a collection (default 300 seconds), but this is the faster way (the way I use).

1) Navigate to the SPA directory, if you installed on an x64 system it will be under "Program Files (x86)", otherwise just "Program Files\Server Performance Advisor"

2) Since I want just a system overview report I ran spacmd start "system overview"

a) At this point the collection starts and you should see some processes labeled plahost running in task manager. You can let this run for 300 seconds but in my case I just needed a quick 30 second snapshot since the repro was constantly happening.

b) If you installed this on a domain controller you could do spacmd start "active directory" or spacmd start * which would start all the templates you have installed.

3) Now stop the collection: spacmd stop "system overview"

a) At this point as long as you left the defaults during install you should see a new folder under c:\perflogs with the server name and a few files underneath that.
C:\PerfLogs\Data\System Overview\Current\BRAD-SERVER_200706211545>dir
Volume in drive C is C_Drive
Volume Serial Number is 70C4-9FFD
Directory of C:\PerfLogs\Data\System Overview\Current\BRAD-SERVER_200706211545
06/21/2007 03:49 PM    <DIR>          .
06/21/2007 03:49 PM    <DIR>          ..
06/21/2007 03:49 PM             1,673 global_reg.xml //Some registry settings are checked by SPA there saved here
06/21/2007 03:49 PM         1,441,792 system_kernel.etl //A trace file that SPA analyzes during the capture.
06/21/2007 03:49 PM         1,638,400 system_perf.blg //Perfmon binary log file that SPA analyzes from the capture.
               3 File(s)      3,081,865 bytes
               2 Dir(s)     960,020,480 bytes free

4) Now we need to compile the data we captured into a report: spacmd compile "system overview"

a) Once this is complete, you should see the report in the reports directory. If using the GUI then the report will show up under reports under System Overview.
C:\PerfLogs\report\System Overview\Current\BRAD-SERVER_200706211545>dir
Volume in drive C is C_Drive
Volume Serial Number is 70C4-9FFD
Directory of C:\PerfLogs\report\System Overview\Current\BRAD-SERVER_200706211545
06/22/2007 09:35 AM    <DIR>          .
06/22/2007 09:35 AM    <DIR>          ..
06/22/2007 09:24 AM             1,721 global_reg.xml
06/22/2007 09:35 AM             2,365 obelisk.ip
06/22/2007 09:35 AM           608,594 report.xml //Double click this one.
06/22/2007 09:34 AM            62,417 report.xsl
06/22/2007 09:35 AM               656 summary.xml
06/22/2007 09:24 AM         6,881,280 system_kernel.etl
06/22/2007 09:24 AM         6,094,848 system_perf.blg
               7 File(s)     13,651,881 bytes
               2 Dir(s)     963,108,864 bytes free

Analyzing the report

So now that we have the report we can open it up and start looking at it, just double click report.xml and IE should open. You'll want to allow scripts and ActiveX so that you can adjust the data in the xml doc as it is dynamic. For example, if you look in the second JPG below on the top right its says "3 of 15" if you wanted to see the top 15 of 15 you could just click the 3 and type in 15, and the report would change.

The first part of the report is a summary, and links to other sections pertaining to CPU, Network, Disk, and Memory. Below that is any performance advisories that SPA flagged for you and then how each of the components were doing. In the first JPG below, on the right there is a little help icon, if you click the icon it will open a chm file with further steps you can take to narrow down the issue.

I can't go through each area of concern but you get the idea. As I was going through the network section I noticed this:

This seemed odd so I filtered my network monitor capture that I took during the same time period for vm-lab-machine and it came back with a ton of 1F registrations and releases for the 1F record for that server like so:

13861    5.703125    BRAD-SERVER    VM-LAB-MACHINE         NbtNs    NbtNs: Registration Response, Success for VM-LAB-MACHINE      <0x1F> NetDDE Service, xxx-xx-xxxx-xx
13863    5.703125    VM-LAB-MACHINE         BRAD-SERVER    NbtNs    NbtNs: Release Request for VM-LAB-MACHINE      <0x1F> NetDDE Service, xxx-xx-xxxx-xx
13864    5.703125    BRAD-SERVER    VM-LAB-MACHINE         NbtNs    NbtNs: Release Response, Success for VM-LAB-MACHINE      <0x1F> NetDDE Service, xxx-xx-xxxx-xx
13865    5.703125    VM-LAB-MACHINE         BRAD-SERVER    NbtNs    NbtNs: Registration Request for VM-LAB-MACHINE      <0x1F> NetDDE Service, xxx-xx-xxxx-xx
13866    5.703125    BRAD-SERVER    VM-LAB-MACHINE         NbtNs    NbtNs: Registration Response, Success for VM-LAB-MACHINE      <0x1F> NetDDE Service, xxx-xx-xxxx-xx
13867    5.703125    VM-LAB-MACHINE         BRAD-SERVER    NbtNs    NbtNs: Release Request for VM-LAB-MACHINE      <0x1F> NetDDE Service, xxx-xx-xxxx-xx
13868    5.703125    BRAD-SERVER    VM-LAB-MACHINE         NbtNs    NbtNs: Release Response, Success for VM-LAB-MACHINE      <0x1F> NetDDE Service, xxx-xx-xxxx-xx
13869    5.703125    VM-LAB-MACHINE         BRAD-SERVER    NbtNs    NbtNs: Registration Request for VM-LAB-MACHINE      <0x1F> NetDDE Service, xxx-xx-xxxx-xx
13870    5.703125    BRAD-SERVER    VM-LAB-MACHINE         NbtNs    NbtNs: Registration Response, Success for VM-LAB-MACHINE      <0x1F> NetDDE Service, xxx-xx-xxxx-xx
13871    5.718750    VM-LAB-MACHINE         BRAD-SERVER    NbtNs    NbtNs: Release Request for VM-LAB-MACHINE      <0x1F> NetDDE Service, xxx-xx-xxxx-xx
13872    5.718750    BRAD-SERVER    VM-LAB-MACHINE         NbtNs    NbtNs: Release Response, Success for VM-LAB-MACHINE      <0x1F> NetDDE Service, xxx-xx-xxxx-xx
13873    5.718750    VM-LAB-MACHINE         BRAD-SERVER    NbtNs    NbtNs: Registration Request for VM-LAB-MACHINE      <0x1F> NetDDE Service, xxx-xx-xxxx-xx
13874    5.718750    BRAD-SERVER    VM-LAB-MACHINE         NbtNs    NbtNs: Registration Response, Success for VM-LAB-MACHINE      <0x1F> NetDDE Service, xxx-xx-xxxx-xx
13875    5.718750    VM-LAB-MACHINE         BRAD-SERVER    NbtNs    NbtNs: Release Request for VM-LAB-MACHINE      <0x1F> NetDDE Service, xxx-xx-xxxx-xx
13876    5.718750    BRAD-SERVER    VM-LAB-MACHINE         NbtNs    NbtNs: Release Response, Success for VM-LAB-MACHINE      <0x1F> NetDDE Service, xxx-xx-xxxx-xx
13877    5.718750    VM-LAB-MACHINE         BRAD-SERVER    NbtNs    NbtNs: Registration Request for VM-LAB-MACHINE      <0x1F> NetDDE Service, xxx-xx-xxxx-xx
13878    5.718750    BRAD-SERVER    VM-LAB-MACHINE         NbtNs    NbtNs: Registration Response, Success for VM-LAB-MACHINE      <0x1F> NetDDE Service, xxx-xx-xxxx-xx
13879    5.718750    VM-LAB-MACHINE         BRAD-SERVER    NbtNs    NbtNs: Release Request for VM-LAB-MACHINE      <0x1F> NetDDE Service, xxx-xx-xxxx-xx
13880    5.718750    BRAD-SERVER    VM-LAB-MACHINE         NbtNs    NbtNs: Release Response, Success for VM-LAB-MACHINE      <0x1F> NetDDE Service, xxx-xx-xxxx-xx
13881    5.718750    VM-LAB-MACHINE         BRAD-SERVER    NbtNs    NbtNs: Registration Request for VM-LAB-MACHINE      <0x1F> NetDDE Service, xxx-xx-xxxx-xx
13882    5.718750    BRAD-SERVER    VM-LAB-MACHINE         NbtNs    NbtNs: Registration Response, Success for VM-LAB-MACHINE      <0x1F> NetDDE Service, xxx-xx-xxxx-xx
13883    5.718750    VM-LAB-MACHINE         BRAD-SERVER    NbtNs    NbtNs: Release Request for VM-LAB-MACHINE      <0x1F> NetDDE Service, xxx-xx-xxxx-xx
13884    5.718750    BRAD-SERVER    VM-LAB-MACHINE         NbtNs    NbtNs: Release Response, Success for VM-LAB-MACHINE      <0x1F> NetDDE Service, xxx-xx-xxxx-xx
13885    5.718750    VM-LAB-MACHINE         BRAD-SERVER    NbtNs    NbtNs: Registration Request for VM-LAB-MACHINE      <0x1F> NetDDE Service, xxx-xx-xxxx-xx
13886    5.718750    BRAD-SERVER    VM-LAB-MACHINE         NbtNs    NbtNs: Registration Response, Success for VM-LAB-MACHINE      <0x1F> NetDDE Service, xxx-xx-xxxx-xx
13887    5.718750    VM-LAB-MACHINE         BRAD-SERVER    NbtNs    NbtNs: Release Request for VM-LAB-MACHINE      <0x1F> NetDDE Service, xxx-xx-xxxx-xx
13888    5.718750    BRAD-SERVER    VM-LAB-MACHINE         NbtNs    NbtNs: Release Response, Success for VM-LAB-MACHINE      <0x1F> NetDDE Service, xxx-xx-xxxx-xx

I then popped the query 1F Wins Server into live.com and the first hit was the issue.

SPA roles:

There is more than just the "system overview" template, there are templates for AD, print servers, terminal servers, etc. Each one of these templates focuses on that role and collects different counters depending on the role. For example, on a DC SPA will capture the DS perfmon counters and then analyze the output from those counter and flag issues it finds for follow-up.

Conclusion:

Using SPA I was able to easily find the network client causing the issue on our WINS server and then correlate that with the network capture. This is only one example of where SPA has really assisted in narrowing down the issue for me. One caveat, SPA is CPU intensive when it compiles the report, so if the system is already pegged at 100% its best to compile the report off the the system in question.

If you run into any issues with SPA (only supported on Win2k3), send me an e-mail or drop a comment and I'll try to help you out.

Technorati tags: Windows 2003, SPA

IceRocket tags: Windows 2003, SPA

How to easily calculate your system availability (uptime).

Brad Rutkowski — Tue, 24 Apr 2007 20:39:17 GMT

We all have managers and ALL managers love that little word called metrics, and ALL managers like to know that their service has 99.99% uptime. So how can you easily get this information off a Windows server? Well there is a simple tool out there called uptime.exe.

To get the uptime of a server use: Uptime.exe /s <servername>. If the system log has "wrapped" then your not going to get the events you need to get a good uptime report from the beginning. Uptime will show you each reboot and then at the bottom report give you valuable data that you can then put in a nice spreadsheet and send up to management.

Perhaps I should start a new section called metrics, because there are quite a few other easy things you can do to get relevant information to managers...

C:\localbin>uptime /s brad-dc-02
Uptime Report for: \\brad-dc-02

Current OS: Microsoft Windows Server 2003, Service Pack 1, Multiprocessor Free.
Time Zone: Pacific Daylight Time

System Events as of 4/24/2007 10:27:16 AM:

Date: Time: Event: Comment:
---------- ----------- ------------------- -----------------------------------
2/5/2007 1:50:11 PM Boot Prior downtime:0d 0h:1m:54s
2/5/2007 2:01:34 PM Shutdown Prior uptime:0d 0h:11m:23s
2/5/2007 2:03:30 PM Boot Prior downtime:0d 0h:1m:56s
2/5/2007 2:11:31 PM Abnormal Shutdown Prior uptime:0d 0h:8m:1s
2/5/2007 2:15:30 PM Boot Prior downtime:0d 0h:3m:59s
2/5/2007 2:15:31 PM Bluescreen STOP 0x0000000a
2/6/2007 12:22:12 PM Shutdown Prior uptime:0d 22h:6m:42s
2/6/2007 12:23:57 PM Boot Prior downtime:0d 0h:1m:45s
2/12/2007 1:08:06 PM Shutdown Prior uptime:6d 0h:44m:9s
2/12/2007 1:09:54 PM Boot Prior downtime:0d 0h:1m:48s
2/12/2007 1:54:09 PM Shutdown Prior uptime:0d 0h:44m:15s
2/12/2007 1:55:57 PM Boot Prior downtime:0d 0h:1m:48s
2/12/2007 2:15:49 PM Service Pack Service Pack 3 removed
2/12/2007 2:16:21 PM Shutdown Prior uptime:0d 0h:20m:24s
2/12/2007 2:18:05 PM Boot Prior downtime:0d 0h:1m:44s
2/12/2007 2:31:20 PM Shutdown Prior uptime:0d 0h:13m:15s
2/12/2007 2:33:06 PM Boot Prior downtime:0d 0h:1m:46s
2/12/2007 2:45:34 PM Service Pack Service Pack 3 removed
2/12/2007 2:48:06 PM Shutdown Prior uptime:0d 0h:15m:0s
2/12/2007 2:50:29 PM Boot Prior downtime:0d 0h:2m:23s
2/24/2007 12:09:42 AM Shutdown Prior uptime:11d 9h:19m:13s
2/24/2007 12:12:13 AM Boot Prior downtime:0d 0h:2m:31s
3/27/2007 12:26:16 PM Shutdown Prior uptime:31d 11h:14m:3s
3/27/2007 12:27:57 PM Boot Prior downtime:0d 0h:1m:41s
4/7/2007 12:04:53 AM Shutdown Prior uptime:10d 11h:36m:56s
4/7/2007 12:07:20 AM Boot Prior downtime:0d 0h:2m:27s
4/20/2007 9:45:10 PM Shutdown Prior uptime:13d 21h:37m:50s
4/20/2007 9:47:11 PM Boot Prior downtime:0d 0h:2m:1s

Current System Uptime: 3 day(s), 12 hour(s), 40 minute(s), 38 second(s)

--------------------------------------------------------------------------------

Since 11/3/2005:

System Availability: 99.9860%
Total Uptime: 460d 5h:51m:10s
Total Downtime: 0d 1h:48m:13s
Total Reboots: 31
Mean Time Between Reboots: 17.31 days
Total Bluescreens: 1

If you only want to see the uptime for the last 30 days then you can use the p: switch, there are other switches too.

C:\localbin>uptime /s brad-dc-01 /p:30
Uptime Report for: \\brad-dc-01

Current OS: Microsoft Windows Server 2003, Service Pack 1, Multiprocessor Free.
Time Zone: Pacific Daylight Time

System Events as of 4/24/2007 10:28:53 AM:

Date: Time: Event: Comment:
---------- ----------- ------------------- -----------------------------------
3/27/2007 12:26:16 PM Shutdown Prior uptime:31d 11h:14m:3s
3/27/2007 12:27:57 PM Boot Prior downtime:0d 0h:1m:41s
4/7/2007 12:04:53 AM Shutdown Prior uptime:10d 11h:36m:56s
4/7/2007 12:07:20 AM Boot Prior downtime:0d 0h:2m:27s
4/20/2007 9:45:10 PM Shutdown Prior uptime:13d 21h:37m:50s
4/20/2007 9:47:11 PM Boot Prior downtime:0d 0h:2m:1s

Current System Uptime: 3 day(s), 12 hour(s), 42 minute(s), 14 second(s)

--------------------------------------------------------------------------------

Since 3/25/2007: (Last 30 Days)

System Availability: 99.9860%
Total Uptime: 30d 10h:22m:44s
Total Downtime: 0d 0h:6m:9s
Total Reboots: 3
Mean Time Between Reboots: 10.15 days
Total Bluescreens: 0

Technorati tags: Windows, Vista, Windows 2003

del.icio.us tags: Windows, Vista, Windows 2003