Otto Helweg - Management Matters

Forwarding Security Events from Windows XP, Server 2003, and Vista/Server 2008

ottoh — Mon, 22 Jun 2009 20:54:00 GMT

Security events are different than other Windows events because they require a special level of authentication/credentials in order to read or forward these events. Different configurations are required depending on which Windows platform is the client. The event collector functionality is only implemented in Windows Vista or Server 2003 R2 (or later).

The following table illustrates the special configurations required for each platform:

Platform	Configuration Requirement
XP SP2+	The “Windows Remote Management” Service needs to run as “Local System” (make sure you're okay with this elevation - it does have potential security ramifications)
Server 2003	The following “CustomSD” key needs to be set within “HKLM/SYSTEM/CCS/Services/EventLog/Security” to “O:BAG:SYD:(A;;CC;;;NS)”
Vista, Server 2008, and beyond	Add “Network Service” to the “Event Log Readers” Local Security Group

Note: A popular scenario includes forwarding Security Events from a Domain Controller in order to get an enterprise view for auditing and security monitoring. Due to the large number of Security Events that tend to be generated on Domain Controllers, the Event Forwarding subscription should not request that Forwarded Events be "Rendered". Event rendering for a large number of events will consume a large amount of processing resources on the client. The "ContentFormat" of the subscription needs to be set to "Events" rather than the default "RenderedText". This change can be made via "WECUTIL.EXE".

<ContentFormat>Events</ContentFormat>

Note: In addition, Security events are typically considered time sensitive and it's desirable to forward them immediately, rather than at a set interval. The following "MaxItems" setting for the subscription insures that events are forwarded as they occur (only valid for "Push" subscriptions).

wecutil ss <subscription name> /cm:Custom /dmi:1

More details can be found below:

WMI Blog on this topic (adds a little more background info): http://blogs.msdn.com/wmi/archive/2009/04/06/forwarding-security-related-events-from-xp-win2k3-vista-using-winrm-wsman-event-forwarding.aspx
Setting Event Log Security via Group Policy (KB Article): http://support.microsoft.com/default.aspx/kb/323076
Event Forwarding Plug-In for XP SP2+ and Server 2003 SP1+: http://www.microsoft.com/downloads/details.aspx?displaylang=en&FamilyID=845289ca-16cc-4c73-8934-dd46b5ed1d33

Hyper-V Virtual Identification

ottoh — Mon, 04 Aug 2008 19:00:00 GMT

Since virtual machines can be easily moved between physical hosts (parents), it becomes important to track where virtual machines are physically residing for both asset management as well as troubleshooting purposes. The following post focuses on discovering the relationship between virtual hosts (parents) and the virtual machines (children) from both the perspective of the parent as well as the perspective of the child.

Note: The following examples use the WinRM and WinRS command-line utilities which shipped with Windows Vista and Windows Server 2008, but are also available as an Out-Of-Band install for Windows XP SP2+ and Windows Server 2003 SP1+ here.

Query the Parent:

Most commonly used for asset collection, this model gathers the names (and other virtual machine characteristics) of all the children running on a virtual host. This method queries the Hyper-V specific WMI provider/class by using the following command.

winrm enumerate wmi/root/virtualization/msvm_computersystem /r:<remote Hyper-V Host>

With the following sample output:

Msvm_ComputerSystem
    AssignedNumaNodeList = 0
    Caption = Virtual Machine
    CreationClassName = Msvm_ComputerSystem
    Description = Microsoft Virtual Machine
    ElementName = PROV-XP
    EnabledDefault = 2
    EnabledState = 2
    HealthState = 5
    InstallDate
        Datetime = 2008-07-01T21:47:02Z
    Name = 31F497F1-2437-4E89-8308-BE07FB5C14C2
    NameFormat = null
    OnTimeInMilliseconds = 432464839
    OperationalStatus = 2
    OtherEnabledState = null
    PrimaryOwnerContact = null
    PrimaryOwnerName = OTTOH-HOST\Administrator
    ProcessID = 2628
    RequestedState = 12
    ResetCapability = 1
    Status = null
    TimeOfLastConfigurationChange
        Datetime = 2008-07-30T17:07:06Z
    TimeOfLastStateChange
        Datetime = 2008-07-30T17:07:06Z

Query the Child:

Most commonly used for troubleshooting scenarios where a virtual machine is being evaulated and needs to be queried in order to determine its physical host (parent). The following command queries the registry on the child in order to determine its host (parent):

Remote Access Method #1 (the /f parameter merely structures the output in XML – handy for scripting, especially in PowerShell):

winrm invoke GetStringValue wmi/root/default/StdRegProv @{hDefKey="2147483650";sSubKeyName="Software\Microsoft\Virtual Machine\Guest\Parameters";sValueName="PhysicalHostNameFullyQualified"} /r:<Remote VM> /u:<Username> /p:<Password> /f:pretty

Remote Access Method #2:

winrs /r:<Remote VM> /u:<Username> /p:<Password> reg query "HKLM\Software\Microsoft\Virtual Machine\Guest\Parameters" /v PhysicalHostNameFullyQualified

Note: The first method demonstrates a powerful way to access the value of any registry key using the ‘StdRegProv’ WMI provider via WS-Man/WinRM for remote transport. Other registry hives can be accessed with the following hDefKey values: HKLM=2147483650, HKCU=2147483649, HKCR=2147483648, HKEY_USERS=2147483651.

Quick and Dirty Large Scale Eventing for Windows

ottoh — Tue, 08 Jul 2008 18:25:00 GMT

One of the least known yet most powerful management features to ship with Windows Vista and Windows Server 2008 is built-in Event Forwarding which enables large scale health and state monitoring of a Windows environment (assuming health and state can be determined from Windows Events - which they usually can). Not only is this feature built into the latest versions of Windows, but it's also available for down-level OSs like Windows XP SP2+ and Windows Server 2003 SP1+ (here).

Note: True enterprise class Windows eventing is included with enterprise monitoring solutions like System Center Operations Manager.

This new Windows Event Forwarding (also known as Windows Eventing 6.0) is exceptional for the following reasons:

Standards Based: No really! It leverages the DMTF WS-Eventing standard which allows it to interoperate with other WS-Man implementations (see OpenWSMAN at SourceForge).
Agentless: Event Forwarding and Event Collection are included in the OS by default
Down-Level Support: Event Forwarding is available for Windows XP SP2+ and Windows Server 2003 SP1+
Multi-Tier: Forwarding architecture is very scalable where a "Source Computer" may forward to a large number of collectors and collectors may forward to collectors
Scalable: Event Collection is very scalable (available in Windows Vista as well) where the collector can maintain subscriptions with a large number of "Source Computers" as well as process a large number of events per second
Group Policy Aware: The entire model is configurable by Group Policy
Schematized Events: Windows Events are now schematized and rendered in XML which enables many scripting and export scenarios
Pre-Rendering: Forwarded Windows Events can now be pre-rendered on the Source Computer negating the need for local applications to render Windows Events
Resiliency: Designed to enable mobile scenarios where laptops may be disconnected from the collector for extended periods of time without event loss (except when logs wrap) as well as leveraging TCP for guaranteed delivery
Security: Certificate based encryption via Kerberos or HTTPS

This implementation will walk through the following example design where via Group Policy a domain computer group will be configured to forwared Windows Events to a single collector:

Implementation steps are as follows:

Step 1: Create Event Forwarding Subscription
Step 2: Configure WinRM Group Policy
Step 3: Configure Event Forward Group Policy
Step 4: Test

Step 1: Create the Event Forwarding Subscription on the Event Collector

In the Windows Event Forwarding architecture, the subscription definition is held and maintained on the Collector in order to reduce the number of touch-points in case a subscription needs to be created or modified. Creating the subscription is accomplished through the new Event Viewer user interface by selecting the 'Create Subscription' action when the 'Subscriptions' branch is highlighted. The Subscription may also be created via the "WECUTIL" command-line utility.

Note: Both Windows Vista and Windows Server 2008 can be event collectors (this feature is not supported for down-level). Although there are no built-in limitations when Vista is a collector, Server 2008 will scale much better in high volume scenarios.

Although the above subscription is configured to leverage Group Policy, the subscription can be configured in a stand-alone mode (see the "Collector Initiated" option). In addition, this subscription is designed to gather all events from the "Application" and "System" logs that have a level of "Critical", "Error", or "Warning". This event scope can be expanded to gather all events from these logs or even add additional logs (like the "Security" log).

Lastly, the subscription is configured to forward events as quickly as possible with the advanced settings delivery option of "Minimize Latency". The default setting of "Normal" would only forward events every 15 minutes (which may be more desirable depending the the Collector and Source Computer resources).

If Group Policy is not being used, configure the "Subscription type" to be "Collector Initiated". In this case Source Computers will need to be manually added to the Subscription either through the Subscription configuration or the "WECUTIL" command-line utility (which can also be scripted using PowerShell, but that's another topic).

Note: In cases where there Source Computer is generating a large volume of forwarded events (e.g. Security events from a Domain Controller), use WECUTIL on the collector to disable event rendering for the subscription. The task of pre-rendering an event on the source computer can be CPU intensive for a large number of events.

Step 2: Configure Group Policy to enable Windows Remote Management on the Source Computers (clients)

Group Policy can be used to enable and configure Windows Remote Management (WinRM or WS-Man) on the Source Computers. WinRM is required by Windows Event Forwarding as WS-Man is the protocol used by WS-Eventing. The following shows the Group Policy branch locations for configuring both WinRM and Event Forwarding:

The following GP setting will enable WinRM on the client as well as configure a Listener that will accept packets from ANY source.

Note: This Listener configuration should only be used in a trusted network environment. If the environment is not trusted (like the Internet), then configure only specific IP Addresses or ranges in the IPv4 and IPv6 filters.

To configure WinRM outside of Group Policy, run the following command on the Source Computer (also see the above Note):

winrm quickconfig

Step 3: Configure Group Policy to enable Windows Event Forwarding on the Source Computers

As with WinRM, Group Policy can be used to configure Source Computers (Clients) to forward events to a collector (or set of collectors). The policy is very simple. It merely tells the Source Computer to contact a specific FQDN (Fully Qualified Domain Name) or IP Address and request subscription specifics. All of the other subscription details are held on the Collector.

If Group Policy is not being used, then there is nothing to do here as the "Collector Initiated" Subscription will proactively reach out to the Source Computer.

Step 4: Test Event Forwarding

If all of the Event Forwarding components are functioning (and there's minimal network latency), a test event created on the Source Computer should arrive in the Collector's "Forwarded Events" log within 60 seconds. Create a test event with the following command:

eventcreate /id 999 /t error /l application /d "Test event."

This event should appear on the Collector as follows:

If the event doesn't appear, perform the following troubleshooting steps:

Troubleshooting Step 1: Has Policy Been Applied to the Source Computer?

This can be forced by running the following command on the Source Computer:

gpupdate /force

Troubleshooting Step 2: Can the Collector Reach The Source Computer via WinRM?

Run the following command on the Collector

winrm id /r:<Source Computer> /a:none

Troubleshooting Step 3: Is the Collector Using the Right Credentials?

Run the following command on the Collector

winrm id /r:<Source Computer> /u:<username> /p:<password>

Note: These are the credentials defined in the Subscription on the Collector. The credentials don't need to be in the local Administrators group on the Source Computer, they just need to be in the "Event Log Readers" group on the Source Computer (local Administrators will also work).

Troubleshooting Step 4: Has the Source Computer Registered with the Collector?

Run the following command on the Collector

wecutil gr <subscription name>

This will list all the registered Source Computers (note if the Subscription is "Collector Initiated" then this will list all configured Source Computers), their state (from the Collector's perspective), and their last heartbeat time.

Enjoy!

Reference The Event That Triggered Your Task

ottoh — Sat, 10 Nov 2007 00:29:00 GMT

Task Scheduler 2.0 (which shipped with Vista and Windows Server 2008) is a complete re-write of the built-in Windows Task Scheduler. Task Scheduler 2.0 is a fantastic automation tool that includes new powerful triggers like "from Event". From a task's perspective, I'm going to outline how to determine which event is responsible for triggering a task. This can be helpful when the task may want to branch depending on the type of event, or the task may want to report more information about the specific event content.

For example, suppose you want a task to notify you every time a Service is Stopped on your server. This would require a task to inspect the details of the specific Service Control Manager event in order to figure out the affected Service (by name) as well as the state change (was it stopped or started?). This is also a good example because the Service Start/Stopped events use the same EventID, so further event inspection of the event content is required.

Step 1: Create the Task

Create your task (either by selecting "Attach A Task To This Event..." from the Event Viewer, or manually selecting the "On an event" Trigger when building the Task inside the Task Scheduler.

Event Viewer Example:

Task Scheduler Example:

Step 2: Export the Task

From within Task Scheduler, export the newly created task (as an XML file)

Step 3: Modify the Task

Use Notepad (or your text editor of choice - keep in mind the text editor must honor unicode which notepad does) to add the Event parameters you which to pass along to your task. The event parameters below are the most useful for event identification. Notice the entire node <ValueQueries> and its children need to be added to the EventTrigger branch.

Step 4: Pass the Values to Your Action

The following example shows how the event values can be used the simple Message Box action.

Step 5: The Result

Now the following message box will fire every time the Event Trigger condition is met. Notice the 3 Event parameters that are displayed in the message box.

Step 6: Next Steps - Display The Whole Source Event

With the above parameters, the specific event can be tracked down for ever more information about the event. For example, the following WEVTUTIL.EXE utility can be used to display the specific event (substitute the variables $eventLog and $eventRecordID as needed):

wevtutil qe $eventLog /f:RenderedXML /e:EVENTS /q:"<QueryList><Query Id='0' Path='$eventLog'><Select Path='$eventLog'>*[System[(EventRecordID=$eventRecordID)]]</Select></Query></QueryList>"

Step 7: Next Steps - Find Other Event Parameters

Any Event Parameter can be passed to a task. Event parameters can be found by inspecting the specific Event's Properties using the Event Viewer.

Example Task:

<?xml version="1.0" encoding="UTF-16"?>
<Task version="1.2" xmlns="http://schemas.microsoft.com/windows/2004/02/mit/task">
<RegistrationInfo>
    <Date>2007-11-09T09:21:33.3102706</Date>
    <Author>Otto Helweg</Author>
</RegistrationInfo>
<Triggers>
    <EventTrigger>
      <Enabled>true</Enabled>
      <Subscription><QueryList><Query Id="0" Path="Application"><Select Path="Application">*</Select><Select Path="Security">*</Select><Select Path="System">*</Select></Query></QueryList></Subscription>
      <ValueQueries>
        <Value name="eventChannel">Event/System/Channel</Value>
        <Value name="eventRecordID">Event/System/EventRecordID</Value>
        <Value name="eventSeverity">Event/System/Level</Value>
      </ValueQueries>
    </EventTrigger>
</Triggers>
<Principals>
    <Principal id="Author">
      <UserId>SomeUser</UserId>
      <LogonType>InteractiveToken</LogonType>
      <RunLevel>LeastPrivilege</RunLevel>
    </Principal>
</Principals>
<Settings>
    <IdleSettings>
      <Duration>PT10M</Duration>
      <WaitTimeout>PT1H</WaitTimeout>
      <StopOnIdleEnd>true</StopOnIdleEnd>
      <RestartOnIdle>false</RestartOnIdle>
    </IdleSettings>
    <MultipleInstancesPolicy>Queue</MultipleInstancesPolicy>
    <DisallowStartIfOnBatteries>true</DisallowStartIfOnBatteries>
    <StopIfGoingOnBatteries>true</StopIfGoingOnBatteries>
    <AllowHardTerminate>true</AllowHardTerminate>
    <StartWhenAvailable>false</StartWhenAvailable>
    <RunOnlyIfNetworkAvailable>false</RunOnlyIfNetworkAvailable>
    <AllowStartOnDemand>true</AllowStartOnDemand>
    <Enabled>true</Enabled>
    <Hidden>false</Hidden>
    <RunOnlyIfIdle>false</RunOnlyIfIdle>
    <WakeToRun>false</WakeToRun>
    <ExecutionTimeLimit>P3D</ExecutionTimeLimit>
    <Priority>7</Priority>
</Settings>
<Actions Context="Author">
    <ShowMessage>
      <Title>Event Parameters</Title>
      <Body>Record ID = $(eventRecordID)
Log = $(eventChannel)
Severity = $(eventSeverity)</Body>
    </ShowMessage>
</Actions>
</Task>

Quick and Dirty Web Site Monitoring with PowerShell

ottoh — Thu, 23 Aug 2007 19:44:00 GMT

The other day Mark noticed that redirections for our http://www.sysinternals.com/ URL were intermittently failing. In order to get more objective data, I built a script that tested the URL every 5 seconds, and reported back Success or Failure as well as performance (how long it took to completely download base HTML content). I found that PowerShell provided an easy way to use the WebClient .Net object and evaluate the returned HTML content.

Don't have PowerShell? Get it here.

Example 1: Single Site Monitoring

The following example opens a URL every 5 minutes, tests the content, and measures the time it took to download the HTML for the page. Notice that all the HTML is dumped into a big fat string. The string is then searched for specific text that is known to be in the requested page. Note that this script runs forever and can be stopped with a <Ctrl> 'C'.

Example PowerShell script:

$webClient = new-object System.Net.WebClient
$webClient.Headers.Add("user-agent", "PowerShell Script")

while (1 -eq 1) {
   $output = ""

   $startTime = get-date
   $output = $webClient.DownloadString("http://www.sysinternals.com/")
   $endTime = get-date

   if ($output -like "*Mark Russinovich*") {
      "Success`t`t" + $startTime.DateTime + "`t`t" + ($endTime - $startTime).TotalSeconds + " seconds"
   } else {
      "Fail`t`t" + $startTime.DateTime + "`t`t" + ($endTime - $startTime).TotalSeconds + " seconds"
   }

sleep(300)
}

Example 2: Monitoring and Alerting for Multiple Web Sites

This script monitors multiple URLs (or web sites), and incorporates e-mail alerting and logging. Unlike the above script, it is designed to be triggered from the Windows Task Scheduler (or some other job scheduler) rather than running forever in a loop. Notice that one of the URLs is actually a zipped file and PowerShell has no problem evaluating it as a string.

Example PowerShell script:

# Collects all named paramters (all others end up in $Args)
param($alert)

# Display Help
if (($Args[0] -eq "-?") -or ($Args[0] -eq "-help")) {
   ""
   "Usage: SysinternalsSiteTest.ps1 -alert <address> -log"
   "       -alert <address>      Send e-mail alerts"
   "       -log                  Log results"
   ""
   "Example: SysinternalsSiteTest.ps1 -alert somebody@nospam.com -log"
   ""
   exit
}

# Create the variables
$global:GArgs = $Args

$urlsToTest = @{}
$urlsToTest["Sysinternals Redirect"] = "http://www.sysinternals.com"
$urlsToTest["TechNet Redirect"] = "http://www.microsoft.com/sysinternals"
$urlsToTest["Sysinternals Home"] = "http://www.microsoft.com/technet/sysinternals/default.mspx"
$urlsToTest["Sysinternals Forum"] = "http://forum.sysinternals.com"
$urlsToTest["Sysinternals Blog"] = "http://blogs.technet.com/sysinternals"
$urlsToTest["Sysinternals Downloads"] = "http://download.sysinternals.com/Files/NtfsInfo.zip"

$successCriteria = @{}
$successCriteria["Sysinternals Redirect"] = "*Mark Russinovich*"
$successCriteria["TechNet Redirect"] = "*Mark Russinovich*"
$successCriteria["Sysinternals Home"] = "*Mark Russinovich*"
$successCriteria["Sysinternals Forum"] = "*Sysinternals Utilities*"
$successCriteria["Sysinternals Blog"] = "*Sysinternals Site Discussion*"
$successCriteria["Sysinternals Downloads"] = "*ntfsinfo.exe*"

$userAgent = "PowerShell User"
$webClient = new-object System.Net.WebClient
$webClient.Headers.Add("user-agent", $userAgent)

foreach ($key in $urlsToTest.Keys) {
   $output = ""

   $startTime = get-date
   $output = $webClient.DownloadString($urlsToTest[$key])
   $endTime = get-date

   if ($output -like $successCriteria[$key]) {
      $key + "`t`tSuccess`t`t" + $startTime.DateTime + "`t`t" + ($endTime - $startTime).TotalSeconds + " seconds"

      if ($GArgs -eq "-log") {
         $key + "`t`tSuccess`t`t" + $startTime.DateTime + "`t`t" + ($endTime - $startTime).TotalSeconds + " seconds" >> WebSiteTest.log
      }
   } else {
      $key + "`t`tFail`t`t" + $startTime.DateTime + "`t`t" + ($endTime - $startTime).TotalSeconds + " seconds"

      if ($GArgs -eq "-log") {
         $key + "`t`tFail`t`t" + $startTime.DateTime + "`t`t" + ($endTime - $startTime).TotalSeconds + " seconds" >> WebSiteTest.log
      }

      if ($alert) {
         $emailFrom = "computer@nospam.com"
         $emailTo = $alert
         $subject = "URL Test Failure - " + $startTime
         $body = "URL Test Failure: " + $key + " (" + $urlsToTest[$key] + ") at " + $startTime
         $smtpServer = "somesmtpserver.nospam.com"
         $smtp = new-object Net.Mail.SmtpClient($smtpServer)
         $smtp.Send($emailFrom,$emailTo,$subject,$body)
      }
   }
}

Quick and Dirty Software Inventory with PsInfo and PowerShell

ottoh — Sun, 04 Mar 2007 23:30:00 GMT

PsInfo is great for gathering asset information from Windows computers, both locally and remotely. PowerShell is great for automation and cleaning up output (among other things) as well as working with database driven data.

The following examples show how to gather an itemized list of the installed software on remote machines, process the data, then either display it to the screen or store it in a database. It's worth noting that PsInfo can also work on multiple remote computers from its native command line, or even read a list of computers from a file (check out the PsInfo site for more info). Since the final example seeks to show PsInfo in a database driven envoriment, PowerShell comes in very handy.

Note: In order for this example to work the necessary network connectivity and credentials will need to be in place.

Consider the following examples:

1 - The output is merely displayed on the screen. With this method the output can be redirected to a file and imported into an application like Excel for further analysis or record keeping.

2 - A database is used to drive the computers polled as well as store the output. The database table is very flat (one table) with 2 fields: 'Computer' and 'Software'. For large amounts of data, this will need to be normalized.

With the following output (imported into Excel):

Example 1: Standard Screen Output

The following PowerShell script gathers a software inventory from 3 remote computers ('happyhour', 'shaken', and 'extradry'). Presumably, your computer names will be different. After gathering and parsing the data, it's then displayed on the screen for all machines successfully queried.

Before running this script, test your connectivity and credentials with a single PsInfo command:

PsInfo -s Applications \\somecomputer

Example PowerShell script:

$computersToQuery = ("happyhour","shaken","extradry")

$softwareInventory = @{}
foreach ($computer in $computersToQuery) {
   $psinfoOutput = ./psinfo.exe -s Applications \\$computer

   $foundSoftwareInventory = 0
   $computerName = ""
   foreach ($item in $psinfoOutput) {
      if ($foundSoftwareInventory -eq 1) {
         # Force the results to a string
         # Remove any single quotes which interfere with T-SQL statements
         # Load the result into a hash whereby removing any duplicates
         [string]$softwareInventory[$computerName][$item.Replace("'","")] = ""
      }

      if ($item -like "System information for *") {
         $computerName = $item.Split("\")[2].TrimEnd(":")
      } elseif ($item -eq "Applications:") {
         $foundSoftwareInventory = 1
         $softwareInventory[$computerName] = @{}
      }
   }
}

foreach ($computer in $softwareInventory.Keys) {
   foreach ($softwareItem in $softwareInventory[$computer].Keys) {
      $computer + ":" + $softwareItem
   }
}

Your output should look something like:

Example 2: Save Output to a Database

This example is additive to the first in that it adds the following 3 items:

1- Pulls the list of computer to query from a database table
2- Adds the current data and time to the result
3- Records the audit results into a database

The following is the database schema for this example:

Example PowerShell script:

# Open the database connection
$dbConn = new-object System.Data.SqlClient.SqlConnection "server=kcdb;database=Inventory;Integrated Security=sspi"
$dbConn.Open()
$sqlQuery = $dbConn.CreateCommand()

# Get all known computers
$sqlQuery.CommandText = "select * from Inventory..Computers"
$reader = $sqlQuery.ExecuteReader()
$computersToQuery = @()
while ($reader.Read()) {
   $computersToQuery += $reader["Computer"]
}

# Close the database connection
$dbConn.Close()

$softwareInventory = @{}
foreach ($computer in $computersToQuery) {
   $psinfoOutput = ./psinfo.exe -s Applications \\$computer

   $foundSoftwareInventory = 0
   $computerName = ""
   foreach ($item in $psinfoOutput) {
      if ($foundSoftwareInventory -eq 1) {
         # Force the results to a string
         # Remove any single quotes which interfere with T-SQL statements
         # Load the result into a hash whereby removing any duplicates
         [string]$softwareInventory[$computerName][$item.Replace("'","")] = ""
      }

      if ($item -like "System information for *") {
         $computerName = $item.Split("\")[2].TrimEnd(":")
      } elseif ($item -eq "Applications:") {
         $foundSoftwareInventory = 1
         $softwareInventory[$computerName] = @{}
      }
   }
}

$dbConn = new-object System.Data.SqlClient.SqlConnection "server=kcdb;database=Inventory;Integrated Security=sspi"
$dbConn.Open()
$sqlQuery = $dbConn.CreateCommand()

foreach ($computer in $softwareInventory.Keys) {
   foreach ($softwareItem in $softwareInventory[$computer].Keys) {
      "Loading-" + $computer + ":" + $softwareItem

      # Try an Insert than an Update
      trap {
         $sqlQuery.CommandText = "update Inventory..SoftwareInventory set AuditDate = getdate() where Computer = '" + $computer + "' and Software = '" + $softwareItem + "'"
         $result = $sqlQuery.ExecuteNonQuery()
         continue
      }

      $sqlQuery.CommandText = "insert into Inventory..SoftwareInventory (Computer,Software,AuditDate) values ('" + $computer + "','" + $softwareItem + "',getdate())"
      $result = $sqlQuery.ExecuteNonQuery()
   }
}

$dbConn.Close()

For more information:

Get PsInfo bits and info: http://www.microsoft.com/technet/sysinternals/utilities/psinfo.mspx
Get PowerShell bits and info: http://www.microsoft.com/technet/scriptcenter/hubs/msh.mspx
The PowerShell blog: http://blogs.msdn.com/powershell/

Automate E-Mail Alerts with Windows Vista

ottoh — Thu, 01 Mar 2007 22:24:00 GMT

The Windows Task Scheduler as been completely re-written for Windows Vista to be a very powerful automation tool. The Task Scheduler now has the ability to launch tasks from a wide variety of useful triggers rather than being limited to the standard time based trigger. One of the most powerful triggers is the 'On an event' trigger and this example can easly be extended to leverage it for even more alerting options. In addition, tasks can now execute notifications (message box and e-mail) where they were previously limited to the standard command line action.

This example sets up a simple e-mail alert using Task Scheduler and PowerShell which sends an e-mail every time somebody connects to (login, remote desktop, or unlocks) the local computer. This might be useful for auditing access to critical systems in a data center.

Consider the 2 options for sending an e-mail alert from the new Task Scheduler:

1 - E-Mail Action using PowerShell:

Since this PowerShell script uses the SmtpClient object in Windows, it has more flexibility when dealing with SMTP server authentication requirements. This example sends the alert e-mail through an SMTP server that does not require authentication.

2- E-Mail directly through a Task Scheduler action:

Although the new Task Scheduler supports direct e-mail actions via SMTP, it is limited to fairly specific scenarios. It will only work if authentication is required by the SMTP server and the credentials used for the SMTP authentication need to be the same credentials used to run the task. For these reasons, this example will focus on option #1.

Step 1: Open 'Task Scheduler' and create a task to run whether the user is logged on or not

Step 2: Create 3 triggers ('At log on', 'On connection to user session', 'On workstation unlock'). Notice that an 'On event' trigger may be used instead to track a wide variety of state changes in a computer.

Step 3: Create the action to run the PowerShell e-mail script

Step 4: Create the PowerShell script (for this example it's called 'ConnectionAlert.ps1')

$emailFrom = "computer@nospam.com"
$emailTo = "somebody@nospam.com"
$subject = "Connection Alert"
$body = "Somebody has connected to the computer."
$smtpServer = "somesmtpserver.nospam.com"
$smtp = new-object Net.Mail.SmtpClient($smtpServer)
$smtp.Send($emailFrom,$emailTo,$subject,$body)

Step 5: Test the PowerShell Script

C:\> PowerShell ./ConnectionAlert.ps1

Step 6: Test the Task by running it directly from the Task Scheduler, then by locking and unlocking the computer.

For more information:

Windows Vista Task Scheduler: http://msdn2.microsoft.com/en-us/library/aa383614.aspx
Get PowerShell bits and information: http://www.microsoft.com/technet/scriptcenter/hubs/msh.mspx

WS-Man Ping Test

ottoh — Wed, 21 Feb 2007 22:24:00 GMT

WS-Man (Windows Remote Management) is a new protocol and framework first delivered in Windows 2003 R2 and is more prevalent in Windows Vista and beyond.

There's an easy way to test WS-Man reachability that is similar to the network ping concept by testing both the network connectivity as well as the end-point service. The main component that is not tested by 'winrm id...' is security (authentication). The following examples use an end-point system called 'wsman.msft.net' which may or may not exist. At the time of this posting, it was active.

Consider the 2 basic network configurations.

1 - Direct connectivity:

Direct connectivity implies that there are no intermediate network proxies or that the intermediate network devices attempt to be transparent. Transparent devices include Network Address Translators (NATs), routers, etc. In the direct connectivity model, the following WS-Man Id command should work as is, therefore successfully testing the end point's WS-Man service availability.

winrm id -auth:none -remote:wsman.msft.net

2- Indirect connectivity:

When there is a non-transparent network device like a proxy between both end points, WinRM uses the proxy identified in the winhttp agent configuration (this is different than the proxy defined in IE). Much like IE needs to be aware of proxies in the environment for Internet or external connectivity, so does WinRM. This can be accomplished by using Net Shell to configure winhttp to route through a proxy as follows (run from an elevated command prompt).

netsh winhttp set proxy proxy-server=http://someproxy/

Now the following WS-Man ping should work.

winrm id -auth:none -remote:wsman.msft.net

A Few Good Vista WS-Man (WinRM) Commands

ottoh — Sat, 10 Feb 2007 02:57:00 GMT

In Vista, a lot of instrumentation, configuration, and utilization information is exposed via WS-Man. Vista WS-Man (aka: WS-Management, Windows Remote Management, and WinRM) incorporates many features, but I like to think of it as the management protocol/framework of the future (look out SNMP!). What makes WS-Man so great is the fact that it's all standards based, rides on HTTP/HTTPS (very firewall/NAT friendly), and packages its data in SOAP/XML packets (easy to shove into a database or use with a script).

Out of the box; Vista WS-Man exposes WMI information as well as Windows Remote Shell capabilities. What this means is that with WS-Man it's much easier to get instrumentation from remote machines as well as use that info in scripts.

Here are some sample commands to play with. If you cannot get the ‘Test WS-Man...' step to work, none of the steps following will work either (you're probably not using the right credentials to access the remote machine). One more caveat, the remote commands work best on domain joined machines. For workgroup machines, the WinRM service needs additional configuration.

Description	Command
Run from an Elevated* Command prompt*
Quickly configure the WS-Man service	winrm QuickConfig
Quickly delete the WS-Man listener	winrm invoke Restore winrm/Config @{}
Run from an standard* Command prompt*
Display your machine's basic hardware info	winrm enumerate wmicimv2/Win32_ComputerSystem
Display your operating system properties	winrm get wmicimv2/Win32_OperatingSystem
Output your OS info in XML	winrm get wmicimv2/Win32_OperatingSystem -format:pretty
Ping WS-Man on a remote machine	winrm id -auth:none -remote:<some machine>
Test WS-Man access to a remote machine**	winrm id -remote:<some machine>
Grab a remote machine's WS-Man config	winrm get winrm/Config -r:<some machine>
Grab a remote machine's CPU load	winrm g wmicimv2/Win32_Processor?DeviceID=CPU0 -fragment:LoadPercentage -r:<some computer>
Grab a remote machine's free memory	winrm g wmicimv2/Win32_OperatingSystem -fragment:FreePhysicalMemory -r:<some computer>
Stop a service on a remote machine	winrm invoke stopservice wmicimv2/Win32_Service?name=w32time -r:<some computer>
Start a service on a remote machine	winrm invoke startservice wmicimv2/Win32_Service?name=w32time -r:<some computer>
Reboot a remote machine	winrm invoke reboot wmicimv2/Win32_OperatingSystem -r:<some computer>
Run a command on a remote machine (this uses winrS, not winrM)	winrs -r:<some computer> ipconfig /all
Run from PowerShell
Use PowerShell to grab the WS-Man Win32_OperatingSystem XML output	[xml]$osInfo = winrm get wmicimv2/Win32_OperatingSystem /format:pretty
Display the OS version property	$osInfo.Win32_OperatingSystem.Version
Display the last boot time	$osInfo.Win32_OperatingSystem.LastBootupTime.DateTime
Put free memory metric into an XML variable	[xml]$freemem = cmd /c "winrm get wmicimv2/Win32_OperatingSystem -fragment:FreePhysicalMemory -f:pretty -r:<some computer>"
Display the free memory value	$freemem.XMLFragment.FreePhysicalMemory

**Note: This step verifies that you have good connectivity to the remote machine, WS-Man is running and properly configured on the remote machine, AND you have the correct permissions to fully leverage WS-Man on the remote machine. If this step fails, it's probably a permissions issue.

Details:

WS-Man (WinRM) Architecture

The following diagram shows a high-level overview of the WS-Man (WinRM) architecture. In the diagram the 'Client' is querying the 'Server' for WS-Man information. Note that HTTP.sys and WinHTTP support the HTTP(s) transport for WS-Man, not IIS. In addition, IIS (or another web publishing service) can co-exist with WS-Man and share port 80.

Quickly configure the WS-Man service

Remember, this needs to be run from an 'Elevated' Command Prompt.

As you can see, this simple command does quite a bit. Please note every modification (hightlighted) since this might increase the attack surface of your computer. For example, Quick Config configures a listener that accepts connections from every network interface. This is probably not ideal for edge machines that connect to unsecure networks (like the Internet). In addition, this command only needs to be run once.

Quickly delete the WS-Man listener

Although this command only deletes all WinRM listeners, it effectively turns off any WS-Man communication to a machine.

Display your machine's basic hardware info and operating system properties

Win32_ComputerSystem and Win32_OperatingSystem are common WMI classes and useful for asset information and configuration information as well as some utilization metrics.

Output your OS info in XML

XML output makes the data much easier for storing in a database or dealing with programmatically (like with a script). PowerShell makes this that much easier since it works VERY well with XML (see below for a sample PowerShell script).

Ping WS-Man on a remote machine and test authorization credentials

Testing WS-Man on a remote machine is very useful in troubleshooting lots of connectivity and configuration issues. When pinging WS-Man without 'auth', it allows for the testing of the connectivity and basic service configuration. Using the 'auth' parameter tests the necessary authorization. Generally the credentials need to be in the 'Administrators' group for 'auth' to work. In this case no credentials are provided so the current credentials are used (this can be over-ridden). Notice that the OS version is included when successfully using 'auth' to test WS-Man.

Grab a remote machine's WS-Man config

Successfully completing this step pretty much insures that one has complete access to WS-Man on the remote computer.

Grab a remote machine's CPU load, free memory, and restart a service

WS-Man allows for gathering WMI properties (reading and writing although we're only reading in this example) as well as invoking methods (starting and stopping a service; as well as rebooting!). Notice the minimal return code for the invoke commands (0 = success).

Run a command on a remote machine (this uses winrS, not winrM)

WinRS is another utility that leverages WS-Man. WinRS allows for the execution of local, non-interactive command-line commands on a remote machine and returns the output. In other words, if the command can be run at the CMD prompt without any required input and it only accesses local resource (no network shares for example), then it will most likely work. There are ways to get around the 'local resource' issue, but that is out of scope for this blog.

Use PowerShell to grab instrumentation via WS-Man

PowerShell is great for consuming data from WS-Man since it works very well with XML. Notice that in the following example it's easy to 'surf' an object in PowerShell. In this case the XML object 'osInfo' is displayed on its own, and then expanded to the 'Win32_OperatingSystem' branch.

A simple PowerShell script

The following script shows how easy it is to automate the collection of WS-Man information using PowerShell.

$machines = ("machine1","machine2","machine3")
foreach ($machine in $machines) {
[xml]$osInfo = winrm get wmicimv2/Win32_OperatingSystem /format:pretty /r:$machine
$machine + ": " + $osInfo.Win32_OperatingSystem.LastBootupTime.DateTime
}

More Info

For more information on WS-Man, please see the following articles:

Microsoft: About Windows Remote Management
Microsoft: Using Windows Remote Management
Microsoft: Windows Remote Management Reference
Sun: A Java Implementation of WS-Man
SourceForge: Open WS-Man Project
DMTF (PDF): WS-Management Fact Sheet
DMTF (PDF): WS-Management Standard Document