Goatee PFE

Blog of Microsoft Premier Field Engineer Ashley McGlone featuring PowerShell scripts for Active Directory.

PowerShell Get-WinEvent XML Madness: Getting details from event logs

I hope to meet you at

PowerShell Summit North America 2014

PowerShell Get-WinEvent XML Madness: Getting details from event logs

  • Comments 11
  • Likes

Announcements

Before we jump into today’s script here are some current events:

  • This blog post celebrates three years of PowerShell blogging on TechNet as GoateePFE.  It has been a great ride, and I am far from done.  See the most popular posts here.  Thank you for making this blog successful.
  • The PowerShell Deep Dives book is out now.  I contributed a chapter on Active Directory token bloat taken from my SID history blog series.  This book has a ton of great chapters by a ton of great people. All the proceeds go to Save The Children.  Buy your copy today.
  • If you haven’t had a chance to watch the Microsoft Virtual Academy recordings Getting Started with PowerShell 3.0 Jump Start and Advanced Tools & Scripting with PowerShell 3.0 Jump Start then you need to put them on your list.  Jeffrey Snover and Jason Helmick do a fantastic job of covering everything you need to know to get started with PowerShell.  Make time for this over several lunches or knock it out in a couple training days.  These videos will seriously boost your career.  You could even gather the family around with a bowl of popcorn.
  • PowerShell Saturday 005 is coming up October 26th in Atlanta, Georgia.  My session is titled It’s Time To Part With Blankie: Moving from command line tools to PowerShell for Active Directory.  If you’re in the area stop by for a good time with several PowerShell celebrities.  I’m looking forward to Ed Wilson’s session PowerShell Workflows for Mere Mortals.

Now for today’s topic…

XML vs. IT Pro

Maybe I haven’t looked hard enough, but I’ve just not found any clear documentation aimed at IT Pros for what I am posting today.  As an IT Pro type guy (not a .NET type guy) I have avoided XML for years.  CSV and HTML are so much easier.  XML seems to be a labyrinth of complexity in my mind, and it still is, at least from a PowerShell perspective.  The object model is convenient, but trying to navigate it loses me.  Yeah, I know XML makes the world a happy place, but I’m just not there yet.

Despite this disparaging disclaimer I believe I have drafted a script that will help many of us IT Pros as we weed through event logs (or ETL trace files or EVTX files).

The Backstory

A couple years ago a distinguished peer of mine, Matthew Reynolds, invited me to contribute to a project parsing ETL trace data with PowerShell.  You can hear him talk about the fruits of this labor in his recent TechEd talk, How Many Coffees (New 2013 Edition) Can You Drink While Your PC Starts?  Through this project I became very familiar with parsing XML event log data.  Most of this learning happened for me while I was parsing GPO processing events.

Recently I had a request from a customer who wanted to parse logon audit events and filter deep into the event message body.  I had to dust off my code from the former project and dive a little deeper.  I figured if I’ve had this much trouble with the topic, then I should blog it so that I have something to refer back to when I forget all this again in a few months.

Events:  The good, the bad, and the ugly

The good:  PowerShell works with event logs out of the box.  You have two cmdlets:  Get-EventLog and Get-WinEvent.  Get-WinEvent is the one we’re all supposed to use now.

The bad:  All of a sudden reading event logs gets complicated.  The filtering in particular requires some crazy syntax.  We are far removed from the simplicity of DUMPEL.  PowerShell team blog posts from 2009 here and here attempt to make this look routine.  Um… yeah.

The ugly:  All of the juicy nuggets of event data in the message body are stored in XML.  And nearly every combination of event ID and provider has a unique event schema for storing the data we want.  Neo’s MSDN blog post gets us most of the way there.  AskDS and Hey Scripting Guy show how we can use the GUI to help write the XML filter syntax.  Now my head is spinning.  This is the farthest point from intuitive.  Don’t even get me started on XPATH.

Note:  In all fairness to the product this data structure is necessary.  All events have a few common properties like provider, ID number, date/time, source, etc.  But in order to capture the unique details of each event we needed a way to store a variable number of properties.  So the design is good, just a bit complicated to script.

In the life of every scripter you will come to challenges like this.  You just have to cowboy up and dive in.  I recommend that you cruise through these other articles linked above for some good details on filtering XML event log data.

The thing I’ve not seen in these blog posts is how to dump out the event message data in a CSV file where I can easily report and manipulate the data I need.  For example, if I’m collecting logon failure event 4625, then I want the guts of the message body in separate columns where I can easily summarize and report on the user and computer accounts involved.  While I can harvest event logs from multiple servers in the GUI, it is just not friendly for mass reporting, sorting and visualization like Excel.  This is the problem I am trying to solve. 

image

The individual nuggets of interest are represented as XML in the EventData portion of the message:

image

 

Finding Event Message Schemas

Use the lines below to discover the XML schemas behind the events you want to parse.  Run each line and substitute the event IDs and providers you want to investigate.  These lines serve as a bit of an event log deep dive when you run them on your own machine.  Note that some of these lines may need to run in an elevated shell (Run As Administrator).

            
# List all event providers            
Get-WinEvent -ListProvider * | Format-Table            
            
# List all policy-related event providers.            
Get-WinEvent -ListProvider *policy* | Format-Table            
            
# List the logs on the machine where the name is like 'policy'            
Get-WinEvent -ListLog *policy*            
            
# List all possible event IDs and descriptions for the provider            
(Get-WinEvent -ListProvider Microsoft-Windows-GroupPolicy).Events |            
   Format-Table id, description -AutoSize            
            
# List all of the event log entries for the provider            
Get-WinEvent -LogName Microsoft-Windows-GroupPolicy/Operational            
            
# Each event in each provider has its own message data schema.            
# Use this line to find the schema of each event ID.            
# For a specific event            
(Get-WinEvent -ListProvider Microsoft-Windows-GroupPolicy).Events |
   Where-Object {$_.Id -eq 5314}
            
# For a keyword in the event data            
(Get-WinEvent -ListProvider Microsoft-Windows-GroupPolicy).Events |            
   Where-Object {$_.Template -like "*reason*"}            
            
# Find an event ID across all ETW providers:            
Get-WinEvent -ListProvider * |            
   ForEach-Object { $_.Events | Where-Object {$_.ID -eq 4168} }            

Notice that the Template property holds the XML definition of the event message body.  This is where our event log goodness hides.  The trick is parsing these individual XML values for our reporting.  I’ve highlighted the data entries and their corresponding placeholders in the GPO event schema example below. 

PS C:\> (Get-WinEvent -ListProvider Microsoft-Windows-GroupPolicy).Events |
 Where-Object {$_.Id -eq 5314}


Id          : 5314
Version     : 0
LogLink     : System.Diagnostics.Eventing.Reader.EventLogLink
Level       : System.Diagnostics.Eventing.Reader.EventLevel
Opcode      : System.Diagnostics.Eventing.Reader.EventOpcode
Task        : System.Diagnostics.Eventing.Reader.EventTask
Keywords    : {}
Template    : <template
              xmlns="http://schemas.microsoft.com/win/2004/08/events">
              <data name="BandwidthInkbps" inType="win:UInt32"
                  outType="xs:unsignedInt"/>
              <data name="IsSlowLink" inType="win:Boolean"
                  outType="xs:boolean"/>
              <data name="ThresholdInkbps" inType="win:UInt32"
                  outType="xs:unsignedInt"/>
              <data name="PolicyApplicationMode" inType="win:UInt32"
                  outType="xs:unsignedInt"/>
              <data name="ErrorCode" inType="win:UInt32"
                  outType="win:HexInt32"/>
              <data name="LinkDescription" inType="win:UnicodeString"
                  outType="xs:string"/>
              </template>

Description : A %6 link was detected. The Estimated bandwidth is %1 kbps. The
              slow link threshold is %3 kbps.

 

Cracking the XML Nut

So how do I pull those values out of the event message?  This time we’re going to look at a log from one of our domain controllers (DCs).  Notice the filter syntax.  DO NOT pipe the entire event log to a Where-Object if you want results returned in this century.  Let’s grab one event entry to examine:

# Prompts for creds            
$cred = Get-Credential Contoso\Administrator            

# Grab the events from a DC            
# Target DC needs firewall rule enabled:            
# Remote Event Log Management (RPC)            
$Event = Get-WinEvent -ComputerName 2012DC -Credential $cred ` 
 -FilterHashtable @{Logname='Security';Id=4625} ` 
 -MaxEvents 1

# View the event properties.
$Event | Format-List *

# View the array of message body values.
# But the property names are missing.
$Event.Properties

# Convert the event to XML
$eventXML = [xml]$Event.ToXml()

# Drill down through the XML to the message goodness
# Ah ha! This is what we want.
$eventXML.Event.EventData.Data

# You have to index each data element to access it.
$eventXML.Event.EventData.Data[0].name
$eventXML.Event.EventData.Data[0].'#text'

 

PS C:\> $eventXML.Event.EventData.Data

Name                      #text                          
----                      -----                          
SubjectUserSid            S-1-5-18                       
SubjectUserName           2012DC$                        
SubjectDomainName         CONTOSO                        
SubjectLogonId            0x3e7                          
TargetUserSid             S-1-0-0                        
TargetUserName            DanPark                        
TargetDomainName          CONTOSO                        
Status                    0xc000015b                     
FailureReason             %%2308                         
SubStatus                 0x0                            
LogonType                 4                              
LogonProcessName          Advapi                         
AuthenticationPackageName Negotiate                      
WorkstationName           2012DC                         
TransmittedServices       -                              
LmPackageName             -                              
KeyLength                 0                              
ProcessId                 0x390                          
ProcessName               C:\Windows\System32\svchost.exe
IpAddress                 -                              
IpPort                    -

  

My Inefficient Magic XML to CSV Event Reporting Machine

The code in this example pulls event 4625 from the Security log on a domain controller, and then it copies each of the XML message body properties into their own event object property for easier reporting.  Note that the DC must have the firewall rule enabled to allow Remote Event Log Management (RPC).

# Prompt for creds            
$cred = Get-Credential Contoso\Administrator            
            
# Grab the events from a DC            
$Events = Get-WinEvent -ComputerName 2012DC -Credential $cred ` 
    -FilterHashtable @{Logname='Security';Id=4625}            
            
# Parse out the event message data            
ForEach ($Event in $Events) {            
    # Convert the event to XML            
    $eventXML = [xml]$Event.ToXml()            
    # Iterate through each one of the XML message properties            
    For ($i=0; $i -lt $eventXML.Event.EventData.Data.Count; $i++) {            
        # Append these as object properties            
        Add-Member -InputObject $Event -MemberType NoteProperty -Force ` 
            -Name  $eventXML.Event.EventData.Data[$i].name ` 
            -Value $eventXML.Event.EventData.Data[$i].'#text'            
    }            
}            
            
# View the results with your favorite output method            
$Events | Export-Csv .\events.csv            
$Events | Select-Object * | Out-GridView            

I call this inefficient, because it must go back through the event log data a second time to process the XML message body properties.  This is OK for smaller data sets, but your performance will be slower with larger data sets.

Note that this approach is only appropriate for events that share the same schema.  Some events within a provider will have the same schema, even though the event IDs are different.  In those cases it would be OK to include multiple event IDs in your query.

image

This is my solution to the problem called out at the beginning of the article.  Now that I have all of the XML data in native event object properties I can easily group, filter, sort, and report.  Yes, there is fancy XML syntax that will filter deep into the message body, but that does not give me visibility to all of the values.  Now I have the data exposed in a CSV spreadsheet where I can quickly visualize patterns and trends.  Add some pivot tables and charts to make your reporting management-friendly.

The Big Finish

You could really amp this up by running it as a workflow in parallel against all of your target computers.  Or you could use Invoke-Command -AsJob for multi-threading.  Unfortunately remote sessions and workflows return their results as deserialized objects that lack the .ToXML() method.  I’m still working on a way to get this working over these remoting technologies.  In the meantime you can scale this out across multiple servers by using the cmdlet Start-Job to spin up a thread for each server you query.

Am I Wrong?

One of the things I like about working in IT is constantly learning new things.  I’ve been doing PowerShell for three years, but I still have much to learn.  If I’m missing something here please let me know in the comments below.  Have you found an easier way to work with XML event message bodies from Get-WinEvent?  I’d love to hear about it.

Can you help me?  Yes!

If you would like to have me or another Microsoft PFE visit your company and assist with the ideas presented in this blog post, then contact your Microsoft Premier Technical Account Manager (TAM) for booking information.

For more information about becoming a Microsoft Premier customer email PremSale@microsoft.com.  Tell them GoateePFE sent you.

Sharing Links
Comments
  • <p>Awesome post! &nbsp;...and yes I have to agree working with that XML data from a scripting perspective is headache inducing. &nbsp;You had asked if anyone found an &quot;easier&quot; way to work with it. &nbsp;Unfortunately I can&#39;t offer an easier way, but a fairly similar methodology. &nbsp;Check out the &quot;Advanced Examples&quot; as a part of this older (and no longer updated) project: &nbsp;<a rel="nofollow" target="_new" href="http://pseventlogwatcher.codeplex.com/wikipage?title=Advanced%20Examples&amp;referringTitle=Documentation">pseventlogwatcher.codeplex.com/wikipage</a></p> <p>More specifically Example 1-B and Example 2. &nbsp;Although they are relate the some of the cmdlets in the module, you can see the basic logic used on the XML there. &nbsp;As you already mentioned this only works if you keep to one event ID type for each CSV generated.</p> <p>I am curious on the FOR loop to index $eventXML.Event.EventData.Data.Count in your example. &nbsp;Did you find Events with multiple Data nodes in your work? &nbsp;For the most part I was working with a subset of audit events when building the project, so I only scratched the surface of event schemas in my travels.</p> <p>Definitely agree though that it would be nice to find a more effecient way to break apart the details in the message details of the event.</p>

  • <p>I use export-clixml and convertto-xml cmdlets for turning deserialized objects into XML. &nbsp;I use PSRemoting in place of RPC everywhere in my environment so working with deserialized objects is in my bailiwick.</p>

  • <p>Thanks for the feedback. Good to hear.</p> <p>@sgrinker, I&#39;ve not found multiple data nodes in any of the providers or events I&#39;ve explored. Your mileage may vary.</p> <p>@jason, glad you&#39;re comfortable with XML. The only problem with deserialization is that you always lose the methods. Comes with the territory.</p> <p>Keep scripting,</p> <p>Ashley</p> <p>@GoateePFE</p>

  • <p>Running </p> <p>Get-WinEvent -FilterHashtable @{Logname=&#39;Security&#39;;Id=4625} on a local computer returns an error to say that parameter is incorrect at char13, which is -FilterHashtable</p> <p>It&#39;s in the examples if you do a get-help get-Winevent, so am not sure what is going wrong now. </p> <p>even copy-pasting the examples on FilterHashtabel don&#39;t work</p>

  • Hello Anonymous, I had the same result on my local machine. First, make sure you launch the PowerShell console as Administrator if you want to query the Security event log. Second, it is not likely that event 4625 is present on your local machine. Try event 4624. Let me know if those two changes help. Ashley (GoateePFE)

  • hi Ashley,<br/><br/>I been trying to run your script on my local server but keep getting errors. Its look like failing to index each data element to access it? its server 2012 and running PS as admin!<br/>here is my code:<br/><br/>$Event = Get-WinEvent -FilterHashtable @{Logname='ForwardedEvents'} <br/> <br/># Parse out the event message data <br/>ForEach ($Event in $Events) { <br/> # Convert the event to XML <br/> $eventXML = [xml]$Event.ToXml() <br/> # Iterate through each one of the XML message properties <br/> For ($i=0; $i -lt $eventXML.Event.EventData.Data.Count; $i++) { <br/> # Append these as object properties <br/> Add-Member -InputObject $Event -MemberType NoteProperty -Force -Name $eventXML.Event.EventData.Data[$i].name -Value $eventXML.Event.EventData.Data[$i].'#text' <br/> } <br/>} <br/># View the results with your favorite output method <br/>$Events | Export-Csv .\events.csv <br/>$Events | Select-Object * | Out-GridView <br/><br/><br/>and this is the error Iam getting:<br/>Add-Member : Cannot bind argument to parameter 'Name' because it is null.<br/>At line:10 char:79<br/>+ Add-Member -InputObject $Event -MemberType NoteProperty -Force -Name $e ...<br/>+ ~~<br/>

  • Hello zafman,<br/><br/>I think the error is in your first line:<br/>$Event = Get-WinEvent -FilterHashtable @{Logname='ForwardedEvents'} <br/><br/>Instead make the variable plural:<br/>$Events = Get-WinEvent -FilterHashtable @{Logname='ForwardedEvents'} <br/><br/>Let me know if that fixes it.<br/>Ashley<br/>@GoateePFE<br/>

  • I am still getting the same error!<br/>Add-Member : Cannot bind argument to parameter 'Name' because it is null.<br/>At line:10 char:70<br/>+ Add-Member -InputObject $Event -MemberType NoteProperty -Force -Name $eventXML.E ...<br/>+ ~~~~~~~~~~~<br/> + CategoryInfo : InvalidData: (:) [Add-Member], ParameterBindingValidationException<br/> + FullyQualifiedErrorId : ParameterArgumentValidationErrorNullNotAllowed,Microsoft.PowerShell.Commands.AddMemberCommand<br/><br/>ForwardedEvents are coming from server 2003 (event ID=680)<br/><br/>my results are little bit different then from yours for this events (680)<br/><br/>XML in the EventData portion of the message: <br/><br/>- <Event xmlns="<a href="http://schemas.microsoft.com/win/2004/08/events/event">http://schemas.microsoft.com/win/2004/08/events/event</a>"><br/>- <System><br/> <Provider Name="Security" /> <br/> <EventID Qualifiers="0">680</EventID> <br/> <Level>0</Level> <br/> <Task>9</Task> <br/> <Keywords>0xa0000000000000</Keywords> <br/> <TimeCreated SystemTime="2014-04-01T16:13:36.000Z" /> <br/> <EventRecordID>243224405</EventRecordID> <br/> <Channel>Security</Channel> <br/> <Computer>AD01</Computer> <br/> <Security UserID="S-1-5-21-217390450-0000000000-95347372-0000" /> <br/> </System><br/>- <EventData><br/> <Data>MICROSOFT_AUTHENTICATION_PACKAGE_V1_0</Data> <br/> <Data>John.Doe</Data> <br/> <Data>PC07</Data> <br/> <Data>0x0</Data> <br/> </EventData><br/> <RenderingInfo Culture="en-US" /> <br/> </Event><br/><br/><br/>PS C:\Windows\system32> $Event.Properties<br/><br/>Value <br/>----- <br/>MICROSOFT_AUTHENTICATION_PACKAGE_V1_0 <br/>John.Doe<br/>PC07 <br/>0x0 <br/><br/><br/><br/>PS C:\Windows\system32> $eventXML.Event.EventData.Data<br/>MICROSOFT_AUTHENTICATION_PACKAGE_V1_0<br/>John.Doe<br/>PC07<br/>0x0<br/><br/><br/>any idea how can I pull these type event logs?

  • Hmmm. Get-WinEvent is not compatible with Windows Server 2003, although it can read the classic log format. I would have to repro this in my lab to completely analyze the scenario. For now...<br/>Set a debug break point in the PowerShell ISE, then use F11 to step one line at a time. Use the command pane to inspect the value of these two items:<br/>$eventXML.Event.EventData.Data[$i].name<br/>$eventXML.Event.EventData.Data[$i].'#text'<br/>You could even substitute numbers for the variables like this:<br/>$eventXML.Event.EventData.Data[0].name<br/>$eventXML.Event.EventData.Data[0].'#text'<br/>Then change the 0 to 1,2,3 respectively to check each one of your expected name/value pairs.<br/>That should help you identify the null name.<br/>Hope this helps,<br/>Ashley<br/>@GoateePFE<br/>

  • I used the Event Log Collector to pull all of my logs to a single server. They ALL wind up in a very busy 'Forwarded Events' folder, but from there I can manipulate them. <br/><br/>There are filtering options when setting up the Subscriptions. You can even apply an XML filter that you have written or copied from another source. <br/><br/>Does that address the problem with de-serialized return results? The events that I am receiving seem complete and contain the source Computer name.

  • Great job Mr. McGlone

Your comment has been posted.   Close
Thank you, your comment requires moderation so it may take a while to appear.   Close
Leave a Comment