Hey, Scripting Guy! Question

Hey, Scripting Guy! I would like to parse the firewall log on my computer and find out the kinds of packets that are going out. You know, how many Web page packets, how many DNS lookups, how many pings. That kind of stuff. I figure that I can look at these logs and pick up diagnostic information. For example, if I see a bunch of DNS queries going to our old DNS server, I know we have some configuration issues to deal with, and I need to track down the computers that have the old DNS information on their machines and find out why. Can you figure out how to do this?

- CG

SpacerHey, Scripting Guy! Answer

Hi CG,

It is amazing the things we think we know that we really do not know at all. Take song lyrics for example. There are entire Web sites devoted to misheard song lyrics. The problem is so great there is actually a term for the phenomenon—mondegreen. One of my favorite cities (Munich, Germany) has a similar problem with their downtown area, Marienplatz. Most people see the new city hall (Neues Rathaus), and they think it is really old:

Image of the new city hall in Munich, Germany

 

Then they see the old city hall (Altes Rathaus), which was first built in 1310, and they think it is the new city hall:

Image of the old city hall in Munich, Germany

 

To really understand what is going on, we need information that goes beyond just using our senses. Similarly, we cannot simply use our five senses (or even our sixth sense) to determine what is going on with our computer. We might see the hard drive light flicker or hear the drive clattering, but that is about it. CG, you correctly identified one cause of "slow Internet access" in your e-mail—problems with name resolution because of bad DNS configuration. The really cool thing is we are using the Windows Firewall, which is usually thought of as a security feature that aids troubleshooting.

This week we are focusing on regular expressions. There are some VBScript examples in the Script Center. Here is a good introduction from the 2008 Winter Scripting Games (by the way, in the 2009 Summer Scripting Games, I can pretty much guarantee you will need to be able to do something with regular expressions for one of the events). The Regex .NET Framework class from the System.Text.RegularExpressions namespace is documented on MSDN. This is one of the main classes we use in Windows PowerShell when working with regular expressions. You also will find some information about regular expressions in the Microsoft Press book, Windows PowerShell Scripting Guide. Here is a very good article about regular expression use in VBScript. In this week's articles, we are using Windows PowerShell for our samples. Please refer to the Windows PowerShell Scripting Hub for more information about this exciting new technology.

We decided to write a script called ScanFirewallLogGatherStats.ps1 that will look through the Windows Firewall and collect information about the different kinds of packets that are going through it. Because you asked about searching for a specific IP address, we added that capability too. We do not have a script similar to this one on the Script Center, so I cannot refer you to a VBScript example. I could not easily write a VBScript to perform the same activity either because VBScript does not have a command that is the same as the Switch statement. To use the script, you can run the script and it will produce default information. If you are interested in obtaining information about the prevalence of a particular IP address, you add that value to the $IpAddress variable. The ScanFirewallLogGatherStats.ps1 script is shown herehere.

Function New-TempFile
{
  [io.path]::GetTempFileName()
} #end Get-TempFile

Function Show-Output
{ 
 Param($TempFile)
 Notepad $TempFile | Out-Null 
 Remove-Item -path $TempFile
} #end Show-Output

Function Search-LogGatherStats()
{
  Param(
       $logFile
      )
 $fwLog = Get-Content -path $logFile
 "Scanning firewall log ....."
 Switch -regex ($fwLog)
 {
  $ipaddress { $addressCount+=1 }
  "tcp"     { $tcp+=1 }
  "udp"    { $udp+=1 }
  "icmp"  { $icmp+=1 }
  "\s53"   { $dns+=1 }
  "\s80"   { $web+=1 }
  "\s443" { $ssl+=1 }
 } 

if($IPaddress)
 {
  $hash = `
     @{
     "tcp" = $tcp ; "udp"= $udp ; "icmp"= $icmp ; 
     "dns"= $dns ; "web" = $web ; "ssl" = $ssl ;
     $IpAddress = $addressCount 
   } #end hash
  } #end if
ELSE
 {
  $hash = `
   @{
    "tcp" = $tcp ; "udp"= $udp ; "icmp"= $icmp ; 
    "dns"= $dns ; "web" = $web ; "ssl" = $ssl 
    } #end hash
 } #end else
 $hash
} #end Search-LogGatherStats

# *** Entry Point to script ***

$log = "C:\Windows\pfirewall.log"
$IpAddress = $null
Search-LogGatherStats -log $log |
Out-File -filepath (New-TempFile | Tee-Object -variable TempFile) | 
Show-Output -tempfile $TempFile

The first function we come to in our script is the New-TempFile function. This function is used to create a new temporary file, and was discussedyesterday’s “Hey, Scripting Guy!” article:

Function New-TempFile
{
  [io.path]::GetTempFileName()
} #end Get-TempFile

The next function we come to is the Show-Output function, which was also discussed in yesterday’s article. Here is the Show-Output function:

Function Show-Output
{ 
Param($TempFile)
Notepad $TempFile | Out-Null 
Remove-Item -path $TempFile
} #end Show-Output

See how easy it is to write a script when you organize your code into functions? It makes it easy to reuse the code. Now we come to the meat of our new script. The Search-LogGatherStats function begins by using the Function keyword to create a new functionfunction:

Function Search-LogGatherStats()
{

We create an input parameter to the function by using the Param statement and specifying the name of the variable that will be used to hold the value that is supplied when the function is called. This is seen here:

Param(
       $logFile
      )

It is time to collect the content of the firewall log. To do this, we use the Get-Content cmdlet, which will read all the text in the firewall log and store it in a variable named $fwLog. You will need to pay a bit of attention to the script when you use this technique because if the firewall is very big, the amount of memory required to store the contents of the log will grow proportionately. On my computer, the firewall log is set to a maximum size of 4 MB. However, if the log was allowed to grow to 2 GB, I would have to rewrite this section of the code.

$fwLog = Get-Content -path $logFile

Next we display a simple message that says we are scanning the firewall log. In Windows PowerShell, we do not have to use anything like Wscript.Echo, the way we do in VBScript. We only need to put the value to be emitted inside quotation marks. There is a Write-Host cmdlet, but generally, I only use it when I want to format the color of the text. Refer to this article for more information about using Windows PowerShell to display output in color.

"Scanning firewall log..."

Now we come to the Switch statement. The Switch statement is the most powerful statement in Windows PowerShell. It is the Swiss Army Knife of Windows PowerShell commands. For more information about using the Switch statement, refer to this Windows PowerShell article. Here we are using the Switch statement to parse a text file by using a regular expression pattern. The opening Switch command is seen here:

Switch -regex ($fwLog)

Inside the code block for the Switch statement, we have a series of regular expression patterns that will be evaluated. If one of these patterns matches, we increment the value of a variable by one. Each of these variables is used to keep track of the various protocols and other information that we are looking for in our firewall log file. The first patterns we look for are literal text values:

{
  $ipaddress { $addressCount+=1 }
  "tcp"     { $tcp+=1 }
  "udp"    { $udp+=1 }
  "icmp"  { $icmp+=1 }

The next pattern we have to match is the number 53, when it is preceded by any white space. The number 53 is a literal pattern match, and the "\s" is a special character that means any white space. Port 53 is the port that DNS traffic uses, and we use the variable $dns to count the pattern matches. This line of code is shown here:

"\s53"   { $dns+=1 }

The same is true for the numbers 80 (HTTP protocol) and 443 (HTTP over secure sockets layer):

"\s80"   { $web+=1 }
  "\s443" { $ssl+=1 }
}

If we specify a value for the $IPaddress variable, we have to display it when we display our  data:

if($IPaddress)
{

Now we create a hash table, which we use to create a nice table display. The hash table is similar to the dictionary object that is used in VBScript. For more information about hash tables, refer to this Windows PowerShell article.

$hash = `
     @{
     "tcp" = $tcp ; "udp"= $udp ; "icmp"= $icmp ; 
     "dns"= $dns ; "web" = $web ; "ssl" = $ssl ;
     $IpAddress = $addressCount 
   } #end hash
  } #end if

If there is no $IpAddress variable, we do not have to display it. We use this pattern for our hash table instead:

ELSE
{
  $hash = `
   @{
    "tcp" = $tcp ; "udp"= $udp ; "icmp"= $icmp ; 
    "dns"= $dns ; "web" = $web ; "ssl" = $ssl 
    } #end hash
} #end else

Now we display the contents of the $hash variable. To do this, we just call the $hash variable, and the contents of the hash table are returned to the script:

$hash
} #end Search-LogGatherStats

The entry point to the ScanFirewallLogGatherStats.ps1 script is almost the same to the entry point of yesterday's script. The only difference is the name of the function that was used to collect the information from the Windows Firewall log file. In our script today, that function is called Search-LogGatherStats. To remind yourself about the entry point to the script, see yesterday's "Hey, Scripting Guy!" article.

$log = "C:\Windows\pfirewall.log"
$IpAddress = $null
Search-LogGatherStats -log $log |
Out-File -filepath (New-TempFile | Tee-Object -variable TempFile) | 
Show-Output -tempfile $TempFile

This concludes our article on parsing firewall logs. It also brings to a conclusion our discussion of regular expressions. Join us tomorrow as we open the e-mail bag and answer questions that do not require such long answers. Because the answers are short, the topics are varied. Yep, tomorrow is Quick-Hits Friday. Until then, peace.

 

Ed Wilson and Craig Liebendorfer, Scripting Guys