Hey, Scripting Guy! Blog

Scripting Wife Comments on Beginner Event 5

The Scripting Guys — Thu, 23 May 2013 18:59:00 GMT

Summary: The Scripting Wife reveals her impressions of 2013 Scripting Games Beginner Event 2.

This event was tough—at least for me. You see, the thing is that I can read text files, and I can get listings of text files. And these log files are just text, after all. The trouble is the part about finding IP addresses. That had me stumped. I tried using the filter with Get-Content, but I could not make it work for me. So in the end, I had to ask the Scripting Guy for help.

Lots of luck there. But what he told me was just enough to get me moving in the right direction. He said I need to use a regular expression to find the IP address in the log files. He told me to search the Hey, Scripting Guy! Blog for regular expression and IP address. So I did.

I did not find exactly what I needed; but then again, I guess I did. What I was looking for was a Regex pattern that defines an IP address. I actually found the pattern in an old Scripting Guy blog post from 2007—written in VBScript script. It does not matter, because it is the pattern I needed. Once I had the pattern, I needed to find a cmdlet that would accept a pattern, so I used Get-Command like this:

Get-Command -ParameterName pattern

Then I needed to look up Help about the cmdlet I found, and figure out how to give it the Regex pattern. When I did that, I got my matches, and then I reduced the matches to unique ones. That did it. Hope this helps you. It is actually pretty cool, and it can be done as a one-liner.

2013 Scripting Games: Advanced Event 5

The Scripting Guys — Thu, 23 May 2013 07:03:00 GMT

Summary: Scripting Games Advanced Event 5: Dr. Scripto needs to create a tool to find client IP addresses from a bunch of IIS logs.

Microsoft Scripting Guy, Ed Wilson, is here. Today is the release of Event 5 for the 2013 Scripting Games. Remember that they are being run and hosted at www.powershell.org. You will need to go to that webpage to submit your entry.

The Logfile Labyrinth

Dr. Scripto finds himself in possession of a bunch of IIS log files. (It is much like the one at http://morelunches.com/files/powershell3/LogFiles.zip, if you need one to practice with.) He’s keeping all of the log files in C:\Reporting\LogFiles, and he’s left the log files with their default file names, which he’s given a .log file name extension. All of the files are for a single website, on a single web server.

He’d like you to write a tool that accepts a path, and then simply scans through each file in that path, generating a list of each unique client IP address that have been used to access the website. No IP address should appear more than once in your output, and you don’t need to sort the output in any way.

Your tool should optionally accept an IP address mask, such as “192.0.1.*” and only display IP addresses that match the specified pattern. If run without a pattern, display all IP addresses.

As described in IIS Log File Example, the client IP address is the first item in each log file entry. Regardless of the addresses found in the sample file, you should assume that any legal IP address may appear in the files Dr. Scripto needs to scan. Your command should scan all of the files in the folder (and the folder doesn’t contain any other kind of file) and produce a single set of results. If an IP address appears in multiple log files (it’s likely that will be the case), your final output should still only list that IP address one time.

I invite you to follow me on Twitter and Facebook. If you have any questions, send email to me at scripter@microsoft.com, or post your questions on the Official Scripting Guys Forum. See you tomorrow. Until then, peace.

Ed Wilson, Microsoft Scripting Guy

2013 Scripting Games: Beginner Event 5

The Scripting Guys — Thu, 23 May 2013 07:01:00 GMT

Summary: Scripting Games Beginner Event 5: Dr. Scripto needs to find client IP addresses from IIS logs.

The Logfile Labyrinth

He’d like you to simply scan through each file, and generate a list of each unique client IP address that’s been used to access the website. No IP address should appear more than once in your output, and you don’t need to sort the output in any way.

As described in IIS Log File Example, the client IP address is the first item in each log file entry. Regardless of the addresses found in the sample file, you should assume that any legal IP address may appear in the files that Dr. Scripto needs to scan. Your command should scan all of the files in the folder (and the folder doesn’t contain any other kind of file) and produce a single set of results. If an IP address appears in multiple log files (it’s likely that will be the case), your final output should still only list that IP address one time.

Dr. Scripto is getting a little tired, so he’s asked you to keep your command as short and concise as possible, although you’re welcome to use full command names and full parameter names to make things easy to read.

Ed Wilson, Microsoft Scripting Guy

PowerTip: Find Format Data Used by Current PowerShell Session

The Scripting Guys — Wed, 22 May 2013 18:59:00 GMT

Summary: Find the Windows PowerShell format data files that are used in your current Windows PowerShell session.

How can I find the format data files that are being used in my current Windows PowerShell session?

Use the Get-FormatData cmdlet in Windows PowerShell 3.0, and to browse this information, pipe the results to the More command:

Get-FormatData | more

Variable Substitution in a PowerShell Script Block

The Scripting Guys — Wed, 22 May 2013 07:01:00 GMT

Summary: Microsoft Scripting Guy, Ed Wilson, talks about performing variable substitution inside a Windows PowerShell script block.

Hey, Scripting Guy! I am trying to create a command. The command contains variables that I would like to assign prior to creating the command. However, when I get to the script block portion of my code, it does not do the variable substitution for the value the way an expanding string normally works. Can you help me?

—SW

Hello SW,

Microsoft Scripting Guy, Ed Wilson, is here. Last week at the first ever Northern Virginia PowerShell User Group, the Scripting Wife and I found a great little tea shop. I can’t help but thinking about the scone I got there—it was pretty good.

Expanding variable values

One of the really cool things about Windows PowerShell is the expanding strings—I absolutely love them. It surely beats having to do lots of string concatenation like I had to do back in the VBScript days. To illustrate this technique, I assign a string to the value of a variable—in this case, the $a variable. I look at the value contained in the variable. This code is shown here.

PS C:\> $a = "This is a string"

PS C:\> $a

This is a string

Now, by using the expanding string, I can see the value that is contained inside the $a variable. To suppress the variable expansion, I escape it with the grave accent character as shown here.

PS C:\> "The value of `$a is $a"

The value of $a is This is a string

The problem with a script block

If, on the other hand, I want to expand the value of a variable inside a script block, it does not work. This is shown in the code that follows.

PS C:\> $a = "This is a string"

PS C:\> $a

This is a string

PS C:\> $b = {"The value of `$a is $a"}

PS C:\> $b

"The value of `$a is $a"

Solving the problem with variable expansion in a script block

The solution to expanding a variable inside a script block is to do two things. First create the script block as an expanding string. This is shown here:

PS C:\> $a = "This is a string"

PS C:\> $a

This is a string

PS C:\> $b = "The value of `$a is $a"

PS C:\> $b

The value of $a is This is a string

Now, I use the static Create method from the [scriptblock] class. This will create a script block. To do this, I use the [scriptblock] class and then call the Create method while passing the string contained in the $b variable. This is shown here.

PS C:\> [scriptblock]::Create($b)

The value of $a is This is a string

I can confirm that it is in fact a script block by piping the results to the Get-Member cmdlet as shown here:

PS C:\> [scriptblock]::Create($b) | gm

TypeName: System.Management.Automation.ScriptBlock

Name MemberType Definition

---- ---------- ----------

CheckRestrictedLanguage Method void CheckRestrictedLanguage(System.Collection...

Equals Method bool Equals(System.Object obj)

GetHashCode Method int GetHashCode()

GetNewClosure Method scriptblock GetNewClosure()

GetObjectData Method void GetObjectData(System.Runtime.Serializatio...

GetPowerShell Method powershell GetPowerShell(Params System.Object[...

GetSteppablePipeline Method System.Management.Automation.SteppablePipeline...

GetType Method type GetType()

Invoke Method System.Collections.ObjectModel.Collection[psob...

InvokeReturnAsIs Method System.Object InvokeReturnAsIs(Params System.O...

ToString Method string ToString()

Ast Property System.Management.Automation.Language.Ast Ast ...

Attributes Property System.Collections.Generic.List[System.Attribu...

File Property string File {get;}

IsFilter Property bool IsFilter {get;set;}

Module Property psmoduleinfo Module {get;}

StartPosition Property System.Management.Automation.PSToken StartPosi...

Now, the cool thing about this is that I can also store the script block into another variable. This is shown here:

$sb = [scriptblock]::Create($b)

After I have stored the script block into the variable, I can also call any of the methods or properties of the script block. For example, here is the AST:

PS C:\> $sb.Ast

ParamBlock :

BeginBlock :

ProcessBlock :

EndBlock : The value of $a is This is a string

DynamicParamBlock :

ScriptRequirements :

Extent : The value of $a is This is a string

Parent :

It is not horribly exciting in this example, but for more complex code, it is definitely exciting stuff. We have a great Hey, Scripting Guy! Blog post written by Bartek Bielawski, which offers several good ideas for further exploration: Learn How IT Pros Can Use the PowerShell AST.

SW, that is all there is to using variable expansion and substitution in a Windows PowerShell script block. Join me tomorrow when I will talk about way cool Windows PowerShell stuff.

Ed Wilson, Microsoft Scripting Guy

PowerTip: Create a PowerShell Script Block on the Fly

The Scripting Guys — Tue, 21 May 2013 18:59:00 GMT

Summary: Create a Windows PowerShell script block on the fly.

How can I convert a string into a Windows PowerShell script block?

Use the static Create method from the [scriptblock] class:

PS C:\> [scriptblock]::Create("this is a string") | gm

TypeName: System.Management.Automation.ScriptBlock

Name MemberType Definition

---- ---------- ----------

CheckRestrictedLanguage Method void CheckRestrictedLanguage(System.Collection..

Equals Method bool Equals(System.Object obj)

GetHashCode Method int GetHashCode()

GetNewClosure Method scriptblock GetNewClosure()

GetObjectData Method void GetObjectData(System.Runtime.Serializatio..

GetPowerShell Method powershell GetPowerShell(Params System.Object[..

GetSteppablePipeline Method System.Management.Automation.SteppablePipeline..

GetType Method type GetType()

Invoke Method System.Collections.ObjectModel.Collection[psob..

InvokeReturnAsIs Method System.Object InvokeReturnAsIs(Params System.O..

ToString Method string ToString()

Ast Property System.Management.Automation.Language.Ast Ast ..

Attributes Property System.Collections.Generic.List[System.Attribu..

File Property string File {get;}

IsFilter Property bool IsFilter {get;set;}

Module Property psmoduleinfo Module {get;}

StartPosition Property System.Management.Automation.PSToken StartPosi..

PowerShell Hash Tables that Contain Keywords...hmm...

The Scripting Guys — Tue, 21 May 2013 07:01:00 GMT

Summary: Microsoft Scripting Guy, Ed Wilson, talks about dealing with Windows PowerShell hash tables that contain keywords. Dude!

Hey, Scripting Guy! I have a problem in that I want to create a hash table from an array. This is easy enough to do, but if the array contains keywords, then everything goes pear shaped. Any ideas?

—GP

Hello GP,

Microsoft Scripting Guy, Ed Wilson, is here. Things are certainly becoming exciting around here. The Scripting Wife and I have been busy planning for TechEd North America in New Orleans and TechEd Europe in Madrid. As it turns out, we will have a Scripting Guys booth in both New Orleans and Madrid. In addition, we are sharing our booth with the folks from PowerShell.org; so, that means that Windows PowerShell people like Don Jones will be there. I am also going to be working with the Windows PowerShell team on a couple of instructor lead labs that (to be quite honest) will absolutely knock your socks off —they are that cool. More details to follow.

Creating a hash table that contains keywords

Well GP, I have never actually run across your problem before. This is because I avoid using Windows PowerShell keywords. I refer to the Help topic, about_Language_Keywords, from time to time to refresh my mind when it comes to reserved keywords.

Note As an aside, I do the same thing for automatic variables. For more information, see about_Automatic_Variables in the Script Center.

Here is the easy way to bring up the about_Language_Keywords Help topic:

help -Category helpfile -Name *keyword*

So, this is all on the preventive side of things. To experiment with this, the first thing I do is create an array that contains a few keywords—in addition to other words. I then use the ForEach command to add the items in the array to a hash table. This technique is shown here:

PS C:\> $a = "key","keys","property","properties","red","blue","green"

PS C:\> $h = @{}

PS C:\> foreach ($wd in $a) {$h[$wd] = $true}

If I look at the hash table that is contained in the $h variable, I see the following:

PS C:\> $h

Name Value

---- -----

red True

property True

green True

keys True

properties True

blue True

key True

So far, so good. But suppose I want to look at the properties or the keys of my hash table. I try the following:

PS C:\> $h.Keys

True

PS C:\> $h.key

True

PS C:\> $h.property

True

PS C:\> $h.properties

True

So I am wondering what the properties really are for this hash table. I can find out by using the Get-Member command as shown here.

PS C:\> Get-Member -InputObject $h -MemberType Properties

TypeName: System.Collections.Hashtable

Name MemberType Definition

---- ---------- ----------

Count Property int Count {get;}

IsFixedSize Property bool IsFixedSize {get;}

IsReadOnly Property bool IsReadOnly {get;}

IsSynchronized Property bool IsSynchronized {get;}

Keys Property System.Collections.ICollection Keys {get;}

SyncRoot Property System.Object SyncRoot {get;}

Values Property System.Collections.ICollection Values {get;}

There really is a property called Keys, and I really cannot seem to access it. So what do I do?

I imagine that I will take a look at the base Windows PowerShell object. To do this, I use the PSBase property. All Windows PowerShell objects have a base object, but most of the time, it is not necessary to fool with it. Here, however, I suspect that I will be able to get past the keys keys kind of thing. Here is the base object:

PS C:\> $h.psbase

IsReadOnly : False

IsFixedSize : False

IsSynchronized : False

Keys : {red, property, green, keys...}

Values : {True, True, True, True...}

SyncRoot : System.Object

Count : 7

Now, I see there is a Keys property on the base object. So I attempt to access it. Here is how I do it:

PS C:\> $h.psbase.keys

red

property

green

keys

properties

blue

key

GP, that is all there is to using PSBase when dealing with hash tables. By the way, I tested this on Windows PowerShell 2.0 and on Windows PowerShell 3.0, so you should be safe to use this technique. Join me tomorrow when I will talk about more cool Windows PowerShell stuff.

Ed Wilson, Microsoft Scripting Guy

PowerTip: Create an XML Representation of a Registry Key with PowerShell

The Scripting Guys — Mon, 20 May 2013 18:59:00 GMT

Summary: Use Windows PowerShell to create an XML representation of a registry key.

How can I easily create an XML representation of a registry key by using Windows PowerShell?

Use the Get-ChildItem cmdllet and the registry provider to get the registry key, and pipe the results to the Export-CliXML cmdlet:

Get-ChildItem 'HKCU:\Software\Microsoft\Active Setup' -Recurse | Export-Clixml -Path c:\fso\active.xml

Use PowerShell to Copy Only Folders that Contain Files

The Scripting Guys — Mon, 20 May 2013 07:01:00 GMT

Summary: Microsoft Scripting Guy, Ed Wilson, talks about using Windows PowerShell to copy only folders that contain files.

Microsoft Scripting Guy, Ed Wilson, is here. In some respects, it seems like I have been using Windows PowerShell for a long time. In other respects, it seems like the journey begun only a short while ago. I mean, after writing five books about Windows PowerShell, it would seem that I know the product pretty well. And yet, I continue to amaze myself with the power of Windows PowerShell.

What am I talking about? Well, a few months ago, I created a whole bunch of folders based on dates in preparation for the Scripting Wife’s and my trip to Europe. Unfortunately, due to certain time issues, I did not always upload pictures every day into the folders. This caused some folders to not have pictures in them. Now when you are talking about 40 or so folders, it is annoying to click and not see anything. So I decided to delete folders that were not being used. At first I was going to do it by hand, but that was painful. I thought the code to clean up the empty folders might be too difficult to write, but in the end, it was not.

Delete empty folders

I opened the Windows PowerShell ISE, and created two variables to hold the source path and to hold the destination path. These two variables are shown here (note that you will not have these directories unless you have created them):

$path = 'C:\data\mypics\2012'

$destination = 'e:\data\mypics\archive\2012’

Now I use the Get-ChildItem cmdlet to collect all of my directories. I use the Directory switch that is available in Windows PowerShell 3.0 to do this. I send the directory objects down the pipeline. This is shown here.

Get-ChildItem $path -Directory |

Now I use the Foreach-Object cmdlet to work with each directory object. I call the EnumerateFiles method from the DirectoryInfo object to see if there are any files. I send the results to Measure-Object to count the files. I am sure this code could be cleaner, but it works:

Foreach-Object {if (($_.enumeratefiles() | measure).count -gt 0)

If there are files, I copy the folder (and files) to the destination as shown here:

Copy-Item -path $_.fullname -Destination $destination -Recurse}

That is it. It did not take me very long at all to create the script. And it saved a whole lot of clicking. Here is the competed CopyOnlyFoldersWithFiles script.

CopyOnlyFoldersWithFiles.ps1

$path = 'C:\data\mypics\2012'

$destination = 'e:\data\mypics\archive\2012’

Get-ChildItem $path -Directory |

Foreach-Object {if (($_.enumeratefiles() | measure).count -gt 0)

{

Copy-Item -path $_.fullname -Destination $destination -Recurse}

}

Join me tomorrow when I will talk about more cool Windows PowerShell stuff.

Ed Wilson, Microsoft Scripting Guy

PowerTip: Toggle Outline View in the Windows PowerShell ISE

The Scripting Guys — Sun, 19 May 2013 18:59:00 GMT

Toggle the Outline view in the Windows PowerShell ISE.

...(read more)