Make Windows PowerShell Your Web Client

Make Windows PowerShell Your Web Client

  • Comments 2
  • Likes

 

Summary: Interact with websites from Windows PowerShell with this how-to post that wraps .NET Framework classes.

 

Hey, Scripting Guy! QuestionHey, Scripting Guy! I would like to be able to download files and things from the Internet via Windows PowerShell. Is there an easy way to do this? I have not been able to find a Windows PowerShell cmdlet, and I would like to share whatever I come up with other users.

-- BR

 

Hey, Scripting Guy! AnswerHello BR,

Microsoft Scripting Guy Ed Wilson here. James Brundage began talking about wrapping .NET Framework classes to work with the web in yesterday’s Hey, Scripting Guy! post. That post is a great introduction to wrapping .NET Framework classes to work with the Internet. Let’s let him complete the discussion.

James Brundage is the founder of Start-Automating (http://www.start-automating.com), a company dedicated to saving people time and money by helping them automate. Start-Automating offers Windows PowerShell training and custom Windows PowerShell and .NET Framework development. James formerly worked on the Windows PowerShell team at Microsoft and is the author of the PowerShellPack, a collection of Windows PowerShell scripts to enable building user interfaces, interact with RSS feeds, retrieve system information, and more. You can follow James on Twitter @JamesBru.

 

A Series on Splatting (and Stuff)

Part 5: Splatting .NET Framework conclusion

In previous posts, we have covered how to build commands that wrap other commands with splatting and building bridges between two commands with splatting.  Yesterday we looked at improving a Windows PowerShell script that invoked a .NET command. Today, we will learn how to apply those skills to make some usable Windows PowerShell scripts to replace some hard-to-use .NET.

Great! Let’s take the New-WebClient function that was developed yesterday, because we can think of the next command we will write as bridging New-WebClient and a quick function to invoke the method. 

Our begin block will also contain this New-WebClient function and a helper function called Invoke-Method that lets us invoke .NET methods in a Windows PowerShell syntax.

The $BeginBlock variable will contain the two helper commands, and nothing else.  If these helper commands were going to be used by a larger group of commands, I’d put them into the same module as the command I’m working on. However, you can also embed a private command this way. The $BeginBlock and its two functions are seen here.

$BeginBlock = {
    function New-WebClient
    {
        [CmdletBinding(DefaultParameterSetName="Easy")]
        param(
        [string]$BaseAddress,
        [Parameter(ParameterSetName='Hard')]
        [Net.ICredentials]$Credentials,
        [Parameter(ParameterSetName='Easy')]
        [Management.Automation.PSCredential]$Credential,
        [Parameter(ParameterSetName='Hard')]
        [Net.WebHeaderCollection]$Headers,
        [Parameter(ParameterSetName='Easy')]
        [Hashtable]$Header,
        [Parameter(ParameterSetName='Hard')]
        [Collections.Specialized.NameValueCollection]$QueryString,
        [Parameter(ParameterSetName='Easy')]
        [Hashtable]$Query,
        [Parameter(ParameterSetName='Hard')]
        [bool]$UseDefaultCredentials,
        [Parameter(ParameterSetName='Easy')]
        [Switch]$Anonymous    
        )
       
        process {
            if ($psCmdlet.ParameterSetName -eq "Hard") {
                New-Object Net.Webclient -Property $psBoundParameters
            } else {
                $newWebClientParameters = @{} + $psBoundParameters

                $newWebClientParameters.UseDefaultCredentials = -not $Anonymous
                $null = $newWebClientParameters.Remove("Anonymous")

                if ($newWebClientParameters.Credential) {
                    $newWebClientParameters.Credentials = $newWebClientParameters.Credential.GetNetworkCredential()
                    $null = $newWebClientParameters.Remove("Credential")
                }

                if ($newWebClientParameters.Header) {
                    $newWebClientParameters.Headers = New-Object Net.WebHeadercollection
                    foreach ($headerPair in $newWebClientParameters.Header.GetEnumerator()) {
                        $null = $newWebClientParameters.Headers.Add($headerPair.Key, $headerPair.Value)
                    }
                    $null = $newWebClientParameters.Remove("Header")
                }

                if ($newWebClientParameters.Query) {
                    $newWebClientParameters.QueryString = New-Object Collections.Specialized.NameValueCollection
                    foreach ($QueryPair in $newWebClientParameters.Query.GetEnumerator()) {
                        $null = $newWebClientParameters.QueryString.Add($QueryPair.Key, $QueryPair.Value)
                    }
                    $null = $newWebClientParameters.Remove("Query")
                }
                New-WebClient @newWebclientParameters
            }
        }   
    }
       
    function Invoke-Method
    {
        param($Object, [string]$Method, $Parameters)
       
        process {
            $realMethod =$object."${method}"
            if ($realMethod) {
                $realMethod.Invoke($Parameters)
            }       
        }
    }     
}

The parameter block is next.  Remember how we said that most of what you do in .NET can be expressed as creating a class, setting lots of properties, and running lots of methods? The begin block took care of the first two.  The parameter block and process block will handle the rest.

You can think of any Windows PowerShell command as one or more things that you can do.  Each parameter set may be a small variation on the thing.  Many commands actually contain many related operations in one, with each parameter set representing a different operation.

When wrapping something from .NET in Windows PowerShell, it is easy to go down the list and turn methods into parameter sets.

Here are the .NET methods we want to wrap:

  • # string DownloadString(System.Uri address)
  • # byte[] DownloadData(System.Uri address)
  • # void DownloadFile(System.Uri address, string file)
  • # byte[] UploadData(System.Uri address, string method, byte[] data)
  • # byte[] UploadFile(System.Uri address, string method, string file)
  • # byte[] UploadString(System.Uri address, string method, string string)
  • # byte[] UploadValues(System.Uri address, string method, NameValueCollection values)

Turning these into parameters is tedious, but easy.  Here is a list of guidelines that describe how we'll do it.

  • Each method becomes a parameter set.
  • Each parameter becomes a mandatory parameter for that parameter set.
  • Any parameter that is in multiple methods will be in multiple parameter sets (it is easy, just add another [Parameter()] attribute).
  • Make sure that you give common parameters a position (i.e. address and method).
  • If there is a case in which the only way the methods are different is by the type of data that they return, make an -AsTYPE parameter to let the user decide (i.e. DownloadString and DownloadData).
  • Select the most common operation as the default parameter set.
  • Add comments for user help.
  • Add the ValueFromPipelineByPropertyName attribute to each parameter (unless you have a good reason not to, it opens up some interesting possibilities if you do).

If you are writing one of these from scratch in the Windows PowerShell Integrated Scripting Environment (ISE), you might want to try using the Add-Parameter function from IsePack (part of PowerShellPack) because it simplifies some repetitive typing.  The Add-Parameter function has several switches that let you add mandatory parameters, select a parameter set, create the parameter from the pipeline or from the pipeline by property name.

Here is the parameter block for Send-WebRequest, which combines the ”Easy” parameter set of New-WebClient and the parameters that match each newly added operation.

$paramBlock = {
    <#
    .Synopsis
        Sends web requests to download or upload information from the web
    .Description
        Uses the System.Net.WebClient class to download or upload information from the web
    .Example
        Send-WebRequest "http://www.start-automating.com/"
    #>
    [CmdletBinding(DefaultParameterSetName="DownloadString")]
    param(
    # The Uri to download or upload
    [Parameter(ParameterSetName='DownloadData',
        Position=0,
        Mandatory=$true,
        ValueFromPipelineByPropertyName=$true)]
    [Parameter(ParameterSetName='DownloadString',
        Position=0,
        Mandatory=$true,
        ValueFromPipelineByPropertyName=$true)]
    [Parameter(ParameterSetName='DownloadFile',
        Position=0,
        Mandatory=$true,
        ValueFromPipelineByPropertyName=$true)]
    [Parameter(ParameterSetName='UploadData',
        Position=0,
        Mandatory=$true,
        ValueFromPipelineByPropertyName=$true)]
    [Parameter(ParameterSetName='UploadFile',
        Position=0,
        Mandatory=$true,
        ValueFromPipelineByPropertyName=$true)]   
    [Parameter(ParameterSetName='UploadString',
        Position=0,
        Mandatory=$true,
        ValueFromPipelineByPropertyName=$true)]   
    [Parameter(ParameterSetName='UploadValues',
        Position=0,
        Mandatory=$true,
        ValueFromPipelineByPropertyName=$true)]   
    [Uri]   
    $Address,
   
    # The method used to send an upload request (i.e. Get or Post)
    [Parameter(ParameterSetName='UploadData',Position=1,Mandatory=$true,ValueFromPipelineByPropertyName=$true)]   
    [Parameter(ParameterSetName='UploadFile',Position=1,Mandatory=$true,ValueFromPipelineByPropertyName=$true)]   
    [Parameter(ParameterSetName='UploadString',Position=1,Mandatory=$true,ValueFromPipelineByPropertyName=$true)]
    [Parameter(ParameterSetName='UploadValues',Position=1,Mandatory=$true,ValueFromPipelineByPropertyName=$true)]       
    [string]$Method = "Get",

    # The data to upload   
    [Parameter(ParameterSetName='UploadData',Mandatory=$true,ValueFromPipelineByPropertyName=$true)]   
    [Byte[]]$UploadData,
   
    # The file to upload
    [Parameter(ParameterSetName='UploadFile',Mandatory=$true,ValueFromPipelineByPropertyName=$true)]   
    [string]$UploadFile,
   
    # The string to upload
    [Parameter(ParameterSetName='UploadString',Mandatory=$true,ValueFromPipelineByPropertyName=$true)]    
    [string]$UploadString,
                         
    # The string to upload
    [Parameter(ParameterSetName='UploadValues',Mandatory=$true,ValueFromPipelineByPropertyName=$true)]   
    [Hashtable]$UploadValue,
                                  
    # If set, the information will be returned as Data
    [Parameter(ParameterSetName='DownloadData',Mandatory=$true,ValueFromPipelineByPropertyName=$true)]
    [Switch]$ASData,
   
    # If set, the infromation will be downloaded to a file
    [Parameter(ParameterSetName='DownloadFile',Mandatory=$true,ValueFromPipelineByPropertyName=$true)]
    [String]
    $ToFile,
   
    # The Base Address.   The Uri will be appended to this address
    [Parameter(ValueFromPipelineByPropertyName=$true)]
    [string]$BaseAddress,

    # The Credential used to send the web request
    [Parameter(ValueFromPipelineByPropertyName=$true)]
    [Management.Automation.PSCredential]$Credential,

    # A collection of header information   
    [Parameter(ValueFrom      PipelineByPropertyName=$true)]
    [Hashtable]$Header,

    # A set of data for GET or POST query
    [Parameter(ValueFromPipelineByPropertyName=$true)]
    [Hashtable]$Query,

    # If this is set, then the connection will be anonymous.  Otherwise, the default credentials will be used.
    [Parameter(ValueFromPipelineByPropertyName=$true)]
    [Switch]$Anonymous
    )
}

It’s a piece of cake right? (Well maybe the kind that Marie Antoinette would serve).

The process block is less tedious, but follows a similar pattern.  The start of the command becomes a bit of common code (creating the web request) followed by either a switch statement (if you prefer readability) or lots of else-ifs (if you prefer speed) that set up the input for the method, followed by a calling the method.

Because the parameter set name is the method name, and the webclient is always the object, the only thing that has to be in the switch statement is the part that determines the arguments.  Here’s what our process block looks like.

$processBlock = {
    # Create the web client
    $newWebClientParameters = @{}   
    foreach ($p in "BaseAddress", "Credential", "Header", "Query", "Anonymous") {
        if ($psBoundParameters[$p]) {
            $newWebClientParameters[$p] = $psBoundParameters[$p]
        }
    }   
    $webClient = New-WebClient @newWebClientParameters
   
    $parameters = @()
    switch ($psCmdlet.ParameterSetName)
    {
        DownloadString {
            $parameters = $Address
        }
        DownloadData {
            $parameters = $Address
        }
        DownloadFile {
            $unresolvedFile = $psCmdlet.SessionState.Path.GetUnresolvedProviderPathFromPSPath($ToFile)
            $parameters = $Address, $unresolvedFile
        }       
        UploadData {
            $parameters = $Address, $Method, $UploadData
        }
        UploadFile {
            $resolvedFile = $psCmdlet.SessionState.Path.GetResolvedProviderPathFromPSPath($uploadFile)
            $parameters =  $Address, $Method, $resolvedFile
        }
        UploadString {
            $parameters = $Address, $Method, $UploadString
        }
        UploadValues {
            $nameValueCollection = New-Object Collections.Specialized.NameValueCollection
            foreach ($pair in $value.GetEnumerator()) {                
                $null = $nameValueCollection.Add($Pair.Key, $Pair.Value)
            }
            $parameters = $Address, $Method, $nameValueCollection
        }
    }
    $invokeMethodParameters = @{
        Object = $webClient
        Method = $psCmdlet.ParameterSetName
        Parameters = $parameters
    }
    Invoke-Method @invokeMethodParameters
}

Now let's create our function from the three different parts we just finished making. The Send-WebRequest function is seen here.

$functionDefinition = "function Send-WebRequest {
    $paramBlock
   
    begin {
        $beginBlock
    }
    process {
        $processBlock
    }
}
"

$functionDefinition | Set-Content .\Send-WebRequest.ps1
. .\Send-WebRequest.ps1

Let’s try downloading the Hey, Scripting Guy blog:
$rss = Send-WebRequest "http://blogs.technet.com/b/heyscriptingguy/rss.aspx"
$rss = [xml]$rss
$rss.rss.channel.item | Select-Object Title, PubDate

It even has help! Cool!

Get-Help Send-WebRequest –Full

The complete Send-WebRequest function can be downloaded from the Scripting Guys Script Repository. You can paste the function into the Windows PowerShell ISE, run the script, and then use the code seen here to download the RSS feed for the Hey, Scripting Guy! Blog.

$rss = Send-WebRequest "http://blogs.technet.com/b/heyscriptingguy/rss.aspx"
$rss = [xml]$rss
$rss.rss.channel.item | Select-Object Title, PubDate

When you run this code, the output shown in the following image appears.

 

If you save the Send-WebRequest function to a Windows PowerShell script file, you can dot-source it into your new script. This is seen in the UseSendWebRequestFunction.ps1 script.

UseSendWebRequestFunction.ps1

. C:\fso\Send-WebRequestFunction.ps1
$rss = Send-WebRequest "http://blogs.technet.com/b/heyscriptingguy/rss.aspx"
$rss = [xml]$rss
$rss.rss.channel.item | Select-Object Title, PubDate

BR, that is all there is to using Windows PowerShell to wrap .NET Framework commands. Guest blogger week has ended. Thank you James for all this information and scripts.  Join us tomorrow for the Weekend Scripter.

We invite you to follow us on Twitter or Facebook. If you have any questions, send email to us at scripter@microsoft.com or post them on the Official Scripting Guys Forum.. See you tomorrow. Until then, peace.

 

Ed Wilson and Craig Liebendorfer, Scripting Guys

Your comment has been posted.   Close
Thank you, your comment requires moderation so it may take a while to appear.   Close
Leave a Comment
  • Hello,

    thank you for the nice script.

    Could you please tell me how i can send a post request?

    Is that right?

    Send-WebRequest -Address "example.com/api_update.asp" -Method "Post" -Query @{"value1" = "test"}

    Thank You

    Max

  • Just a few correction to use the POST method (ie UpdateValues) :

    1/In the param block replace

    [Hashtable]$UploadValue,

    by

    [Hashtable]$UploadValue,

    2/In the process block replace

    foreach ($pair in $value.GetEnumerator()) {

    by

    foreach ($pair in $UploadValues.GetEnumerator()) {

    Now to use it to post values :

    Send-WebRequest -Address "example.com/api_update.asp" -method "POST" -UploadValues @{"value1" = "test"}