Automation–The New World of Tenant Provisioning with Windows Azure Pack (Part 3): Automated Deployment of the Identity Workload as a Tenant Admin

Automation–The New World of Tenant Provisioning with Windows Azure Pack (Part 3): Automated Deployment of the Identity Workload as a Tenant Admin

  • Comments 14
  • Likes

Aaaaaand we’re back!

Okay, I realize the title of this post is very similar to Part 2 of this blog series - and there is very good reason for that. Though they look similar, they are different. Different personas; different scripts; different context.

Once again, I will call on a little image assistance, to clear things up (the scope of this post is illustrated by the portion of the image with the red outline):

image

Note     Find Part 1: Intro & TOC here and Part 2: Automated Deployment of Tenant Network and Identity Workload (Isolated Tenant Virtual Network & Active Directory VM Role; from the Service Admin Persona) here.


And if that didn’t clear it up, let’s try expanding on the title for Part 3, (now with more sub-title!)

Automated Deployment of the Identity Workload as a Tenant Admin

(Active Directory VM Role; from the Tenant Admin Persona)

So, as you know, Part 2 of this blog series was all about deployment as the Service Administrator persona. Now, in Part 3, it is all about deployment as the Tenant Administrator.

I think it is time for another quick rehash (with image):

In the above image, we see my view of these two personas, as well as what we will be getting into for this post (the first two bullets in the box on the right).


The Options

How about another image?

This time, to visualize the high-level process for the Tenant Admin:

image

So, as you can see, the Tenant Admin has options.

Note     Automatically adding the Isolated Tenant Virtual Network (Isolated Software Defined Network (SDN)) is completely optional, as the Service Admin can simply recommend that the Tenant Admin manually create Virtual Networks from the Tenant Portal before deploying VM Roles.

Reminder     These options are the same for any VM Role Deployment, Active Directory happens to be the first one to be deployed, so it gets all the attention.


The Process

Remember that multi-step process I described in Part 2?

Well, it is basically the same.

So, to save you from going back and forth, here is the representative image for “The Process”:


What’s the difference?

Primarily, it is the Web Service Endpoint PORT and Authorization method.

  • As a Service Admin, you can leverage Bearer Token authorization and make WS calls against the Tenant API (non-Public), Port 30005
  • As a Tenant Admin, you leverage Certificates and make WS calls against the Public Tenant API, Port 30006

In fact, if you are familiar with Automating Windows Azure with PowerShell, the pre-requisite process has a very similar look and feel.

Care to venture a guess what we leverage to setup/use the Certificate?

The Windows Azure PowerShell Module (It is one of the options anyway.)

That’s right. The very same – and if you haven’t checked it out, or downloaded the latest version, you may have missed all the “new-ish” commands that contain *-WAPack*

Which is a great segue to…


The Pre-Requisites

  1. Valid Account and Subscription to a Plan within WAP (have the ability to login to the WAP Tenant Portal and provision Gallery Items (VM Roles)):
    image
  2. The latest Windows Azure PowerShell module (contains the latest *-WAPack* Commands):
    image
  3. Ability to Execute Windows Azure PowerShell Commands
  4. Creation and Usage of a Management Certificate (derived from WAP Publish Settings File – see the Pre-Requisite Setup steps below)

The Pre-Requisite Setup

  1. Download and Install the Windows Azure PowerShell Commands
  2. Get the Published Settings File for your Subscription – Navigate to your version of the following URL: https://tenant.portal.com:30081/publishsettings
    image

    This will prompt to download the file locally – My recommendation is to download to the machine where the PowerShell Scripts will be executed.

    Note
         If the target portal leverages ADFS, do not include the port (30081) in the address while attempting to retrieve the .publishsettings file. At least this is what I experienced between the two environments we have, one without ADFS (required port in URL), and one with ADFS (port not used in URL).


    (Example Path: C:\LocalPath\SubscriptionName-MM-d-YYYY-credentials.publishsettings)
    image
  3. Open PowerShell and ensure the Windows Azure PowerShell module is loaded
  4. Execute the following command (modify the command text based on the name and where the publishsettings file is located):

    Import-WAPackPublishSettingsFile C:\WAP\Collaboration Workloads (Tenant Deployed)-3-11-2014-credentials.publishsettings

Note    These steps not only import the certificate into the Management Certificates section in the Tenant Portal:

image

But ALSO add it to the Certificate Store for the Current User:

image

Resulting in the following, when Get-WAPackSubscription is executed from PowerShell:

image

Reminder    The security best practice for the publishsettings file is to store it temporarily, and then delete it after the settings have been imported. A malicious user gaining access to the publishsettings file can edit, create, and delete your Windows Azure Pack resources.

So, there you go - All the theory and pre-requisites you can handle, right?

The good news…With these pre-requisites out of the way, we can now get on to the actual deployment script!


Example Active Directory Gallery Item VM Role Deployment Script

(as a Tenant Admin against the Public WAP API)

001
002
003
004
005
006
007
008
009
010
011
012
013
014
015
016
017
018
019
020
021
022
023
024
025
026
027
028
029
030
031
032
033
034
035
036
037
038
039
040
041
042
043
044
045
046
047
048
049
050
051
052
053
054
055
056
057
058
059
060
061
062
063
064
065
066
067
068
069
070
071
072
073
074
075
076
077
078
079
080
081
082
083
084
085
086
087
088
089
090
091
092
093
094
095
096
097
098
099
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
#region GetWAPConnectionData

# Get WAP Subscription Information

$WAPSubscription = Get-WAPackSubscription

# Set Subscription
$SubscriptionID = $WAPSubscription.SubscriptionId

# Get Management Certificate Info
$CertThumb = $WAPSubscription.Certificate.Thumbprint
$CertPath = "Cert:\CurrentUser\My\{0}" -f 
$CertThumb
$Cert
 = Get-Item $CertPath

# Set Tenant Portal Address
$TenantPortalAddress = $WAPSubscription.ServiceEndpoint.Host

# Set Port
$Port = $WAPSubscription.ServiceEndpoint.Port

#endregion GetWAPConnectionData

#region SetVariables

# Set Gallery Item Name and Version for Match and Deploy

$GalleryItemName = "DomainController"
$GIVersion = "1.0.0.0"

# Set Common Gallery Item Parameters
$UserID = "tenant@company.com"
$VMRoleNetwork = "Tenant Network ({0})" -f 
$UserID
$CloudServiceName
 = "CloudService-4-{0}" -f 
$SubscriptionID
$VMRoleName
 = "ActiveDirectory"
$VMRoleNamePattern = "DC##"
$VMRoleSize = "ExtraSmall"
$VMRoleTZ = "Pacific Standard Time"
$OSDisk = "Windows Server 2012 Datacenter"
$OSDiskVersion = "1.0.0.0"
$Password = "Password"

#endregion SetVariables

#region GetResDef

# Get Gallery Item Reference

$GIReferenceUri = "https://{0}:{1}/{2}/Gallery/GalleryItems/$/MicrosoftCompute.VMRoleGalleryItem?api-version=2013-03" -f $TenantPortalAddress,$Port,
$SubscriptionID
$GIReferenceData
 = [xml](Invoke-WebRequest -Certificate $Cert -Uri $GIReferenceUri | Select-Object -ExpandProperty Content)
$GalleryItemREF = $GIReferenceData.feed.entry.content.properties.resourcedefinitionUrl | ? {$_ -match $GalleryItemName}

# Get Gallery Item Resource Definition
$GIResDEFUri = "https://{0}:{1}/{2}/{3}/?api-version=2013-03" -f $TenantPortalAddress,$Port,$SubscriptionID,
$GalleryItemREF
$GIResourceDEFJSON
 = Invoke-WebRequest -Certificate $Cert -Uri $GIResDEFUri | Select-Object -ExpandProperty Content

#Convert ResDef JSON to Dictionary
[System.Reflection.Assembly]::LoadWithPartialName("System.Web.Extensions") | Out-Null
$JSSerializer = New-Object System.Web.Script.Serialization.JavaScriptSerializer
$ResDef = $JSSerializer.DeserializeObject($GIResourceDEFJSON)

#endregion GetResDef

#region SetResDefConfig

# Create Gallery Item Parameter Hashtable (for Common Data)

$GIParamList = @{
    VMRoleVMSize = $VMRoleSize
    VMRoleOSVirtualHardDiskImage = "{0}:{1}" -f $OSDisk,$OSDiskVersion
    VMRoleAdminCredential = "administrator:{0}" -f $Password
    VMRoleTimeZone = $VMRoleTZ
    VMRoleComputerNamePattern = $VMRoleNamePattern
    VMRoleNetworkRef = $VMRoleNetwork
    }

# Add to Gallery Item Parameter Hashtable (for GI Specific Data)
if ($GalleryItemName -eq "DomainController")
{
    $GIParamList += @{DomainControllerWindows2012DomainDNSName = $UserID.Split("@")[1]} 
    $GIParamList += @{DomainControllerWindows2012DomainNETBIOSName = ($UserID.Split("@")[1]).Split(".")[0]}
    $GIParamList += @{DomainControllerWindows2012SafeModeAdminPassword = $Password}
}
   
# Convert Gallery Item Parameter Hashtable To JSON
$ResDefConfigJSON = ConvertTo-Json $GIParamList

#Add ResDefConfig JSON to Dictionary
$ResDefConfig = New-Object 'System.Collections.Generic.Dictionary[String,Object]'
$ResDefConfig.Add("Version",$GIVersion)
$ResDefConfig.Add("ParameterValues",$ResDefConfigJSON)

#endregion SetResDefConfig

#region GenerateGIPayloadJSON

# Set Gallery Item Payload Variables

$GISubstate = 
$null
$GILabel
 = 
$VMRoleName
$GIName
 = 
$VMRoleName
$GIProvisioningState
 = 
$null
$GIInstanceView
 = $null

# Set Gallery Item Payload Info
$GIPayload = @{
    "InstanceView" = $GIInstanceView
    "Substate" = $GISubstate
    "Name" = $GIName
    "Label" = $GILabel
    "ProvisioningState" = $GIProvisioningState
    "ResourceConfiguration" = $ResDefConfig
    "ResourceDefinition" = $ResDef
    }

# Convert Gallery Item Payload Info To JSON
$GIPayloadJSON = ConvertTo-Json $GIPayload -Depth 7

#endregion GenerateGIPayloadJSON

#region GetOrSetCloudService

# Get Cloud Services

$CloudServicesUri = "https://{0}:{1}/{2}/CloudServices?api-version=2013-03" -f $TenantPortalAddress,$Port,
$SubscriptionID
$CloudServicesData
 = [xml](Invoke-WebRequest -Uri $CloudServicesUri -Certificate $Cert | Select-Object -ExpandProperty Content)
$CloudService = $CloudServicesData.feed.entry.content.properties.Name | ? {$_ -match $CloudServiceName}
if (!$CloudService) {
    # Set Cloud Service Configuration
    $CloudServiceConfig = @{
        "Name" = $CloudServiceName
        "Label" = $CloudServiceName
        }

    # Convert Cloud Service Configuration To JSON
    $CloudServiceConfigJSON = ConvertTo-Json $CloudServiceConfig

    $CloudServicesData = [xml](Invoke-WebRequest -Uri $CloudServicesUri -Certificate $Cert -Method Post -Body $CloudServiceConfigJSON -ContentType "application/json")
    $CloudService = $CloudServicesData.entry.content.properties.Name | ? {$_ -match $CloudServiceName}
}

#endregion GetOrSetCloudService

#region DeployGIVMRole

# Set Gallery Item VM Role Deploy URI

$GIDeployUri = "https://{0}:{1}/{2}/CloudServices/{3}/Resources/MicrosoftCompute/VMRoles/?api-version=2013-03" -f $TenantPortalAddress,$Port,$SubscriptionID,$CloudService

# Deploy Gallery Item VM Role
$VMRoleDeployed = Invoke-WebRequest -Uri $GIDeployUri -Certificate $Cert -Method Post -Body $GIPayloadJSON -ContentType "application/json"
$VMRoleDeployed

#endregion DeployGIVMRole

Note(s)     I have several notes about the above script. So I will list them here:

  • This is just an example script.
  • It has been tested against our Demo/Test/Dev environment multiple times.
  • It absolutely requires the pre-requisites discussed above. And while there are alternatives to getting the required variable data based on the Get-WAPackSubscription command, I have found this to be the most efficient/dynamic method.
  • The $TenantPortalAddress Variable may need to  be set to a specific string, rather than being extracted from the information available from the Get-WAPackSubscription command, specifically if the public portal address is different than the FQDN of WAP Admin Server.
  • If you are getting errors during deployment like, “Disk with Name (Windows Server 2012 Datacenter) and Version (1.0.0.0) not found while translating Resource Definition for cloud resource.” it is likely that the $OSDisk and/or $OSDiskVersion Variables have incorrect data. The Tenant API does not care so much for the actual OS Disk name, instead, it appears to require the Family Name of the OS Disk.
  • I chose to leave all the Variable settings within the script. These could very easily be presented as Script Parameters, and fed into the script to make it a bit more generic.
  • Just like the Service Admin example script, there is a portion of the Resource Definition Configuration (ResDefConfig) that is tied directly to the ResDef/ResDefExt for a given Gallery Item VM Role. When generating the Gallery Item Parameter Hashtable, I have separated the GI specific data from the common GI data. In most cases (especially for the Gallery Item VM Roles produced by my team), the only portion of the script that has to change per VM Role is this GI specific data (and, of course any specific Variable data).
  • This script (for the Tenant Admin) should look nearly identical to the script from Part 2 (for the Service Admin). This is by design, as I wanted to keep as may synergies in play as possible. Remember, there are only subtle differences (Ports and Auth).
  • While the steps in the Gallery Item VM Role deployment process will likely remain the same, the actual script could be improved in various ways: Addition of Script Parameters, Separated into Functions, Transformed into a Set of Cmdlets, etc. If anyone takes these improvements on, I will be happy to reference / endorse the published work here in this blog.
  • Finally, as you can see, I have broken the script up into “regions”, each of which builds on the last, to eventually complete all the data collection / command execution for the final Invoke-WebRequest POST to deploy the Gallery Item VM Role. Here is an image illustrating the seven regions:
    image

All that said, if everything goes right during the execution of this example script, you should see something very similar to this:

image


What’s Missing?

Reminiscent of one of the final sections in Part 2, this is where we check to see if I have “missed anything” – of course, strictly based on the in scope deliverables called out at the top.

So, let’s check those checkmarks…

image

Looks good to me!


But wait, WHY would I want to do this?

Great question! Here are some use cases (from both the Service Admin and Tenant Admin personas):

  • Use Case 1: As a Tenant – Simple avoidance of manual clicking to deploy Gallery Item VM Roles
  • Use Case 2: As a Tenant – Develop scripts to fully deploy a set of multiple concurrent (and/or dependent) Gallery Item VM Role Deployments (with scripts like this, you have complete control over the “what” and “when”)
  • Use Case 3: As a Service Provider (or Enterprise acting like one) – Create a custom set of cmdlets encapsulating the parameters and logic into easily consumable/executable commands
  • Use Case 4: As a Service Provider (or Enterprise acting like one) – Enabling your Tenants/End Users to automate their own Gallery Item VM Role deployments (external to any SMA efforts on the Service Admin side)

Note     These are just some of the use cases I could come up with off the top of my head. I am sure you have many more scenarios in mind.


What’s Next?

Well, at this point, based on Parts 2 and 3 of this blog series, you really have everything necessary to deploy not only the Gallery Item VM Role for Active Directory, but any Gallery Item VM Role you have built (or pulled down off of WebPI (or our Blog)).

But hey, I am not going to leave you hanging. The very next blog post is all about Tenant Workloads, where I will be providing the necessary script updates (to what you have seen so far) to deploy the Gallery Item VM Roles for Lync, SharePoint, and Exchange (for both personas).

AND A VIDEO (an actual 8-Minute-Demo Video)!

AND A TECHNET GALLERY CONTRIBUTION (for all the scripts)!

How exciting….


Oh, and have you seen this blog post yet?

Windows Azure Pack–Gallery Item VM Role–References for Creation, Configuration, and Automation

If not, check it out.

And while it does cross-reference back to this post, it covers the entire Gallery Item VM Role Lifecycle (well the important bits, anyway).


Blog Series Table of Contents

  1. Part 1: Intro & TOC
  2. Part 2: Automated Deployment of Tenant Network and Identity Workload (Isolated Tenant Virtual Network & Active Directory VM Role; from the Service Admin Persona)
  3. Part 3: Automated Deployment of the Identity Workload as a Tenant Admin (Active Directory VM Role; from the Tenant Admin Persona)
  4. Part 4: Automated Deployment of Tenant Workloads (Lync, SharePoint, and Exchange) (Lync, SharePoint, and Exchange VM Roles; from both Service Admin and Tenant Admin Personas)
  5. Part 5: Working with the SQL Server resource provider, and the ITIL dilemma (by Bruno Saille)
  6. Part 6: TBD (We hope to have something around: Value Added Services/Offerings and Ongoing Automated Maintenance/Operations)

TechNet Gallery Contribution and Download

The download (Windows Azure Pack Tenant Provisioning Automation Toolkit.zip) includes (14) files.

For more information please refer to Part 4 of this blog series (near the bottom of the post), or in the description of the TechNet Gallery Contribution itself.

Download the Windows Azure Pack Tenant Provisioning Automation Toolkit from TechNet Gallery here:

BC-DLButtonDark


Thanks for checking out my latest blog series! For more information, tips/tricks, and example solutions for Automation within System Center, Windows Azure Pack, Windows Azure, etc., be sure to check out the other blog posts from Building Clouds in the Automation Track!

enJOY!


Your comment has been posted.   Close
Thank you, your comment requires moderation so it may take a while to appear.   Close
Leave a Comment
  • it's a very good reason for us....thanks.

  • Hey Charles Great Job done on putting this together!!!

  • Great stuff Charles!! I have been working with the VM Role script you build and it helps out big time. I made a small change to $GalleryItemREF variable to prevent it from failing if there are multiple versions of the same Role. // $GalleryItemREF = $GIReferenceData.feed.entry.content.properties.resourcedefinitionUrl | ? {$_ -match $GalleryItemName -and $_ -match $GIVersion} // Also can you think of an easy :) way to let the script wait for the deployment to complete. That way it is easier to make dependencies in a larger script. The New-WAPackVM already does this, but this script will complete once the deployment starts. Once again. Awesome!!

  • This is truly genius. I am having an issue however. When I run the script I get an error at line 75, char 68 saying that invoke-webrequest cannot bind the parameter 'Uri' (cannot convert value). Other than the necessary changes I am using the script verbatim. Any thoughts?

  • @Dale - First, thank you for your interest and kind words. Second - Hmm. It is hard to say, I would start by identifying which Invoke-WebRequest command the URI is failing on... Then output the URI, so you can see what "it thinks" the value is (before it is leveraged in the Invoke-WebRequest). This is really the only way to start to troubleshoot this type of error. Let me know what you find.

  • Actually, I ended up solving that issue, and an issue with SSL/TLS connectivity (we were using self signed certs). Then, finally, I get an error on the second last line:

    $VMRoleDeployed = Invoke-WebRequest -Uri $GIDeployUri -Certificate $Cert -Method Post -Body $GIPayloadJSON -ContentType "application/json"

    It says that the Name parameter is invalid because it is over 100 characters or contains illegal characters. Lol, what Name parameter? There are many. I know that this is basically taking the JSON script built from all the information gathering and concatenation and prepping it to run a deploy. I have gone over the JSON script that the powershell pumps out, and I cannot find what it is complaining about. I am extremely green though, and this stuff is way over my head. I very much appreciate you replying to my previous post btw.

  • Turns out there is an issue with SMA. Somewhere there is a disconnect between a service (runbook worker maybe) and the databases. Its not an issue with the script, its an issue with runbook functionality as they all stall in the queue. Nothing ever picks them up and moves them into ISE. So, we are looking into that now. Not sure if you're curious how this turns out or not, but again thank you for your excellent blogs!

  • @Dale - You are welcome. And yes, I am interested to know that you were able to get things up and running...it is always interesting to me to see the various experiences people have getting the same (or similar) thing working. With this many moving parts, and environmental differences, it is always interesting.

    BTW - What did you mean by this, "Nothing ever picks them up and moves them into ISE." I am just curious, I am not sure what context this specific statement is based...

    Either way, let me know how things progress.

  • So, I implemented your subscription-create-dispatcher script (for admin-side automatic deploy of network and domain controller) and the problem we are having is that it launches every time we create a user because they are automatically assigned a plan. I noticed in your teched video that you went into the tenant portal and added a subscription there, and that's what triggers the automation runbook. When I do this what I end up with is a whole lot of subscription-create-dispatcher runbooks queued. I have taken to telling them to output anything at all so that I know they are actually running instead of being instantly queued, and there is never any output. Any thoughts as to why they are just queuing?

  • @Dale - The event/triggers within WAP/SPF are universal for that Object/Action combination. This means that for the Subscription/Create combination, any time a user is assigned to a plan, the event/trigger fires. If it is associated with a Runbook, the Runbook will be invoked.

    I am not sure why they would be queuing. It is possible that the raw number of event/triggers could be overwhelming the system. It would take further debugging.

    To that end, I would suggest adding the example runbook from this post: http://blogs.technet.com/b/privatecloud/archive/2014/05/13/reusing-event-data-when-developing-sma-runbooks.aspx and linking it to the Subscription/Create combination event/trigger. It serves two purposes, it is small/fast and captures a bunch of information. If this is what you are already doing, and it is still queuing, I would see about restarting SMA, and/or adding more SMA Runbook Workers.

    I hope this helps.

  • You rock sir. I will do this. The system shouldn't be getting overwhelmed as its really just a test deploy of Azure Pack. Its not doing anything yet except for me trying to set up a "press here to get a sandbox for your devs" button. Lol. That said, I do just have the one runbook worker for now. I get what you are saying, that the runbook will be invoked because of the SPF flag regardless of whether the subscription is added in the tenant or admin side, at least I think that's what you're saying. I tried attaching a simple runbook to this event before as a test and got no output, but I might have done something wrong because I am so green at this. Its really a miracle that we have this monstrosity in place at all, and totally HA as well. I will try your suggestion though. When you say "restart SMA" are you referring to the rb services etc? You are kind to answer my comments with helpful advice and I appreciate it very much.

  • @Dale - Happy to help!

    Yep, you caught my intention correctly - Any time "a Subscription is Created" (the combination of User + Plan) and that event is active, its associated Runbook is invoked. This is actually the entire reason for the concept of a "Dispatcher" Runbook - to be invoked and dispatch accordingly. Sometimes, it is just to figure out what kind of event triggered the invocation, and end with no further action, other times it is to actually initiate the entire downstream process.

    RE: Restarting SMA - Yes, restart the service, possibly do an IISRESET for the SMA Web Service, if they are on the same machine, reboot. Sometimes things just get hung up.

    Deploying more SMA Runbook Servers will help with load overall, and remember, if you deploy a Runbook Server, keep the following in mind: http://technet.microsoft.com/en-us/library/dn530618(v=sc.20).aspx

  • So I tried the eventlogger runbook sample you suggested and the same thing happens. It instantly queues. I checked on orchestrator and the runbooks are definitely being processed, but when I checked the runbook worker I see this error under SMA in the eventviewer: RunbookService storage is unavailable - System.ArgumentException: Format of the initialization string does not conform to specification starting at index 132. I can't find any reference to runbookservice storage, how to test the configuration or anything else for that matter. Any thoughts?

  • @Dale - I have not seen this error before. I have a mail out asking the product team for guidance, I provide an update as soon as I have more information.