• Remember to vote for your AlwaysOn Availability Groups

    While visiting a customer and helping them set up AlwaysOn Availability Groups we came across a surprising "feature". On my laptop I had build a demo lab consisting of one domain controller, three Windows 2008R2 Core member servers with SQL 2012 SP1 installed and a full Windows 2008 R2 installation with SQL Server management studio installed. I had setup the cluster, enabled AlwaysOn and created an empty database to create an availability group. I tested the availability group by manually failing over using management studio and this worked fine.

    But that's in the demo world. Usually we have all kind of conditions that cause our demo's not to work with our customers. In this case the customer had set up an availability group as well and the failover was working when initiated from management studio. But stopping the SQL Server service caused the Availability Group to stay in a resolving state and never failover. I tried this on my machine and it failed over perfectly. The only difference between my setup and the customer was that I was using a three node cluster and they were using a two node + file share cluster. To match the customers setup I removed the third replica from the availability group and evicted this node from the cluster. This left me with a two node cluster. What is important to remember is that this is not a recommended setup. In this setup you will never have majority and the cluster will go offline when one of the nodes goes offline. This basically defeats the purpose of a cluster. But to satisfy my curiosity I stopped the SQL Server service. To my surprise the availability group failed over to the other node.

    A quick recap:

    • Three node cluster with an availability group: Stopping the service fails over the availability group
    • Two node cluster (not recommended) with an availability group: Stopping the service fails over the availability group
    • Two node with file share witness: Stopping the service leaves the secondary in a resolving state and the listener remains offline

    This is a known issue and if you read the signs along the road you would have noticed this. The reason I missed it at first is that in the earlier builds the link in the warning did not work. Below is a screenshot of the warning you will get when you perform a failover in the third scenario.

     

    There is a KB article that describes this behavior (http://support.microsoft.com/kb/2761129) and this references the same KB article (http://support.microsoft.com/kb/2494036) that is mentioned in the link in the warning message. In short it turns out that the nodes do not have a vote. You can see this by querying sys.dm_hadr_cluster_members or clicking the View Cluster Quorum Information link in the AlwaysOn dashboard. This will show NULL or Not Available. This means the nodes do not have a vote and we need the vote to determine if there is quorum.

    After applying the hotfix we get the desired behavior and the dmv and dashboard show 1 indicating each node has a vote. Stopping the service results in a failover leaving the new primary in a Not Synchronizing state. This is the expected state but leaves the availability group at risk. Another replica must come online as soon as possible to start synchronizing. In this state the transaction log will not be able to truncate and reuse the space within the log file.

  • From 0 to SQL Demo lab in a few easy steps – Part 4

    I have not been able to follow up my post for a long time. Work has to be done as well. The good thing is that it gives me a lot of new ideas for blog posts and as a (late) resolution for the new year I will write more blog posts

    But back to part 4 of building a SQL Demo lab. There are a few things you have to consider building an entire lab. Some are easy and some require creativity. First let me list the steps you would take to create an AlwaysOn demo manually.

    • Build a Domain Controller
    • Provision some users to the domain
    • Install 2 or more member servers
    • Install SQL Server 2012
    • Configure SQL Server
    • Build a cluster on top of the member servers
    • Enable AlwaysOn

    And that should leave you with a ready-to-go demo machine.

    In the previous part we have done all the individual steps but the complexity lies in the sequence. We cannot add a member server if there is no domain controller and we cannot enable AlwaysOn if there is no Windows Cluster build on top of the member servers. And what would be the use of automating things if we have to execute three or four scripts manually.

    Key Value Pairs to the rescue

    Hyper-V virtual machines have this concept of exchanging data through a dedicated registry portion. There is some good information out there but the best I found so far was this blog post by Taylor Brown; http://blogs.msdn.com/b/taylorb/archive/2012/12/05/customizing-the-key-value-pair-kvp-integration-component.aspx. By leveraging this concept I can write some PowerShell code that will read this data in a loop and exit when a certain value has been saved to the registry key. This will enable me to create one script that will call different scripts to install the various servers but in the proper sequence. In my VM I will set a status. Using a function on the host I read the value and depending on the status I will proceed to the next step.

    There had to be challenges

    Of course there had to be. What I wanted was to alter the unattend.xml files. I could have taken the easy road and use placeholders like @computernamehere@ and just do a textual replace. But I choose to use XML to navigate through the unattend.xml and change the proper value. To give you an example of some funny PowerShell behavior. Imagine this piece of XML:

    <root>
    <data type="database">
    <name>master</name>
    </data>
    <data type="job">
    <id>1</id>
    </data>
    </root>

    If you navigate through this piece of XML in PowerShell, for example $xml.root.data.name it will return master. Great! A new world of possibilities arises. Things get ugly if you want to change the value. You get an error telling you the value cannot be set. It took some time but it turns out to be related to the fact the data element is there twice but the name element only once. When you navigate through the XML PowerShell uses late binding to figure this out but when you write to it, it wants to know exact. I solved this in my script by using Xpath queries to get to the correct element.

    On my SkyDrive you can find the scripts I used: http://sdrv.ms/Y0oUfv

    The main script I used is Create-Lab which will create a Domain Controller and configure it. Installs 1 management member server and 3 SQL Servers. When the installation of the third server is finished a cluster is build and Availability Groups are enabled. The instances are restarted using WMI. This proved to be the most reliable method. If you want a shorter example of what the script does check out Create-Lab-Demo. This creates a lab with SQL 2008 R2 and SQL 2012 installed on it.

    Important! In the Create… scripts there are names and paths I used on my machine. Evaluate each one and set this to your environment. Also in my script I refer to two *.wim files for the base installation. Due to legal and other reasons I cannot provide you these files but you can use a *.wim file from the Windows setup media or create your one using the method I described in one of my earlier posts. Also the iso files I used for SQL Server are of course not included and you need to provide your one. Finally, the keys included are setup keys the installer uses when you would choose Skip Installation during the Windows Setup process. Check one of my earlier posts to see where to retrieve these keys.

    Feel free to contact me for any further information or improvements. And remember, do not use these scripts in production environments. They were built to create quick demo labs.

  • From 0 to SQL Demo lab in a few easy steps – Part 3

    In part 1 showed how you could install Windows on a virtual disk and use that disk to create a new virtual machine. We improved on part 2 to show how to set the administrator password or the computer name during installation to minimize user input.

    In part 3 we will take another step. It's nice to have a clean Windows installation with the desired network configuration or computer name but what if I need a domain controller setup with some default settings or have a SQL server joined to an existing domain.

    The magic is in the unattend file as well. Besides setting the computer name or network we can also autologon for a specified number of times. We can also specify commands we want to run when the logon takes place.

    First the autologon. To get this going add the following XML right before the UserAccounts segment but after the OOBE.

    <AutoLogon>
    <Password>
    <Value>p@ssw0rd</Value>
    <PlainText>true</PlainText>
    </Password>
    <Username>Administrator</Username>
    <LogonCount>1</LogonCount>
    <Enabled>true</Enabled>
    <Domain>CONTOSO</Domain>
    </AutoLogon>

    Again, not the best method for a production environment but it fits my purpose. If you use WAIK to generate the XML file the password will be encrypted.

    We can now specify some commands we want to execute during the logon process. It is important to know how to install what you want using commands only. For active directory this is easy. You run DCPROMO and specify an unattend file. In this file you specify the domain name, level and dns settings. Some examples of the unattend file are on technet.

    To copy it over I add another Copy-Item to my PowerShell script to copy my ad installation file to the destination system. Finally you must add the command to execute:

    <FirstLogonCommands>
    <SynchronousCommand wcm:action="add">
    <CommandLine>dcpromo.exe /unattend:c:\ad.txt</CommandLine>
    <Description>Installation of AD</Description>
    <Order>1</Order>
    </SynchronousCommand>
    </FirstLogonCommands>

    If all goes well you will have a shiny new domain controller. Now for step two. Adding a member server to the domain. There are several ways to add a machine to the domain. I have tried a few but my knowledge of active directory is not at the same level as SQL so I ran in some issues using the autojoin. Eventually what worked for me was just specifying the user to join the domain. Again, using the autojoin is much safer than putting passwords in text files. This is what I added after the network settings in the unattend file.

    <component name="Microsoft-Windows-UnattendedJoin" processorArchitecture="amd64" publicKeyToken="31bf3856ad364e35" language="neutral" versionScope="nonSxS" xmlns:wcm="http://schemas.microsoft.com/WMIConfig/2002/State" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
    <Identification>
    <UnsecureJoin>false</UnsecureJoin>
    <Credentials>
    <Domain>CONTOSO</Domain>
    <Username>bob</Username>
    <Password>p@ssw0rd</Password>
    </Credentials>
    <JoinDomain>contoso.com</JoinDomain>
    <MachineObjectOU>OU=Servers,DC=contoso,DC=com</MachineObjectOU>
    <DebugJoin>true</DebugJoin>
    </Identification>
    </component>

    The next thing was installing SQL server. There are several ways to do this. One is to use the sysprep feature and include SQL in your base image and then finalize the installation. This proved to be limited for me. This only works for some components of SQL and also not for all versions of SQL. I wanted a solution that would work regardless of the version. My solution was to add the DVD during the VM creation and run a cmdline setup of SQL Server. This gives me much more control and flexibility. To install SQL server I created a cmd file with a very basic setup:

    d:\Setup.exe /q /ACTION=Install /FEATURES=SQLEngine,ADV_SSMS /INSTANCEID=SQL2012 /SQLSVCACCOUNT="NT Service\MSSQL$SQL2012" /ADDCURRENTUSERASSQLADMIN="True" /AGTSVCACCOUNT="NT Service\SQLAgent$SQL2012" /AGTSVCSTARTUPTYPE=Automatic /BROWSERSVCSTARTUPTYPE=Automatic /TCPENABLED=1 /IACCEPTSQLSERVERLICENSETERMS /INDICATEPROGRESS /UPDATEENABLED=0

    And using the logon command I showed earlier I started this installation after the first logon. I also added a LOGOFF command to the cmd file just to return to the logon screen.

    This works perfectly. I did notice the installation of SQL took a long time because of some .NET requirements. I have included these in my base image and things got a lot faster afterwards. Check your installation logs to see where most time is spend and see if you can speed things up by including these in the base image.

    In the final part I will provide you with the definite PowerShell scripts and command files to recreate what I showed here.

  • From 0 to SQL Demo lab in a few easy steps – Part 2

    This post is part of a series of posts about buidling a lab environment using some of the new PowerShell cmdlets and features in Windows 8.

    In the previous post I explained how to create a virtual disk, apply a standard Windows Server 2008 image to it and attach the disk to a new VM. If you followed my steps you should have noticed that it was not a fully automated experience. During the installation I was asked to provide the regional settings, set or skip a license key, accept the EULA and provide an administrator password. And when I finally was logged in I still had to do a lot of configuration like setting an ip-address and renaming the machine. In this post we will get these steps out of the way as well.

    The first step is to do some primary settings. In my case I enable PowerShell and Remote Desktop. This is also a good time to download updates and .NET frameworks you might need later. Try to keep things as basic as possible. Only set the ip-address if that's needed. We will change it later on.

    Once the machine is to your liking it's time to sysprep it. Sysprep is located in the sysprep folder inside the system32 folder. Sysprep removes all personal information from the machines and makes it possible to create an image for redeployment. Start sysprep and choose OOBE Experience, Shutdown and make sure the box at Generalize is checked or better run it from the commandline: sysprep /generalize /oobe /shutdown. Make sure you specify the generalize option. This will create a new machine identifier for every machine. If you miss that one you could get into weird problems.

    After sysprep is done it shuts down the machine. Now it's time to create an image that we can use as a base for deployement. You only have to do this once for every base image you want to create. I'm using a Windows 2008 and Windows 2008 Core template. Using DISM you can create an image (WIM) file. First you need to mount the image using Mount-VHD <pathtovhd>. Now capture the image with: DISM.exe /Capture-Image /ImageFile:D:\vm\base.wim /CaptureDir:G:\ /name:Base-Image. Replace the paths and name with your own. After some time a file is created. Make sure to dismount the vhd using Dismount-VHD. This concludes step 1 of this task. Next up is to automate the installation. There is an easy way and a hard way. The easy way is to use the Windows Automation Installation Kit. This tool has an editor to create unattended files. Another option is to create one with notepad. Of course you can also copy from others. Here is an example of the unattend.xml I'm using to create a domain controller. Read through it, I will explain the values in a moment.

    <?xml version="1.0" encoding="utf-8"?>
    <unattend xmlns="urn:schemas-microsoft-com:unattend">
    <settings pass="specialize">
    <component name="Microsoft-Windows-Shell-Setup" processorArchitecture="amd64" publicKeyToken="31bf3856ad364e35" language="neutral" versionScope="nonSxS" xmlns:wcm="http://schemas.microsoft.com/WMIConfig/2002/State" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">\
    <ComputerName>DC-01</ComputerName>
    <TimeZone>UTC</TimeZone>
    <RegisteredOrganization>Contoso Corp.</RegisteredOrganization>
    <RegisteredOwner>Contoso Corp.</RegisteredOwner>
    <ProductKey>7P8GH-FV2FF-8FDCR-YK49D-D7P97</ProductKey>
    </component>
    <component name="Microsoft-Windows-DNS-Client" processorArchitecture="amd64" publicKeyToken="31bf3856ad364e35" language="neutral" versionScope="NonSxS" xmlns:wcm="http://schemas.microsoft.com/WMIConfig/2002/State" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
    <Interfaces>
    <Interface wcm:action="add">
    <DNSServerSearchOrder>
    <IpAddress wcm:action="add" wcm:keyValue="1">127.0.0.1</IpAddress>
    </DNSServerSearchOrder>
    <Identifier>Local Area Connection</Identifier>
    </Interface>
    </Interfaces>
    </component>
    <component name="Microsoft-Windows-TCPIP" processorArchitecture="amd64" publicKeyToken="31bf3856ad364e35" language="neutral" versionScope="nonSxS" xmlns:wcm="http://schemas.microsoft.com/WMIConfig/2002/State" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
    <Interfaces>
    <Interface wcm:action="add">
    <Ipv4Settings>
    <DhcpEnabled>false</DhcpEnabled>
    <Metric>10</Metric>
    <RouterDiscoveryEnabled>false</RouterDiscoveryEnabled>
    </Ipv4Settings>
    <Identifier>Local Area Connection</Identifier>
    <UnicastIpAddresses>
    <IpAddress wcm:action="add" wcm:keyValue="1">192.168.200.1/24</IpAddress>
    </UnicastIpAddresses>
    <Routes>
    <Route wcm:action="add">
    <Identifier>1</Identifier>
    <Metric>10</Metric>
    <NextHopAddress>192.168.200.1</NextHopAddress>
    <Prefix>0.0.0.0/0</Prefix>
    </Route>
    </Routes>
    </Interface>
    </Interfaces>
    </component>
    </settings>
    <settings pass="oobeSystem">
    <component name="Microsoft-Windows-International-Core" processorArchitecture="amd64" publicKeyToken="31bf3856ad364e35" language="neutral" versionScope="nonSxS" xmlns:wcm="http://schemas.microsoft.com/WMIConfig/2002/State" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
    <InputLocale>132105</InputLocale>
    <SystemLocale>en-US</SystemLocale>
    <UILanguage>en-US</UILanguage>
    <UserLocale>en-US</UserLocale>
    </component>
    <component name="Microsoft-Windows-Shell-Setup" processorArchitecture="amd64" publicKeyToken="31bf3856ad364e35" language="neutral" versionScope="nonSxS" xmlns:wcm="http://schemas.microsoft.com/WMIConfig/2002/State" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
    <OOBE>
    <HideEULAPage>true</HideEULAPage>
    </OOBE>
    <UserAccounts>
    <AdministratorPassword>
    <Value>p@ssw0rd</Value>
    <PlainText>true</PlainText>
    </AdministratorPassword>
    </UserAccounts>
    </component>
    </settings>
    <settings pass="windowsPE">
    <component name="Microsoft-Windows-Setup" processorArchitecture="amd64" publicKeyToken="31bf3856ad364e35" language="neutral" versionScope="nonSxS" xmlns:wcm="http://schemas.microsoft.com/WMIConfig/2002/State" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
    <UserData>
    <AcceptEula>true</AcceptEula>
    </UserData>
    </component>
    </settings>
    </unattend>

    There are several sections in the xml. One section is the pass phase. This tells the setup routing at what point to use the settings. Some settings are only valid in one phase while other can be in multiple. The Windows Automation Kit knows and will only choose the right ones.

    In the first part of the file the computer name is set along with the time zone and other details. The product key is not a real key but a temporary one. It's actually the same key that's entered when you click skip during the setup process. You can find this key inside a file in the Windows installation dvd. So, it's not a real key, just one that will get the setup process automated.

    In the second part the network is set. In this example the ip 192.168.200.1 is used with netmask 255.255.255.0. The DNS is set to 127.0.0.1 because this machine will take the role of domain controller with DNS installed. The gateway is set to the 192.168.200.1 as well.

    The last part is where the locale settings are set. The values can be retrieved from this table. Also the EULA is accepted and the administrator password is set. If you use the Windows Automation Kit the password will be encrypted. As a best practice you should not include the password in plain text.

    So far for preparation, now it's show time. The trick is to get the unattend.xml we created located at the rootdrive of the machine. Using PowerShell and the commands from the previous post this task is easy. As you might recall I mounted the disk to apply the image and set the bootrecords. Run through the steps from the first post up until the point where the drive is dismounted. With the disk still mounted you can copy whatever file you want to the drive, including an unattend. If you add this PowerShell one-liner before you dismount the disk the file will be copied: Copy-Item source destination –force. This will copy the file from A to B. Now dismount the disk and you are done.

    Once the autounattend is copied you can start the VM. After a few minutes and with no user interaction you should be able to log into the machine.

    This brings us one step closer to a complete unattended installation of a lab environment. In the next post I will explain how to perform post install actions like installing and configuring AD and setting up SQL Server.

  • From 0 to SQL Demo lab in a few easy steps – Part 1

    As a Premier Field Engineer I get into a different challenge every week. These challenges can be anything; performance issues, architecture review or migration scenarios. If I'm at a customer site it is not always possible (or recommended) to test my solution on the production systems. And if it's a possible bug I want to reproduce it so it can be investigated. For this you will need a lab environment. But building a lab environment can be a time consuming task. Most of the times I need different SQL versions, clustering, AlwaysOn or another Microsoft product relying on SQL server. And when I'm ready to present and prove my solution I don't want any data from other customers to be visible. For these reasons I always build new and fresh environments. If it's a performance issue I can get away with just a single VM but if it is a more complex scenario I might have to install a domain controller and a series of other machines as well.

    I used to run Windows Server 2008 R2 on my laptop as my primary OS. Basically because I needed Hyper-V. But with Windows 8 I can run Hyper-V on a client OS and as a bonus I get a lot more PowerShell cmdlets to automate recurring tasks. Do you see where I'm going? There is some great software available to automate the deployment of VM's. You can use Windows Deployment Services or System Center Virtual Machine Manager. But I wanted an out-of-the-box solution that didn't require any additional tooling other than Windows 8 with Hyper-V enabled. And there is.

    This post will be part of a series of posts because getting a complete AlwaysOn cluster running is a lot of preparation and understanding all the steps will make it easier to customize this to your purpose. Basically I will break it down in a few steps:

    • Part 1: Installing Windows on a virtual disk and boot it in Hyper-V
    • Part 2: Automating and customizing the installation using sysprep and unattend
    • Part 3: Running post deployment tasks like AD configuration or SQL Server installation
    • Part 4: Putting it together

    Installing Windows on a virtual disk and boot it in Hyper-V

    Before you can start to automate anything with PowerShell you will need the steps to do it. Imagine you would create a new VM? What steps do you need to take?

    1. Create the VM
    2. Insert the DVD
    3. Install Windows

    If we are going to do this in PowerShell you will need to reorder some of the items. In PowerShell we will create the disk first, install Windows on it and create the VM in the final step.

    Create a vhd(x) with PowerShell

    There are several ways to create a virtual disk. You can use DISKPART but that can be a bit of a drag and is harder to automate, or you can use PowerShell. In Windows 8 you can use some new and cool cmdlets to create a virtual disk. The next script will create a vhd and format it. I will explain them after the script.

    $vhdpath = "D:\vm\dc-01.vhdx"
    $vhd = New-VHD -Path $vhdpath -SizeBytes 30GB

    Mount-DiskImage -ImagePath $vhdpath
    $disk = Get-DiskImage -ImagePath $vhdpath | Get-Disk

    Initialize-Disk -Number $disk.Number -PartitionStyle MBR
    $volume = New-Partition -DiskNumber $disk.Number -UseMaximumSize -AssignDriveLetter -IsActive | Format-Volume -Confirm:$false
    $drive = [string]$volume.DriveLetter + ":\"

    We start with setting a variable to make things easier later on. The variable $vhdpath hold the path to our virtual disk. The first step is to create the disk using New-VHD with the specified size, in my case 30GB is enough. The disk will be dynamically by default but you can create a fixed or differencing disk as well. Refer to the documentation for more options: http://technet.microsoft.com/en-us/library/hh848503.

    The following step is to mount the image using Mount-DiskImage specifying the path. When the disk is mounted you can format it using PowerShell instead of DISKPART. Formatting is a three-step action; first initialize the disk, partition it and finally format it. This is not different from DISKPART. Once it's formatted we can get a handle to the drive which we will need later on.

    With a freshly formatted drive we are ready to install Windows. For those of you that are not familiar with deployment or images; we can deploy Windows directly to a disk. No need to boot into a DVD. When you install Windows an image file (*.wim) is applied to the destination. The proper boot records are written and you are off to go. Prior to Windows 8 you would need a tool called ImageX to capture and apply images to a destination. Being part of the Windows Automation Deployment Toolkit this was another download and dependency and does not fit in my 'only use what's default' policy. But with Windows 8 this is not a problem. The command line tool DISM is extended to capture and apply images as well. Being part of the operating system it can be used for my purpose. Check out the next piece of PowerShell.

    Install the DVD

    $image = "F:\sources\install.wim"
    $dismArgs = @("/Apply-Image", "/ImageFile:$image", "/index:4", "/ApplyDir:$drive")
    Start-Process -FilePath "DISM.exe" -ArgumentList $dismArgs -NoNewWindow -Wait -PassThru

    There is no DISM cmdlet so we will resort to invoking the process. The source is set to my DVD drive with the Windows Server 2008 R2 DVD loaded. The index is set to 4 because I'm installing the Enterprise Core edition. You need to supply an index if the image contains multiple installations. I will not elaborate much on this because in the next post I will show another method. If you want to find out the index number of the installation use DISM /Get-WimInfo /wimfile:<pathtowimfile>. Finally we apply this to the drive we obtained after the formatting.

    When that's finished the proper boot records need to be written to the disk. The tool is bcdboot and we pass the drive and boot type as parameter.

    $bcdargs = @((Join-Path $drive "Windows"), "/s $drive", "/f BIOS")
    Start-Process -FilePath "bcdboot.exe" -ArgumentList $bcdargs -NoNewWindow -Wait -PassThru

     

    Create the VM

    And basically that's it. All that is left is to create the VM, start it and open it. And as you might have guessed, Windows 8 comes loaded with all you need cmdlets.

    $vmname = "DEMO-01"
    Dismount-DiskImage -ImagePath $vhdpath

    $vm = New-VM -name $vmname -MemoryStartupBytes 512MB -SwitchName "Hyper-V Internal Network" -VHDPath $vhdpath
    Set-VMMemory $vm -DynamicMemoryEnabled $true -MinimumBytes 512MB -MaximumBytes 1GB
    Start-VM $vm
    vmconnect localhost -G $vm.VMId

     

    First another variable is loaded with the name of the VM. Then the disk is dismounted and a new VM is created. It's connected to my Hyper-V switch named "Hyper-V Internal Network". You will need to adjust this to your environment. In the second step dynamic memory is configured. I use this a lot. Using dynamic memory enables me to have 6 VM's running with 8GB internal memory and still not run into any issues. The final two steps are starting the vm and connecting to it using vmconnect. The latter can be discarded of course. Also notice I'm using the identifier to connect to the VM. During my testing I found out it is possible to create multiple VM's with the same name. Connecting to the identifier makes sure you connect to the right one.

    After a few minutes you should have a shiny Windows installation awaiting you to create a new administrator password. In the next post I will go into customizing the installation, sysprepping it and automating the installation process.