...building hybrid clouds that can support any device from anywhere
I’ve been getting questions about the overall architecture of MAT and then specifically how the PowerShell functions work together. Of course, it’s all right there in the scripts, but it would probably take you a while to sort through the 2,000 or so lines of code. So in the spirit of trying to make the MAT easy to use and assessable, let me explain these two items.
The MAT can be run on a single server, or using lots of servers. Here’s a list of the components and a neat diagram that puts them all together. If you are only running one system (the ‘Control’ server) it needs to have the following:
If you want to run more than one conversion server, you add ‘Helpers’. These don’t require much:
If you want to understand more about the general process that the three phases perform have a look at the first MAT post
Since the MAT uses the same set of scripts for both the Control Server and the Helper servers, it’s good to understand how the two behave differently and in what sections of the script. Now these scripts are all v2 of PowerShell and so, the term ‘workflow’ is a little misleading. No, this isn’t a workflow in the PowerShell sense, just the regular sense. Words are funny like that, especially in a living language. I miss Latin.
Anyway what follows is the workflow for the actual conversion, this doesn’t cover the preparation, VM listing, DB updating, etc.
On the Control Server the process starts by checking to see if the Variable.xml contains entries for remotes (i.e. if you added them). If it does, it will create a job using the task scheduler of the remote system and immediately start that job. Once all the remotes have been serviced, the Convert function will be triggered.
The Convert function will call CheckSemaphore. CheckSemaphore, oddly enough, will check the server’s semaphore. It does this by calling SQL to see how many conversions are listed for that machine (Queuelength). As long as the value of the semaphore is greater than the Queuelength the process continues. If it is not, the process will loop until one of the current conversions finishes and thereby freeing up a ‘slot’.
Next the process checks for unassigned VMs from the database. If it finds one, it assigns the VM (using GetNextVM) to itself and thereby increments the Queuelength by one. VMs are taken using a TOP 1 SQL call but they are ordered by ‘Priority’ which is established when (or if) you re-order the VMlist.txt. If no more VMs are found in the table, we are done and we exit.
At this point we have the data we need to convert and the script will call the Convert.ps1 using the Start-Process cmdlet. The point of starting another process is to let one Convert.ps1 run for every MVMC.exe that runs. Ideally this would have been done with PowerShell v3 workflow but some of the vagaries of the MVMC.exe made this approach more practical. This approach also allows you to extend the Convert.ps1 without messing with the overall flow.
Once Convert.ps1 is called, the script starts back over at the Convert function (sorry, when I wrote it I didn’t consider the challenge in explaining it in a blog.) and the process continues anew.
While the MAT has looped and moved one, the Convert.ps1 will call Logging.ps1 (assuming you use default settings). One Logging.ps1 is called for every MVMC.exe (both tracking it by PID) and it’s job is to watch the contents of mvmc.log (or mvmc1.log, or mvmc2.log…) and ‘translate’ the messages there into useful information and pass that information to the main log (and the database in case you are watching using the Show-Status functions). Again, ideally we would capture the stdout messages from the MVMC.exe, but it doesn’t actually make any, so we are forced to parse the log.
When Logging.ps1 sees that the PID for its particular MVMC.exe exits, it will spin down and exit too. Shortly thereafter Convert.ps1 will exit too. It sees that the PID exists and then it ‘waits’ for Logging.ps1 before closing. It does this by waiting X seconds, where X is the value of LogMonitorDelay multiplied by the value of the SleepMulitplier (both set in Convert.ps1). The default delay is 39 seconds (30 x 1.3).
BONUS: If you want to extend the functionality of the MAT for any post conversion functions, the right place to do this would be at the end of Convert.ps1. It can also be done between the GetNextVM and Convert functions but this isn’t as desirable as it would slow your conversion flow down (again, sorry for the two Converts, one’s a script, one is a function – I’ll probably change that in the next version. Stultus verba!). Adding it to Convert.ps1 leaves it on the “set it and forget it” side of the fence and will allow any additional tasks to run in parallel.
The helper servers are started using the command “ .\main.ps1 Convert” which bypasses the remote logic and jumps directly into the Convert function where it checks the Queuelength and gets underway just like above.
I hope this helps clarify the logical flow that these scripts take. As always I’m interested in any improvements that you might conceive of, so please feel free to share those with me and other here on the blog.
Keep calm and migrate,
Go Social with Building Clouds! Building Clouds blog Private Cloud Architecture Facebook page Private Cloud Architecture Twitter account Building Clouds Twitter account Private Cloud Architecture LinkedIn Group Cloud TechNet forums TechNet Cloud and Datacenter Solutions Site Cloud and Datacenter Solutions on the TechNet Wiki