In case you missed the announcement last week at the Microsoft Management Summit in Las Vegas, the beta for VMM VNext (now called VMM 2008) is publicly available - you can download the software here: http://www.microsoft.com/systemcenter/scvmm/default.mspx
The beta is nearly feature complete meaning that in one form or another, we have implemented all of the new functionality including Hyper-V support and VMware support. Of course, good ol' Virtual Server is still fully supported as well. It is of course a beta so it's still a little rough around the edges so you can expect the final version to have better overall performance, reliability and scale but the beta should get you pretty good mileage in the mean time. I was able to demo VMM 2008 in front of 4000 IT Professionals last week and was interrupted several times for spontaneous applause. Assuming that wasn't just our team, people liked what they saw :-)
While the beta has nearly all of the features baked in, there are a few things that we're still working on for the final release later this year so stay tuned and I'll run some of them by you in future posts. Until then, download the beta and don't be shy, give us honest and direct feedback.
Rakesh
Not necessarily a glamorous topic I know, but absolutely essential as part of any deployment.
Contrary to popular belief (or what other vendors might have you believe) you don't need to re-engineer your backup infrastructure or license new technology if you're using virtualization. One of the benefits of virtualization with Windows as the host OS is that we have 20+ years of partners who have worked with us to ensure you get a well integrated backup and recovery solution. That's not to say that virtualization doesn't provide new opportunities for efficiencies, it most certainly does, but our philosophy is that you should be able to back up physical and virtual machines with a single tool (one you probably already have) while still taking advantage of the new benefits.
One of the things that virtualization makes easier is what is commonly called "bare metal recovery". This is the recovery situation you'd find yourself in if you wanted to completely rebuild a server after hardware failure or a disaster. With physical servers this can be tricky since the server that you backed up might not be identical to the server that you're restoring to. In addition, unless you're using an imaging tool, recovery typically involves laying down a fresh copy of the OS before you can even begin to get your application and data back which can kill your SLA. With VMs, you have the opportunity to back up the entire virtual machine (at the VHD level) so that bare metal recovery simply involves copying your VHDs to any host server. There's a not so subtle catch here though - how do you ensure that your VHDs are in a "consistent" state so that recovery will actually work? You could shut down the VMs before backup or save state but that's not particularly appealing for obvious reasons. You could just grab the VHDs while the VM is running and hope that the recovery works....which is equally unappealing. What you really need is a way to back up the VMs at the host (VHD) level but ensure that that the data within the VHDs is consistent and recoverable without downtime.
The Windows platform provides just such a mechanism and it's called the Volume Shadow Copy Service or VSS. VSS allows for application-coordinated backups using snapshot technology. Every one of our server applications and subsystems (SQL, Exchange, Sharepoint, registry, DNS etc.) comes with a VSS plug-in known as a "VSS Writer". When a backup system asks for a snapshot of a server, the VSS infrastructure goes through all of the VSS writers on a system and asked them to get their on-disk data into a consistent state (flush buffers, finish or rollback mid-flight transactions etc.) so that backup applications can grab the files. There's a much more orchestration than that involved but for the purposes of this discussion, the simplification I've made is sufficient. We actually use the same infrastructure on our desktop OS versions starting with XP....if you're really bored, open a command prompt and type "vssadmin list writers" and you'll get back a list of VSS writers installed on that system. The reason we have application vendors build the writers is because only the application knows how to get itself into a consistent state and every application is different.
Starting with Virtual Server R2 SP1 and also available in Hyper-V, we've included a VSS writer for virtual machine backup. This has allowed products like System Center Data Protection Manager (DPM) and products from our partners to install a single agent on the virtual machine host and ensure that you get consistent, recoverable copies of your VMs. Using the VSS framework and the application-specific writers, when the backup agent initiates a backup request, the host OS automatically communicates with all of the applications running inside of the VMs to ensure that they get themselves into a consistent state so that the data contained within the VHD is application consistent. That way, the recovery is not only guaranteed to work but you won't have to sit through a 2 hour chkdsk or consistency check session.
DPM has a pretty slick UI to make backup and recovery easy and includes the ability to backup to disk and tape in an integrated way. If you've got SCVMM, you're already licensed for DPM so give it a try. If you're not using DPM, check with your backup vendor to see if they include support for virtual machine backup on Virtual Server and Hyper-V, most of them already do.
Bottom line - if you're virtualizing on Windows, backup and recovery will snap into place seamlessly with your existing infrastructure.
Rakesh
Sorry for the large gap between posts here folks but as you might imagine, we've been pretty heads down working on the next release of SCVMM. Our immediate goal is to get the beta out so that you all can start managing your Hyper-V deployments and also start consolidating your management.
A few weeks ago we had a customer event here in Redmond for our Windows Server early adopters to give them the first glimpse of VNext and some of the functionality that we're going to support. The number one piece of feedback was "I need this yesterday". Point taken.
Aside from the VMware and Hyper-V support, there are a bunch of other things we're adding to the product so I thought I'd run through a few.
Resource Optimization - With VNext we've taken integration with Operations Manager to the next level. You'll be able to take a subset of application specific alerts that Operations Manager has generated while monitoring your environment and surface them directly into the SCVMM console (no more "alt-tabbing") in a feature we call "Physical Resource Optimization" or "PRO" for short. As Ops Manager finds opportunities for optimization within your data center, the alert is forwarded to SCVMM where you can view these "PRO tips". You can even have SCVMM automatically implement PRO tips based on rules that you can defined. The really cool part about PRO is that it is extensible and we're working with our industry partners (hardware and software) so that you get the best data available to make a fully informed decision when you go to optimize your data center. Even on VMware's hypervisor, PRO will help you make better decisions resulting in fewer mistakes and less reconfiguration.
Host Clusters - I know that lots of you will cluster your Hyper-V hosts and we wanted to simplify the management experience around clustering. Server 2008 has plenty of improvements with respect to clustering that make it easier to configure. Once you have configured a host cluster, SCVMM recognizes this and lets you manage this group of servers as a unit within your host group. You can then specify how many node failures you want the cluster to be able to withstand while keeping the VMs running and SCVMM's intelligent placement system will ensure that you don't over commit the resources on your cluster so that this SLA is maintained. In addition, when you deploy a new VM and want it to be highly available, you simply specify this requirement by checking a check box and we configure the appropriate cluster settings and resource groups for you. We wanted to keep things simple while ensuring that we maintain your SLA.
Delegated Administration - In SCVMM V1, we basically had two types of users - self-service users and administrators. Customers have been telling us that their virtualization deployments are getting very large and they need the ability to sub-delegate chunks of their environment to regional or departmental administrators. With SCVMM VNext, we've included this capability so now you'll be able to use a single SCVMM server to manage very large enterprise environments in a flexible way. You'll also find that our published scale numbers for SCVMM V1 are roughly double that of our competition. We've heard you loud and clear - fewer management servers and consoles is better. As a guy who builds management consoles and servers, that took a second to digest ;-)
There are plenty of additional features as well and I'll be blogging about them over the coming weeks. In addition, we'll have lots of sessions and discussion about the product at the upcoming Microsoft Management Summit (more info about that here) which I encourage you to attend.
Rakesh
Since we announced a few months back that our next release of VMM (VNext) would manage VMware, I've received lots of questions with respect to why we made this decision and what "manage VMware" really means. Let me walk you through some of the reasoning and thinking.
1) Customers - The primary reason that we decided to manage VMware was, quite frankly, because customers were asking for it. Personally I take this as a compliment to our management suite given that most if not all VMware customers are already using VMware's management tools. The fact that they want our tools to work on top of VMware's OS platform tells me that there's a definite opportunity to innovate and do a better job and we're fully committed to doing this....which leads to my next point.
2) We think we can do a better job - At the end of the day, it's not about managing virtual machines. It's about managing applications and services and today, many if not most of those run on Windows. Understanding the application in detail is absolutely critical to making intelligent management decisions. For example, migrating a VM when the CPU spikes to 80% for 10 minutes is not a particularly smart way to make that decision but if the VM is a 'black box' to you, it's the only choice that you have. With our management tools, you'll be able to set policies and rules based on application specific criteria. For example, if the average amount of time it takes for your order entry system to process an order exceeds 10 seconds and CPU is the reason, add more CPU capacity to the VM. Our customers are telling us that this is much more powerful and relevant. We feel strongly that with Hyper-V, our platform and our management tools provide an excellent end to end solution. With that said, we know that you have investments in VMware but even in that case, our management 'engine' can make better decisions on the VMware platform. In addition, the System Center family of products gives you the ability to manage physical servers right alongside your virtual machines with a single set of integrated tools rather than creating a new silo or island within your organization.
3) Flexibility in Hypervisors with a single management solution - As I said above, we feel very confident that our hypervisor provides the best platform in the vast majority of customer use cases (that's for you to decide of course) but regardless, customers want to use a single management tool in mixed environments. You'll also be able to automate across hypervisors using a single Powershell interface that we provide. VMM will abstract the difference in hypervisor APIs for you. You simply run the "Stop-VM" cmdlet and we make sure that regardless of the hypervisor platform, the VM is stopped. No more code blocks that read "If (VMware)....elseif (VirtualServer)......elseif (Hyper-V)......"
Finally, I want to emphasize that when we say "manage VMware", we mean that day to day, you'll be able to use our console and command line interface to fully manage your Virtual Infrastructure environment (including live migration), Virtual Server and Hyper-V environments seamlessly. In addition, we'll be able to extend the management capabilities that VMware offers today so you'll get an enhanced solution even on a non-Windows OS.
We'll share more details, screenshots and demos very soon....
Rakesh
I've been getting lots of questions about SCVMM and the recently announced Hyper-V beta (more info about that here).
The version of SCVMM that we released a couple of months back supports Virtual Server running on Windows Server 2003 only. Support for Hyper-V and Windows Server 2008 is being worked on as we speak and we plan to have a beta out in the second quarter of 2008 and the final release will coincide with the final release of Hyper-V. We're working as fast as humanly possible to release the Hyper-V support (and the VMware support) . You don't need to wait though, start kicking the tires on Hyper-V now and when you're ready to set up a large Hyper-V environment, I promise that we'll have your back with a sweet management solution that snaps seamlessly into place. We will of course continue to support Virtual Server so you'll be able to manage Hyper-V, VMware and Virtual Server with a single management tool and a single Powershell scripting interface. Our engine will abstract the differences between the platforms but will still allow you to take full advantage of the distinguishing features of each.
If you're wondering why the beta of SCVMM isn't available at the same time as the Hyper-V beta, let me give you some background. We had two options with respect to delivering SCVMM to the market:
1) Deliver a V1 based on Virtual Server and then a V2 roughly a year later with Hyper-V support
2) Deliver a V1 aligned with the Hyper-V schedule (which includes both Virtual Server and Hyper-V support)
Had we chosen option 2, we probably could have aligned the betas of SCVMM and Hyper-V more closely. That's because we could have kept writing more code for longer chunks of time instead of stabilizing the product and running the vigorous test processes that are required at the tail end of the software release cycle. There is a fixed cost associated with the "end game" or "home stretch" of a release (particularly with Enterprise software) and the more releases you do in the same period of time, the fewer features you ultimately can include since you have less time to write new code. We ultimately decided to chose option 1 because, frankly, customers were screaming for a management solution and they couldn't afford to wait an extra year so neither could we. Also, we're still aligned very well with the final Hyper-V release but unfortunately the betas don't align as nicely as we would have liked. With that said, I haven't heard too many people complaining about Hyper-V beta being release earlier than expected ;-)
Bottom line - we're working on it and nothing is higher in priority for our team than to deliver a world-class management solution for Hyper-V so stay tuned.
Rakesh
One of the things we heard fairly early on while planning VMM was that offline storage of VHDs was going to become an increasingly large management task. Based on this need, we created the "VM Library" feature to allow users to save virtual machines that weren't currently running to a managed location. As we developed the requirements for this feature, we quickly realized that people wanted to store more than VMs, they wanted to store all of the stuff that they used to create VMs as well (ISOs, templates, OS profiles, Hardware profiles etc.) so we changed the feature's name to "library" instead of "VM library". Before defining the feature in detail, we had to make some key design decisions and at the risk of boring you, I'll walk through some of that decision making process.
Does the library contents reside in the file system or in a database?
This is an age old debate and ultimately we decided to keep meta data in our SCVMM database but we kept all of the VHDs, ISOs etc. on file shares. While this made our life a little bit harder (we were now responsible for designing a system that kept meta data and the file system data in sync) it gave customers more flexibility. They could continue to use existing file replication technology and still had the ability to use Windows explorer or other file management tools to organize the information in any way that they wanted. It also made it easier to integrate with existing systems.
Is the library a physical thing or just a logical thing?
We decided that the answer to this question was "both". If you go to the library section of our UI, you'll see that you can either view the library as a logical collection of items (my favorite view is to click on the library server tree-node on the left and then group by "type") or you can view the actual physical folder hierarchy within the library share. Logical views are very convenient but sometimes the physical location of a file is important as well so you can use either view interchangeably. Templates, OS profiles and hardware profiles are all purely logical constructs so you won't find those items on disk, they're in our database. It's likely that we'll ultimately have on disk representations of those objects as well in the future though.
SCVMM Library Logical View
SCVMM Library Physical View
Does the library contain every object the SCVMM knows about or just stuff physically in the file shares that the library is comprised of?
This one we went back and forth on but ultimately decided that the library isn't really an "inventory" of everything in the world that we manage, it's where you keep stuff that is not running or associated with a host. You can think of it as a virtual 'shelf' to store stuff that you're not currently using. With this decision, there are only two places an object can live and be managed by SCVMM - either registered to a host or stored in the library. If you put something in a random folder or unmanaged file share, you're on your own.
What about co-location of library objects with the host for rapid deployment?
This decision was pretty easy. We decided to allow a host server to also be a library server. In this manner, you can store templates or other library objects local to a physical host so that deployment from the library to that host would be very fast.
Do we build Powershell cmdlets to organize library contents?
We initially started working on "move" and "copy" and "create" cmdlets to help users organize the library directly from our UI but that turned into a very slippery slope. One of the benefits of providing direct access to our library files via the file system is that you can organize directly through the Windows explorer interface (create directories, copy, move etc.). Aside from being a lot of extra work for us to expose all of that through our UI as well, it was unlikely that we'd get parity with the full set of file system operations. As a result, we made the call to cut this functionality from our UI but we added some requirements to ensure that as stuff was moved around in the file system, SCVMM would track the changes and have them reflected in our UI. By default we "refresh" the library share every hour but you can change this value or just refresh manually in the library UI or via cmdlet. We also tag each of objects in the library with GUID so that if you move things between shares, we just update the path property since we recognize the object even as the location changes.
Once we made these decisions, the rest of the functionality worked itself out pretty seamlessly but some of our beta feedback showed that we had overlooked one key scenario.
Throughout the design process, we had assumed that customer would typically have a handful of library servers (probably less than five) but lots of hosts (probably tens or hundreds). This is true for most data centers as admins like to consolidate and it wasn't uncommon to see large library servers attached to SAN arrays with fast storage for rapid deployment. With retail customers however, it was a different story. These guys had thousands of stores worldwide and needed to have a library server, often the same physical machine as the host computer, in each location so they effectively had at at least as many library servers as hosts. Some customers had library servers attached to regions of world and again they ended up with many more libraries than we had designed for. Typically, each of the library servers had the same file content replicated to across many sites (in the same way that they published software installation files or content).
Taking this feedback into consideration, we did a couple of things:
1) Began testing the system with lots and lots of library servers :-)
2) Added the "Library Group" concept.
If you right click on a library server in the library section of our UI, you'll see a property called "library group" which lets you associate a library server to a host group. Now, when you go to create a new virtual machine and you want to chose a base VHD from the library to start with, you can have the SCVMM UI scope your choices to library servers associated with the host group that you want to put the VM into.
As an example, you might create 25 geographic regions and assign a host group to each region. If you are creating a VM for the "Pacific northwest" region/host group, you'll want to use library objects from the library server associated with that region to prevent large file copy operations across very long distances. To enable this use case, you right click on the library file share and set the "library group" equal to "Pacific northwest". Now, when you launch the new VM wizard, you'll notice that when you select a library object to build a VM, you can scope the objects down by "library group" to ensure you use the closest copy of the file you need. In my example, you'd select "Pacific northwest" for this value. Similarly, from the command line, you can query for a library object and filter by library host group to ensure that you use the closest copy of the file.
The rest of the library concepts are pretty self-explanatory and there is plenty of additional content on the SCVMM library here as well.
Rakesh
P.S. - Happy Holidays!
Sorry for the gap between posts. I was on the east cost for the thanksgiving holiday and then stayed for the week to talk to some customers about our future road map and get their feedback. Anyhow, today we're gonna talk about checkpoints.
One of the things that I personally used to find confusing about virtual server was the usage of both differencing disks and undo disks. Both were used to do pretty much the same thing - roll back a VM to a previous state - but neither were particularly easy to use. After some deliberation with customers, we decided we'd only support one of the two in SCVMM and we decided on differencing disks since they provided a general superset of the functionality of undo disks. If you try to important a VM with differencing disks into VMM, it will prompt you to either discard the undo disk or merge it first.
The second key decision we made was with respect to granularity of creating of the differencing disks. Although virtual server allows you to create a differencing disk at the VHD level, we decided to allow creation of differencing disks only at the VM level. This simplifies much of the management and frankly, if you create a differencing disk on only a subset of the VHDs associated with a VM and then try to delete/recover a subset of the VHDs, you're taking your life into your own hands. The whole point of reverting a VM back in time is to revert the entire system state, application state and data. If you recover a subset of your drives, you have a high probability of these three components being out of sync and the VM won't run properly.
The last thing we had to figure out (and probably the most important thing) was how to make differencing disks easier to use. Managing them the the virtual server web UI isn't particularly easy and you as an administrator need to create and migrate the differencing disks on your own through the file system. To simplify this experience, we created the UI concept of a "checkpoint". A checkpoint is a logical entity that refers to a set of differencing disks associated with a VM and SCVMM manages the disks and their relationships for you. If you right click on a VM and click on "manage checkpoints", this opens up our checkpoint management dialog.
When you create a new checkpoint in our UI or use the the New -VMCheckpoint cmdlet, underneath the covers, SCVMM creates a set of differencing disks for each of the VHDs associated with the VM that you are checkpointing. If you actually go to Windows explorer and take a look, you'll see that the new disks are there and the naming convention that we use is the parent VHD name + the Date/Timestamp + VHD sequence number. SCVMM stores the fact that a new checkpoint exists in our database but it can also parse back the VHD name to figure out that the differencing disk is actually associated with a checkpoint object. (If you import VMs that already have differencing disk, SCVMM won't recoginize them as checkpoints since the metadata is missing.)
With virtual server, the VM needs to be stopped in order for checkpoints to work. If the VM is running and has VM additions installed, SCVMM will perform a shutdown for you, otherwise it is stopped directly. With the upcoming hypervisor, the VM can be checkpointed even while running which is a nice improvement. The hypervisor UI will call checkpoints "snapshots" but in order to be consistent and compatible with our first version, we decided to keep the term 'checkpoint' so we can use the same cmdlet names and the same UI.
You can also 'merge' checkpoints in our UI. We called this 'delete' in our beta since technically it removes the differencing disk chain by merging all changes into a single VHD but for obvious reasons, this caused users to think that data would actually be lost. If you merge a checkpoint, all future checkpoints must also be merged and you'll get that warning if you try this in the UI.
Restore has similar semantics to merge but it actually does delete the differencing chain thus reverting the state of a VM to the point in time represented by the checkpoint. When you restore to a checkpoint, we keep the diff disk/checkpoint for the point you restored back to. This enables training lab scenarios where you build a set of VMs, have students run through their training and then revert back to the original VM over and over again.
One thing we know from customers about restores is that typically people restore to the most recent checkpoint (not always but this is by far the most common case). Taking this into account, we add a '-MostRecent' parameter to the Get-VMCheckpoint cmdlet so you can have easy script access to the most recently created checkpoint. If you want to restore a VM to the most recent checkpoint, you'd run something like this:
Get-VMCheckpoint -MostRecent | where { $_.VM -eq "VM01" } | Restore-VMCheckpoint
Of course you'd change the name of the the VM to the one you're targeting.....if you don't specify a particular VM, this cmdlet will restore all of your VMs to the most recent checkpoint so be careful.
One of the parameters for New-VMCheckpoint that we regrettably didn't get time to include in V1 is the -RunAsynchronously flag. This means that in a script, VMCheckpoint operations run serially. As a workaround, the UI does let you select multiple VMs and create a new checkpoint which launches several checkpoint jobs in parallel. Such is the nature of shipping software though, you try to get everything in but ultimately you need to make some tough calls. Given that the hypervisor has a much improved mechanism for differencing disk creation, this feature will get even better in our next release but it's already proved to be pretty popular in V1.
Rakesh
Even though I'm writing a lot about V1, the engineering team here in Redmond is hard at work on the next release of SCVMM slated for next year. In V1 we had a lot of 'plumbing' code to write including a task infrastructure, the Powershell integration model, WS-Man etc. etc. I feel really good about the product we delivered and the early customer feedback has been tremendous. One of the things we didn't have a lot of time in the schedule for is what I would characterize as "polish" and we're gonna spend a few more cycles on that in this release. As some new screens come to life in the product, I'll post screenshots here on the blog.....keep in mind none of the screens are final and are subject to change (or being cut completely).
This is a screenshot of what our next autorun screen might look like.....internally we call the next release "Carmine VNext" (Carmine was the codename for V1). In case you're curious, we chose Carmine because it's a complimentary color to Viridian which is the codename for our new hypervisor.
The marketing people will ultimately slap the proper branding and name on the product. They tend to change their minds often so we just throw placeholders into the UI. Personally I think 'Carmine' sounds better than "Microsoft System Center Virtual Machine Manager 2007" but I digress....
Well I'm off to IT Forum in Barcelona for the rest of the week. If you're at the show, drop by our booth or attend one of our sessions and introduce yourself to me or one of my team members.
Rakesh
Placement is the term we use to describe the processes of matching VM requirements to available host resources. This is useful when you initially create a VM, migrate a VM or deploy a VM from the library.
We broke down VM placement requirements into two buckets:
1) Hard Requirements - these are requirements that a VM absolutely needs a host to meet in order to run - memory and storage.
2) Soft Requirements - these requirements, if met by the host, allow a VM to perform more optimally and include CPU allocation, network bandwidth, network availability, disk IO bandwidth and free memory.
In the user interface, VMM gives you a star rating system for hosts with a range of zero to five stars. If a hard requirement is not met (i.e. not enough memory), the host automatically get zero stars and the product will block you from placing a VM on that host. If a soft requirement is not met, this also affects the star rating to various degrees depending upon the weightings and placement algorithm that you use (see below). You can also scope the placement process to have SCVMM look only at a subset of hosts by selecting the target host group for placement at the top of the screen.
For networking, if you're migrating a VM between hosts and the network a VM is currently connected to is not available on the new host (we do this comparison based on virtual network names to figure this out), the host gets zero stars but we don't block you from selecting that host since the technically the VM can run, it just won't have the same network connectivity. In the next release, we'll also let you specify whether a VM requires high availability or not in which case we'll look for hosts that are configured in a cluster. You'll also notice that there is a "SAN" column on the ratings screen. You'll see a check mark here if the target host is properly configured to enable SAN-based transfers which are much quicker than standard LAN transfers. I'll have a separate posting on SAN transfers in the future but if you want more details on this, you can get it here.
While building this feature, customers told us that they wanted good visibility into the placement process so we added features to improve the visibility into why a host gets the rating it does in the 'rating explanation' tab below the placement screen.
You can also customize the placement algorithm by adjusting performance parameters for a particular VM. Just click on the "customize ratings' button on the placement wizard screen. The first thing you'll notice is that we support a couple of algorithms - load balance and resource maximization. You'd use the load balance algorithm if you have a pool of hosts that you want SCVMM to evenly distribute VMs across. You'd use resource maximization if you'd prefer to fully saturate a host completely before moving to a new one. This is useful if you're looking to really max out the utilization of a machine. As I mentioned in a previous post, at the host group level, you can also set host reserves so that the system doesn't exceed a certain allocation threshold. This way, you can ensure your environment has enough 'spare' capacity around for emergency deployments in the event of failures or new requirements. The other slider bars in the UI let you modify the weighting/importance various soft requirements. Ultimately, the algorithm we use to generate the host rating, we use a formula something like this:
Host Rating = (Free CPU * CPU Weight) + (Free Memory * Memory Weight) + (Free Disk * Disk Weight) + (Free Network * Network Weight)
In order to do this placement accurately and across a variety of hardware, we use technology originating from Microsoft Research which helps normalize performance characteristics across a hardware types using models. When the technology was first developed, it was kind of a solution looking for a problem....and along came virtualization which was the perfect problem. Here's a slide from a technical presentation we often give on placement that describes the various inputs and outputs of the process. As you can see, it's pretty sophisticated but at the end of the day, you just need to deal with the rating that it produces and leave the heavy lifting to us.
In the next version of SCVMM, we'll be improving placement in several ways. Since we'll be supporting multiple virtualization platforms (Virtual Server, our hypervisor and VMware), placement will need to ensure that it picks hosts appropriately based on the capabilities of the platform. For example, if you want a 64-bit VM, Virtual Server-based hosts will be given zero stars. More superficially, the user interface for the star rating screen is pretty crammed with information and we're looking at ways to make it easier to read and perhaps change the layout. As the new UI comes on board, I'll post some early screen shots here as well.
Rakesh
Sorry for the delayed post....I was visiting my family in Canada and didn't get a chance to post before I left.
In this post I'll get into how you go about building a VM. We wanted to make the product as flexible as possible and allow for the creation of virtual machines from a variety of sources without making it overly complicated. Most customers told us they have a handful of 'gold images' that they create VMs from - typically these are sysprepped VHDs. We centered a lot of the functionality around this concept but we also support alternative VM creation methods. I'll walk you through them some of our thinking.
Creating a VM from a Template
If you're planning on deploying lots of VMs, it's a good idea to use templates. In SCVMM, a template has three parts:
1) A sysprepped VHD - if you click on 'new template' in the library section of the UI we can create one of these for you from a VM if you don't already have one in the library. We basically sysprep the source VM and stash the resulting VHD into the library and it becomes part of the template. If the source VM for your template has multiple VHDs, you should select the VHD that contains the OS. You should also make sure you have the VM additions baked into your template to have the sysprep process go smoothly. Since the source machine is converted into a template, if you want to keep your original source VM in tact, make a copy before creating the template. The current version of SCVMM supports Windows 2000 and Windows Server 2003 templates. More operating systems will be supported in the next release (see the end of this post).
2) OS/Sysprep Settings - These are settings that tell SCVMM how to configure your OS. If you want to reuse the same sysprep setting across a set of templates (i.e. same product key, same admin password, timezone etc.), you can create and save what we call an "operating system profile" and store it in the library. When you create a new template (or a VM from a template), you can either specify the sysprep settings manually or reuse a pre-defined operating system profile. We did this to keep the number of templates you'd need manageable (or as manageable as possible). OS profiles are kept in our database. Logically you'll see them show up in the library and you can find them in our UI but you won't find them in your library file shares. If you use sysprep.inf files, you can associate this file to an OS profile and avoid directly specifying all of the settings as well.
3) Hardware Settings - These are settings that tell SCVMM what the virtual hardware for your VM should look like. Similar to OS profiles, we also support hardware profiles that you can save and associate with a template. When creating a new template (or VM from a template) you can either specify the virtual hardware settings or reuse an existing hardware profile from the library. Like OS profiles, these are logical entities that are stored in our DB. As a point of interest, you'll notice a CPU drop-down list that we provide during specification of virtual hardware. This actually came from our colleagues in the Capacity Planner team and was originally dreamed up by MS Research. At the time, it was somewhat of a science project to come up with software models of hardware but then along comes virtualization to put the technology to good use.....good thing for us.
We arrived at this overall user model for templates after lots of customers told us that server build processes typically involve selecting hardware, choosing an OS and configuring the OS so we added SCVMM user elements for each of these steps. There are often additional build steps and we do support sysrep's ability to run individual commands once after customization is complete (this is a setting on the OS profile). You can independently save off a bunch of hardware and OS profiles even without templates so the system is pretty flexible. Personally I worried about the fact that this flexibility added too much complexity to the product but I think it worked out well since for simpler environments, you don't need to touch OS profiles or HW profiles at all and everything still works.
Now that you have a template in the SCVMM library, you can use it as the starting point for new VMs. When you launch the new VM wizard, just select a template from the library and all VM settings are defaulted to what was specified in the template. You can override the settings if you so choose. SCVMM builds the VM per your virtual hardware specs and apply the sysprep settings for you. If you go to the jobs UI can click on the running operation, you can see the steps involved in customizing a new VM in all of its glory. (You might notice that we create and remove a virtual floppy drive....we use this mechanism to inject the sysprep answer file into the build process).
If you create a new VM from a template, you'll notice that you cannot store the VM in the library and you have to actually place it on a host. That's because the customization step can't be done offline (yet anyways). If you want to pre-build a bunch of VMs from a template and stash them in the library via scripting, a good approach is to create a host group with one or more hosts and call it "maintenance hosts" or something like that. Have your script scope placement to that host group and once customization completes, you can script the saving of the resulting VM into your library. You can use a similar approach for patching images in the library but I'll provide a post with more prescriptive help there.
Creating a VM from an Existing VM or VHD
When you launch the new VM wizard, you're also given the option of creating a VM from a VM that you already have. The UI is slightly misleading here - if you hit the 'select' button after launching the wizard, you'll see a list of VHDs, templates and VMs in your library and you'll also see all stopped VMs on managed hosts. SCVMM always creates a copy of the source objects in this type of VM creation so your original VM, VHD or template is never moved or modified, it is 'cloned'. If you want to deploy the actual VM stored in the library rather than a copy of the VM, you can go to the library, select the VM you want and click the 'deploy' action which will physically move the VM from the library to a managed host for execution.
The key difference between using a template to create a VM versus a VM or VHD is that you cannot specify operating system configuration information (sysprep settings) with the latter.
Creating a VM from an Blank VHD
This is also supported in the event that you want to lay down an OS with PXE or perhaps drop an ISO image into a virtual DVD-ROM and install from scratch. This is a nice way to build a source image for a template. As a side note, you can attach an ISO from the library to a VM's virtual DVD-ROM and and select the option to use the library's copy rather than actually copying it local to the VM. This saves you from proliferating hundreds of ISOs with identical content all over your environment and sharing the central library copy instead. We got this feature request during one of our betas for customers using SCVMM to deploy VM additions via ISO images inserted into virtual DVD-ROMs.
Creating a VM from a VMware Virtual Machine
We had lots of customers asking for this functionality but didn't think we had time to get it in. However, due to the work of a couple of our hotshot developers and testers, we were able to ship this in V1. Just copy your VMs to a library share and then go to the library section of our UI and click on 'convert virtual machine'. You might need to refresh the library first since by default we only refresh new library objects every hour. You then point us at the .vmx file and walk through the rest of the familiar wizard and SCVMM does the conversion work for you. I should point out that this isn't just a format conversion.....vmdk to VHD is a trivial process. Actually ensuring the VM will run once converted can involve driver replacement, a HAL swap and disabling VMware tools (we disable the service, we don't uninstall). We reuse a good chunk of our P2V code and infrastructure to make this conversion process work.
Since this is often a scripted operation, here's an excerpt from our scripting guide that provides some guidance (Okay, it kind of uses a back door and goes directly against a file path rather than a library path....technically this isn't supported and you should use a library path to your .VMX file but this should work)
####################################################################
# Connect to the Virtual Machine Manager server.
####################################################################
# Substitute the name of your VMM server and domain in this command:
Get-VMMServer -ComputerName "VMMServer1.Contoso.com"
####################################################################
# Define variables.
####################################################################
# Substitute the name of your host server and domain in this command:
$VMHost = Get-VMHost -ComputerName "VMHost01.Contoso.com"
# Substitute the path to your .vmx file in this command:
$LegacyVM = "E:\VMWare\Windows Server 2003 Enterprise Edition (Dynamic)\Windows Server 2003 Enterprise Edition.vmx"
$Memory=256
# Substitute the name of your virtual machine for VM01 in this command:
$VMName = "VM01"
$VMPath = $VMHost.VMPaths[0]
####################################################################
# Run a VMM cmdlet that creates a job - in this example script, the
# cmdlet is New-V2V, so the job is the creation of a new VM from an
# existing VMware VM.
####################################################################
$VM = New-V2V -VMXPath $LegacyVM -VMHost $VMHost -Name $VMName -Path $VMPath -Memory $Memory -Runasynchronously -Jobvariable "Job"
If you want to use VMware VMs in your library as a source, the following Powershell script should do the trick:
get-vmmserver –computername “VMMSERVER.CONTOSO.COM”
$vmhost = get-vmhost –computername “VSHOST1.CONTOSO.COM”
$library = get-libraryserver –computername “VMMSERVER.CONTOSO.COM”
new-v2v –LibraryServer $library –vmxpath “\\VMMSERVER.CONTOSO.COM\MSCVMMLibrary\VMDKS\ConvertMe.vmx” –vmhost $vmhost -name “DemoV2V” –path “C:\VHDs”
You do have to provide a host for this conversion to work since we need to fire up the VM to finalize the conversion. You can make a library server a host as well (just add it as a host and as a library in our UI) and have everything run local to the machine to expedite things. Alternatively, you could use the maintenance host group concept that I described earlier.
In the next version of SCVMM, we'll have the same core set of features and mechanisms to create VMs but we'll support new functionality available with our upcoming hypervisor currently not available in Virtual Server including multi--proc VMs, 64-bit VMs, direct disk access, higher memory limits and much more. Since we're supported VMware, you'll also be able to create these types of VMs on their platform too. We also plan to support templatizing (is that a word?) Vista and Windows Server 2008. We may have time to fit in Windows XP template support as well but we're not certain at this stage but a couple of customers have already asked for this.
You can get lots of additional information on creating new VMs and templates here.
I didn't discuss the P2V process in this post and I'll cover that in the future as it warrants a separate discussion. With P2V you can create a VM from a physical server and then use that as the basis for your template or other VMs as well so it just adds an additional source for new VMs.
rakesh
Kind of a long post but helpful I hope....
When we started working on this product, we tried to wrap our heads around what it really means to provide a "VM Management Solution". A great way to figure it out is to spend a few days doing the work of a Windows Server admin and walk a mile in their shoes. We went out on the road to work with customers and here's what we found they spent most of their time doing:
- Managing/configuring host computers to run Virtual Server
- Managing images (that includes VHDs, ISOs, scripts, templates...the list goes on)
- Configuring Virtual Machines
- Converting physical machines into virtual machines
- Figuring out which physical host to run the VM on (Excel spreadsheets were used here...Excel is great but there had to be a better way)
- Monitoring, Patching and Backing up their VMs and hosts
- Automating and scripting all of the above
After returning to Redmond, we decided to assign a group of engineers to focus specifically on each of these areas of management with one goal - drive down the pain and cost associated with all of these tasks so people can get more real work done. Over the next few posts I'll walk you through some of the details on each of these areas, how VMM helps you out and what you can expect moving forward. Lets start with host management.
After firing up the VMM UI for the first time, the first thing you typically do is add a host or two to manage. Sometimes you're just adding 1, other times you're importing a large chunk of an existing environment. There are several ways we made this easier:
- "Add host" wizard walks you through this process step by step
- You can search and bulk add directly from Active Directory (heck, you've already got a catalog of hosts in AD, might as well use it)
- Automatically install the agent software on added hosts if required - you can also locally install the agent if you want to bake it into an image
- Automatically install and configure Virtual Server if required - we don't' enable the Web UI or IIS since you won't need them with SCVMM
- Enable the addition of hosts located in a perimeter network - lots of web servers sit out there so this enables support for them as well
- Use the add-vmhost PowerShell cmdlet (you can step through the wizard and then click on "view script" at the end if you want some hints...we'll be publishing a scripting reference manual and guide in November so stay tuned for that)
Before any of this works though, you'll need to make sure your host is running Win2K3 SP1+ and you need to ensure that WinRM (aka WS-Management) is installed. WS-Man is the protocol we use to communicate between the SCVMM server and the host computers. WS-Management is a web-services protocol and by being one of the first enterprise products to fully embrace it, it opens up a world of interoperability. You can learn more about the guts of WS-Man here if you're interested. One of the cool things about being a brand new product is that you get to use some of the most cutting edge technologies...I'll talk about more of them in later posts.
After you run the add-vmhost cmdlet or run through the wizard, SCVMM kicks off a bunch of jobs to actually add the hosts you requested. You can go to jobs tab in the UI to see the progress. This should work smoothly but if for some reason it doesn't, we've assembled some troubleshooting help based on the most common issues we customers hit during our early testing. When you go back to the VMs or Hosts view, you should see your recently added hosts begin to appear.
If you've got lots of hosts, you might want to 'bucketize them' into that we call "host groups". Why would you do this? Well, host groups are important as an organizing unit plus they are the unit of delegation for self service. You can also set host reserves at a host group level. Just right-click and hit properties on a host group. When you set reserves on hosts, our intelligent placement feature honors them so we keep some spare capacity around in the host group (for stuff like fail-over or other emergencies). If you reserve lots of capacity, you'll see the star ratings during placement affected as a result....I'll have more about this when I do my placement posting.
You can also double-click on a host to expose host properties and there are some interesting, non-obvious ones worth calling out
The hardware tab allows you to see what type of iron you're running on and lets you create new virtual networks. If you give multiple virtual networks across hosts the same name, SCVMM assumes that they are identical. This is helpful as you move VMs around and want to ensure they stay connected to the required network. For example, you can create a network called "ITLAN" on each of your hosts and associate it with a physical NIC connected to your main network and then hook it up to VMs on that host. If you later want to migrate that VM to a different host, our intelligent placement will make sure that the target host has a network called "ITLAN" to wire the VM back up to else you'll lose network connectivity. This does require a little planning up front and personally, I'd just run a PowerShell script to add networks to a group of hosts up front to prevent the chances of human error and ensure all of my hosts have the right networking configured. One feature idea that has arisen from this is the concept of 'host templates' but I'm not sure when that might get into the product. In the mean time, Powershell gets the job done just as well.
The placement property tab let's you add physical paths (typically one per data volume) to store the VMs deployed to that host and also mark a host as unavailable for placement. You might do this if you're got a host down for maintenance and don't want VMs deployed there.
The "VMs" tab lets you 'register hosts". You'd use this if you have a VMC and VHDs sitting local to the hosts file system (not in the VMM library) and you want to fire them up. This isn't a typical thing that should happen but if it does, this is where you go. You can also use the 'register-vm' cmdlet for this.
The options tab lets you enable things like multiple VMRC connections which is off by default as of Virtual Server 2005 R2 SP1 because it allows snooping across sessions. The downside of disabling this is that live thumbnails cannot be shown on the same screen as a full VMRC session (since that would be considered 2 sessions) nor can multiple users VMRC into one box. Convenience versus privacy...your call depending on where/how you're deployed.
The custom tab is common to many of our objects including hosts, templates etc. It lets you attach custom property values to a host. You can then group, search, sort or organize with this custom property. I've sen customers often use things like "business unit" or "chargeback ID" for values here. You can add columns to the UI to show these custom properties as well by right-clicking on the column bar in any of our center panes.
In the next version of SCVMM, we'll obviously allow you to add hosts running our new hypervisor and there will be several more improvements that we are planning including:
-
Smaller install footprint - The virtualization stack and other required bits will be part of Windows Server, we'll configure it for you...we're hoping to truly be agent-less
-
New Networking Options - Virtual Switches, VLANs.....networking is about to get very interesting
-
Support for non-trusted domain joined hosts - we've had several requests for this already
-
Adding VMware environments - I'll have more details on this later
-
Full cluster-awareness - today we work with Virtual Server clusters but we're not specifically cluster aware. This gets a lot easier for you.
-
A bunch of refinements I can't remember off the top of my head
There's lots of additional detail about host management on our technet site located here.....that's it for now....rakesh
Hi - I'm Rakesh Malhotra and I'm the Group Program Manager for SCVMM. My team is responsible for prioritizing features, staying in touch with customers and partners, understanding the competition and driving the overall project forward. With the first release of SCVMM behind us, I thought for the next release I'd start a blog so anyone who is interested can keep up with our progress and thinking in real time.
Version 1 is all about managing Virtual Server based environments and has a pretty extensive feature set - if you haven't played with it, you can download the eval copy at http://technet.microsoft.com/en-us/scvmm/bb679927.aspx. If you don't want to spend time installing/configuring a server to test with, I'd recommend downloading the VHD and running it as a VM. There's lots of great content describing all of the things that SCVMM can do for you on the site but let me hit some of the highlights:
- Rapid template-based provisioning of Virtual Machines - I've done many demos of this over the past year and you can push out a new Win2K3 box in about 35 seconds
- Intelligent Placement - You tell us the type of VM you need and we'll find the best spot to put it based on the resources you have. SCVMM uses sophisticated modeling technology (thanks to Microsoft Research) to match performance requirements to the available capacity in the data center.
- Library Management - Put all of the stuff you you need to create virtual machines into the library and have SCVMM manage it for you. You can centralize your library (with a big Windows File Server) or distribute it across the edges of your network like many of our retail customers do.
- Virtual Machine Self Service - Want to use VMs in your dev and test lab? This feature lets you delegate the ability to create and manage VMs to developers/testers or regional administrators. They just need to access the SCVMM self-service web portal and they'll have this ability. Before you freak out, we put lots of controls in place to prevent VM sprawl and general unmitigated chaos (quotas, granular permissions, template limited access etc. etc.)
- PowerShell Automation - When we first started working on SCVMM, I had a large enterprise customer tell me that we needed to build and document our command line interfaces much better (his language was a little bit more colorful and a bit more forceful) - so here you go - everything you can do through the UI you can do through the command line via PowerShell. In fact, our developers first build PowerShell cmdlets and then built our UI directly on top of it so our API is your API. And oh yeah, we have a boatload of documentation and sample scripts.....I know I'm biased but PowerShell makes existing scripting environments look and feel their age (seriously, it's 2007 and it's about time...thank you Jeffrey snover)
We're already heads down working on the next release which will coincide with the Windows Hypervisor (codename Viridian) release. We'll obviously 'light up' all of the new hypervisor features and we also recently announced that we plan to support 3rd party hypervisors including VMWare and Xen. Lots of customers tell me that they want 1 management tool that works across platforms so this is in direct response.
Over the next few posts, I'll dig into individual feature areas in more detail and describe exactly how stuff works under the covers, why we made the decisions we made and the direction that we plan to take our features.
Rakesh