Storage Automation in VMM 2012

Hi, I am Hector Linares, Senior Program Manager on the SCVMM team. This blog is an overview of the storage automation feature set in System Center Virtual Machine Manager (VMM) 2012. The goal of this feature is to simplify and automate the discovery, allocation, and assignment of storage assets in a virtualized environment.

Summary

VMM 2012 introduces enhanced networking, storage, and physical server management that enables enterprises to create on premise private clouds. The key investments in storage include modeling and automation. Modeling helps administrators understand what storage assets they have and how the underlying SAN infrastructure relates to their Hyper-V environment. Automation helps streamline common storage workflows – provisioning new storage, adding capacity to a Hyper-V or cluster, and rapid provisioning of new VMs using the SAN.

The automation of storage in the context of a Hyper-V environment helps reduce the time to market of new virtual servers, reduce the complexity of using the SAN in a virtualization environment, and take advantage of all the capabilities of the underlying SAN while delivering excellent performance. This level of integration will help customers fully leverage the SAN for their Hyper-V environment.

VMM 2012 uses a new storage service that communicates to SMI-S based storage providers for active management of storage arrays. Leveraging an industry standard like SMI-S helps VMM deliver value-add functionality to customers across a wide range of storage devices while minimizing the engineering effort and time to market.

Overview

Virtualized workloads consume compute, storage, and network resources in a datacenter. Virtualization tends to consume large amounts of storage, requiring sufficient infrastructure to deliver the capacity and performance required. There are two types of storage that VMM 2012 is aware of – local and remote. Local storage is great for low cost virtualization solutions, while remote storage delivers higher levels of capacity and performance while being cost effective. Which type of storage is used depends on the needs of the virtualized workload. Some environments might even deploy both types of storage. For this reason, VMM 2012 can model, deploy, and manage virtualized workloads in a datacenter efficiently and at scale, independent of which storage type is used. With this objective in mind, it is also important for VMM 2012 to help simplify and automate storage operations common in a virtualized environment. Please keep in mind that the storage automation features only apply to Hyper-V.

Storage Types

There are two types of storage infrastructure that gets deployed in a datacenter – local and remote. Local storage represents storage capacity available inside of a server chassis or directly attached to a server. Local storage is typically cheaper to deploy with minimal management required. Workloads deployed on local storage have less stringent requirements on having hardware-based backup, recovery, replication, block copy, de-duplication, or thin provisioning. Using local storage depends more on the operating system and other software to deliver some or all of these services and capabilities.

Remote storage on the other hand, offloads work from the operating system to a storage array. In this case, it’s the “hardware” that delivers advanced capabilities. SAN arrays deliver large amounts of capacity, performance, and centralization of workloads. A SAN array might require additional infrastructure as well. Unfortunately, over the years, the “SAN” has become a black box that virtualization administrators have very little visibility into and minimal influence on how or what gets implemented. This perception will change with VMM 2012.

Benefits to the Customer

Virtualization together with automation makes it easy to deploy virtual machines at scale. With more enterprises focusing on private (on-premise) cloud computing - minimizing human error, lowering costs, and increasing efficiency are critical requirements.

Features

Virtualization together with automation makes it easy to deploy virtual machines at scale. With more enterprises focusing on private (on-premise) cloud computing - minimizing human error, lowering costs, and increasing efficiency are critical requirements from the storage perspective.

• Discovery -Most virtualization administrators have minimal visibility into the underlying storage infrastructure. This is especially true with remote storage, typically seen as a “black box.” VMM 2012 can discover both local and remote storage. The information discovered includes arrays, pools, logical units (this document will refer to LUNs as logical units), disks, volumes, and virtual disks.

• Resource Mapping - With all of the storage assets discovered, VMM 2012 can map a virtual machine to its respective storage, creating a full end-to-end map directly accessible in the administrator console and PowerShell. There is no need to rely on outdated diagrams or additional consoles to have this information.

• Classification - Even with all the storage knowledge available through VMM 2012 to the administrator, it is still necessary to expose a simplified model of storage to the end users who are creating virtual machines through the private cloud. For this, storage can be classified using a friendly descriptive name. The administrator knows the capabilities of the underlying storage. However, the end user only needs to know that the storage is “High Performance”.”

• Allocation - VMM 2012 is aware of all the local storage available on a hypervisor host and the remote storage that the administrator makes VMM 2012 aware of. Unlike local storage which is always available to the host, remote storage first needs to be assigned to a host. VMM 2012 provides administrators with the ability to first allocate available storage to a host group before assigning it to a host. This additional step allows administrators to request storage and allocate it to a business group with or without having to assign it to a host.

• Assignment - When a business group is ready to consume new storage, they simply look up what is allocated to them and assign it to a Hyper-V host. For remote storage, assignment involves operations on the remote storage device and on the host. VMM 2012 automates this process by exposing the storage to the host, initializing the disk, and formatting a new volume. In the cluster case, VMM 2012 will create all the necessary cluster resources and even set up the volume so it is shared across a cluster.

• Provisioning - VMM 2012 can assign existing storage to a host or cluster and provision new storage as a new logical unit. There are three ways to provision new logical units – from available capacity, creating a writeable snapshot of an existing logical unit, or creating a clone of a logical unit.

Creation of a new logical unit from available capacity is very useful when a pool of storage is available to you. This means you have control of how many logical units you create and the size of each logical unit.

Creating a writeable snapshot of an existing logical unit is one way to rapidly create many copies of an existing virtual disk. This allows you to provision many virtual machines in a small amount of time with minimal load on the hosts. Depending on the array, snapshots can be created almost instantaneously and are very space efficient.

Creating a clone of an existing logical unit offloads the work of creating a full copy of a virtual disk to the array. Depending on the array, clones are typically not space efficient and can take some time to create.

Which of these provisioning methods you decide to use depends on the array you have and the virtualization workload you need to deploy. VMM 2012 supports all of these methods to give you the flexibility to choose the best method.

• Decommission  - VMM 2012 can also decommission the storage it creates. This is important to avoid running out of storage capacity over time.

Basic Scenarios

VMM 2012 leverages the storage automation features explained in this document to deliver unique value-add in a private cloud environment. The scenarios described below are supported directly in the VMM 2012 administrator console and PowerShell interface.

• Adding Capacity to Hosts or Clusters - For managed Hyper-V hosts, VMM can assign additional storage to a host or cluster. VMM automates the unmasking and preparation of the volume. In the cluster case, VMM also creates the cluster resources.

• Simple Rapid Provisioning - Storage arrays can create copies of virtual disks very efficiently with minimal load on the host. Using VMM 2012 you can leverage this capability to rapidly create virtual machines. VMM 2012 knows about the capabilities of the storage array. It knows when a logical unit contains a file system and a virtual disk. Through VMM 2012, you can create a template with a virtual disk on a logical unit. Finally, VMM 2012 can instruct the array to create a copy of a virtual disk by provisioning new storage on the array using snapshot or cloning technology. VMM 2012 then takes care of exposing the storage to the host, mounting the file system, and associating the virtual disk to the virtual machine.

In the administrator console, you can create stand-alone virtual machines or service- based virtual machines using rapid provisioning. You can also integrate rapid provisioning into your own provisioning tools using PowerShell.

Advanced Scenarios

VMM 2012 can provision one VM at a time through the administrator console.

• Cluster Creation - VMM 2012 can create a cluster with up to 16 Hyper-V nodes. In addition, VMM 2012 can automate the assignment of cluster shared storage as part of the same workflow. Simplifying the creation of new Hyper-V clusters with shared storage is important in the private cloud. In VMM 2012 building blocks of VM capacity is now possible for a single console.

• Large Scale Rapid Provisioning - Using VMM’s PowerShell interface, customers can go further with rapid provisioning. Hyper-V clusters can be created with storage that is already populated with golden virtual disks. VMM can use these existing virtual disks to create many new VMs concurrently.

Requirements

So what do you need to take advantage of all of these capabilities? There are both hardware and software requirements you need to be aware of.

• Hardware Requirements - The storage automation feature set in VMM 2012 is only available for remote storage. You will need to use one of the following arrays supported for BETA: NetApp FAS, EMC Symmetrix, EMC Clariion CX, HP EVA

• Software Requirements  - VMM 2012 uses the Storage Management Initiative (SMI) standard produced by the Storage Networking Industry Association (SNIA) to deliver the storage automation feature set. The SMI specification (SMI-S) standardizes the storage assets model and the active management operations that you can execute against an array. Each storage vendor is required to create a provider that implements SMI-S for their arrays. These providers are the ones that VMM 2012 interacts with to automate storage operations. Therefore, a supported storage provider must be installed on a server that can communicate with the VMM server. VMM 2012 requires that the provider implement version 1.4 of the SMI specification. http://go.microsoft.com/fwlink/?LinkID=209605.

Windows Server 8 introduces a new WMI-based API called the Storage Management API (SMAPI) and corresponding set of PowerShell Cmdlets.  These provide storage management primitives to manage direct attach storage on the OS as well as external storage arrays.  The PowerShell Cmdlets replace tools like diskpart and diskraid.  The API is comprised of a WMI object model along with the corresponding set of methods and properties.   Storage partners plug into the new API either by:

1.Implementing the SNIA (Storage Networking Industry Association) industry standard for storage management called SMI-S (Storage Management Initiative –Specification) 

2.Implementing a new provider model called the Storage Management Provider (SMP)

Future versions of SCVMM will use the SMAPI and be able to use both SMI-S and SMP providers.

For more information on standards based storage management, check out read Jeffrey Snover’s blog post: http://blogs.technet.com/b/server-cloud/archive/2011/10/14/windows-server-8-standards-based-storage-management.aspx