Disaster Recovery – not a nightmare with virtualization

Disaster Recovery – not a nightmare with virtualization

  • Comments 6
  • Likes

Hello all, my name is Chris Steffen and I am the Principal Technical Architect at Kroll Factual Data. 


Kroll Factual Data is a company that provides business information to mortgage lenders, consumer lenders, property management firms and other businesses.  We have been a part of the Microsoft Technology Adoption Program for virtualization products for the past several years, and we have about 1,500 Virtual Machines (VM) across our computing environment.


As companies experiment with server virtualization, one of the benefits that is seemingly overlooked (or not emphasized as often as it should be) by those discussing virtualization technologies is the impact that virtualization can have when creating & maintaining a disaster recovery environment.


Put simply, the more that your production environment is virtualized, the easier it is to build a disaster recovery environment using the golden images created for your production systems.


For years, I had worried about my company’s ability to recover from a catastrophic disaster.  But over the last 18 months, we have utilized Microsoft Virtual Server to create the situation that we are in today:


We have a production data center environment that is 85% virtualized.  All of the golden images are replicated to our Disaster Recovery (DR) site (the golden image store is constantly updated to the DR site, and the updated / patched / modified golden images are automatically copied to DR as soon as they are changed).  We have standing virtual machine hosts awaiting a VM image (these machines are running and used for testing).  Our Internet circuits and vendor connections are “hot” and tested constantly.  Databases and all other non-VM machines are already duplicated and maintained at DR (we’re talking a couple dozen machines, not the 600 that are located in our production environment today).


Were we to experience a major disaster, the IT staff at our DR site would immediately start the process of copying VM images to selected hosts.  We would change our circuit routing and externally hosted Domain Name Service to the DR IP range.  After the VMs have been copied to the host machines, we would start to bring up the “new” data center.  Our testing has shown that this can realistically be completed in less than a day, but we have a 24 hour Recovery Time Objective (RTO).


Our DR site is populated by our hand-me-down equipment.  But with server virtualization, this is not a problem.  Sure, you may not be able to run 20 VMs on a single host machine due to the older hardware configs, but even if you can only run a quarter of that number, you are gaining additional life out of “end of life” equipment, and getting your production systems up and running in pretty much the same condition that they were running in your real production environment.


I still worry about business continuity situations (who doesn’t?).  But the DR plan that we have established for our core technology finally lets me rest a little bit better.  And using the Microsoft virtualization software and their System Center management tools has made the entire disaster recovery project more cost effective, not only from a licensing and equipment side, but from a manpower / administrative perspective. As you embrace a virtualization technology, keep disaster recovery in the back of your mind as another solution to your DR maintenance nightmare.


I welcome your questions or comments. 




Your comment has been posted.   Close
Thank you, your comment requires moderation so it may take a while to appear.   Close
Leave a Comment
  • PingBack from http://projectmanagementsoftware.reviewedhere.info/2008/04/03/disaster-recovery-%e2%80%93-not-a-nightmare-with-virtualization/

  • Question submitted over the weekend: "Would you mind explaining how you make these Golden images?  Are you just using Trumbull's script to suspend VMs then VSS it then resume the VMs?"

    Chris' reply:

    "In regards to making golden images, the process is relatively straight forward (more on that in a moment).  

    With that background, when we are developing a program for the first time, we provision a clean / fresh VM for creation in our development environment.  If we are updating an existing program, we will often use the existing production golden image and begin development at this point.

    The VM purpose will determine the starting configuration of Windows Server 2003 that we install (we have a pre-existing image of an IIS server, for example, that is already hardened and correctly patched).  The application is developed on the VM, and the same VM is promoted through dev, then to QA and finally into production.  At the time the VM is ready to be released to production, the VM is added to the golden image store as the most recent and updated release.  As each application usually requires more than a single instance running in production, all instances of the VM are copied from the golden image VM store.  This is also the point when the new golden image is replicated to the disaster recovery environment.

    I would also add that when we initially adopted virtualization (this was over 5 years ago now, so the circumstances have changed a bit), we had to decide the best methodology for managing the individual VMs in our environment.  We realized almost immediately that the simplified provisioning process using Microsoft Virtual Server could create a situation where rouge VMs could be provisioned with next to no effort (this was a specific issue in our QA and development environments) especially since management tools such as SCVMM did not yet exist.

    Therefore, we made a decision to treat all VMs just as we would treat every other physical machine in our environment:  as a separate and wholly different instance of the operating system, at least from a provisioning and management point of view."

  • Greetings! Chris Steffen here again from Kroll Factual Data. I want to share some thoughts on what I

  • Kevin Fogarty: One of the major drawbacks of Microsoft 's virtualization pitch is the lack of good

  • Hi, I’m Mike Neil, general manager of virtualization at Microsoft. It’s been a while since I’ve blogged here, but today’s post is worth a read. Ever since we released Windows Server 2008 Hyper-V and Terminal Services, System Center Virtual Machine Manager