HPC, Virtualization and Random Thoughts

Posts
  • Generic Resource Scheduler for Azure

    GeReS (Generic Resource Scheduler) for Windows Azure is now available as a beta release on Codeplex . It is a simple batch job manager written in C# (or Python for the older version). Geres provides: Command line utilities (e.g. qsub, qlist...
  • Python and Azure VMs

    I have been experimenting lately with the Azure SDK for Python, in particular with the service management API. I found some points in the documentation that were unclear to me, so I am posting what I discovered in the process. If you want to create...
  • Using CloudBlitz to Submit Jobs

    In the second post about Cloudblitz ( http://cloudblitz.codeplex.com ) we'll examine how to use it to submit jobs to a deployed Azure cluster. Keep in mind that in Microsoft's HPC scheduler implementation, jobs are just containers. They contain one...
  • Cloudblitz: A Tool to Deploy HPC Clusters in Azure

    I'm happy to announce the availability of Cloudblitz, a tool (framework) to deploy HPC clusters in Azure programmatically via Powershell, then submit jobs to them via Powershell as well. You can find it at http://cloudblitz.codeplex.com The tool...
  • Endpoints, firewalls and other annoyances

    When you deploy a HPC cluster on Azure, you typically want to run some application in it besides those provided in the azure samples. Those applications may require their own ports to be opened on the internal network and endpoints to be established for...
  • Getting data and applications to / from Azure

    After Wenming’s post on deploying an azure hpc cluster with powershell, I have been looking for an easy way to transfer applications & data to / from such cluster. I have found that in HPC SP3 the hpcpack command includes the option to upload and...
  • SQL Server in Windows Azure

    It is certainly possible to run SQL Server 2008 R2 in Azure virtual machines, but keep in mind that they are not persistent between deployments, hence you want to use them for testing only and be aware of potential data loss. In order to mitigate that...
  • Azure Connect - a vpn between Azure and your machines

    Azure Connect is a service that lets you establish a vpn tunnel (for want of a better word) between a virtual machine running in Azure and another running on premises. The connection is point-to-point, meaning that you will need to configure it on every...
  • Azure billing clarification

    A quick clarification: Azure allocates resources to your role as long as it is deployed, whether it is running or not. Billing starts when the deployment is complete and finishes when it is deleted. Roles that are deployed will be billed for the deployed...
  • Put a VM on Azure

    I have summarized here all the steps you need to take in order to deploy an Azure VM. Step 1: Get your certificates I assume that you have an active Azure subscription and you have installed visual studio 2010, the azure sdk and tools and activated...
  • The Hyper-V Cloud - no clusters?

    Microsoft has recently published a set of guides to build your own private cloud solution using Hyper-V, System Center Virtual Machine Manager and its Self-Service Portal 2.0 They cover planning, deployment and operations. You can find them here. ...
  • Patches and KB articles for Hyper-v R2

    I'd like to share with you some tips to mitigate a few frustrating intermittent problems with hyper-v and system center. 1. Data Protection Manager 2010 BEFORE installing DPM agents on systems running Hyper-V R2, you must install hotfixes KB975921...
  • Live Migration, Cluster Shared Volumes & Networks

    The recommendation for people setting up live migration clusters is to isolate different kinds of traffic on their own networks: Public network to access the cluster and the virtual machines running on it “Private” cluster heartbeat network “live migration...
  • P2V with SCVMM – a quick tip

    System Center Virtual Machine Manager (SCVMM) has been offering a relatively simple way of doing physical-to-virtual migrations (P2V) for a while. You just click on the “Convert Physical Server” icon and off you go. Despite the name, it also works with...
  • VHD Boot

    With Windows 7 and Server 2008 R2 you get the opportunity to boot directly from a vhd file. The operating system in the vhd file will have direct access to the machine hardware. It will not run as a virtual machine with synthetic or emulated adapters...
  • A Free Book on Microsoft Virtualization

    Understanding Microsoft Virtualization Solutions from Microsoft Press is available as a FREE download.   This 15MB E-Book gives an overview of all current Microsoft Virtualization technologies: Hyper-V, the Microsoft Enterprise Desktop Virtualization...
  • Updated Infiniband on Server 2008 Paper

    I have finally updated my notes on the installation of Infiniband on Windows Server 2008. They now cover the released version 2.0 of Mellanox WinOF stack. You can find the document in my skydrive public folder . Let me know if you find it useful. ...
  • Faking Networks

    On a Windows HPC Server 2008 head node, that is... 1. No Infiniband on the head node In many cases people want to save themselves some money by not installing an Infiniband adapter on the head node, thereby also sparing a port on that expensive...
  • Live Migration in R2

    I've got a lot of questions about Live Migration in 2008 R2. Rather than writing a long post on it, I thought I'd point you at some resources I found useful whilst setting up my test environment, so you can build one too: - Frank Cicalese's (virtualization...
  • Turning hyper-v on and off

    I use hyper-v on my laptop. When I know I don't need VMs for the day, I can squeeze a bit more performance out of the machine by turning hyper-v off with: bcdedit /set hypervisorlaunchtype off and a reboot. To turn it back on: bcdedit /set hypervisorlaunchtype...
  • What is new in virtualization with Windows Server 2008 R2?

    There are some quite interesting improvements in Windows Server 2008 R2 (what was wrong with W7 as a name?) that help us progress toward a dynamic infrastructure. Three of them are worthy of highlighting: live migration of virtual machines in hyper-v...
  • Upgrading from an evaluation version

    I have received a few questions about upgrades from the evaluation version that you can download from microsoft.com/hpc to a full version. The good news is that the evaluation version is fully functional, so you won't need a complete re-installation....
  • Proxies and Compute Nodes

    You’ve prepared your templates, configured your network, your firewalls and everything you could think of, yet your automated provisioning takes forever and eventually fails… Well, check if you have a patching task in your node template. If you do, you...
  • Upgrading to HPC Server 2008 RC1

    Well, there is no upgrade path, so the quickest way is to re-image. Download the RC build of the HPC software from connect.microsoft.com. If you have Infiniband cards, download the latest WinIB-ND drivers (1.4.0.2577) from http://www.mellanox.com.   ...
  • Stop Climate Change?! – Part 2

    I have been investigating some more in the area of Green IT, S+S. Some ideas and a lot of questions have come to mind. Please read on and let me know if they make any sense. By the way, Part 1 is here :-) 1. How do you understand the status quo...