HPC, Virtualization and Random Thoughts

Posts
  • Turning hyper-v on and off

    I use hyper-v on my laptop. When I know I don't need VMs for the day, I can squeeze a bit more performance out of the machine by turning hyper-v off with: bcdedit /set hypervisorlaunchtype off and a reboot. To turn it back on: bcdedit /set hypervisorlaunchtype...
  • Azure Connect - a vpn between Azure and your machines

    Azure Connect is a service that lets you establish a vpn tunnel (for want of a better word) between a virtual machine running in Azure and another running on premises. The connection is point-to-point, meaning that you will need to configure it on every...
  • Windows Server 2008 High-Availability Clusters

    It is no secret that HPC Server 2008 will offer the option to make the head node of a HPC cluster highly available. This feature is not in beta 1, but it is being developed. It will exploit fail-over mechanisms provided by Server 2008 (enterprise edition...
  • Python and Azure VMs

    I have been experimenting lately with the Azure SDK for Python, in particular with the service management API. I found some points in the documentation that were unclear to me, so I am posting what I discovered in the process. If you want to create...
  • Put a VM on Azure

    I have summarized here all the steps you need to take in order to deploy an Azure VM. Step 1: Get your certificates I assume that you have an active Azure subscription and you have installed visual studio 2010, the azure sdk and tools and activated...
  • Generic Resource Scheduler for Azure

    GeReS (Generic Resource Scheduler) for Windows Azure is now available as a beta release on Codeplex . It is a simple batch job manager written in C# (or Python for the older version). Geres provides: Command line utilities (e.g. qsub, qlist...
  • Live Migration, Cluster Shared Volumes & Networks

    The recommendation for people setting up live migration clusters is to isolate different kinds of traffic on their own networks: Public network to access the cluster and the virtual machines running on it “Private” cluster heartbeat network “live migration...
  • Cloudblitz: A Tool to Deploy HPC Clusters in Azure

    I'm happy to announce the availability of Cloudblitz, a tool (framework) to deploy HPC clusters in Azure programmatically via Powershell, then submit jobs to them via Powershell as well. You can find it at http://cloudblitz.codeplex.com The tool...
  • How Many Cores for the Job?

    That is quite a common question. Experienced systems engineers have accumulated knowledge over the years that they distill into a few rules of thumb, e.g.: Given a certain hardware configuration, for software package A with an input size of X, on...
  • Using CloudBlitz to Submit Jobs

    In the second post about Cloudblitz ( http://cloudblitz.codeplex.com ) we'll examine how to use it to submit jobs to a deployed Azure cluster. Keep in mind that in Microsoft's HPC scheduler implementation, jobs are just containers. They contain one...
  • The Hyper-V Cloud - no clusters?

    Microsoft has recently published a set of guides to build your own private cloud solution using Hyper-V, System Center Virtual Machine Manager and its Self-Service Portal 2.0 They cover planning, deployment and operations. You can find them here. ...
  • Faking Networks

    On a Windows HPC Server 2008 head node, that is... 1. No Infiniband on the head node In many cases people want to save themselves some money by not installing an Infiniband adapter on the head node, thereby also sparing a port on that expensive...
  • VHD Boot

    With Windows 7 and Server 2008 R2 you get the opportunity to boot directly from a vhd file. The operating system in the vhd file will have direct access to the machine hardware. It will not run as a virtual machine with synthetic or emulated adapters...
  • Gaming and HPC?

    Did you think that High Performance Computing was boring? Well, think again: we spent most of our time last week on a videogame :-) At our last HPC Lab we had CCP Games, maker (and hoster) of Eve-Online (http://www.eve-online.com/), a massive multi-player...
  • Patches and KB articles for Hyper-v R2

    I'd like to share with you some tips to mitigate a few frustrating intermittent problems with hyper-v and system center. 1. Data Protection Manager 2010 BEFORE installing DPM agents on systems running Hyper-V R2, you must install hotfixes KB975921...
  • Stop Climate Change?! – Part 2

    I have been investigating some more in the area of Green IT, S+S. Some ideas and a lot of questions have come to mind. Please read on and let me know if they make any sense. By the way, Part 1 is here :-) 1. How do you understand the status quo...
  • Accelerating Excel by Parallelization

    Note that Excel 2007 is able to use multiple threads to parallelize calculations on the workstation. Previous versions were not. When you hit “recalculate”, Excel will identify independent areas in your spreadsheet and allocate computations in those to...
  • SQL Server in Windows Azure

    It is certainly possible to run SQL Server 2008 R2 in Azure virtual machines, but keep in mind that they are not persistent between deployments, hence you want to use them for testing only and be aware of potential data loss. In order to mitigate that...
  • A Free Book on Microsoft Virtualization

    Understanding Microsoft Virtualization Solutions from Microsoft Press is available as a FREE download.   This 15MB E-Book gives an overview of all current Microsoft Virtualization technologies: Hyper-V, the Microsoft Enterprise Desktop Virtualization...
  • Endpoints, firewalls and other annoyances

    When you deploy a HPC cluster on Azure, you typically want to run some application in it besides those provided in the azure samples. Those applications may require their own ports to be opened on the internal network and endpoints to be established for...
  • P2V with SCVMM – a quick tip

    System Center Virtual Machine Manager (SCVMM) has been offering a relatively simple way of doing physical-to-virtual migrations (P2V) for a while. You just click on the “Convert Physical Server” icon and off you go. Despite the name, it also works with...
  • Dual-boot Linux and HPC Server 2008

    In the past few weeks I had the pleasure of working with UGS, a division of Siemens that develops engineering product lifecycle management solutions. They were looking for a way to dual-boot an existing Linux cluster into Windows. The process we came...
  • Azure billing clarification

    A quick clarification: Azure allocates resources to your role as long as it is deployed, whether it is running or not. Billing starts when the deployment is complete and finishes when it is deleted. Roles that are deployed will be billed for the deployed...
  • Proxies and Compute Nodes

    You’ve prepared your templates, configured your network, your firewalls and everything you could think of, yet your automated provisioning takes forever and eventually fails… Well, check if you have a patching task in your node template. If you do, you...
  • Getting data and applications to / from Azure

    After Wenming’s post on deploying an azure hpc cluster with powershell, I have been looking for an easy way to transfer applications & data to / from such cluster. I have found that in HPC SP3 the hpcpack command includes the option to upload and...