The Microsoft HPC & Batch Team Blog

The blog for the Big Compute team in Microsoft Azure, working on HPC & Batch services and technologies

Browse by Tags

Related Posts
  • Blog Post: HPC Pack 2012 R2 Update 1 and MS-MPI v5 are now available

    We are pleased to announce that the official releases of Update 1 for HPC Pack 2012 R2, and the newest version of Microsoft MPI (MS-MPI) are now available for download. The installation package for HPC Pack 2012 R2 Update 1 is available for download here . As always, HPC Pack includes the latest version...
  • Blog Post: Azure Batch Preview

    I am excited to announce the preview of Azure Batch , a new Azure platform service that provides job scheduling and large-scale resource management for developing and delivering Big Compute applications native in the cloud. What is Batch? Azure Batch originated from seeing our customers repeatedly...
  • Blog Post: Creating Submission and Activation Filters

    There were a lot of questions about how to use Activation and Submission Filters to help customize queue managmeent and do things like license-aware scheduling. That, on top of some changes made in a QFE , led us to do an updated doc on using filters. You can check it out here: http://technet.microsoft...
  • Blog Post: Issue with HPC Event Logs with HPC Pack SP1

    Some customers have reported issues with the HPC Job Scheduler's event logs after installing Service Pack 1 for the HPC Pack. After some internal investigation, it looks like there is an issue with the SP1 Patch Installer which fails to correctly update the event log manifests (making it so you don't...
  • Blog Post: What the heck is a “Job Template” and why should I care?

    Job Templates are one of the most whiz-bang new features in HPC Server 2008, but we've gotten a lot of feedback from the community that you don't really know what they're for or how to use them. So I figured a quick post here might help solve that problem by introducing you to one of the most powerful...
  • Blog Post: Welcome to the Job Scheduling Blog!

    Not to be outdone by some of the other parts of the Windows HPC Server product, we wanted to start a blog to get you information on the Windows HPC Server job scheduler, including details on: What's new in HPC Server 2008 How to use different interfaces to the scheduler How the various job...
  • Blog Post: Integrating Windows HPC Server 2008 with Linux

    We find many of our Windows HPC Server 2008 deployments are going into environments where there are existing Linux (and Linux HPC) solutions. It is possible to configure these two environments to achieve integrated authentication, file sharing and job submission. The following technical documents...
  • Blog Post: How to set environment variables on computes nodes

    For ISV application you have to set some environments variables on the compute node. You can set this variable by hand, but if you have many nodes it can be long. So, the idea of this post is how can set them attaumaticaly ? Use the HPCS tools : You can set cluster wilde variable by using cluscgf...
  • Blog Post: Submitting and Managing Jobs with Windows PowerShell

    For me, one of the most exciting new features in Windows HPC Server 2008 is our Windows PowerShell interface. It's not only incredibly powerful, it's quite easy to learn! And it's very useful for posting demos and examples up here. So I figured it made pretty good sense to have my first real posting...
  • Blog Post: Making a Clean Sweep with Windows HPC Server 2008

    Parameter Sweeps are one of the most common types of jobs that get run on HPC clusters, and we've done some work in Windows HPC Server 2008 to make them easier (and faster) than ever. Today I'll dig in a little on what's different and how to take advantage of these new features. For those familiar...
  • Blog Post: MPI Process Placement with Windows HPC Server 2008

    We get an awful lot of questions about how to go about getting the desired process placement across nodes in an MPI job (just check our forums if you don’t believe me), so I thought I’d post here to shed some light on the things that are possible. Requesting Resources The first thing you need...
  • Blog Post: HPC Basic Profile Web Service

    Do you want to access the HPCS 2008 Job Scheduler from other environments such as Java, Linux, etc? Or build Job Submission tools around a standard web services based interface? Well, in the latest Community Technology Preview (CTP) released this week we have included a new feature – the HPC Basic...
  • Blog Post: How to Submit a Job

    I was asked today where there was a quick explanation of how to submit a job from the command line, and I realized that I didn't have a really good answer. So I figured I'd do a blog post (first in a while, I know!) on how to do this. Starting Simple Let's say your have an application called...
  • Blog Post: 7 Minute Job Scheduling Thrill-Ride

    Shahrokh was nice enough to film and post a video of me walking through the HPC Job Scheduler and some of its new features at SuperComputing 2008. You can check out the video up on YouTube . In the video I cover many of the different scheduler interfaces and policies, so it's a great primer for those...
  • Blog Post: Process Affinity and Windows HPC Server 2008 SP1

    An issue came to our attention after Windows HPC Server 2008 shipped regarding the way we set affinity on the processes within a job. There are actually two places where we set affinity: 1. The Node Manager (the service on each node responsible for starting jobs and tasks) sets processor affinity...
  • Blog Post: Run Batch Workload on a Mixed Infrastructure (Windows Azure Worker Nodes & On-Premise HPC Server 2008 R2 Compute Nodes)

    With the introduction of SP1 of HPC Server 2008 R2 it is possible to run workload on Windows Azure. If you want to be able to off-burst your batch application (= extending the infrastructure from classic on-premise servers to cloud based Azure worker nodes) you need to set specific environment variables...
  • Blog Post: Submit job from SUA (Subsystem for Unix based Application)

    SUA is a very nice tool for porting Unix/Linux application, but it can also be used to submit job. You just have to set the path to the HPCS binary into your PATH variable : With BASH : export PATH=$PATH:"/dev/fs/C/Program Files/Microsoft HPC Pack/Bin" With Korn Shell : set PATH $PATH:"/dev/fs...
  • Blog Post: a powershell problem

    We've gotten reports of an installation glitch that makes it so that node management cmdlets won't work when run from a remote client (they'll work just fine run on the headnode) due to a registry key that doesn't always get set up correctly. Fortunately, there is a simple work around! If you are trying...
  • Blog Post: Job Template White Paper is Now Available

    We've posted a white paper on how Job Templates can help you manage your cluster. You can check out the white paper here: https://windowshpc.net/Blogs/jobscheduler
  • Blog Post: Clusrunning with Windows HPC Server 2008

    One of our most popular features in the Compute Cluster Pack was clusrun (known to you GUI users as “Remote Command Execution”), which allowed you to run a command line command across a set of cluster nodes in parallel, with their output piped back to you on the client. Not content to rest on our laurels...
  • Blog Post: Dryad and DryadLINQ on HPC Server

    Microsoft Research has an interesting project called Dryad which is investigating programming models for distributed data-parallel problems. If you've heard some of the hype around map/reduce programming or the Hadoop framework, you might be familiar with these type of problems. If you have gigabytes...
  • Blog Post: How that Node/Socket/Core thing works

    This week, I’d like to take some time to explain how a new feature, Multi Level Resource Allocation, can help you get the most out of your applications. The basic explanation for this feature is that when creating a job, you can choose at what granularity your job gets scheduled. This is as simple...
  • Blog Post: Troubleshooting Windows HPC WCF/SOA Issues

    HPC uses HPC sessions to support the service-oriented architecture (SOA) programming model based on Windows Communication Foundation (WCF). Sometimes troubleshooting errors from this SOA based applications could be challenging. However this tip I'm about to share should be helpful to figure out exactly...