The Microsoft HPC & Batch Team Blog

The blog for the Big Compute team in Microsoft Azure, working on HPC & Batch services and technologies

What the heck is a “Job Template” and why should I care?

What the heck is a “Job Template” and why should I care?

  • Comments 1
  • Likes

Job Templates are one of the most whiz-bang new features in HPC Server 2008, but we've gotten a lot of feedback from the community that you don't really know what they're for or how to use them. So I figured a quick post here might help solve that problem by introducing you to one of the most powerful features of the v2 Job Scheduler.

 

What is a Job Template?

Simply put, a job template is a custom submission policy configured by the admin. Admins can create a number of different job templates and then let users pick the one that is right for them and their job (assuming they have the necessary permissions).

 

Job Templates vs. Job Queues

In many ways, HPC Server job templates are the same the queues found in other scheduling products (like Platform's LSF), in that they allow you to:

  • Partition the cluster
  • Give different permissions to different jobs and users
  • Provide handling for different types of jobs

That being said, job templates aren't queues because of the simple fact that in the end, all jobs submitted to the system end up in the same queue. We think that's a great design, because there is a single place to view all of the jobs, and a single, ordered queue of all the jobs waiting to execute.

 

How Job Templates Work

Job templates work by allowing administrators to provide Defaults and Constraints to every job that comes into the system. They are also ACL'd, meaning that Administrators can control which sets of users can submits which types of jobs.

This diagram quickly explains how Job Templates are applied to a job:

Figure 1: The Scheduler Validates a Job Using a Job Template

 

How to Use Job Templates

The easiest way to explain how to use job templates is to make up a scenario and then show you how to enforce the policies required by that scenario. So let's do just that . . .

 

Say you have two groups of users:

  • Paying Customers- These are the guys who paid for the cluster to be installed, and they get nearly unlimited rights to use the cluster as they see fit.
  • Freeloaders- These are other employees at your company. They are allowed to use the cluster, but only in limited amounts and only if they don't get in the Paying Customers way.

 

Let's go ahead and create two templates, one for each group, with the necessary settings and permissions to make everything work out nicely.

 

Step 1: Create a template for the Paying Customers group

First, let's head into the product and great a template for the "Paying Customers:"

  • Click on "Configuration" in the lower left of the HPC Cluster Manager, and then select "Job Templates" in the Navigation Pane.
  • Now select "New" in the Action Pane to create a new Job Template.
  • On the Welcome page, set the template name to "Paying Customers Template."
  • Accept the defaults on the Job Run Times page, since these guys should be able to do whatever they want.
  • On the Job Priorities page, set the Maximum priority to Maximum; this will allow paying customers to submit jobs with any priority that they'd like.
  • Accept the defaults on the Project Names and Node Groups pages.
  • Click finish to complete the wizard and create this job template.

 

Step 2: Create a template for the Freeloaders groups

1.    Click on "New" again to create another new Job Template.

2.    Enter the name "Freeloaders" on the Welcome page.

3.    On the Job Run Times page, enter a maximum run time of 1 hour to prevent Freeloaders from submitting long-running jobs.

4.    On the Job Priorities page, set a default priority of Lowest and a Maximum Priority of Below Normal. This will ensure that Freeloader's jobs always get a nice low priority . . . so Paying Customers will pass them in the queue and even pre-empt them if the pre-emption scheduling policy is enabled.

5.    Accept the defaults for Project Names and Node Groups.

6.    Click finish to complete the wizard and create this job template.

 

Step 3: Set permissions

The final step is to set the appropriate permissions so that no one can use job templates that they shouldn't.

  • In the job templates view, highlight the "Default" template and say "Set Permissions."
    • Remove the "Users" group from this ACL so that no users can use the Default template any more.
  • Now highlight the "Paying Customers" template and say "Set Permissions."
    • Remove the "Users" group from this ACL.
    • Add the Paying Customers group (you can create a new local group and manage the users in it by doing to Computer -> Right-click -> Manage -> Users and Groups from the start menu), and give them the Submit Job permission.
  • Now highlight the "Freeloaders" template and say "Set Permissions."
    • Remove the "Users" group from this ACL.
    • Add the Freeloaders group (you can create a new local group and manage the users in it by doing to Computer -> Right-click -> Manage -> Users and Groups from the start menu), and give them the Submit Job permission.

 

Step 4: Profit

You're actually done. At this point, members of your local "Freeloaders" group can only submit using the Freeloaders template. This means they can't submit jobs with a priority above Below Normal, and they can't submit jobs that run for more than 1 hour. Sucks to be them, huh? Meanwhile members of the "Paying Customers" group can pretty much do whatever they want. Members of both groups can use either job template.

 

Step 5: Getting more advanced

There are actually many more advanced things that you can do with Job Templates, since they can be used to default and constrain any job property. For an example, let's say you wanted to change things up by saying that Freeloaders couldn't submit jobs which required Exclusive use of a node. This can easily be done!

1.    Highlight Freeloaders in the job template view and click "Edit."

2.    Click the Add button to add a constraint, and select the Exclusive constraint.

3.    Highlight the Exclusive constraint in the Job Template Details window to show the settings for this Job Property.

1.    In the Details for the Exclusive constraint, set the Default Value to "False," and set the Valid Values to include only "False."

4.    Hit Save.

 

 

Figure 2: Editing a Job Template

Now when Freeloaders submit a job, it will always be marked as non-exclusive (because Exclusive is False by default). If they try to mark it as exclusive, job submission will fail (since True isn't in the list of valid values for the Exclusive property).

 

Comments
Your comment has been posted.   Close
Thank you, your comment requires moderation so it may take a while to appear.   Close
Leave a Comment