The Windows HPC Team Blog

HPC Pack SOA Tutorial V: Diagnose a SOA Service

Yumin Guo[MSFT] — Thu, 14 Mar 2013 09:00:00 GMT

It’s not an easy job to write an HPC application with complex algorithms, but it’s even more difficult to troubleshoot the application. This tutorial is to introduce how to diagnose the SOA application.

Get SOA logs

In HPC Pack 2012, it is much easier to get the SOA-related logs from HPC Cluster Manager. To enable SOA logging, in Configuration -> Services, click Set Event Logging Level to set the log level. See the following figure.

Then you can try to call the service. After the SOA job finishes, you can right-click on the last job executed, then click Export SOA Trace to export all the SOA-related logs. Note that these actions are only applicable to SOA jobs.

Export SOA Trace exports a zip file which contains several .csv files and .evtx files. The .csv files have all the SOA-related logs of all the compute nodes which ran the SOA tasks. Each .csv file corresponds to one compute node. The .evtx files have more log information generated by HPC Pack to help further diagnostics.

Writing to a log

The next question is how to write logs in your own SOA service. HPC Pack 2012 provides a logging framework. See the accompanying code sample to follow these steps to use the framework.

In a SOA service you can use code like the following to add your own log, and the log can then be accessed from HPC Cluster Manager.

ServiceContext.Logger.TraceInformation("Get a request to factorize {0}", n);

In above sample, the log level is Information. So you need to make sure the log level of the service is set to information or verbose. You can use HPC Cluster Manager to set the level, as we did previously, or you can change the service configuration file to set the log level.

<microsoft.Hpc.Session.ServiceRegistration>

<serviceassembly="\\head.contoso\SOAServices\Diagnostic.PrimeFactorization.dll"

stdError=""soaDiagTraceLevel="Information, ActivityTracing" />

</microsoft.Hpc.Session.ServiceRegistration>

Now invoke the service again. To view the log, click View Message Details dialog for the job.

In the dialog box that appears you can see all the requests. Click one of the requests, and you can see your log information.

If you want to export the log files, as was shown previously, the .csv files will contain all your service log data.

Debug

HPC Pack provides a Visual Studio plug-in to help debug the service. With that you can attach to all the compute nodes for debugging. For more information, see Microsoft HPC Debugger Add-ins for Visual Studio.

HPC Pack SOA Tutorial IV – Common Data

Yumin Guo[MSFT] — Thu, 14 Mar 2013 08:39:00 GMT

With multiple tasks running on an HPC cluster, often, many tasks work with a shared data set. For example, a stock risk analysis program can run a large number of simulations against a set of historical stock market data. In this example, each SOA request can have different parameters, but all analyses use the same historical data. In HPC, the shared data set is called common data.

Common data must be transferred to services running on compute nodes. Since the data is static, it will be inefficient to transfer the data within each SOA request. A good solution is to send the data set to the cluster and store it in a centralized place, where all services have access to. In this blog post, we’ll see how to do it. (Note: You could also store this data in a database that that HPC cluster has access to.)

We will still be using our prime factorization example (see this blog post for details). To accelerate the algorithm, we want to employ a prime number table, so that we can look up the table for prime factors instead of traversing all numbers. Obviously, a prime number table should be common data among all factorization requests. We’ll follow these steps to handle it:

1. Implement a data manager

a. Send common data

b. Release common data

2. Implement the service and get common data into the service

3. Implement the client

4. Test the common data service

We also discuss how to configure common data storage.

See the accompanying code sample to follow the steps in this article.

1. Implement a data manager

First, we create a data manager to handle the common data’s lifecycle.

First, generate a prime number table as our common data. The implementation of CreatePrimeNumberTable is not relevant to our topic, so we won’t go into detail here.

//create prime number table of 200000 prime numbers

List<int> PrimeNumberTable = CreatePrimeNumberTable(200000);

Common data is managed by the DataClient type in HPC. To create a data client, we should prepare the following information.

conststring headnode = "head.contoso.com";

string dataId = "PRIME_NUMBER_TABLE";

The data ID is used to identity each data client, so it must be a unique value.

Create a data client:

//create DataClient to send data

DataClient dataClient = DataClient.Create(headnode, dataId);

If a data client with same ID already exists, an exception will be thrown. In this case, if the existing common data is still needed, we’ll have to change our data ID; otherwise, just delete it as follows:

//delete the data client

DataClient.Delete(headnode, dataId);

Send the common data to the service by simply invoking WriteAll method.

//Send data to service.

//WriteAll() can only be called once on a DataClient object

dataClient.WriteAll<List<int>>(PrimeNumberTable);

We use WriteAll to send structured data to the cluster. Any serializable data can be sent this way.

Also, raw data can be sent by invoking WriteRawBytesAll method. For example, if we get prime numbers directly from a file PrimeNumbers, we can send data like this:

//WriteAllRawBytesAll can be called only once per data client.

dataClient.WriteRawBytesAll(File.ReadAllBytes("PrimeNumbers"));

However, WriteAll or WriteRawBytesAll can be called only once per data client. On each data client, only one “write” operation is allowed, but multiple “read” operations can be performed.

Typically, we send common data in the client and read them in the services. In this scenario, the common data is read-only to services.

By now, the prime number table has been sent to the HPC cluster, and all the service requests can access it by providing the correct data ID. The data will remain in the cluster until it is explicitly deleted.

2. Implement the service and get common data into the service

When implementing our service, the service contract remains the same.

As mentioned before, we can get the data client through the data ID, and get common data by invoking the ReadAll method.

List<int>PrimeNumberTable;

using (DataClient dataClient = ServiceContext.GetDataClient("PRIME_NUMBER_TABLE"))

{

PrimeNumberTable = dataClient.ReadAll<List<int>>();

}

Or use the ReadRawBytesAll method to send raw data.

byte[] PrimeNumberTableRaw;

using(DataClient dataClient =

ServiceContext.GetDataClient("PRIME_NUMBER_TABLE"))

{

PrimeNumberTableRaw = dataClient.ReadRawBytesAll();

}

To avoid reading common data into memory every time a new service object is created, reading it in a static constructor is a good practice.

With the prime number table, factorization can be implemented in a quicker way like this.

public List<int>Factorize(intn)

{

List<int> factors = newList<int>();

//When factors are in PrimeNumberTable

for (inti = 0; i < PrimeNumberTable.Count; )

{

if(n % i == 0)

{

factors.Add(i);

n /= i;

}

else

{

i++;

}

//When factors are not in PrimeNumberTable

for (inti = PrimeNumberTable.Max() + 1; i <= n; )

{

if(n % i == 0)

{

factors.Add(i);

n /= i;

}

else

{

i++;

}

return factors;

}

3. Implement the client

We’ll create a simple client just to test the service.

//Change headnode here

const string headnode = "head.contoso.com";

conststring serviceName = "PrimeFactorizationWithCommonData";

SessionStartInfo info = newSessionStartInfo(headnode, serviceName);

Random random = newRandom();

try

{

//create an interactive session

using (Session session = Session.CreateSession(info))

{

Console.WriteLine("Session {0} has been created", session.Id);

using (BrokerClient<IPrimeFactorization>client = newBrokerClient<IPrimeFactorization>(session))

{

//send request

int num = random.Next(1, Int32.MaxValue);

FactorizeRequest request = newFactorizeRequest(num);

client.SendRequest<FactorizeRequest>(request, num);

client.EndRequests();

//get response

foreach (BrokerResponse<FactorizeResponse>response in client.GetResponses<FactorizeResponse>())

{

int number = response.GetUserData<int>();

int[] factors = response.Result.FactorizeResult;

Console.WriteLine("{0} = {1}", number,

string.Join<int>(" * ", factors));

}

session.Close();

Console.WriteLine("done");

Console.WriteLine("Press any key to exit");

Console.ReadKey();

}

catch (System.Exceptionex)

{

Console.WriteLine(ex.Message);

}

4. Test the common data service

Run DataManager.exe to send common data to the cluster.

Now start Client.exe. We can see that the service runs correctly.

5. Configure common data storage

Common data is stored in a shared folder which can be accessed by all compute nodes. This path is determined by the environment variable HPC_RUNTIMESHARE. You can check it by using HPC Pack command line tool cluscfg.

Type cluscfg listenvs in a command window, and you’ll see a list of environment variables. By default, the value of HPC_RUNTIMESHARE is \\COMPUTE_NAME\Runtime$. This is a shared path, which is mapped to C:\HPCRuntimeDirectory on the head node.

If you need to, you can change the value of HPC_RUNTIMESHARE by using cluscfg setenvs. Ensure that all compute nodes have read access to this path, and clients should have write access if they need to send common data.

For more information, see Configuring the Runtime Data Share.

HPC Pack SOA Tutorial III – Interactive mode

Yumin Guo[MSFT] — Thu, 14 Mar 2013 08:11:00 GMT

In tutorial II we discussed how to deal with a time-consuming service by employing a durable session. However, batch mode is not the only computation mode in the HPC world. Some computations can be finished within a few seconds to a few minutes. The end user may expect almost real-time response.

That mode brings different challenges when compared to batch mode. Response time is more critical. Therefore the session startup overhead cannot be ignored. In a typical HPC cluster, it takes a few seconds to start a new session. If there are other jobs running on the cluster ,the newly created session must wait until the resources are available, which makes the startup time much longer. Fortunately, HPC Pack provides a way to deal with this situation and cut down the unnecessary cost.

See the accompanying code sample to follow the steps in this article.

Implement the service

We use the same service as in tutorial II – the prime factorization service. To satisfy the real-time requirements, we’ll just pass small numbers to the service.

Here’s the service contract:

[ServiceContract]

publicinterfaceIPrimeFactorization

{

[OperationContract]

List<int>Factorize(intn);

}

And here’s the service implementation:

public List<int>Factorize(intn)

{

List<int> factors = newList<int>();

for (inti = 2; n > 1; )

{

if(n % i == 0)

{

factors.Add(i);

n /= i;

}

else

{

i++;

}

return factors;

}

Implement the client

To save the time of starting new job, the client must reuse the existing session instead of creating a new one, because creating a new session means starting a new job. To reuse the existing session, we need to create the session as follows:

const string headnode = "head.contoso.com";

conststring serviceName = "PrimeFactorizationService";

SessionStartInfo info = newSessionStartInfo(headnode, serviceName);

//Enable session pool

info.ShareSession = true;

info.UseSessionPool = true;

You may notice that there are two new properties of SessionStartInfo being assigned here.

Setting ShareSession to true means that any user can send requests to the broker, not just the one that creates the session.

Setting UseSessionPool to true ensures that the every new client uses the existing session instead of creating another one. The session pool is maintained on the server side - it guarantees that when a client connects to the same service with the flag set to true, it always returns the same session as long as it’s still alive.

Now we can create the session. We don’t want to use a durable session because it may affect the performance.

//create an interactive session

using (Session session = Session.CreateSession(info))

{

Console.WriteLine("Session {0} has been created", session.Id);

…

}

Create a broker client to send requests and get responses.

In the case of the previous code, we now have a situation where there can be many broker clients in a single session. In this case, we should assign a unique ID to the client.

//in one session, each broker client should have a unique id

string ClientId = Guid.NewGuid().ToString();

using (BrokerClient<IPrimeFactorization> client = newBrokerClient<IPrimeFactorization>(ClientId, session))

{

Console.WriteLine("BrokerClient {0} has been created", ClientId);

Random random = new Random();

int num = random.Next(1, Int32.MaxValue);

//Send request

FactorizeRequest request = newFactorizeRequest(num);

client.SendRequest<FactorizeRequest>(request, num);

client.EndRequests();

//Get response

foreach (BrokerResponse<FactorizeResponse> response in client.GetResponses<FactorizeResponse>())

{

int number = response.GetUserData<int>();

int[] factors = response.Result.FactorizeResult;

Console.WriteLine("{0} = {1}", number, string.Join<int>(" * ", factors));

}

Now run the client twice. You’ll see that the clients share the same session ID, as a result of the enabled session pool. Also, the first client runs much longer than then second one, which indicates that the new client reuses the created session.

Since GetResponses is a synchronous function, the client will be blocked and kept waiting for the results. This is not a welcome situation in a real-time system, so let’s try another way to get responses.

We can set an asynchronous callback for the client by using SetResponseHandler as follows:

//use this event sync main thread and callback

AutoResetEvent done = newAutoResetEvent(false);

//set callback function. this handler will be invoke before service replies.

client.SetResponseHandler<FactorizeResponse>((response) =>

{

int number = response.GetUserData<int>();

int[] factors = response.Result.FactorizeResult;

Console.WriteLine("{0} = {1}", number,

string.Join<int>(" * ", factors));

//release the lock

done.Set();

});

Thus after normally sending requests, the client can continue with other work. When the responses are ready, the response handler will be called to display the results.

Deploy and test the service

You may follow this blog post to deploy and test the case step by step.

You can run multiple clients. The output will be like this:

Notice that all the clients share the same session ID.

Cluster autogrow and autoshrink

A common situation of the interactive mode is to run a long-running service serving multiple clients. To respond to each request as quickly as possible, we should keep the session alive. But, on the other hand, having a SOA job occupy a large number of resources during off-peak times is wasteful.

HPC Pack has a feature to grow and shrink resources based on the number of requests. If there are no requests it will shirnk the number of resources to the minimum number specified by the job. When receiving requests, it will autogrow the resources to handle them.

Note: A session will time out if there’s no client connecting to it for a period of time. To make a session a long running service, you can change the SessionIdleTimeout when starting the session.

HPC Pack SOA Tutorial II – Batch mode

Yumin Guo[MSFT] — Thu, 14 Mar 2013 07:34:00 GMT

In the last blog post, we implemented a simple SOA client and service. However in most cases, the HPC algorithm is not as simple as adding two numbers. Some services are more complex and run for hours. For such services, the end user normally submits all the requests, then retrieves the responses in a couple of hours. This represents a challenge for the developers. For example, can we close the client application after sending all the requests, and then start another client to get the responses the next day? This blog post is to guide you to design the application to handle such scenarios.

The key requirement of the batch mode is to make sure the requests won’t be lost after being successfully sent, and the result generated by the service won’t be lost either.

Let’s create a new service to explain how to create such reliable computations. We create a service which can do prime factorization. Then, we will write a client application to submit 100 big numbers to the service, and then start another client application to retrieve the results.

Implement the service

Download the accompanying code sample to build the service.

It’s as simple as the first example.

Here is the service contract:

[ServiceContract]

publicinterfaceIPrimeFactorization

{

[OperationContract]

List<int> Factorize(intn);

}

Here is the service implementation:

public List<int>Factorize(int n)

{

List<int> factors = newList<int>();

for (inti = 2; n > 1; )

{

if(n % i == 0)

{

factors.Add(i);

n /= i;

}

else

{

i++;

}

returnfactors;

}

Implement the client to send requests

HPC Pack provides a feature called DurableSession. It makes sure all the requests and the responses are persisted.

Let’s write a client application to send the requests.

a. First we need to prepare the session info. This is the same as in the previous tutorial.

SessionStartInfo info = new SessionStartInfo("head.contoso.com", "PrimeFactorizationService");

b. Now we need to create a DurableSession object, like the following code.

//Create a durable session

DurableSession session = DurableSession.CreateSession(info);

Console.WriteLine("Session {0} has been created", session.Id);

c. Create a BrokerClient object to send requests.

//Send batch request

Random random = newRandom();

constintnumRequests = 100;

using (BrokerClient<IPrimeFactorization>client = newBrokerClient<IPrimeFactorization>(session))

{

Console.WriteLine("Sending {0} requests...", numRequests);

for (inti = 0; i < numRequests; i++)

{

int number = random.Next(1, Int32.MaxValue);

FactorizeRequest request = newFactorizeRequest(number);

//The second param is used to identify each request.

//It can be retrieved from the response.

client.SendRequest<FactorizeRequest>(request, number);

}

client.EndRequests();

Console.WriteLine("All the {0} requests have been sent", numRequests);

}

Here we call EndRequests() to indicate that we have submitted all the requests. After calling EndRequests() you cannot use this client object to send more requests.

Now we can finish this application. Because we used a durable session, it’s fine to close the client. All the requests are still on service side and will be calculated by the compute nodes in the cluster.

Monitor the progress

In HPC Cluster Manager, we can monitor the progress of the requests we just submitted. Go to Job Management to find your job, and click View Job. You can see how many requests have been processed.

Implement the client to retrieve responses

After a couple of hours, the user can come back and retrieve the results. (In our case you don’t have to wait that long.)

a. Prepare the session info. Note that you need the session ID of the previous session to help the client to attach the session.

//Input sessionId here

intsessionId;

Console.Write("Input the session id : ");

sessionId = Int32.Parse(Console.ReadLine());

//Change the headnode name here

SessionAttachInfo info = newSessionAttachInfo("head.contoso.com", sessionId);

b. Instead of creating a new session, attach to the existing session.

//Attach to session

DurableSession session = DurableSession.AttachSession(info);

c. Get responses from the session.

//Get responses

using (BrokerClient<IPrimeFactorization> client = newBrokerClient<IPrimeFactorization>(session))

{

foreach (BrokerResponse<FactorizeResponse>response in client.GetResponses<FactorizeResponse>())

{

int number = response.GetUserData<int>();

int[] factors = response.Result.FactorizeResult;

Console.WriteLine("{0} = {1}", number, string.Join<int>(" * ", factors));

}

d. Close the session.

Tips:

- Because of persistence of requests and responses, the performance of the durable session is slightly slower than that of the interactive session.

- session.Close(true) purges all the persisted data related to this session. It should only be called when the session and its data are not needed any more. If the session will be used going forward, you should call session.close(false).

- If EndRequests() is not called, the service will keep waiting for the upcoming request. After a period of time (by default, 5 mins), it will remove the client if it does not receive more requests. All the requests sent by this client will be removed.

HPC Pack SOA Tutorial I – Write your first SOA service and client

Yumin Guo[MSFT] — Thu, 14 Mar 2013 07:03:00 GMT

This series of blog posts introduces the SOA programming model for Microsoft HPC Pack 2012. This is the first blog post of the series and provides guidance to write your first SOA service and client. See the accompanying code sample to follow the steps in this article.

Note: Most of the general considerations for SOA for HPC Pack 2012 also apply to earlier versions of HPC Pack.

What is a SOA service?

Service-oriented architecture (SOA) is an approach to building distributed, loosely coupled systems.

With SOA, distinct computational functions are packaged as software modules called services. Unlike traditional HPC applications, an HPC SOA service exposes its functionality through a well-defined service interface which allows any application or process on the network to access the functionality of the service. Developers can implement service functionality by using any programming language, and can write original services or package existing dynamic-link libraries (DLLs) and IT investments as services.

Why SOA services?

HPC applications are typically classified as “message intensive” or “embarrassingly parallel.” Message-intensive applications comprise sequential tasks. Embarrassingly parallel applications can be easily divided into large numbers of parallel tasks, with no dependency or communication between them.

SOA services are designed to solve the embarrassingly parallel problem. Compared with traditional HPC computation models, SOA services provide the following advantages:

- Easier programming model: The interface of the SOA service is just a remote method. It makes the client application able to easily access the service.

- Easier message exchange: Traditional HPC jobs require dealing with input and output files. SOA services allow client applications to easily transfer data between the client application and service.

See http://www.microsoft.com/en-us/download/details.aspx?id=11540 for a technical overview of cluster-enabled SOA application development.

Getting started

To write a SOA program to run on an HPC Pack 2012 cluster, you need the following software:

HPC Pack 2012 Client Utilities

HPC Pack 2012 SDK

.NET 4.0 or later

To test your service you need to set up an HPC Pack Cluster to run your service. You can download the HPC Pack 2012 from here.

Now we can start writing the SOA application. Let’s say we need a service running on the HPC cluster to do math calculations. Then we need a client program for end users, because they cannot directly access the cluster. The client program submits the calculation request to the service and returns the result to the end users. To simplify the code, let’s just use a simple addition calculation as an example.

Here are the steps to write a SOA application:

- Step 1: Implement the service

- Step 2: Deploy the service

- Step 3: Implement the client

- Step 4: Test the service

Step 1: Implement the service

Following code can be found in the CalculatorService project in the sample code.

A SOA service is a Windows Communication Foundation (WCF) service running on the HPC cluster. The SOA service is ideal for writing interactive, embarrassingly parallel applications, especially for calculation of complex algorithms.

A SOA service does not require too much extra effort beyond your algorithm. It’s just a standard WCF service. The first step is to define the service contract, like the following code:

[ServiceContract]

publicinterfaceICalculator

{

[OperationContract]

double Add(double a, doubleb);

}

The only extra code is to add the attribute, which can be recognized by WCF, of the interface and its methods. With the service contract the client can understand how to invoke the method and what parameters are required.

The next step is to implement the algorithm of the service:

public class CalculatorService : ICalculator

{

publicdouble Add(double a, doubleb)

{

returna + b;

}

Step 2: Deploy the service

We need to deploy the service we just created to the HPC cluster. Before we deploy the service, we need one more step which is creating the SOA service configuration file. Create an XML file like the following:

<?xmlversion="1.0"encoding="utf-8" ?>

<sectionGroup name="microsoft.Hpc.Session.ServiceRegistration"

type="Microsoft.Hpc.Scheduler.Session.Configuration.ServiceRegistration, Microsoft.Hpc.Scheduler.Session, Version=2.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35">

<section name="service"

type="Microsoft.Hpc.Scheduler.Session.Configuration.ServiceConfiguration, Microsoft.Hpc.Scheduler.Session, Version=2.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"

allowDefinition="Everywhere"

allowExeDefinition="MachineToApplication"

</sectionGroup>

</configSections>

<microsoft.Hpc.Session.ServiceRegistration>

<serviceassembly=”\\fileshare\SOAServices\CalculatorService.dll”/>

</microsoft.Hpc.Session.ServiceRegistration>

</configuration>

Note that the name of the XML file must be as same as the service name. In this case, name it CalculatorService.config.

The only configurable part of the above configuration file is the path of the assembly. Change the highlighted part to the path of the DLL file in your cluster. It is recommended to put the DLL file in a shared folder which can be accessed by all compute nodes. This will save you time manually copying the DLL to all the computer nodes.

Now copy the service configuration file to the configuration file folder, which is under %CCP_HOME%\ServiceRegistration on the head node of the cluster. After copying the file, go to HPC Cluster Manager and navigate to Configuration -> Services to confirm that the CalculatorService is there. Notice that the service name is as same as the file name of the configuration file.

Now the next step is to enable HPC Pack to locate the dll file. Copy the DLL file to the folder you specified in the configuration file. (Note: If you specified a local path, you need to copy the DLL file to the corresponding path on all the compute nodes.)

Now you have successfully deployed the service. To verify that the deployment is successful, you can use a diagnostic tool in HPC Cluster Manager console to test the service. Right click CalculatorService, and then select Run SOA Service Loading Diagnostic Test. You can check the diagnostic result. Click Diagnostics, select the test result, and you can see the test result and the detailed information.

A successful diagnostic result looks like this:

Step 3: Implement the client

Following code can be found in the Clientproject in the sample code.

Now we need a client program to submit the calculation request. The client needs to invoke the service we just created remotely, so we need to leverage WCF to generate the proxy class for us. Visual Studio can simplify the whole process by using its service references functionality.

The suggested project organization in Visual Studio is to create different projects for the service and client programs, as shown in the following figure:

We need to generate the proxy class for the client program based on the CalculatorService. Make sure the service project has been successfully built then click Add Service Reference, click Discover, and you should see the service you just created. Select the service and click Advanced.

Important: In the advanced Service Reference Settings dialog box, ensure that Always generate message contracts is selected.

Now we can write the HPC client code:

a. Prepare the session info, which includes the head node address and the service name. Let’s assume the head node host name is head.contoso.com and we are using the CalculatorService.

SessionStartInfo info = new SessionStartInfo(“head.contoso.com”, “CalculatorService”);

b. With the SessionStartInfo object we can create a session to connect to the head node.

using (Session session = Session.CreateSession(info))

{

……

}

c. To be able to send requests and receive responses, you need to create a BrokerClient object.

using (Session session = Session.CreateSession(info))

{

using (BrokerClient<ICalculator>client = new BrokerClient<ICalculator>(session))

{

……

}

d.With the BrokerClient, you can send requests and receive responses.

using (BrokerClient<ICalculator>client = new BrokerClient<ICalculator>(session))

{

//send request

AddRequest request = newAddRequest(1, 2);

client.SendRequest<AddRequest>(request);

client.EndRequests();

//get response

foreach (BrokerResponse<AddResponse>response in client.GetResponses<AddResponse>())

{

doubleresult = response.Result.AddResult;

Console.WriteLine("Add 1 and 2, and we get {0}", result);

}

Step 4: Test the service

Now you can test your service using your client program.

Run the client in the Visual Studio. If everything is working fine, you should see output like the following:

Open HPC Cluster Manager, navigate to Job Management, and you can see the corresponding job.

Congratulations! You have successfully created and run your fisrt SOA service. In our next blog post, we will introduce way of writing a service to handle a batch of client requests.

Updating SOA service for HPC Pack 2012

Don Pattee — Mon, 11 Feb 2013 19:15:00 GMT

If you are running service oriented architecture (SOA) services on an HPC Pack 2008 R2 cluster, the 2012 release of HPC Pack requires the SOA services to be compiled with .NET Framework 4 or a later version. If you have any SOA services compiled with a version of .NET Framework earlier than 4, it may not be able to run on HPC Pack 2012.

There are two ways to solve the issue:

Recompile the service binary using .NET Framework 4 or a later version.
Change the configuration file of HPCServiceHost to make it support a previous version of .NET Framework. Here are the detailed steps:

a. Find the HPCServiceHost.exe file (for 64-bit services) and HPCServiceHost32.exe file (for 32-bit services) on each compute node. Both reside under %CCP_HOME%\bin on each compute node.
b. Create a file, and name it “HPCServiceHost.exe.config” or “HPCServiceHost32.exe.config”. You only need to create one file, depending on whether your compute node is 64-bit or 32-bit.
c. In the new created config file, add the following xml:

<?xml version="1.0" encoding="utf-8" ?>

</startup>

</configuration>

With the change, the legacy SOA services should run on HPC Pack 2012. If you have any questions, head over to the Windows HPC Discussion forums and let us know.

Announcing the release of HPC Pack 2012

Don Pattee — Fri, 14 Dec 2012 08:04:00 GMT

Announcing the immediate availability of the next major version of Windows HPC software: HPC Pack 2012

The fourth 'major version' of the HPC software brings improved performance and reliability to the HPC platform, simplifies the installation experience, and adds some great new features such as:

support for Windows Server 2012 (head nodes, compute nodes, broker nodes, and unmanaged server nodes) and Windows 8 (as clients and workstation nodes)
the ability to customize the tuning of MPI operations for improved performance on your hardware
the ability to increase cluster robustness by adding more than 2 head nodes in a failover cluster group
the ability to install a head node on a Windows Azure VM, so an entire cluster can run 'in the cloud'
all features previously split between the 'Express,' 'Enterprise,' and 'Workstation' or 'Cycle Harvesting' editions now available through a single installation
additional features are described in the What’s New in HPC Pack 2012 document

If you are using the Windows Azure HPC Scheduler SDK, this update comes with the same reliability and performance improvements that the HPC Pack is getting, and also adds compatability with the Windows Azure SDK October 2012 / v1.8 release.

Before installing either, take a look at the What’s New in HPC Pack 2012 document and Release Notes over on our Windows HPC Technical Library.

More details and additional documentation will arrive in January, in the mean time head over to the Windows HPC Discussion forums if you have any questions or comments. We'll be happy to hear from you!

Note: The single HPC Pack 2012 download contains all the features previously seperated in the Express, Workstation, Cycle Harvesting, and Enterprise editions. Standalone 'Client Utilities' and 'MS-MPI' packages have also been updated for HPC Pack 2012.

Congrats to our Visual Studio buddies!

Don Pattee — Wed, 12 Sep 2012 20:49:00 GMT

Visual Studio 2012 and .NET 4.5 have launched. The live keynote videos are over, but will be available for on-demand viewing tomorrow on http://www.visualstudiolaunch.com/vs2012vle/Home

You can read about Visual Studio Express 2012 over on the VS team's blog http://blogs.msdn.com/b/visualstudio/archive/2012/09/12/visual-studio-express-2012-for-windows-desktop-is-here.aspx

And you can download Express or trial versions of the other editions from http://www.microsoft.com/visualstudio/eng/downloads

Announcing the HPC Pack 2012 Beta Program

Don Pattee — Thu, 06 Sep 2012 17:45:00 GMT

I am pleased to announce the start of the HPC Pack 2012 Beta Program!

The 4th major version of our HPC software line following the Compute Cluster Pack, HPC Pack 2008, and HPC Pack 2008 R2 releases, the HPC Pack 2012 release brings a batch of new functionality and support for new software.

Windows Server 2012 (including 'Server Core'), the Microsoft .NET Framework 4, Microsoft SQL Server 2012, and Windows 8 are all now supported
A single version of HPC Pack 2012 now includes all the functionality that used to be split across the HPC Pack 2008 Express, Enteprise, Workstations and Cycle Harvesting editions
HPC Pack now supports extending your enterprise network to Windows Azure via 'Project Brooklyn'
MS-MPI has a new 'message compression' feature to reduce the time spent waiting on communications to complete
The job scheduler now allows for dependencies between jobs (in addition to the task dependencies that already existed) to better integrate with your work flow
Windows Azure nodes can be set to automatically mount a VHD, which could contain some static data or application code, to make using HPC Pack's 'burst' functionality even easier
and more... A more complete list of new functionality is available once you join the beta :)

Sign up for the beta program by going to the Microsoft HPC Connect beta site, logging in with your Windows Live ID and filling out a quick survey. (If this is your first Connect-based beta program you'll also need to complete a short set of 'profile' questions.)

Announcing the HPC Pack 2008 R2 Service Pack 4 and Windows Azure HPC Scheduler updates!

Don Pattee — Tue, 12 Jun 2012 21:32:00 GMT

As part of the spring WIndows Azure release, I'm pleased to announce that the HPC team has also released a set of updates:

Service Pack 4 for the HPC Pack 2008 R2 family of software
Our first update to the Windows Azure HPC Scheduler development kit

In addition to rolling up the existing HPC Pack 2008 R2 Service Pack 3-based updates (aka 'patches' or 'qfes'), we have made a number of robustness fixes to the product. Everyone using the HPC Pack should update to SP4, and if you are using our 'Windows Azure Burst' capabilities, you should definitely upgrade to take advantage of the improved reliability.

If you are using the Windows Azure HPC Scheduler SDK, this update comes with the same reliability fixes that the HPC Pack is getting, and also adds compatability with the updated Windows Azure SDK.

Before installing either, take a look at the What’s New in SP4 document and Release Notes over on our Windows HPC Server 2008 R2 Technical Library.

If you haven't already taken a look at the Windows Azure spring release update, hop over to the Windows Azure blog and the video at the Meet Windows Azure site.

And as always, head over to the Windows HPC Discussion forums if you have any questions or comments. We'll be happy to hear from you!

Note: The single SP4 installer applies to all installations - Express, Workstation, Cycle Harvesting, and Enterprise, as well as the standalone 'Client Utilities' and 'MS-MPI' packages. You can download it from the Microsoft Download Center. Installers for the standalone Client Utilities and MS-MPI packages with the service pack already integrated are also available.

If you do not have an HPC Pack 2008 R2 cluster, you can download a free Windows HPC Server 2008 R2 Suite evaluation version. Before you install, you can try out the new Installation Preparation Wizard which can help analyze your environment for common issues and provide some best practice guidance to help ensure an easy HPC cluster setup.