The Windows HPC Team Blog : Announcements

HPC Team is proud to release our Beta 1 of the Windows HPC Server 2008 R2 solution

DPattee — Tue, 17 Nov 2009 00:02:00 GMT

Our first Beta release is now available! You can read the full press release at http://www.microsoft.com/presspass/press/2009/nov09/11-16SC09PR.mspx if you're in to reading that kind of thing ;)

Windows HPC Server 2008 R2 delivers productivity, performance and ease-of-use improvements in several areas, including the following:

Improved scalability, with Windows HPC Server 2008 R2 offering out-of-the-box support for deploying, running and managing clusters up to 1,000 nodes
New configuration and deployment options such as diskless boot, mixed-version clusters and support for a remote head node database
Improved system management, diagnostics and reporting including an enhanced heat map, multiple customizable tabs, an extensible diagnostic framework and the ability to create richer custom reports
Improved support for service-oriented architecture (SOA) workloads including a new fire-and-recollect programming model, finalization hooks, improved Java interoperability, automatic restart and failover of broker nodes, and improved management, monitoring, diagnostics and debugging
Message Passing Interface (MPI) and networking enhancements including optimizations for new processors, enhanced support for RDMA over Ethernet and InfiniBand, improved MPI debugging, and a pushbutton HPC LINPACK optimization wizard
New ways to accelerate Microsoft Office Excel workbooks such as support for Cluster-Aware User-Defined Functions and the capability to run distributed Excel 2010 for the cluster

Come and join our beta program to give it a try, and you can give us feedback (positive or negative) on it: http://connect.microsoft.com/HPC/content/content.aspx?ContentID=6923

Dryad and DryadLINQ on HPC Server

JVert — Wed, 15 Jul 2009 23:04:00 GMT

Microsoft Research has an interesting project called Dryad which is investigating programming models for distributed data-parallel problems. If you've heard some of the hype around map/reduce programming or the Hadoop framework, you might be familiar with these type of problems. If you have gigabytes, terabytes, or even petabytes of data that you need to churn through, this is a powerful tool. On top of the Dryad framework is a programming model called DryadLinq which brings the power and familiarity of .NET's LINQ syntax to cluster computing. You can write your data analytics algorithm as a LINQ expression in your favorite .NET language and submit cluster jobs directly from Visual Studio.

I am delighted to announce that Microsoft Research has released a distribution of Dryad and DryadLINQ that runs on top of HPC Server 2008. You can download this release from here and contribute feedback and suggestions at the Microsoft Connect site here. If you want to learn more about Dryad and DryadLINQ, you can find more information on the Microsoft Research site. The source for DryadLINQ is also included in this release.

Releasing this has been a great collaboration between the HPC team and Microsoft Research. I am excited to get this work out there and see what people can do with it. So try it out and give us some feedback!

John Vert

Architect

Windows HPC Server

Update - Channel9 has a video interview with Erik Meijer and Roger Barga discussing this: http://channel9.msdn.com/posts/Charles/Expert-to-Expert-Erik-Roger-Barga-Introduction-to-Dryad-and-DryadLINQ/

Coming early November - New windowshpc.net web site!

MichaelCole — Wed, 17 Dec 2008 04:07:00 GMT

What will change?

Http://windowshpc.net will be migrated from its existing Community Server platform to Microsoft Office SharePoint Server 2007.

Existing users and community members will need to re-register using Microsoft Live ID .

We are planning to migrate as much of your user information to the new site as possible. Due to privacy regulations we may not be able to move all information. We appreciate your understanding and cooperation in this matter.

Why are you migrating the site?

We are going to implement new features such as RSS feeds with the new site. Users will be able to receive new information through RSS feeds from other Microsoft premises.

Community members will be presented with information based on their registration profile. Non-profile related content will be still searchable and available.

What will the new site look like?

Stay tuned

Windows HPC @ TechEd 2008

PhilPenn — Sat, 31 May 2008 20:31:00 GMT

Heading to TechEd 2008? Discover some of the exciting technologies the HPC Server 2008 team is working on.

Developer Week Sessions

INF301 Application Development for High Performance Computing on the Windows Platform, June 5, 10:15AM - 11:30AM, S230A

INF302 Parallel Computing and HPC Technologies @ Microsoft, Wednesday, June 4 10:15AM - 11:30AM, S220A

ITPro Week Sessions

SVR57-TLC Introducing Windows HPC Server 2008, June 11, 12:00PM-12:45PM

SPC451 Black Belt in Optimizing High Performance Computing Applications for Windows HPC Server 2008, June 12, 1:00PM-2:15PM, N310E

SPC351 Advanced Management and Deployment, June 13, 10:15AM-11:30AM

SPC450 Advanced SOA-Based Scheduling with Windows High Performance Computing Server 2008, June 13, 4:30PM-5:45PM, S310C

Windows HPC Server 2008 April CTP Released

PhilPenn — Tue, 22 Apr 2008 16:33:00 GMT

Please see the latest Community Technology Preview (CTP) release of Windows HPC Server 2008 at the beta program community site (http://Connect.Microsoft.Com). If you aren't yet a registered beta program participant, then you'll first need to join the program using the same web site. This CTP release includes the HPC Pack which may be installed on any x64 edition of Windows Server (an evaluation version is also available for download) as well as the HPC Pack SDK (for developing HPC applications) and the HPC Pack OPK (for OEMs preparing HPC offerings). Be sure to provide feedback on the CTP features using the Feedback Tool on the Connect website. Also, please leverage the HPC MSDN Forums to obtain product support for ITPro's and Developers (http://forums.microsoft.com/WindowsHPC).

Learn About HPC Server 2008 Features

PhilPenn — Tue, 08 Jan 2008 20:54:00 GMT

Please see the new articles published on both TechNet Edge (for ITPro's) and Channel9 (for Developers).

http://channel9.msdn.com/shows/The_HPC_Show

http://edge.technet.com/

MSMPI v1 symbols are available!

erezh — Thu, 29 Nov 2007 02:34:00 GMT

The symbols for MSMPI v1 RTM and SP1 are available on the Microsoft public symbols server @ http://msdl.microsoft.com/download/symbols.

You can access the symbols from the debugger (and other tools) using the symbols server symbols path

e.g., srv*download_cache_folder*http://msdl.microsoft.com/download/symbols

Enjoy
.Erez

Windows HPC Server 2008 JumpStart for ITPro and Developer

PhilPenn — Thu, 15 Nov 2007 23:08:00 GMT

Consider registering for a 1 day Windows HPC Server 2008 immersion event. Designed to introduce new features and tools, attendees will obtain a JumpStart in their preparation for deployment and development of Windows HPC solutions. World-wide dates and venues. Check-back for events in your locale. http://www.hpcpal.com

Chronicles of a Cluster Troubleshooter

FrankChism — Wed, 24 Oct 2007 18:46:00 GMT

Applications Performance Improved with Snoop Filter Off on Intel 5000x Chipset

Chronicles of a Cluster Troubleshooter Volume One #1

Date: October 5, 2007

Issue: Performance of application, Eclipse, on Windows Compute Cluster Server V1 is below that of the same application run on the same hardware on Linux.

Hardware Platform: HP Proliant DL140 cluster with Voltaire InfiniBand interconnect.

Software Platform: Windows Server 2003 Compute Cluster Edition Service Pack 2 with Compute Cluster Service Pack 1.

Application: Schlumberger Eclipse reservoir simulation package.

Initial Conditions: Cluster hardware set up, O/S and Cluster Pack installed. Application performance under Windows does not match that under Linux. Large test case fails when run on all nodes.

Investigation and Remediation:

Install the uSane cluster sanity test suite on the head node.
Run the uSane MpiHi test.
1. Runs on small case.
2. Fails immediately on testing the whole cluster
  1. Mpiexec error message say that it cannot get credentials from compute node 2.
  2. HP engineer notes that this is the same message that the large application generates. He had assumed this was a Windows CCS issue
Pause node 2 then run MpiHi across the whole network. It runs fine.
Replace node two on theory that the problem may be on the private network NIC or possibly the node software. Fresh image on new hardware is guaranteed to fix either issue in one service action. Post mortem shows the NIC was bad.
Using all nodes in the cluster run the uSane tests: cNodeLate, MPI network latency test; cNodeBand, MPI network bandwidth test; and cFlood, MPI switch flood test with all links active at one time.
1. Latencies near 10 microseconds.
2. Bandwidth near 900 MB/s with “–env MPICH_SOCKET_SBUFFER_SIZE 0” on the mpiexec command.
3. Conclude that the network is fine.
HP engineer adds the “–env MPICH_SOCKET_SBUFFER_SIZE 0” to the application mpiexec line. Performance still subpar.
Discussions with another HP engineer who ran the application on Linux, had revealed that she used the Linux utility ‘setpci’ to improve Linux performance. This utility was not used to set anything that was specific to the PCI bus or the InfiniBand HCAs, but to disable the Intel Greencreek 5000X chipset memory bus Snoop Filter feature.
Used the BIOS to disable Snoop Filter.
Reran the application. Performance was now on par with Linux.

Cores Snoop On Snoop Off Improvement

seconds seconds

2 5490 4520 21%

4 5337 3899 37%

8 2944 2241 31%

16 1721 1298 32%

32 2919 2698 8%

Checked with the application developer and discovered that the test case provided was not expected to scale past 16 cores.

For an interesting article on Snoop Filter performance impact see: http://www.dell.com/downloads/global/power/ps3q06-20060362-Radhakrishnan.pdf

Until next time, good shooting.

Frankie