Kenneth Young's Blog

Case Study: Factors of a Successful Score Card Development

TechNet Archive — Fri, 06 Mar 2009 17:49:00 GMT

I discussed in a previous article how HPC helps in both the data mining phase and the production phase of score card system development. HPC helps to derive score card systems with higher predictive power, and HPC provides the infrastructure to deliver real-time response in production.

Now what factors contributes to a successful score card development?

A successful score card development project requires the involvement of many domain experts. A key task to success, which is often overlooked, is the development the project objective. Do I want the new score card system to help me select candidates who can increase their spending limit? Or do I want the new score card system to reduce delinquencies even at the expense of number of cards issued?

From an IT perspective, on the other hand, some key infrastructure questions crucial to project success are as below:

How do I deploy and manage a big cluster?
How do I troubleshoot and identify the root cause when there is an error condition?
How do I develop an operator manual so that an operator with little privilege or domain knowledge can trigger the correct remedial action?
How do I manage the cluster resources so that I have good utlization and there is no bottleneck in the infrastructure? For example, if I run a program on my grid using 512 cores concurrently, can I be sure I will overload my database and bring it down?
How do I configure the cluster so that concurrent users with different service level agreements are all satisfied? How do I collect the performance data to plan for and justify my hardware and budget requirements?

06 Mar 2009, Kenneth Young

HPC in Fraud Detection in the Credit Card Business

TechNet Archive — Fri, 06 Mar 2009 14:00:00 GMT

The number of credit card transactions exceeded 60 billion in 2007, or over 2 thousands transactions per second. This is not really incredible at roughly 10 transactions per head per year for the world as a whole. With this volume of data moving through the grid, how is fraud detection keeping up?

For both internet vendors and brick-and-mortar vendors, authorization of credit card payments needs to complete on a real-time basis. To be effective, the fraud risk score also needs to be available at the same time as the authorization. Even after a delay as short as one minute, the customer will have walked. To mitigate and eliminate fraud proactively, major authorization vendors are delivering the transaction risk score as part of the authorization message.

One solution utilizes a score card system and this is where HPC comes into play. Once a new transaction event arrives in the system, its risk level can be evaluated against the scoring rules. HPC provides the infrastructure support in two phases: firstly, during the data mining phase to derive better scoring rules; secondly, during the production phase to provide the real-time performance.

As early as April 2006, the super-computing centre Los Alamos National Laboratory in New Mexico, USA, published a paper "Iterative Algorithm for Finding Fraudulent Patterns in Transactional Databases".[1][2] The joint-research between the laboratory and Citigroup benchmarked the effectiveness of different algorithms in detecting frauds via pattern detection. The techniques can be expanded to help detect criminal behaviour and terrorist activities.

Credit risk and fraud management systems that include more data types and more analytical models have an edge in protecting against customers and reducing fraud losses. Credit risks and fraud can be reduced at both the account level and the transaction level.

06 Mar 2009, Kenneth Young

[1] www.lanl.gov/orgs/t/publications/research_highlights_2006/docs/RH06_Berman_iterative.pdf

[2] http://arxiv.org/ftp/cs/papers/0508/0508120.pdf

FAQ: How is Windows HPC Server different from Windows Server Standard?

TechNet Archive — Fri, 06 Mar 2009 10:02:00 GMT

Some common questions from the field include "What applications can I run on Windows HPC?" and "How is Windows HPC Server different from Windows Server Standard?".

1) The short answer is Windows HPC Server is the same as Windows Server Standard except the following differences: HPC Server (a) is only available in 64-bit versions; (b) adds the HPC Pack i.e. cluster utilities including the cluster manager, the job queue, the HPC runtimes, etc.; but (c) disables IIS, Sharepoint, Exchange, MSSQL because the special licensing for HPC is for computational workload.

Other features of Windows Server Standard continue to function, such as Active Directory Domain Services, Terminal Services, etc.. to support the administration and operation of your cluster. And third party applications that are certified on Windows Server will run identically on Windows Server HPC unless they require IIS, Exchange, or MSSQL on the same machine.

For applications that need to access a full unrestricted Windows Server Standard or Enterprise Edition, you should purchase the full OS and add a "HPC Pack". For a detailed explanation of the SKU families, please read on.

2) Let us review the different SKUs. There are 4 basic SKU families related to Windows HPC. (Within each family, you may find variations depending on your OS language choice and depending on whether you include software assurance, etc..) These 4 basic SKU families are:

HPC Server
HPC Server OS
HPC Pack
HPC SDK

3) The "HPC Server" SKU family is the recommended SKU because it includes both the OS and HPC utilities. In other words, "HPC Server" = "HPC Server OS" + "HPC Pack".

4) The "HPC Server OS" is identical to "Windows Server OS Standard" only with further restrictions, as explained by the HPC FAQ from http://www.microsoft.com/hpc/en/us/faq.aspx:

Both Windows HPC Server 2008 and Windows Server 2008 HPC Edition are licensed solely for running clustered HPC (high performance computing) applications. Clustered HPC applications solve complex computational problems or a set of closely related computational problems in parallel, typically using several servers as a group (or cluster). You may run the server software for the primary purpose of running clustered HPC applications, and in conjunction with other software only as necessary to permit security, storage, performance enhancement, and systems management of any devices in the cluster when running the other software on a cluster node for the sole and dedicated purpose of supporting the clustered HPC applications.

You may not use the server software for any other purpose. For example, except as permitted in above, you may not use the server software as a general purpose server, database server, Web server, e-mail server, print server, or file server.

The restrictions are implemented in software. In practice, that means if you try to activate the Windows services for IIS, MSSQL, Exchange, you will encounter errors and these services will fail to start. Terminal services will continue to work to support cluster administration, but terminal services gateway will not function because the dependent IIS service will not start. FTP will also fail because it also depends on IIS. Active directory domain services will continue to function to support domain and security administrations.

5) "HPC Pack" includes the actual cluster utilities that make Windows Server a productive cluster platform. It includes the cluster management console, the cluster node management services, and the client-side runtime. In other words, you need this on your cluster head node, and on every compute node that is part of the cluster, as well as on every workstation or desktop which needs the client runtime to connect to the cluster.

For compute nodes, the recommended SKU is to purchase "HPC Server" which has already incorporated "HPC Pack".
For head nodes the "HPC Server" SKU generally suffices. But for head nodes in failover configuration, Windows Server Enterprise is required and you will have to license Windows Server Enterprise plus HPC Pack for both head nodes.
For workstations and desktops requiring the client runtime, HPC Pack needs to be licensed.

6) "HPC SDK" includes the header files and libraries needed to develop applications targeting HPC Server. This is a free download. You need to install this file to compile MSMPI programs, programs that compile against the HPC scheduler.

06 Mar 2009, Kenneth Young