The Applied Games Group Blog : Development

TrueSkill in F#

apg — Mon, 16 Jun 2008 07:25:00 GMT

Following the publication of the F# source code for the TrueSkill Through Time paper, we have used the same inference library code to demonstrate how the original TrueSkill algorithm can be coded. In the attached source code, only the program.fs file is different to the TrueSkill through Time blog post and it builds up the factor graph for an arbitrary N-player game with an arbitrary draw probability. Both these numbers can be entered on the command line when running the resulting sample. We tried to make sure to stay as close as possible to the description on page 3 of the TrueSkill technical report.

let priorMu = 25.0<SkillPoints>

let priorSigma = priorMu / 3.0

let priorSkills = Array.init noPlayers (fun i -> Gaussian.Create (priorMu, priorSigma))

let (posteriorSkills, logZ) =

NPlayerTrueSkillUpdate (0.5*priorSigma, 0.01*priorSigma, drawProb / 100.0) priorSkills

Note that one can no longer accidentally pass the variance instead of the standard deviation because the former being of type float<SkillPoints^2>. Try yourself!

Update 2: We had some feedback from Andrew Kennedy that we exploited a current "bug" in the units-of-measure code where literals (such as 1.0) can have generic units. We removed the incorrect usages and re-posted the ZIP file.

Update 3: We extended the example to work with draws. Thanks to Chih-Jen Lin for the suggestion!

Ralf Herbrich & Thore Graepel

Statically link with ILMerge

apg — Thu, 29 May 2008 21:49:00 GMT

Today Ralf & I were looking at ways to bundle up a bunch of .Net assemblies into a single assembly for use within our group; the goal to expose only the functionality we want and only need to reference one assembly in our projects. Enter command line app ILMerge by Michael Barnett of Microsoft Research which lets you merge a set of dlls into a single assembly and selectively hide functionality by changing public access modifiers to internal. The first assembly you specify on the command line is the primary one and it's types maintain their access modifiers, the subsequent assembles are the ones that can be hidden by using ILMerge's ExcludeFile parameter where you can specify a file which in turn contains a list of regular expressions for the types you would like to keep public. When you merge the assemblies if you have also written XML code documentation files these get merged too, so you even get to keep the intellisense tooltips on the classes and functions you have documented with /// comments. For C# projects you can specify XML documentation output in the project's Visual Studio build properties, and for F# projects you can use the command line -doc filename option. To quickly check if we were getting the desired results in our target assembly we used the ILdasm tool to walk through our types. While checking the output we noticed that by default in the current F#, triple slash XML comments get embedded in the assembly for use in Visual Studio for tooltips; but if you are already generating seperate XML documentation it is possible to turn off this feature with the --no-interface-data command line option. So in summary ILMerge appears to be a great, highly customisable tool that lets you distribute a single assembly exposing just the functionality you want to expose (just like linking good old .Lib files in C++ :)).

Phil

BabyJack Inline Video Card Game

apg — Mon, 26 May 2008 13:27:00 GMT

Its been yet another wet and windy bank holiday weekend here in the UK, so I decided to involve the family and create a video Blackjack called BabyJack, despite the baby's name being Sean not Jack :). Just press the Play button below to pit your wits against the baby...

For those of you interested in how it was done; the video was shot on a simple digital camera, and then sepia and film age effects were applied in Vista's Movie Maker, with sound generated by my eldest son Thomas playing piano in the background. The stills were created using Microsoft Paint and the Chiller font. The video itself was embedded in the page using the Windows Media Player Control; and the interaction was coded in Javascript using trusty Notepad.

Phil Trelford

Lunar Lander Retro Vector XNA 2.0 Game sample written in F#

apg — Mon, 12 May 2008 00:36:00 GMT

While travelling to Microsoft HQ in Redmond earlier in the year for the annual TechFest event - for some fun I decided to have a go at writing a small retro game in F# using XNA 2.0, and attached are the results. When moving over to Pacific time I usually find myself waking up at 4am for the first few days, leaving me with plenty of wide awake hours I wouldn't normally have to while away, and this is when the sample was born. I decided on an implementation of Lunar Lander basically because it is pretty simple both logically and graphically, and at the same time fun and achievable in a relatively short time. The sample lets you play with a game pad or keyboard ('Z' and 'X' to rotate, Space to thrust), and to complete the game you must land on the long flat section before you run out of fuel and hit the jagged terrain.

For those interested, the source is attached to this post, which comprises a Visual Studio 2005 solution, a C# XNA 2.0 shell project to take advantage of the content pipeline for audio, an F# game implementation project and a small utility project to create terrain vectors from points in an image file. Please feel free to download and play with it, in theory if you have XNA 2.0 and F# installed then you should be able to just load the solution and press F5 to build and run.

Phillip Trelford

TrueSkill Through Time

apg — Sat, 05 Apr 2008 15:59:00 GMT

In December 2007, we published a paper about the application of TrueSkill to all recorded data of professional and semi-professional Chess play from 1850 to 2006. The dataset can be obtained from ChessBase. We are very happy to be able to release the F# source code that performed all these massive computations. A few things to bear in mind when running this code:

The full factor graph has ~30,000,000 factors and ~25,000,000 variables. At this size, the analysis is among the largest applications of (approximate) Bayesian inference to date. Nevertheless, the full message passing schedule allocates no more than 11 GB of memory and is run in under 20 minutes on an Intel Pentium 4 processor. These numbers hold for the adaptive draw margin model run on 3.5 million games.
Due to the huge amount of results (>1,200,000 skills of all players in all their active years) we use SQL server to store the result set. Please let us know if you think a purely text based result output is needed and we will add it.
There is a very interesting, related paper by Prof. Mark Glickman which analyses subset of this data (88 top players of all time).

The attached ZIP file contains the entire source code which should compile without problems with F# 1.9.3.14. You can either load the solution in Visual Studio 2008 or simply run build.bat form the command line - in either case you will end up with a chessanalysis.exe program or an analysis.exe program. If you want to try out the application, we included the first 1,000 lines of match outcomes (this is part of the first 5 years of data we have).

As for running the code, here are the parameter switches of the command line application:

        -no-safe                          Does not save results predictions (default: off)
        -q                                    Verbosity level off; only works on single runs (default: on)
        -N <int>                           First N games only (default: ALL)
        -delta <float>                  Maximum deviation in any marginal for convergence (default: 0.01)
        -muS <float>                   Mean of prior skill belief (default: 1200)
        -sigmaS <float>              Standard deviation of prior skill belief (default: 400)
        -muD <float>                  Mean of prior draw margin belief (default: 300)
        -sigmaD <float>              Standard deviation of prior draw margin belief (default: 100)
        -beta <float>                 Standard deviation of performance distr. (default: 600)
        -tauS <float>                   Standard deviation of skill dynamics distr. (default: 40)
        -tauD <float>                   Standard deviation of draw margin dynamics distr. (default: 10)
        -server <string>               SQL server name for output (default: 'camresapga01')
        -prefix <string>                Table name prefix for output (default: 'Result')
        -db <string>                     Name of the database for output (default: 'ChessBase')
        -mf                                  Fixed draw margin (default)
        -mf2                                 Fixed draw margin with two factors for draw
        -maf                                 Fixed draw margin with ADF (iterate per year)
        -maf2                              Fixed draw margin with pure ADF
        -mv                                  Variable draw margin
        -as                                   Single run (default)
        -am                                  Model selection

So, a good first test is analysis.exe -no-safe SmallChessBase.csv.

We are very interested to hear your feedback - both on the model, our paper and the F# code.

Update: Following the release of the F# September 2008 CTP we have updated the source code to work with F# 1.9.6.2. Also, we added one of the new features of F#, namely units-of-measure. Note that one can no longer accidentally pass the variance instead of the standard deviation beceause the former being of type float<ELOPoints^2>. We have also added full debug support so simply press F5 to experiment with the -no-safe SmallChessBase.csv option mentioned above.

Update: We removed a reduant schedule creation in the function FixedDrawMarginADFAnalyse. Thanks to Henry (Xinhua) Zhang for pointing out this mistake!

Ralf Herbrich & Thore Graepel

F# Custom Exceptions

apg — Fri, 18 Jan 2008 00:07:00 GMT

This post tries to explore exception handling in F# with custom exception types.

Some background on .Net exceptions

Exceptions happen: "Programs must be able to uniformly handle errors that occur during execution. The common language runtime greatly assists the design of fault-tolerant software by providing a model for notifying programs of errors in a uniform way. All .NET Framework operations indicate failure by throwing exceptions." - .NET Framework Developer's Guide: Handling and Throwing Exceptions.
Deal with them: "To build successful and flexible applications that can be maintained and supported easily, you must adopt an appropriate strategy for exception management. You must design your system to ensure that it is capable of the following:
- Detecting exceptions.
- Logging and reporting information.
- Generating events that can be monitored externally to assist system operation."
- Microsoft patterns & practices: Exception Management Architecture Guide.
But: "Do not rely on exceptions in your code. Exceptions can cause performance to suffer significantly, so you should avoid using them as a way to control normal program flow. If it is possible to detect in code a condition that would cause an exception, do so rather than catching the exception itself and handling the condition." - Microsoft TechNet: Developing High-Performance ASP.NET Applications.
And: "In most cases, use the predefined exceptions types. Define new exception types only for programmatic scenarios. Introduce a new exception class to enable a programmer to take a different action in code based on the exception class." - .Net Framework Developers Guide - Best Practices for Handling Exceptions.

Quick guide to F# exception handling keywords

F#	C# equivalent	Description
raise exception	throw exception;	Throws the specified exception.
rethrow ()	throw;	Rethrows the current exception unchanged from a catch block.
try ... with	try-catch	Filters exceptions.
try ... finally	try-finally	Guarantees execution of specified final block.
failwith string		Throws a Failure exception with the specified string.

F#'s predifined library exceptions

Exception	Usage	Description
Failure of string	failwith "Bang!"	General failure.
InvalidArgument of string	invalid_arg "arg1"	Invalid argument exception.
exn	raise (new exn("Bang!"))	System.Exception type abbreviation.

Filtering Failures

Exceptions are caught/filtered using pattern matching and a Failure exception can be matched like a discriminated union type:

    try failwith "Too many errors"
    with | Failure s -> printf "%s" s

Outputs: "Too many errors"

Some reusable System.Exception based classes

Frequently it is possible to simply reuse existing exception classes in your code:

Exception class	Thrown when...
ApplicationException	a non-fatal application error occurs.
ArgumentNullException	a null reference is passed to a method that does not accept it.
ArgumentOutOfRangeException	the value of an argument is outside the allowable range of values.
FormatException	the format of an argument does not meet the parameter specifications.
InvalidOperationException	a method call is invalid for the object's current state.
NotImplementedException	a requested method or operation is not implemented.

Custom F# exceptions

In some cases we have a pragmatic reason to introduce our own custom exceptions, for example if we define our own data layer we may want to have an exception type for this layer. In the same way the .Net framework defines the SqlException class for SQL specific exceptions. F# Custom exceptions provide an extremely concise form definition like discriminated unions which makes them simarly easy to pattern match against:

Defining a custom F# exception
```
exception AppException of string
```
Raising a custom F# exception
```
raise (AppException "Bang!")
```

Catching a custom F# exception

try raise (AppException "Bang!") with | AppException s -> printf "%s" s

Outputs: "Bang!"

Reading the Message property value of a custom F# exception
```
try raise (AppException "Bang!") with | e -> printf "%s" e.Message
```
Outputs: "Exn+AppException[System.String]..."

So this all seems to make F# custom exceptions great for F# projects which are self-contained or which have F# projects as final consumers. However F# custom exceptions current concise construction syntax does not allow us to pass the values for the Message and InnerException properties to the underlying System.Exception. It is possible to override the System.Exception.Message property but then again we lose a lot of our original conciseness. So I think this means then that F# custom exceptions are probably not suitable if you are going to be throwing an exception out of your F# project to say a C# project or to a vanilla exception logging mechanism.

An Alternative - custom exceptions using plain old inheritance

You will see below custom exceptions using inheritance are also pretty concise for definition and for pattern matching and also allow you to easily initialize the System.Exception's Message and InnerException properties. Here we make good use of F#'s constructed classes, optional parameters and inheritance facilities:

Defining an F# custom exception using inheritance

type VanillaException (message:string, ?innerException:exn) =
    inherit ApplicationException (message, 
        match innerException with | Some(ex) -> ex | _ -> null)

Pattern matching exception types in F#

try raise (new VanillaException("Vanilla!")) 
with 
| Failure s -> printf "%s" s
| :? VanillaException as e -> printf "%s" e.Message
| e -> e.Message

Outputs: "Vanilla!"

So it appears then that this approach is more suited to F# originated frameworks and libraries that expose exceptions to other languages and/or logging frameworks. Finally I would recommend that in this case that you catch any F# custom exceptions at the application boundary including Failure exceptions and rethrow them as vanilla .Net exceptions.

Phil Trelford

Further reading check out: Expert F# Chapter 4 Introducing Imperative Programming.

Silicon Minds Challenge (+ submission sample)

apg — Sun, 23 Dec 2007 22:47:00 GMT

On the 8th December 2007 the Silicon Minds challenge was launched at the Machine Learning and Games workshop of the NIPS conference in the picturesque Canadian ski resort of Whistler.

The challenge is to push the limits of Artificial Intelligence while building a game with XNA Game Studio 2.0.
The main prize an opportunity to interview for an internship with Microsoft Research Cambridge, Rare Ltd or Lionhead Studios.
The submission entry period lasts until the 27th January 2008.
Full details of the competition are available at the DreamBuildPlay website: http://www.dreambuildplay.com

The contest is to be judged on 3 criteria:

Game A.I. (60%) – Game A.I. will be evaluated based on how it serves the game design, and how it contributes to the game experience. Novelty and originality of the AI is appreciated but not required: a novel use of an existing state-of-the AI method is in itself considered to be innovative. General applicability of the AI concepts/techniques is also valued.
Fun Factor (20%) – The fun factor will be evaluated based on how the game design creates a positive user experience. This may include how intellectually challenging, relaxing, stimulating or satisfying the game is. A key indicator for the fun factor will be the desire to keep playing.
Production Quality (20%) – Make your game world as polished as possible and hook your judges with exciting, entertaining action. Production quality will be evaluated based on how seamless the overall game play is, the quality of the assets used, and the structure, readability and level of documentation of the code.

The submission must include the following:

Your finished game in .ccgame XNA Creators Club Game Package form.
The source code and content of your game.
A design summary.
Three screenshots of your finished game.

To this end over the last week we have been having fun creating a mini-sample submission attached based on the board game Go demonstrating:

Using Monte Carlo methods to score the board - thanks to David Stern.
Some well documented source - thanks to Ralf Herbrich.
A sample design summary - thanks to Thore Graepel.
Reused code and graphics - thanks to the XNA Minjie board game sample.

Disclaimer: The mini-game sample itself actually uses a rule based AI and should not be seen as an example of AI or game quality - more a sample of documentation quality and the submission files required.

What we do hope is that you will find the sample a useful reference on how to document your code (specifically the XGoLibrary project), how to comment code you have taken from other sources and how to write the design summary.

On coding standards, if you are starting afresh for the competition we suggest you take a look at the MSDN section ".Net Framework General Reference - Design Guidelines for Class Library Developers" for guidance. In summary this is what we would like to see in the code:

Readability chosen over raw performance.
Short understandable classes and functions.
A consistent naming convention.
Good C# XML comments.

Don't forget though 60% of the score is based on Game AI... hope you enjoy the contest - we are really looking forward to seeing your submitted games.

Last but not least a quick special thanks to Joaquin Quiñonero Candela for working so hard organising the challenge.

Update 1: We have noticed that the PDF still had some review comments; they are removed now. Also, rendering during the scoring was accidently reset to constant black (rather than an alpha value proportional to the expected propability of the territory outcome). Finally, we slightly refined the passing algorithm of the AI: If the human player passes then the AI uses the Monte Carlo scorer to estimate whether or not each piece is already determined in colour up to a probability of at least 80%. If so, the AI passes already.

Update 2: We have noticed a couple of further bugs and easy improvements that can be made: The rule-based AI would start to fill in own eyes if the eye is either on the side of the board or in one of the four corners. This was not intended and has been fixe now. Also, the Monte Carlo sampling can be sped up significantly by only checking for move validity and "eye-ness" after a random move from the list of empty vertices has been determined. Checking a vertex for being empty is a lot faster than checking if a move is valid on that vertex.

Commercial Users of Functional Programming (CUFP) 2007

apg — Fri, 26 Oct 2007 00:09:00 GMT

At the beginning of this month (October) I attended and presented at the CUFP event in Freiburg, Germany. The talk was on some of the work our group has been doing over the last year using F# including the XBox 360 title Halo 3 and on predicting ad clicks for Windows Live. The slides (thanks Ralf) are now available online and for those who couldn't make it videos of all the talks are expected soon. There were plenty of interesting talks throughout the day, including Chris Waterson of Liveops "... using continuation passing monads to encapsulate computation state and hide the complexity of the non-blocking I/O layer" with OCaml, which seemed to be in a similar direction to F# asynchronous workflows. The day ended with a discussion led by Don Syme on the subject of hiring functional programmers; where it emerged that the remaining attendees were actively seeking over 40 new hires. Finally it was time to enjoy the remainder of a hot sunny day and some quality German beer.

Phil

Composing a video game in F#

apg — Thu, 25 Oct 2007 22:33:00 GMT

This week it was my opportunity to present a topic for 15 minutes at the MLP group's ritual Tuesday afternoon tea and cakes session (yes - we are based in the UK). I thought it might be fun to knock up a little game with the Microsoft functional programming language F#. With only a few hours to spare at the weekend I decided to go with a retro style game and an old favourite of mine - the light cycle game sequence featured in the 80s movie Tron. So after about 4 hours I had a 2 player game in Windows Forms with optional support for XBox 360 controllers with managed DirectX all in under 200 lines of F#; and come tea time I even had some players. If you fancy having a go yourself the code can be viewed on the F# Wiki.

Have fun,

Phil

Schedule Tasks in Windows XP

apg — Tue, 07 Nov 2006 11:54:00 GMT

We have an F# console app which processes some gigabytes of data over a couple of hours. We wanted to run this on a specific machine daily in the early hours of the morning. The easy answer Windows XP's Scheduled Tasks. Just click

Start > All Programs > Accessories > System Tools > Scheduled Tasks.

We also wanted a simple install mechanism for the script, dependencies and to schedule the task. Managing tasks can be automated from the command line with Schtasks.exe or a COM interface (see this CodeProject article). The command line option was taken for simplicity. A scheduled task could be created and deleted like this:

> Schtasks.exe /Create /TN MorningDewey /TR "c:\windows\explorer.exe http://www.msdewey.com" /SC DAILY /ST 09:00:00
> Schtasks.exe /Delete /TN MorningDewey

When you create the task you may be prompted for your account password. If your task is command line only then you can use the /RU "System" option to run under the system account; this will not work for launching a GUI application.

The whole solution was packaged up using Visual Studio Setup Project and a custom installer. The custom installer implements System.Configuration.Install.Installer with the Install and Uninstall methods invoking Schtasks.exe to create and delete the task respectively. The install directory can be deduced from the assemblypath context parameter:

string installDirectory = Directory.GetCurrentDirectory();

// Try to get install directory from install assembly path

if (this.Context != null)

{

string assemblyPath = this.Context.Parameters["assemblypath"];

installDirectory = Path.GetDirectoryName(assemblyPath);

}

string taskPath = Path.Combine(installDirectory, taskExecutable);

I found that if the path to the task had any spaces the task would not execute under the scheduler; the answer was to convert the path to short name using the Win32 method GetShortPathName and PInvoke. Finally a method to invoke Schtasks using the System.Diagnostics.Process class:

private static void RunCommandLine(string fileName, string arguments)

{

string workingDirectory = Directory.GetCurrentDirectory();

Process proc = new Process();

using(proc)

{

ProcessStartInfo info = proc.StartInfo;

info.UseShellExecute = false;

info.RedirectStandardOutput = true;

info.RedirectStandardError = true;

info.CreateNoWindow = true;

info.FileName = fileName;

info.Arguments = arguments;

info.WorkingDirectory = workingDirectory;

if( proc.Start() )

{

string output = proc.StandardOutput.ReadToEnd();

Debug.WriteLine(output);

proc.WaitForExit();

}

Phillip Trelford

Dealing with Terabytes of Data in F#

apg — Sat, 04 Nov 2006 11:04:00 GMT

In one of our current projects our algorithms we have to process close to 1 TB (Terabyte) of raw (ASCII) logs. Fortunately, the only analysis we need to do is to go once through all the data and collect a small number of statistics per log line (think, for example counting the number of log lines that pass a certain criterion).

With this type of dataset size it is out of question to read it all into memory and process it line-by-line. The central data structure of .Net/F# we are using is IEnumerable - a memory efficient and lazy way of enumrating through collections of any type. Here a short piece of F# code that provides an IEnumerable for all log lines (using the new generate_using function that Don put into the standard library after my posting)

#light

open System.IO

open System.Collections.Generic

/// Creates an IEnumrable through the lines of any text file.
/// The function does not check if the file exists already!

let CreateDataStream (fileName:string) =

    IEnumerable.generate_using

        ( fun () -> new StreamReader (fileName) )

        ( fun reader -> if (reader.EndOfStream) then None else Some (reader.ReadLine()) )

However, during development one often wants to run-and-test the code without having to wait for hours before the full Terabyte is processed - just to find that there is a one-off error in the counting. Of course, one could write a little helper tool that only takes the first, let's say, 10 Megabyte of the full data file and process this much smaller file in the development phase. However, this seems very inelegant and leads to a lot of replication of the same data on the file system. A much better way is to use this short function truncate

module IEnumerable = begin

    /// Truncates a given IEnumerable

    let truncate n (x: #IEnumerable<'a>) =

      IEnumerable.generate

          ( fun () -> ref 0,x.GetEnumerator() )

          ( fun (i,ie) -> if !i >= n or not (ie.MoveNext()) then None else (incr i; Some(ie.Current)) )

          ( fun (_,ie) -> ie.Dispose () )

end

The nice thing with this truncation is that it has practically no computational over-head (other than testing and incrementing an integer) and does not cost any temporary memory. Here is a short piece of test-code for this function

/// Test the truncate.

do [| 0;1;2;3;4;5;6;7;8;9 |] |> IEnumerable.truncate 4 |> IEnumerable.iter (printf "i = %d\n")

do read_line () |> ignore

Ralf Herbrich

P.S.: Thanks to Don Syme and James Margetson for helping us with the truncate function!!!