TrueSkill in F#

apg — Mon, 16 Jun 2008 07:25:00 GMT

Following the publication of the F# source code for the TrueSkill Through Time paper, we have used the same inference library code to demonstrate how the original TrueSkill algorithm can be coded. In the attached source code, only the program.fs file is different to the TrueSkill through Time blog post and it builds up the factor graph for an arbitrary N-player game with an arbitrary draw probability. Both these numbers can be entered on the command line when running the resulting sample. We tried to make sure to stay as close as possible to the description on page 3 of the TrueSkill technical report.

let priorMu = 25.0<SkillPoints>

let priorSigma = priorMu / 3.0

let priorSkills = Array.init noPlayers (fun i -> Gaussian.Create (priorMu, priorSigma))

let (posteriorSkills, logZ) =

NPlayerTrueSkillUpdate (0.5*priorSigma, 0.01*priorSigma, drawProb / 100.0) priorSkills

Note that one can no longer accidentally pass the variance instead of the standard deviation because the former being of type float<SkillPoints^2>. Try yourself!

Update 2: We had some feedback from Andrew Kennedy that we exploited a current "bug" in the units-of-measure code where literals (such as 1.0) can have generic units. We removed the incorrect usages and re-posted the ZIP file.

Update 3: We extended the example to work with draws. Thanks to Chih-Jen Lin for the suggestion!

Ralf Herbrich & Thore Graepel

TrueSkill Through Time

apg — Sat, 05 Apr 2008 15:59:00 GMT

In December 2007, we published a paper about the application of TrueSkill to all recorded data of professional and semi-professional Chess play from 1850 to 2006. The dataset can be obtained from ChessBase. We are very happy to be able to release the F# source code that performed all these massive computations. A few things to bear in mind when running this code:

The full factor graph has ~30,000,000 factors and ~25,000,000 variables. At this size, the analysis is among the largest applications of (approximate) Bayesian inference to date. Nevertheless, the full message passing schedule allocates no more than 11 GB of memory and is run in under 20 minutes on an Intel Pentium 4 processor. These numbers hold for the adaptive draw margin model run on 3.5 million games.
Due to the huge amount of results (>1,200,000 skills of all players in all their active years) we use SQL server to store the result set. Please let us know if you think a purely text based result output is needed and we will add it.
There is a very interesting, related paper by Prof. Mark Glickman which analyses subset of this data (88 top players of all time).

The attached ZIP file contains the entire source code which should compile without problems with F# 1.9.3.14. You can either load the solution in Visual Studio 2008 or simply run build.bat form the command line - in either case you will end up with a chessanalysis.exe program or an analysis.exe program. If you want to try out the application, we included the first 1,000 lines of match outcomes (this is part of the first 5 years of data we have).

As for running the code, here are the parameter switches of the command line application:

        -no-safe                          Does not save results predictions (default: off)
        -q                                    Verbosity level off; only works on single runs (default: on)
        -N <int>                           First N games only (default: ALL)
        -delta <float>                  Maximum deviation in any marginal for convergence (default: 0.01)
        -muS <float>                   Mean of prior skill belief (default: 1200)
        -sigmaS <float>              Standard deviation of prior skill belief (default: 400)
        -muD <float>                  Mean of prior draw margin belief (default: 300)
        -sigmaD <float>              Standard deviation of prior draw margin belief (default: 100)
        -beta <float>                 Standard deviation of performance distr. (default: 600)
        -tauS <float>                   Standard deviation of skill dynamics distr. (default: 40)
        -tauD <float>                   Standard deviation of draw margin dynamics distr. (default: 10)
        -server <string>               SQL server name for output (default: 'camresapga01')
        -prefix <string>                Table name prefix for output (default: 'Result')
        -db <string>                     Name of the database for output (default: 'ChessBase')
        -mf                                  Fixed draw margin (default)
        -mf2                                 Fixed draw margin with two factors for draw
        -maf                                 Fixed draw margin with ADF (iterate per year)
        -maf2                              Fixed draw margin with pure ADF
        -mv                                  Variable draw margin
        -as                                   Single run (default)
        -am                                  Model selection

So, a good first test is analysis.exe -no-safe SmallChessBase.csv.

We are very interested to hear your feedback - both on the model, our paper and the F# code.

Update: Following the release of the F# September 2008 CTP we have updated the source code to work with F# 1.9.6.2. Also, we added one of the new features of F#, namely units-of-measure. Note that one can no longer accidentally pass the variance instead of the standard deviation beceause the former being of type float<ELOPoints^2>. We have also added full debug support so simply press F5 to experiment with the -no-safe SmallChessBase.csv option mentioned above.

Update: We removed a reduant schedule creation in the function FixedDrawMarginADFAnalyse. Thanks to Henry (Xinhua) Zhang for pointing out this mistake!

Ralf Herbrich & Thore Graepel

The Applied Games Group Blog : Bayesian

TrueSkill in F#

TrueSkill Through Time