Reward Driven Racing in Cambridge
I just completed my 12 week internship in the Applied Games Group. I was working with Ralf, Thore and Phil on applying model-based reinforcement learning to race car driving. This is one of the most interesting projects I've worked on. Next to my workstation I had an XBox 360 development kit hooked up to a wide screen television, which gave some of the other interns at Microsoft Research the impression that I was just playing video games all day. While I did play quite a bit, the point of my work was to get the cars in Project Gotham Racing 3 to learn to drive themselves through experience.
I implemented a simplified version of the Adaptive Modeling and Planning System (AMPS) described in my Ph.D. thesis (available here). We found that AMPS learns to drive competently after a single training lap, but the other standard algorithms that I implemented did not perform nearly as well. The strength of AMPS is partly due to its ability to generalize from limited experience through dynamic state abstraction. Currently, most racing games on the market include AI opponents with hard-coded behavior, but I think that opponents that are able to learn and adapt to your driving style and exploit your personal weaknesses can make for some very interesting gameplay.
I would recommend applying for an internship here to any Ph.D. candidate with a passion for games. The projects are fascinating, and the people here are brilliant, coming from all over the world. The atmosphere at Microsoft Research is laid back but intense at the same time, which is exactly what I was looking for in an internship. If anyone has any questions about what it is like to be an intern at MSRC, my current contact information is available on my personal website.
Mykel Kochenderfer
