I came across a fascinating article on Wired which looks into some of the processes that Bungie, the developers of the Halo game series for Xbox and Xbox360, have been using to test the latest iteration, Halo 3.
Thousands of hours of testing with ordinary game players has been recorded, and the footage synchronised so the developers can replay what was on the screen, what the player's face looked like, and what buttons were being pressed, at any point. They even record details of where and how quickly the players are getting killed, so they can figure out if some bits of the game are just too hard.
There have been a series of test waves of Halo3, some of which were for MS and Bungie employees only, and one public beta phase. The public beta was itself unprecedented in scale - over 800,000 users, racking up 12 million hours of online play. Do the maths, and that's about 1,400 years of continuous play...
The internal only tests have been kept necessarily confidential ("It is not cool to talk about the alpha" was a common warning to anyone who thought about leaking screenshots or info). The final phase of testing is underway and the game is said to be very nearly complete.
I'm not going to mention any more about how the game looks, sounds, plays - except to say that you'll all be able to find out just how awesome it is, on the 26th September (in Europe - the US gets it on the 25th). Might as well book the 27th and 28th off as holidays already :)