Contrary to popular belief, we don't decide which tasks to include on an exam by drawing slips of paper out of a hat. The process does include slips of paper—but without the hat. More on that later.

The tasks we test you on are determined by people like you. We invite a group of subject matter experts to Redmond for a three-day session to help us design the exam. During this session, we create what we call the objective domain (OD). The OD identifies the most critical (and testable) tasks for a technology area, and presents them in a logical, hierarchical order. The tasks (we call them objectives, hence the term "OD") are then organized into logical groupings which we call functional groups.

One of the first steps in the OD session is the brainstorming exercise. This is where the slips of paper come in—specifically, sticky notes. I give each SME 15 minutes to write down ten or so of the most critical tasks for that technology area, each on its own sticky note. Then each SME talks through each of their tasks and sticks them to the white board. As the SMEs take turns, we start to move the sticky notes around and organize them into functional groups.

I should probably be ashamed to admit that I use something as mundane as paper rather than some hi-tech tool. But really the process is quite effective, and has the added benefit of acting as a sort of ice-breaking team-builder for a group of people who will be spending three whole days together. Somehow the ability to physically move a sticky note from one place on the white board to another reinforces the idea that we are building a logical structure, while at the same time making it very easy to try out a different structure. (Note: some CDMs on my team use other brainstorming methods. But I'm a die-hard sticky-note fan.) When we’re done, the white board looks something like this:

Just kidding. I don’t have a picture handy of the white board after a real OD brainstorming session, but it looks a lot like this:

After we have sketched out the structure of the exam, I create an initial draft document. We spend the next two days refining the draft. We use various perspectives and filters to ensure that we have identified only the most relevant tasks and that we haven't left out anything critical. For example, one of the filters we use throughout exam development is the “80/80” rule: we ask the SMEs whether 80% of the target audience performs this task 80% of the time. We also add details about the types, methods, and subtasks that might be included in that task.

My absolute favorite part of being a Content Development Manager in the exams group is facilitating OD sessions. The expertise about what skills are critical already exists inside the minds of the OD SMEs—it's my job to draw it out and give it form and structure. We arrive on Monday morning with nothing but an exam title and an audience description. We leave on Wednesday afternoon with a collective feeling of accomplishment. We have created something important together. What's more, it's something visible and tangible, because the OD is not just the roadmap for exam development. It's also the prep guide that all candidates will use to prepare for the exam. To view the OD for an exam, all you have to do is visit the page for any exam on microsoft.com/learning and click on the Skills Measured tab.