Posted by Rob Knies

HoloDesk interactionIf you’re reading this, there’s a good chance you’re already familiar with HoloDesk, research from the Sensors and Devices team at Microsoft Research Cambridge that my colleague Steve Clayton posted on the Next at Microsoft blog a few weeks ago. The work took the tech community by storm, with the HoloDesk video having received more than a million views thus far.

HoloDesk uses a Kinect depth camera to enable the manipulation of virtual, 3-D objects with your hands. It remains a research project at this stage, but the potential to make our interactions with computers more natural is staggering, which is what has people all abuzz—even in the absence of many details. To rectify that absence, I reached out to Otmar Hilliges of the team that produced the technology to fill in a few of the gaps.

“We’re using a regular LCD screen to render perspective graphics that are reflected off a semi-silvered mirror,” he responded, referring to a semi-opaque mirror that partially reflects an image from the front and allows an image to shine through from behind. “The user looks through the semi-silvered mirror so that the scene behind the mirror and the reflected graphics appear combined.”

The use of a Kinect depth camera accelerated the pace of the research considerably.

“We’ve been trying to simulate human grasping within physics simulations for quite a while,” Hilliges said. “Previously, the input data weren’t rich enough to allow for the 3-D interaction fidelity that HoloDesk provides. The Kinect is almost perfectly suited for this sort of task, so we started working on this project as soon as it became available to us—a couple of months before public availability.”
The researchers involved are writing a paper to document their approach, but while that awaits publication, they already are well aware of what they have wrought. I asked Hilliges what makes HoloDesk unique.
“The possibility to directly reach into a 3-D scene,” he responded, “and to directly touch 3-D objects that respond to this natural user input in a physically plausible manner. Modeling this sort of input was the main contribution of the research around HoloDesk.”