Posted by Rob Knies

Mobile Summit logo

You might not be aware of the term “continuous mobile vision,” but I’ll bet there’s a good chance you are aware of one of the scenarios it could enable.

Remember the concept, bandied about in recent years, of technology that can remind you of a person’s name once her or his face has been detected? Yeah, that one. I’m sure that most of us could make use of it once in a while.

The problem, though, is that image sensing takes lots of energy. That’s because modern image sensors lack energy proportionality. They’re power-hungry. That’s fine when a high-resolution, high-frame-rate image is desired. But even when you’re not seeking images of that quality, today’s image sensors still consume a lot of power.

Someday soon, though, things could change for the better. That’s the premise behind the paper Energy Characterization and Optimization of Image Sensing Toward Continuous Mobile Vision, which has been accepted for presentation during MobiSys 2013, the 11th International Conference on Mobile Systems, Applications and Services, being held June 25-28 in Taipei, Taiwan.

The paper was written by Robert LiKamWa and Lin Zhong of Rice University and Microsoft Research Redmond, along with Bodhi Priyantha, Matthai Philipose, and Paramvir (Victor) Bahl of Microsoft Research Redmond.

Bahl, a principal researcher and manager of the Mobility and Networking Research group, spoke June 11 during the MIT Technology Review Mobile Summit 2013, held June 10-11 in San Francisco, discussing the technology behind the MobiSys paper, and he underscored the new area being explored by him and his colleagues.

“Our industry has been focused on building small, higher-resolution image sensors for decades,” he said. “We believe that we are the first to look at systems-level optimizations for improving their energy consumption. We believe the techniques we have designed are fundamental, in that they will stand the test of time.”

The vision behind this work was delivered a year ago, during the Mobile Cloud Computing and Services workshop, in a paper titled Cloud-Powered Sight for All: Showing the Cloud What You See, written by Bahl, Philipose, and Zhong. Now, the researchers are taking the next step.

Current image sensors are optimized for taking photographs and videos, but if paired with a processor, the sensors could do much more, such as reading barcodes or translating text.

The next step would be to enable continuous mobile vision, in which the device in which the sensor is located always seeks ways to serve its user. That could involve remembering faces at a party, but it could also recall food and drink consumption or help the user navigate through a confusing environment—scenarios in which high resolution and high frame rates are not paramount. If, by doing so, such sensors drain less battery power, then that should increase their usefulness over a longer period of time, which is a required part of the continuous mobile vision.

The researchers have invoked a couple of mechanisms for improving the energy efficiency in image sensors. The first optimizes clock scaling, inspired by clock-scaling techniques for processors. In the case of image sensors, reduced resolution and frame rates are analytically determined.

The second technique toward energy proportionality is to use an aggressive standby mode between frames, when the frame rate and resolution are sufficiently low. The energy wasted while the device is in idle mode can be reduced significantly.

In combination, for continuous image registration on video, the paper reports, these mechanisms can achieve a 36-percent power reduction by choosing an optimal clock frequency and a 95-percent power reduction by using aggressive standby. Further power reductions are available via the use of architectural modifications of image sensors, the researchers suggest, and computer-vision benchmarks demonstrate application-quality and energy-efficiency tradeoffs that can be attained.

It all amounts to a novel, promising step toward the dream of continuous mobile vision.

“Researchers and engineers have been working on a similar vision for a long time,” Bahl said. “A case in point is the ongoing work in robotics and the early research pursued by Dan Siewiorek at Carnegie Mellon University and Steve Mann at MIT. Still, we haven’t yet succeeded in scaling up the technology to make it mainstream.

“The research we are pursuing is to take on perennial problems such as battery, computing, latency, connectivity, and bandwidth in the context of this vision. Slowly, but surely, we are making headway, and I am confident that, together, we will build enough technology to make continuous mobile vision a reality for everyone.”

A video of Bahl's Mobile Summit talk is available by clicking on the Day 2 tab of the MIT Technology Review video website for the event.