“All we do is take and process massive amounts of information and use it to do things more effectively,” says Michael Flowers, head of New York City's Office of Policy and Strategic Planning (OPSP) in a weekend New York Times story called The Mayor's Geek Squad.
And data they have.
NYC knows that it takes an average of 546 seconds for NYPD to respond to a crime in progress, that it has 1,985 public drinking fountains in its parks, that its sanitation department picks up 12,000 tons of trash every day, and that an average of 100,156 parking tickets are given out per week. It knows this because of "big data," the hot catch-all term in government and business alike to describe adept number-crunching to better understand phenomena and solve problems.
In New York, such problems that can be tackled with data range from the mundane to the critical to the touchy. Among the mundane, NYC-based Foursquare recently hosted a hackathon in which one product, "Nasdrunk," matched Nasdaq stock exchange closing values with the level of location-based check-ins at NYC bars. Hackathons producing apps with a shelf life of a week or two are seemingly a dime a dozen these days.
But cities such as New York are more mission-focused on putting out proverbial fires and serving their citizens. For example, the New York Times relates how the OPSP used relatively obscure city data and geo-spatial intelligence to identify illegal cooking oil-dumpers:
Last fall, the city’s Department of Environmental Protection wanted, finally, to crack down on restaurants that were illegally dumping cooking oil into sewers in their neighborhoods — congealed yellow grease is responsible, the department says, for more than half of New York’s clogged drains. The question, of course, was how to find the culprits?
The antiquated answer would have been to have the health department send inspectors to restaurants on blocks with backed-up sewers and hope by chance to catch a busboy pouring the contents of a deep fryer into the street.
[The OPSP] dug up data from the Business Integrity Commission, an obscure city agency that among other tasks certifies that all local restaurants have a carting service to haul away their grease. With a few quick calculations, comparing restaurants that did not have a carter with geo-spatial data on the sewers, the team was able to hand inspectors a list of statistically likely suspects.
Other applications of big data, like the NYPD's use of it to guide who it stops-and-frisks, are more controversial. And yet still other challenges - like clearing the 9,662 trees that fell during Hurricane Sandy - are things virtually no one would complain about applying open data and new techniques too.
Open government and physical space
It strikes me that the best and most interesting uses of the one terabyte of data that flows through the OPSP every day, and its equivalent at other entities, usually have to do with the physical spaces of real life.
Meaning: When a new app takes one kind of digital data, mashes it up with other digital data, and then tells you something somewhat interesting yet not very applicable in physical space (i.e., what exactly would one do with the knowledge that there may be a weak correlation between stock prices and happy hour start times?), it doesn't tend to solve real-world problems, and thus it doesn't tend to form the basis of a long-term solution or startup company, and is therefore unlikely to have a long shelf life.
Conversely, an application of open data like tracking cooking oil dumpers and enforcing local laws is just about the opposite of sexy, but it's also extraordinarily important at a hyperlocal level. Think of parallel applications to everything from parking to noise complains to traffic to 911 calls. "What we're really running here is an office of New Yorkology," Flowers told the Times.
San Francisco is now Skillville
Across the country, San Francisco is the seat of American ingenuity in the digital space, and is no stranger to open government, big data, and innovative approaches. I recently had the opportunity to have coffee with Jay Nath, Mayor Lee's Chief Innovation Officer. He approaches "innovation" very broadly, not just considering digital and social technologies, big data analyses, and apps contests, but also - in the vein of Alec Ross' work on innovation in the State Department - in the form of taking creative approaches to solving basic problems.
One thing Jay and I discussed was Living Innovation Zones, which the city sees as an extension of what it's already been doing with open data. If it can open data for public usage, what else can it "open" for use? What if all the rooftops in the Financial District were "open" for solar panel hacking? What if portions of the Tenderloin were designated as "innovative street furniture zones," where citizens could experiment in a Furniture Hack of sorts and beautify the city and change super-hyperlocal dynamics (you see this in NYC a bit along Broadway, with zones of lawn furniture).
Granted, some of these assets are owned or controlled by entities other than the city; my impression is that some applications of Living Innovation Zones would require public-private partnerships, donations, or other arrangements. The larger point is that Nath and the City of SF are thinking outside-the-box on improving the city, and striving to take open approaches to government far beyond open data.
Skillville is another example of solving a real national and local problem - citizens out of work, underemployed, or in need of new job skills - with a creative approach. Here, people looking for jobs and opportunity can work on "micro-jobs" with local government and are rewarded through a bit of gamification with badges from which more benefits can be derived.
As they state on their Knight News Challenge page:
Skillville aligns the resources of city government, job seekers and local employers to overcome challenges for all three groups. Cities get an infusion of new skills and creativity on projects, job seekers gain meaningful work experience, making them more competitive in the job market, and businesses get a filtered pool of candidates who have validated skills. By increasing collaboration between citizens, local government and private businesses, everyone wins.
Another example of public-private partnerships for civic innovation. And another example of a tech-savvy but not tech-centric approach to civic innovation. While online/digital tools are certainly applied to Skillville, a large proportion of the activity occurs in real-life among regular people.
Living in "whine country"
Applying creative approaches and big data to hyperlocal challenges sometimes results in quirky discoveries. For example, back in NYC, the OPSP recently discovered that "a significant percentage of 311 complaints derived from certain neighborhoods in Lower Manhattan — an area they now refer to jokingly as 'whine country.'"
It's not immediately obvious to me what the usefulness of such information about "whine country" is, but one thing that's probably true is that it'll take people with interdisciplinary training and broad interests to figure out the how and why of it all. Michael Flowers worked in the Manhattan DA's office, did tours at law firm Williams & Connolly, the U.S. Senate, and in Iraq before finding his way to the OPSP.
Jay Nath's career is a little more IT-focused, but he has previous private sector experience building things at an Internet company and before that he was a Pricewaterhousecoopers consultant who took a year off to wander through Nepal, India, and Thailand. The aforementioned Alec Ross spent a year of college in Bologna, Italy, participated in Teach for America, and co-founded the global nonprofit One Economy before joining the State Department at the start of the Obama Administration.
Now if someone could just explain the data showing that it's snowing today in Washington, DC, when we're supposed to be in Spring.
Update 1: Slightly different take on SF vs. NYC at TechPresident here.
Noteworthy collaboration between EMS, GOV, and Private Sector: www.pulsepoint.org - the first life saving app developed by Fire Chief in San Ramon Valley, CA now deployed in several states - provides live nearby mobile alerts from 911 data link to mobile volunteers while triangulating closest AED.