Open data — and the art of the possible

The computing power provided by cloud technology, combined with open data and the applications that allow for manipulation of that data, has made building applications and digital tools simpler than ever. All that’s left to really open the floodgates is standardized rules surrounding those data sets.

IBM senior software developer John Arthorne showed off a simple application at the GTEC conference in Ottawa this past October, which he was able to cobble together in a matter of hours. He used publicly available government data sets combined with Google Maps to build an application that allowed him to easily find nearby historical sites where he could take his children.

“It took me a couple of hours to build the app, and then it took me about seven hours to scrub the data,” said Arthorne. “The data was from (an) Excel spreadsheet — it had missing columns, it had a field that should have been a bullion that sometimes said ‘true’ and sometimes said ‘yes.’ It was clear that the data had been manually entered and was fairly inconsistent.”

This lack of consistency, he explained, is the final obstacle before developers are able to truly take advantage of the tremendous amount of data that has become available online.

But the amount of data that is currently available is only the tip of the iceberg. As more data sets become publicly available, the opportunities will become far greater, explained Lewis Wynne-Jones, a data researcher for ThinkData Works, who works closely with Arthorne and IBM’s Bluemix cloud platform.

“As it stands, governments are responsible for a majority of the open data that is being released, and while in both Canada and the United States there are massive federal portals, we are also seeing a huge and very encouraging municipal push,” he said. “Some of these portals have thousands of data sets, some of them have two, but it’s clear this isn’t just something for big government departments anymore.”

Wynne-Jones said the private sector has also recently begun contributing to the open data library, with companies like Thompson-Reuters, for example, releasing their data sets in hopes that it eventually becomes the standard.

“There’s a third tier of open data, and that’s our data, the data that you and I produce every day,” he said. “This is all part of a holistic understanding of how we live in the world, and that data can be opened up and it can provide insight.”

Each new data set provides infinite possibilities for applications, tools and resources. Arthorne and Wynne-Jones outlined some of the applications that will be made possible as a result, with results ranging from smarter cities to new business opportunities.

“My intent here was to give you a flavour of the kinds of things you can do when you take this massive volume of data that we’ve got available to us, this massive at-your-fingertips computing power — you put them together and there’s really no limit,” said Arthorne during his presentation. “We’re barely scratching the surface of what you can with this.”

Image courtesy of Free Digital Photos