Meta

I asked Peter Alexander, Assistant Secretary at AGIMO (Australian Government Information Management Office) about the future of the data.gov.au data portal used to supply data the MashupAustralia competition. This portal also has the url data.australia.gov.au.

I’m speaking on data portals and lessons learned from around the world on Wednesday afternoon in the government 2.0 spatial@gov conference.

The data.gov.au beta portal will be replaced by a production version next month, hosted in the cloud. First priority is discoverability because it doesn’t matter how good it is if you cannot find it.

Second priority is format. The philosophy is to go with whatever is possible (it may be a table in a pdf initially), then improve on it e.g. with web services. AGIMO will be working very hard with agencies to get their data published. Agencies are a lot more comfortable about sharing now they know the sky did not fall with Mashup Australia, on the contrary they saw the benefits.

Agencies will self publish. There will be 5-10 attributes for every dataset, then links through to specialist portals for statistical or geospatial data (eg. Geoscience Australia). These will have more and different metadata.

We agreed on the importance of resourcing the admin, blog etc. But there will be a ‘lag response policy’. If you jump in straight away with an answer it can kill the conversation. Better to encourage comments on comments and let conversation blossom. That’s an interesting take.

Peter has had some discussions with ANZLIC and states. The same technology platform can be used by states.

If you want to promote open data in general, or get access to other people’s data, please take a few minutes to complete one of these surveys by 1st October. It is part of a government study project by Helen Moreland of the Victorian Dept of Transport and www.anzsog.edu.auRespondents can receive a summary of the survey results.

“Economic value of open access to government-held data and information”

We are particularly interested in hearing stories about your experiences with open access to government data and/or information (be they positive, negative or neutral).

GIPA is important for NSW mashups because it will lead to more data being released

The Australian Bureau of Statistics hosted a meeting for Statisticians in Sydney on Thursday 2nd September about GIPA and Open Government. GIPA is the NSW Government Information Public Access which came into force on 1st July 2010. GIPA replaces the Freedom of Information Act. It encourages and authorizes state and local government, ministers and their staff, state owned corporations (including utilities) and universities to pro-actively release data. Exceptions “when there is overriding public interest against disclosure” are clearly specified.

Officials who release data are protected from civil suits.

GIPA covers all sorts of data, not just about documents. It can be emails, tables, transcripts, stats, reports or raw data.

The law doesn’t say anything about the format the data has to come in. It could be on paper, or a pdf which wouldn’t be too helpful for a masher. There may still be a lot of scraping and formatting.

Speakers from the ABS, Office of the Information Commissioner, Dept of Fair trading, police examined the implications, benefits and issues raised with great honesty.

Data could be misinterpreted, misused or abused if it was”just dumped out there”.

On the one side were the policy makers saying “data can then be used creatively in ways and combinations bureaucrats and lawyers have never thought about”, on the other were those who fear that is it precisely because data was collected for one particular purpose, that it could lead to dangerous or erroneous assumptions if it were used for another. For example, the police collect operational data about crime. But where drug dealers are found is not necessarily the same place they live or where the drugs are consumed. Many crimes go unreported, so what if people in a particular suburb are more likely to report more crime than in another? Their suburb then looks more crime-ridden, businesses may not locate in the area, therefore disadvantaging the residents or property values go down.

But like it or not, data will be released and reused. So how do we enable meaningful interpretation and use, while at the same time not tying up public servants in reformatting or explaining data to the extent they can no longer do their regular jobs?

Give an explanation. People won’t necessarily read it but at least the Minister can look at the explanation when questioned about it.

Finding out what people are really asking will help you provide meaningful data and offer additional data if you think it will help.

If you don’t have the information in the form they request, you don’t have to jump through hoops, just be helpful, offer them what you have and explain the limitations.

If the data is incomplete or poor quality, just say so. Tell the story.

Expect questions and criticisms when you release data.

Put frequently requested data on a website. Making it understandable, easy to search and retrieve.

Use the Data Quality Tool from the ABS to create a Quality statement. This tool was demonstrated and helps both producers and users to assess and compare seemingly similar datasets.

Embrace the opportunity to improve data you release from crowdsourcing.

You can release secondary data (i.e.if you are the Dept of Planning and someone else has collected it) but if the requester would be better served by going to the original source you can refer them on. You can also release any data you hold that has come from outside NSW.

Q What if a private consultant asks you for data they will make a profit from? A If this ties you up and thus impacts on your core business you do not have to release the data because if it stops you doing your job that is not in the public interest.

Good news for parents

Your child is going for their first job in a cafe. You want to make sure they will be fairly treated. In about 18 months you will be able to look at a NSW website to check whether they have any harassment claims against them, do they pay penalty rates on Saturdays or how many times they have been inspected.

data.nsw.gov.au

Kate Harrington and Helen Palmer of the Government Chief Information Office run the NSW data portal used by the NSW app4state mashup competition. The goal of data.nsw.gov.au is to be the single point for people to visit for data because it is very hard for outsiders to find what they are looking for. They took us behind the scenes of the portal “what happens when you click on the ‘Request for Data’ button”. (Basically a lot of running around by them to get the data from the custodian).

There is an international wave of improved public information access: from the US where every government agency has to release at least 3 datasets (annually??) to the UK where the two most asked for datasets are public toilets and school catchment areas. Their apps4 prefix has spread around the world from Finland to Africa.

“What a bright spark can do with open data â€“ and the tools to use it”

This is developing into an interesting story. A brilliant ‘live’ tube map showing trains zipping around the London tube in near real time, was launched a couple of weeks ago. It shows what you can do when you get open access to official data according to an article in the Sunday Observer. It was obviously widely read. I couldn’t get on the site at all, but later learned the API has now been suspended due to overwhelming demand.

I enjoyed this article by Matt Ball on the vector1media website. Matt looks at the impact of some profound changes that he says will give GIS users more (and for free) data. Volunteer mapping and quality improvement from crowdsourcing, not just from things like openstreetmap but also mobile platforms. (I was introduced recently to some new mobile location based games like foursquare that collect all sort of features and information as a by-product).

Geospatial technologists will be synthesisers of data. Not a long article so well worth browsing to.

Ross Johnson sent me this fascinating article from the Los Angeles Times about openstreetmap. Crowdsourcing maps is becoming a mainstream social activity. With Google Maps doing it soon (as mentioned in this article), openstreetmap and even crowdsourcing through mobile games how does a user decide where to source data from? Can we talk in terms of a single authoritative source anymore?

http://vimeo.com/11540899 Not your fathers approach to geodata creation and sharing posted by moderator Peter Batty. This is quite a long video and I do plan to make a summary of the key points sometime,but for a start there is a very good explanation of openstreetmap as an example of neogeography and powerful pictures from post earth quake Haiti (the building housing the goverment GIS was destroyed in the earthquake so over a matter of days volunteers created new maps from old paper base maps, high res satellite imagery, even Maps from the CIA if I heard correctly – which made a huge difference to the emergency response and saved lives.

Some amusing banter between neogeographers and traditional GIS people cheekily referred to as paleogeographers.

This conference on metadata management brings together practitioners and managers to discuss issues related to metadata management, with particular emphasis on public sector data and metadata.

I followed the links and found another event on Friday (an â€œunconferenceâ€) covering the wider Information access policy side. Not a lot of notice but you may be able to get copies of the papers or follow the blog. At first I thought it was an online conference but then found it is at ANU