When it comes to open data there’s a gem of a report by Deloitte Canada. ’Understanding government‘ is a belter. It’s an assessment by suits for suits of what open data can achieve. It’s worth downloading.
Think of this post as a review. It’s worth reading the whole thing but here are some bullet points. The sub-headings are mine rather than from the report.
How social media helps open data…
“The social media culture in particular is driving governments to open up while offering the imagination and expertise necessary to improve public services.
“In response, government organisations are embracing the idea that public data should be broadly available in a re-usable format and that governing should be a collaborative enterprise between government and its citizens.
Open data can help…
“The good news is that open government initiatives can help engage the public in making the difficult budgetary choices governments are grappling with.
Open data is not a threat…
“Rather than view the changing relationship between government and its stakeholders as a threat or an inconvenience ncreasingly see it as an opportunity to engage citizens, non-governmental organisations, businesses and other entities in the design of new services and the resolution of old problems.”
Open data may even have stopped the MP’s expenses scandal…
“If the UK had put its database of members’ expense re-mbursements in the public domain in the first place could the scandal have been avoided? Politicians who know that constituents are watching their activities are much more likely to be careful about how they spend public funds.”
The four benefits of open data…
- Better inform the public;
- Enhance accountability;
- Strengthen communities;
- Facilitate markets.
Make a noise about open data…
“Agencies should not quietly put data online. Rather, they should tell the public what they are doing and why, while seeking their participation and engagement. Data that sits in a file are not worth much. Information becomes powerful only as its consumers start to apply it in ways that create value.”
Let people build things…
“This form of user-driven application development, also known as crowd sourcing, user innovation or open sourcing provides governments with an unprecedented opportunity to engage citizens in unlocking the power of public data.”
What to do with user-generated content…
“Encourage users to create applications. Incorporate or adapt user-designed applications into publicly hosted sites. Seek and maintain a dialogue with apps developers. Create methods and channels for listening and responding to user demands for data.”
Don’t just let citizens analyse data. Government needs to be better at it…
“Government leaders recognise that in addition to leveraging community resources to analyse public data they must get better at analysing vast stores of public data – in addition to online resources.
“Leading governments are investing in building a core competency in data analytics.
“This involves acquiring the software tools to manipulate vast stores of public data – often provided by more than one agency – and investing in the people and processes to drive analysis and take action.”
Isn’t data a bit vast? Where do you start to look at data as government?
“Focus analytics on your core mission. Approach data analytics as a new core competency not a new tool set. Enlist key partners inside your agency. Leverage the online community.”
“In an information-driven age, the ability of governments to seize the opportunity may ultimately determine whether a government fails or succeed.”
This post was first published on the Open Data Blog.
Creative commons credits:
Accountant with a computer: LSE Library http://www.flickr.com/photos/lselibrary/4072387390/
Geek tech http://www.flickr.com/photos/modul/4703887615/sizes/m/in/photostream/
Geek in t-shirt http://www.flickr.com/photos/zakwitnij/140788331/
At the Wellcome Trust in London more than 200 people gathered for the International Open Data conference.
David Cameron delivered a recorded message and Minister Francis Maude was there in person. So was uber-geek Tim Berners-Lee.
Arranged by the Open Knowledge Foundation This was a chance to launch the UK Government’s data set of its department’s spend over £25,000.
That’s 194,000 lines of text and £80 billion of spending. The link to it is here.
What’s the point in that? The aim is to open the Government’s books to allow residents, journalists and business a chance to have a look.
Pithily one newspaper commentator posed the question: ‘A great leap forward or masochistic folly?’
It is madness isn’t it?
Actually, no. It’s a movement supported by left and right alike which has the aim of cutting waste, allowing entrepreneurs to flourish and a fairer society.
The event may have been Whitehall focussed but there are powerful golden strands that run through all government. Local and national.
Local government has already been asked to publish items of spend over £500 under the label ‘spending transparency.’
They have until January 1 to do it and as Cameron and Maude 100 of more than 300 odd councils had published.
There is a feeling within Whitehall that some will quietly choose not to publish calculating the flak they get for not completing a slightly arcane process is less than the grief a particular financial skeleton may pose.
It’s unlikely Whitehall will allow this to pass without prompting closer inspection.
It’s also unlikely local government will not be asked to publish more as open data. There is more to come. Much more.
Here are some broad messages from the day for local government:
SO, WHAT’S THE BIG PICTURE?
Open data won’t be an easy ride for people in authority. As Francis Maude said: “It’s going to be very uncomfortable for government and local government. Media outlets will find things that will cause embarrassment.”
It’s not going to go away. It’s easy to like open data in opposition, says Maude. You can shine a light at others’ decisions. However, he pledged there were two key advocates – him and the Prime Minister.
The aim is to move influence away from the traditional centres – “information is power. This is a power shift,” says Maude, “to move the decision making away from Westminister.
It’s FOI turbo charged – It would have taken journalists years of submitting FOI requests to build up the picture revealed in the £25k data sets, the Guardian say.
HOW DOES THIS AFFECT THE PRIVATE SECTOR?
Contracts should allow for open data to be released – The presumption for contracts is transparency, says Maude.
It’ll create wealth – Open government data will create a £6 billion industry, says the Minister.
A website to point the spotlight on the private sector too – Chris Taggart has built opencorporates to shine the light at which big companies are doing well from public sector contracts.
HOW WILL ALL THIS BENEFIT GOVERNMENT – CENTRAL AND LOCAL?
Waste detection – By spotting where the waste is money will be spent better, Francis Maude says.
WHAT IS NEXT?
Historical data will be released – There will be open data from previous administrations. This will help to compare and contrast with the current era.
More public agencies will follow – There are 100,000 public bodies. There’s no timescale for these just yet.
There will be a right to data – David Cameron has pledged that people will ask and receive data for a personal and business use. This is massive for local democracy.
Open data will move from spending into crime – Expect interactive crime maps in the New Year, Maude says.
Journalists – the media needs to be data savvy. Data journalism will become more and more important, says Tim Berners-Lee.
“Chatting people up in pubs was one part of your job,” he told journalists in the room. “Poring over data and equiping yourself with the tools to look for the juicy bits will be important.
“Data journalism will be part of the future.”
Right now, local newspapers haven’t grasped what data journalism is. Don’t hold your breath just yet either.
Traditional news is emergency services calls, court and council agendas. It’s not data mining with csv files.
What may put it on the agenda are national stories re-written with a local.
Hyperlocal bloggers – many bloggers have geek tendancies that will happily work with online tools. Stories from all this will be broken by an 18-year-old rather than a laptop. That’s quite exciting. Tools such as timetric.com where graphs can be built using data and embedded in blogs can help with this.
Geeks – an inexhaustable army of geeks will pore over the data – “what happens when the flashflood of geeks go away?” mused Tim Berners-Lee. “It’s perennial.”
Industry – Data company Spikes Cavell have released spotlightonspend.org to interpret local government data. This hasn’t been without criticism from the opendata community who argue against councils dealing solely with the company and not releasing open data too.
Social entrepreneurs – Chris Taggart has built openlylocal.com as a platform for local government data and has been a pioneer in the field.
Real people – Fascinatingly, The Guardian had a team of four working for four days on the data before it was published. They didn’t think they could glean everything themselves. What they did do was make it possible for the public to use the tools to search for stories. This is the wisdom of the crowd as an extra pair of hands in the newsroom. You can download their app here.
BUT IT’S NOT ALL GOOD NEWS….
There’s no funding for people to cross check the data – As one questioner pointed out the tools that held government to account – journalists – have historically been cross subsidised by other sources such as small ads.
There’s no funding for these resources. There’s a question mark against the sustainability and effectiveness of tools.
Creative commons credits:
Tim Berners-Lee: Paul Clarke via wikipedia. http://en.wikipedia.org/wiki/Tim_Berners-Lee
Open data cutting edge? Like top hats, Christmas trees and giant factories the Victorians got there first.
They may not have built a chimney sweep Google death map. But their approach was similar. Collect the data. Publish it. Draw conclusions. Argue for change.
Don’t believe me?
Look at Florence Nightingale in her funny lace bonnet. Historian Dr Stephen Holliday in BBC History Magazine August 2010 writes about how she used statistics to
revolutionise the care of soldiers in the Crimean War.
By using statistics – data – she painted a picture to show a revolution in care was needed.
“When she reached Scutari the base for casualties from the Crimea,” Halliday writes, “Florence calculated that deaths from disease were seven times those arising in battle and used the campaign to campaign for better food, hygeine and clothing for the troops.”
Battered by the force of Florence’s figures and cutting edge reporting that forged the reputation of The Times the British government was forced into changes.
After the war Nightingale used her Royal connections coupled with arguments based on charts and tables to press for better standards for soldiers who even in peacetime had death rates double that of civilians.
The result? Death rates fell by 75 per cent.
Florence herself said that statistics were “the cipher by way we may read the hand of God.”
We may have lost that religious zeal but it’s an argument Tim Berners-Lee would recognise as a modern-day Florence Nightingale with a passion for data.
Did she get it right all the time?
No. Here’s the warning from history.
By misreading available data Florence Nightingale later helped kill thousands of people.
She used statistics to wrongly argue cholera was an airborne disease. It wasn’t.
It took London GP Dr John Snow to collect his own data on death rates in his patch to argue they were caused by a contaminated water supplies.
So what’s the message to today’s open data pioneers?
That first data visualisation you have in front of you may not be the whole picture.
There may be more to it.
Remember the phrase ‘lies, damn lies and official statistics?’
Statistics were once hailed as the magic cure-all that revealed a hidden truth.
It’s been said that all data in some form or other is political. Let’s not see open data similarly tainted.
Florence Nightingale –http://www.florence-nightingale.co.uk/cms/
BBC History Magazine August2010 http://www.bbchistorymagazine.com/issue/august-2010
Crimean War data visualisation: Wikipedia.
Cholera map: Wikipedia
Tim Berners-Lee: Paul Clarke via Wikipedia
Confucius said that. Only, thing was he never worked in local government.
Speed of change in open data is blisteringly fast and getting faster.
In the Spring I thought all this would be important in 12 months time. Wrong. It was important TWO months later.
Local government in the UK has been asked to publish spending over £500 line by line.
A few months back Maidenhead and Windsor Council were hailed as a shining example of how to do this.
A few months on and the shine is wearing. Yes, they deserve praise for innovation but bright people have pointed out that you can do so much more if you publish a little bit more than a handful of categories.
Change was one of the themes of a session in Birmingham by Vicky Sergeant of SOCITM and hosted by Birmingham Council on the subject of publishing spending transparency open data.
It was a chance for people to bounce ideas and was an alphabetti spaghetti of a gathering with SOCITM, LGA and LeGSB.
Will Perrin from the Local Data Panel that helps shape data.gov.uk policy delivered a clear message:
There will be no spoon feeding from on high.
Eighty per cent of problems have been solved with blog posts such as this, he says.
It’s now down to councils to be brave and stand on their own two feet.
The combined efforts of the groups at the meeting are likely to publish at some stage some valuable advice on how best publishing spend can be put on line. These are things that struck me in the meantime.
Here are 12 key pieces of advice I took from the day
1) Publish open data-related FOI requests. Great idea. Further research shows you’ll have to be careful about publishing personal data not just in the name and address field but also in the text of the response.
2) The size of the dataset would double if it included ALL spend.
3) You can run a programme if you are clever to remove – or redact – at source personal data from social care and children’s services data.
4) You may need to make it clear to suppliers that this change is taking place. Not all are following this whole debate. In fact, I’d be amazed if any of them are.
5) Commercial confidentiality is a grey area. As Will said, the Information Commissioner’s presumption is to publish in the public interest but there is worryingly no case law to show where this has been tested.
6) Publish a unique identifier for your authority when you are publishing open data. Finance people will know what this is. It identoifies the line of data as being from a specific council.
7) Put an email address as a first point of contact for residents queries. Maybe people don’t have to go down a 20-day Freedom of Information response route first to get an answer.
9) Use the licence that can be found at data.gov.uk. It’s been looked at by government lawyers. Creative Commons while great hasn’t really been tested in law in the UK.
10) There are a lot of codes in local government finance. If you don’t know what a CIPFA BVACOP code is make friends with someone who does.
11) Don’t plough a lone furrow. As a council or an officer don’t be alone. The Communities of Practice website is an excellent place to learn and discuss.
12) Guidance maybe getting drawn-up but don’t let this stop you. The LGA, SOCITM and others are looking what would work best. Don’t wait for them, however.
13) Communications is important. You need to explain it internally as well as to elected members, residents and suppliers.
15) Publish monthly. Some in the web community are baffled as to why publication can’t be done at the end of every working day. As a compromise the Local Data Panel are saying publish monthly but within a month of month end.
Ian Carbutt from the LGA made some excellent points at the meeting. He points out there are three main areas that have several sections to them.
- What and who its for: Local authority ID code, directorate, goods and services, service department.
- Payment details: invoices, invoice number, net amount, VAT, gross amount, date of payment.
- Supplier: name, contract title, supplier company number or VAT reference.
Pick from those three paints a better, more complete picture and may lead to fewer FOI requests.
Pezholio blog on the SOCITM Birmingham local data event. A useful summary and some very useful comments.