OPEN DATA: A warning from history

Was this the first data visualisation? Florence Nightingale uses statistical data to argue for healthcare reform for soldiers in the Crimean War.

Open data cutting edge? Like top hats, Christmas trees and giant factories the Victorians got there first.

They may not have built a chimney sweep Google death map.  But their approach was similar. Collect the data. Publish it. Draw conclusions. Argue for change.

Don’t believe me?

Look at Florence Nightingale in her funny lace bonnet. Historian Dr Stephen Holliday in BBC History Magazine August 2010 writes about how she used statistics to

Florence Nightingale - 'funny lace bonnet.'

revolutionise the care of soldiers in the Crimean War.

By using statistics – data  – she painted a picture to show a revolution in care was needed.

“When she reached Scutari the base for casualties from the Crimea,” Halliday writes, “Florence calculated that deaths from disease were seven times those arising in battle and used the campaign to campaign for better food, hygeine and clothing for the troops.”

Battered by the force of Florence’s figures and cutting edge reporting that forged the reputation of The Times the British government was forced into changes.

After the war Nightingale used her Royal connections coupled with arguments based on charts and tables to press for better standards for soldiers who even in peacetime had death rates double that of civilians.

The result? Death rates fell by 75 per cent.

Florence herself said that statistics were “the cipher by way we may read the hand of God.”

We may have lost that religious zeal but it’s an argument Tim Berners-Lee would recognise as a modern-day Florence Nightingale with a passion for data.

Tim Berners-Lee - 'a modern Florence Nightingale'.

Tim Berners-Lee - 'a modern Florence Nightingale'.

Did she get it right all the time?

No. Here’s the warning from history.

By misreading available data Florence Nightingale later helped kill thousands of people.

How?

She used statistics to wrongly argue cholera was an airborne disease.  It wasn’t.

It took London GP Dr John Snow to collect his own data on death rates in his patch to argue they were caused by a contaminated water supplies.

So what’s the message to today’s open data pioneers?

That first data visualisation you have in front of you may not be the whole picture.

The map that Dr John Snow drew to discover that cholera was a waterborne disease.

There may be more to it.

Remember the phrase ‘lies, damn lies and official statistics?’

Statistics were once hailed as the magic cure-all that revealed a hidden truth.

It’s been said that all data in some form or other is political. Let’s not see open data similarly tainted.

LINKS

Florence Nightingale –http://www.florence-nightingale.co.uk/cms/

BBC History Magazine August2010 http://www.bbchistorymagazine.com/issue/august-2010

Creative Commons:

Crimean War data visualisation: Wikipedia.

Cholera map: Wikipedia

Tim Berners-Lee: Paul Clarke via Wikipedia


6 Comments on “OPEN DATA: A warning from history”

  1. John Heaven says:

    Good piece, but isn’t the lesson about data per se, rather than open data? In the example you cite, the GP eventually collected his own statistics and proved Nightingale wrong. If her data, not just conclusions, had been open perhaps through “wisdom of the crowd”, or at least the wisdom of one member of it, this would have come to light sooner.

    I agree with the general point about the danger of statistics, and you’re right to pose the question of how we can get around this issue.

  2. […] This post was mentioned on Twitter by Tiago Peixoto and Jon Fernquest, Adriel Hampton. Adriel Hampton said: RT @jonfernquest RT @participatory: #opendata: A warning from history http://bit.ly/aUYh74 http://bit.ly/dgx6k8 […]

  3. Ben Proctor says:

    Hi Dan

    Good and interesting post. You do raise some important issues. Of course as John says all data can be misinterpreted.

    I’m sure we all know of cases where local authorities talked up their positive performance indicators and talked down poor performance when a more objective observer might have represented the position differently.

    And we all know of cases where journalists honed in on a single piece of data because it looked more interesting than the whole dataset.

    It does make me think that an additional skill for the 2011 press officer is a good grounding in statistics and analysis. And indeed for everyone…

  4. Ady says:

    Very interesting post, which touches on concerns I have with Open Data in the most general sense. (I’m an advocate of Open Data, by the way.)

    The issue is that statistics is not an easy subject to master, and I’m not sure that developers working from their living rooms building mobile phone applications necessarily understand it. In fact, I’d go as far as to say that hardly any of them understand it. (I did Maths and Further Maths ‘A’ levels and Specials in my youth, and statistics was the one area that I never really mastered.)

    Building pretty maps from all this geo-tagged data is fine, but those people who are seeking out trends, correlating open data from multiple sources, or other non-trivial exercises need to make known their credentials if we are to take their output seriously.

  5. […] peu d’histoire et anecdote sur les données et leur interpretation. Puis l’opendata et après ? Quelles suites envisager […]

  6. […] OPEN DATA: A warning from history Ist Tim Berners-Lee eine moderne Florence Nightingale? Open data cutting edge? Like top hats, Christmas trees and giant factories the Victorians got there first. In: danslee.wordpress.com, Nov 5, 2010 […]


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s