Presentation given by Chris Taggart at Open Data session at Future Everything conference, Manchester, May 13, 2010.
Discusses how open data helps change the incentives from big, slow failures to small, fast failures, from which we can learn
5. Me: Chris Taggart
@countculture
✤ Developer of OpenlyLocal.com
✤ Makes local data (esp govt) available
in consistent and accessible form
6. Me: Chris Taggart
@countculture
✤ Developer of OpenlyLocal.com
✤ Makes local data (esp govt) available
in consistent and accessible form
✤ Scrapes council and govt websites
7. Me: Chris Taggart
@countculture
✤ Developer of OpenlyLocal.com
✤ Makes local data (esp govt) available
in consistent and accessible form
✤ Scrapes council and govt websites
8. Me: Chris Taggart
@countculture
✤ Developer of OpenlyLocal.com
✤ Makes local data (esp govt) available
in consistent and accessible form
✤ Scrapes council and govt websites
✤ All open data
9. Me: Chris Taggart
@countculture
✤ Developer of OpenlyLocal.com
✤ Makes local data (esp govt) available
in consistent and accessible form
✤ Scrapes council and govt websites RDF JSON XML
✤ All open data
10. Me: Chris Taggart
@countculture
✤ Developer of OpenlyLocal.com
✤ Makes local data (esp govt) available
in consistent and accessible form
✤ Scrapes council and govt websites RDF JSON XML
✤ All open data
11. Me: Chris Taggart
@countculture
✤ Developer of OpenlyLocal.com
✤ Makes local data (esp govt) available
in consistent and accessible form
✤ Scrapes council and govt websites RDF JSON XML
✤ All open data
✤ Open Election Data project
12. Me: Chris Taggart
@countculture
✤ Developer of OpenlyLocal.com
✤ Makes local data (esp govt) available
in consistent and accessible form
✤ Scrapes council and govt websites RDF JSON XML
✤ All open data
✤ Open Election Data project
13. Me: Chris Taggart
@countculture
✤ Developer of OpenlyLocal.com
✤ Makes local data (esp govt) available
in consistent and accessible form
✤ Scrapes council and govt websites RDF JSON XML
✤ All open data
✤ Open Election Data project
✤ Helping local authorities publish their
election results as open data
14. Me: Chris Taggart
@countculture
✤ Developer of OpenlyLocal.com
✤ Makes local data (esp govt) available
in consistent and accessible form
✤ Scrapes council and govt websites RDF JSON XML
✤ All open data
✤ Open Election Data project
✤ Helping local authorities publish their
election results as open data
✤ Tackling the open local data problem,
one set at a time & learning lessons on
the way
15. Me: Chris Taggart
@countculture
✤ Developer of OpenlyLocal.com
✤ Makes local data (esp govt) available
in consistent and accessible form
✤ Scrapes council and govt websites RDF JSON XML
✤ All open data
✤ Open Election Data project
✤ Helping local authorities publish their
election results as open data
✤ Tackling the open local data problem,
one set at a time & learning lessons on
the way
16. Me: Chris Taggart
@countculture
✤ Developer of OpenlyLocal.com
✤ Makes local data (esp govt) available
in consistent and accessible form
✤ Scrapes council and govt websites RDF JSON XML
✤ All open data
✤ Open Election Data project
✤ Helping local authorities publish their
election results as open data
✤ Tackling the open local data problem,
one set at a time & learning lessons on
the way
✤ Member of Local Public Data Panel
18. Open data: why do we want it?
✤ Stupid question: It’s our data
19. Open data: why do we want it?
✤ Stupid question: It’s our data
20. Open data: why do we want it?
✤ Stupid question: It’s our data
✤ Engagement – reducing barriers to
getting involved
21. Open data: why do we want it?
✤ Stupid question: It’s our data
✤ Engagement – reducing barriers to
getting involved
22. Open data: why do we want it?
✤ Stupid question: It’s our data
✤ Engagement – reducing barriers to
getting involved
✤ Equality of access to information (and
thus power). Much of this data is
already available... for a price
23. Open data: why do we want it?
✤ Stupid question: It’s our data
✤ Engagement – reducing barriers to
getting involved
✤ Equality of access to information (and
thus power). Much of this data is
already available... for a price
24. Open data: why do we want it?
✤ Stupid question: It’s our data
✤ Engagement – reducing barriers to
getting involved
✤ Equality of access to information (and
thus power). Much of this data is
already available... for a price
✤ Efficiency – Endless amount of
duplication in all levels of government
because they can’t/won’t share data
25. Open data: why do we want it?
✤ Stupid question: It’s our data
✤ Engagement – reducing barriers to
getting involved
✤ Equality of access to information (and
thus power). Much of this data is
already available... for a price
✤ Efficiency – Endless amount of
duplication in all levels of government
because they can’t/won’t share data
✤ but most important of all...
31. But wait a minute...
Lord Mandelson photo: http://www.flickr.com/photos/bisgovuk/4382508152/
32. But wait a minute...
Lord Mandelson photo: http://www.flickr.com/photos/bisgovuk/4382508152/
To some people and
companies these are all
reasons for not doing it.
From their perspective,
it’s all downside
33. But wait a minute...
Lord Mandelson photo: http://www.flickr.com/photos/bisgovuk/4382508152/
To some people and
companies these are all
reasons for not doing it.
From their perspective,
it’s all downside
34. But wait a minute...
Lord Mandelson photo: http://www.flickr.com/photos/bisgovuk/4382508152/
To some people and
companies these are all
reasons for not doing it.
From their perspective,
it’s all downside
35. But wait a minute...
Lord Mandelson photo: http://www.flickr.com/photos/bisgovuk/4382508152/
To some people and
companies these are all
reasons for not doing it.
From their perspective,
it’s all downside
36. Open data: why do we want it?
✤ Stupid question: It’s our data
✤ Engagement – reducing barriers to
getting involved
✤ Equality of access to information (and
thus power). Much of this data is
already available... for a price
✤ Efficiency – Endless amount of
duplication in all levels of government
because they can’t/won’t share data
✤ but most important of all...
37. don’t the
Open data: why do we want tit?ve it?
o ha
y want you
✤ Stupid question: It’s our data
✤ Engagement – reducing barriers to
getting involved
✤ Equality of access to information (and
thus power). Much of this data is
already available... for a price
✤ Efficiency – Endless amount of
duplication in all levels of government
because they can’t/won’t share data
✤ but most important of all...
38. don’t the
Open data: why do we want tit?ve it?
o ha
y want you
t; we collected it,
✤ No it’s no It’svourus power
Stupid question:t gi es data
it’s wha
✤ Engagement – reducing barriers to
getting involved
✤ Equality of access to information (and
thus power). Much of this data is
already available... for a price
✤ Efficiency – Endless amount of
duplication in all levels of government
because they can’t/won’t share data
✤ but most important of all...
39. don’t the
Open data: why do we want tit?ve it?
o ha
y want you
t; we collected it,
✤ No it’s no It’svourus power
Stupid question:t gi es data
it’s wha
✤ Engagement – duces power of to
Rereducing barriers olitical/
p
getting involvedpublic service elite
✤ Equality of access to information (and
thus power). Much of this data is
already available... for a price
✤ Efficiency – Endless amount of
duplication in all levels of government
because they can’t/won’t share data
✤ but most important of all...
40. don’t the
Open data: why do we want tit?ve it?
o ha
y want you
t; we collected it,
✤ No it’s no It’svourus power
Stupid question:t gi es data
it’s wha
✤ Engagement – duces power of to
Rereducing barriers olitical/
p
getting involvedpublic service elite
✤ Equality of access to information (and
thus power).goes the competitive
Bang Much of this data is
already available... antage
adv for a price
✤ Efficiency – Endless amount of
duplication in all levels of government
because they can’t/won’t share data
✤ but most important of all...
41. don’t the
Open data: why do we want tit?ve it?
o ha
y want you
t; we collected it,
✤ No it’s no It’svourus power
Stupid question:t gi es data
it’s wha
✤ Engagement – duces power of to
Rereducing barriers olitical/
p
getting involvedpublic service elite
✤ Equality of access to information (and
thus power).goes the competitive
Bang Much of this data is
already available... antage
adv for a price
✤ Efficiency – Endless amount of
Fewer people = difficult d
ecisions,
duplication in all levels of government
less power, less prestige
because they can’t/won’t share data
✤ but most important of all...
42. don’t the
Open data: why do we want tit?ve it?
o ha
y want you
t; we collected it,
✤ No it’s no It’svourus power
Stupid question:t gi es data
it’s wha
✤ Engagement – duces power of to
Rereducing barriers olitical/
p
getting involvedpublic service elite
✤ Equality of access to information (and
thus power).goes the competitive
Bang Much of this data is
already available... antage
adv for a price
✤ Efficiency – Endless amount of
Fewer people = difficult d
ecisions,
duplication in all levels of government
less power, less prestige
because they can’t/won’t share data
✤ but most important of all...
scary
46. The rewards of failure today
✤ Big projects/policy initiatives bring media attention, power &
influence (and the key individuals will have moved on by the time
its failure is publicly realized)
47. The rewards of failure today
✤ Big projects/policy initiatives bring media attention, power &
influence (and the key individuals will have moved on by the time
its failure is publicly realized)
✤ Spread the risk by emphasizing process over results, scoping
everything out to the nth degree (you get very little benefit from
doing good in public service; lots of grief by visibly doing bad)
48. The rewards of failure today
✤ Big projects/policy initiatives bring media attention, power &
influence (and the key individuals will have moved on by the time
its failure is publicly realized)
✤ Spread the risk by emphasizing process over results, scoping
everything out to the nth degree (you get very little benefit from
doing good in public service; lots of grief by visibly doing bad)
✤ Don’t do small, quick, innovative projects (it’s too easy to see whether
they have succeeded... or failed, and difficult to spread the blame)
49. The rewards of failure today
✤ Big projects/policy initiatives bring media attention, power &
influence (and the key individuals will have moved on by the time
its failure is publicly realized)
✤ Spread the risk by emphasizing process over results, scoping
everything out to the nth degree (you get very little benefit from
doing good in public service; lots of grief by visibly doing bad)
✤ Don’t do small, quick, innovative projects (it’s too easy to see whether
they have succeeded... or failed, and difficult to spread the blame)
✤ Outsource everything (see ‘process over results’ above. Bonus:
‘commercial confidentiality’ means the facts can kept secret)
50. no better
The private sector’s
http://www.flickr.com/photos/pingnews/2935021856/
52. http://www.flickr.com/photos/pingnews/2935021856/
The private sector’s
no better
✤ Finance industry incentivized to take
risks, reap short-term rewards and let others bear the consequences
✤ Bigger, more opaque and more complex schemes give protection to
people & companies (until it all falls apart)
53. http://www.flickr.com/photos/pingnews/2935021856/
The private sector’s
no better
✤ Finance industry incentivized to take
risks, reap short-term rewards and let others bear the consequences
✤ Bigger, more opaque and more complex schemes give protection to
people & companies (until it all falls apart)
✤ Senior executives insulated from shareholders by diverse and
diffuse nature of the reporting/ownership system
54. http://www.flickr.com/photos/pingnews/2935021856/
The private sector’s
no better
✤ Finance industry incentivized to take
risks, reap short-term rewards and let others bear the consequences
✤ Bigger, more opaque and more complex schemes give protection to
people & companies (until it all falls apart)
✤ Senior executives insulated from shareholders by diverse and
diffuse nature of the reporting/ownership system
✤ People at the top are incentivized to make big changes (because
intelligent comparison becomes difficult); people at the bottom
incentivized to do as they’re told
56. The rewards of failure done right
✤ The open-source movement has been founded
on failure and fixing that failure, with
amazing results (and amazing stability)
57. The rewards of failure done right
✤ The open-source movement has been founded
on failure and fixing that failure, with
amazing results (and amazing stability)
✤ More than that, it’s failure in public.
Makes all bugs shallow; all faults visible
58. The rewards of failure done right
✤ The open-source movement has been founded
on failure and fixing that failure, with
amazing results (and amazing stability)
✤ More than that, it’s failure in public.
Makes all bugs shallow; all faults visible
✤ It also allows a meritocracy to develop,
incentivizing innovation and quality,
encouraging people to do their best work
59. The rewards of failure done right
✤ The open-source movement has been founded
on failure and fixing that failure, with
amazing results (and amazing stability)
✤ More than that, it’s failure in public.
Makes all bugs shallow; all faults visible
✤ It also allows a meritocracy to develop,
incentivizing innovation and quality,
encouraging people to do their best work
✤ The internet and the web are not only
accepting of failure, the inevitability of
failure is at the heart
61. How can Open Data help?
✤ It helps change the incentives, from Fail Big, Fail Slow to
Fail Small, Fail Fast, and Fail Forward (i.e. building on the failure)
62. How can Open Data help?
✤ It helps change the incentives, from Fail Big, Fail Slow to
Fail Small, Fail Fast, and Fail Forward (i.e. building on the failure)
✤ It helps surface issues/problems making them less likely to be
repeated (learning from other people’s failures)
63. How can Open Data help?
✤ It helps change the incentives, from Fail Big, Fail Slow to
Fail Small, Fail Fast, and Fail Forward (i.e. building on the failure)
✤ It helps surface issues/problems making them less likely to be
repeated (learning from other people’s failures)
✤ It allows you to stand on the shoulders of giants. Don’t reinvent the
wheel – use or improve someone else’s
64. How can Open Data help?
✤ It helps change the incentives, from Fail Big, Fail Slow to
Fail Small, Fail Fast, and Fail Forward (i.e. building on the failure)
✤ It helps surface issues/problems making them less likely to be
repeated (learning from other people’s failures)
✤ It allows you to stand on the shoulders of giants. Don’t reinvent the
wheel – use or improve someone else’s
✤ It increases the quality of data – reuse by other people and combining
with other data identifies (and helps solve) quality issues far more
quickly and cheaply than internal checking
66. We need to recast the debate
✤ Perhaps we need to talk about continuous improvement rather than
failure; evolution (which has failure built in) rather than revolution
67. We need to recast the debate
✤ Perhaps we need to talk about continuous improvement rather than
failure; evolution (which has failure built in) rather than revolution
✤ Be upfront about problems, knowledge gaps, issues
68. We need to recast the debate
✤ Perhaps we need to talk about continuous improvement rather than
failure; evolution (which has failure built in) rather than revolution
✤ Be upfront about problems, knowledge gaps, issues
✤ Less centralised complexity; more distributed innovation
69. We need to recast the debate
✤ Perhaps we need to talk about continuous improvement rather than
failure; evolution (which has failure built in) rather than revolution
✤ Be upfront about problems, knowledge gaps, issues
✤ Less centralised complexity; more distributed innovation
✤ Make public data open by default (and include outsourced work/
data), particularly anything to do with spending money
70. We need to recast the debate
✤ Perhaps we need to talk about continuous improvement rather than
failure; evolution (which has failure built in) rather than revolution
✤ Be upfront about problems, knowledge gaps, issues
✤ Less centralised complexity; more distributed innovation
✤ Make public data open by default (and include outsourced work/
data), particularly anything to do with spending money
✤ Outsource commodity tasks, not skills and knowledge (and avoid the
used-car buyer information trap). Stop the vendoritis
71. We need to recast the debate
✤ Perhaps we need to talk about continuous improvement rather than
failure; evolution (which has failure built in) rather than revolution
✤ Be upfront about problems, knowledge gaps, issues
✤ Less centralised complexity; more distributed innovation
✤ Make public data open by default (and include outsourced work/
data), particularly anything to do with spending money
✤ Outsource commodity tasks, not skills and knowledge (and avoid the
used-car buyer information trap). Stop the vendoritis
✤ Reward small-scale innovation and spending less money (not big
projects and spending more money)
subtitle: These are the things you learn in public sector
subtitle: These are the things you learn in public sector
subtitle: These are the things you learn in public sector
subtitle: These are the things you learn in public sector
Note: Stability is very important when we’re talking about the public sector
Note: Stability is very important when we’re talking about the public sector
Note: Stability is very important when we’re talking about the public sector
Note: Stability is very important when we’re talking about the public sector
Ideally I’d like to reclaim back the concept of failure, back to Edison’s view of it:
“I am not discouraged, because every wrong attempt discarded is another step forward.”
Take the Netflix approach: here’s the data, help us understand it better; show us how to use it to improve things.
The Market for Lemons: George Akerlof, Michael Spence, and Joseph Stiglitz
Ideally I’d like to reclaim back the concept of failure, back to Edison’s view of it:
“I am not discouraged, because every wrong attempt discarded is another step forward.”
Take the Netflix approach: here’s the data, help us understand it better; show us how to use it to improve things.
The Market for Lemons: George Akerlof, Michael Spence, and Joseph Stiglitz
Ideally I’d like to reclaim back the concept of failure, back to Edison’s view of it:
“I am not discouraged, because every wrong attempt discarded is another step forward.”
Take the Netflix approach: here’s the data, help us understand it better; show us how to use it to improve things.
The Market for Lemons: George Akerlof, Michael Spence, and Joseph Stiglitz
Ideally I’d like to reclaim back the concept of failure, back to Edison’s view of it:
“I am not discouraged, because every wrong attempt discarded is another step forward.”
Take the Netflix approach: here’s the data, help us understand it better; show us how to use it to improve things.
The Market for Lemons: George Akerlof, Michael Spence, and Joseph Stiglitz
Ideally I’d like to reclaim back the concept of failure, back to Edison’s view of it:
“I am not discouraged, because every wrong attempt discarded is another step forward.”
Take the Netflix approach: here’s the data, help us understand it better; show us how to use it to improve things.
The Market for Lemons: George Akerlof, Michael Spence, and Joseph Stiglitz
Ideally I’d like to reclaim back the concept of failure, back to Edison’s view of it:
“I am not discouraged, because every wrong attempt discarded is another step forward.”
Take the Netflix approach: here’s the data, help us understand it better; show us how to use it to improve things.
The Market for Lemons: George Akerlof, Michael Spence, and Joseph Stiglitz