Axa Assurance Maroc - Insurer Innovation Award 2024
IOGDC Open Data Tutorial
1. 2012 INTERNATIONAL
OPEN GOVERNMENT DATA
CONFERENCE
Open Government Data Tutorial
Hadley Beeman
Jim Hendler
Jeanne Holm
Presenter Contact Information
Organized by the World Bank and Data.gov
2. Introductions!
• Please introduce yourself
– Name
– Organization
– Three (3) words that explain either why you are
here or what you hope to learn
2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL
9 July 2012 IOGDC Open Data Tutorial 2
3. Understanding the
Foundations of Open Data
• Why do countries and people share data?
• What will citizens, businesses, scientists, and
journalists do with the data?
• How can we manage it?
2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL
9 July 2012 IOGDC Open Data Tutorial 3
4. Why Countries Share Data
• Meet regulatory compliance
• Provide transparency into government
operations
2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL
9 July 2012 IOGDC Open Data Tutorial 4
5. Why Countries Share Data
• Anticipate economic development
• Initiate innovation
2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL
9 July 2012 IOGDC Open Data Tutorial 5
6. 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL
9 July 2012 IOGDC Open Data Tutorial 6
7. Why People Want Open Data
Swati Ramanathan
2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL
9 July 2012 IOGDC Open Data Tutorial 7
8. Real Outcomes = Better Lives
• Open data eases the burden on families in caring for a sick
child or seeking medical information
• More importantly, the data as it’s aggregated empowers
communities to make changes that improve the quality of
life of citizens
– In California, ReLeaf plants trees in areas identified by Together
We Breathe as danger areas for asthma sufferers
– Cities see hot spots that trigger asthma problems for their
citizens
• Each company in Health 2.0 used government data to
innovate and create high-value jobs in America
• Civic Commons has a great collection of good open use
cases: http://civiccommons.org/
2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL
9 July 2012 IOGDC Open Data Tutorial 8
9. Energy Drives Innovation
• Energy.Data.gov
connects
innovators,
industry,
academia, and
government at
federal, state,
and local levels
2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL
9 July 2012 IOGDC Open Data Tutorial 9
10. Challenges Spark Ideas
• Energy.Data.gov
connects works
with challenges
across the
nation to
integrate federal
data and bring
government
personnel to
code-a-thons
2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL
9 July 2012 IOGDC Open Data Tutorial 10
11. Data Drives Decisions
• Apps transform data
in understandable
ways to help people
make decisions
2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL
9 July 2012 IOGDC Open Data Tutorial 11
12. Changing Economic
Equations
Study from Malaysian government:
http://www.transknowformance.com/article.cfm?id=53
2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL
9 July 2012 IOGDC Open Data Tutorial 12
13. Why People Want Open
Data
2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL
9 July 2012 IOGDC Open Data Tutorial 13
14. What Makes Data Open
• Open Format
– The US Government through the Open
Government Directive
(http://www.whitehouse.gov/omb/assets/memor
anda_2010/m10-06.pdf ) defines an open format
as “one that is platform independent, machine
readable, and made available to the public
without restrictions that would impede the re-use
of that information.”
2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL
9 July 2012 IOGDC Open Data Tutorial 14
15. What Makes Data Open
• Example Open Formats:
– PDF for documents (but not data)
– CSV for data
– Web standards for publishing, sharing or linking
• HTML, XML, RDF
– Web standards for syndication
• RSS, Atom, JSON
2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL
9 July 2012 IOGDC Open Data Tutorial 15
16. What Makes Data Open
• Metadata
– The information about the data being shared
• Who produced it
• Where
• When
• Use restrictions
• Etc.
– Use standards such as ADMS or Dublin Core
– New (Search-oriented) Embedded dataset
metadata coming
2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL
9 July 2012 IOGDC Open Data Tutorial 16
17. What Topics of Data Are
Published
• Analytics based on over 1,000,000 datasets
from around the world can be seen at
– http://logd.tw.rpi.edu/iogds_data_analytics
• The examples that follow are from that page
2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL
9 July 2012 IOGDC Open Data Tutorial 17
18. Countries Sharing Data
Important note:
quantity is not really the most
important issue
2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL
9 July 2012 IOGDC Open Data Tutorial 18
19. Countries Sharing Data
Important note:
quantity is not really the most
important issue
2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL
9 July 2012 IOGDC Open Data Tutorial 19
20. Example: US
Data.gov
2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL
9 July 2012 IOGDC Open Data Tutorial 20
21. Example: UK
Data.gov.uk
2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL
9 July 2012 IOGDC Open Data Tutorial 21
22. Example: Spain
2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL
9 July 2012 IOGDC Open Data Tutorial 22
23. Topics (Across All Catalogs)
2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL
9 July 2012 IOGDC Open Data Tutorial 23
24. Topics (Across All Catalogs)
2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL
9 July 2012 IOGDC Open Data Tutorial 24
25. What Topics of Data Are
Published (UK)
2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL
9 July 2012 IOGDC Open Data Tutorial 25
26. Data “Mashups” of Many
Kinds
More than 50 of these at http://logd.tw.rpi.edu
2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL
9 July 2012 IOGDC Open Data Tutorial 26
27. Making Data Open,
Accessible, and Discoverable
• Architecture for systems and technology
• Processes for publishing data
• Policies for ensuring data is open, accessible,
and obtainable
2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL
9 July 2012 IOGDC Open Data Tutorial 27
28. Creating an Open Data
Architecture
• Key components
– Workflow for release approval (often overlooked)
– Dataset storage
• Can be centralized or via linking
– Data Cataloging
• Metadata critical to a good open data site
– Data API
• Can be via download or via access
• Technical issues with syndication, usage rules, etc.
2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL
9 July 2012 IOGDC Open Data Tutorial 28
29. Processes
• Publication (and cleaning)
• Data reuse and integration
• Community input
2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL
9 July 2012 IOGDC Open Data Tutorial 29
30. Policies Become Essential
• Policies are needed to help drive the ecosystem and
“motivate” departments to continue to share data openly
• Build the policies based around issues that are universal
• Licensing, provenance: http://creativecommons.org/licenses/
Open data on food,
security,
transportation, and
transparency
2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL
9 July 2012 IOGDC Open Data Tutorial 30
31. 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL
9 July 2012 IOGDC Open Data Tutorial 31
32. Generations Share Differently
• 1930-50’s era generation
– Focus on society
– Friendships are forged through adversity
• 1960-70’s era generation
– Focus on community
– Friendships forged through identification with a cause
• 1980-90’s era generation
– Focus on the individual
– Friendships forged through individual goal accomplishment
• 2000’s era generation
– Focus on common interests
– Friendships are created or thrive virtually…
2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL
9 July 2012 IOGDC Open Data Tutorial 32
33. Creating Change is a Social
Construct
• The social aspects of an organization that help to
facilitate, create, and ensure that the goals of the
organization are met
– Social networks
– Norms and values
– Social trust
– Networks and communities
– Physical spaces
– Storytelling
– Social capital consists of the stock of active connections among people;
the trust, mutual understanding, and shared values and behaviors that
bind the members of human networks and communities and make
cooperative action possible
• Robert Putnam, Harvard University
2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL
9 July 2012 IOGDC Open Data Tutorial 33
34. Trust and Reciprocity
• Trust can be built on
• Personal experience
• “I know you”
• Shared experience
• “We both worked on
the same project”
• Transfer of trust
• “We know the same
person who trusts us”
• Shared values
• “We agree to operate
by the same rules”
2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL
9 July 2012 IOGDC Open Data Tutorial 34
35. Semantic Web and Linked
Data (UK)
Royal Mail
County Council
9 July 2012 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL
IOGDC Open Data Tutorial 35
Ordnance Survey
36. Linking Data Via Common
Naming (Usually URLs)
2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL
9 July 2012 IOGDC Open Data Tutorial 36
37. Example: Agency Names
2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL
9 July 2012 IOGDC Open Data Tutorial 37
38. Can Be Lots of Things
2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL
9 July 2012 IOGDC Open Data Tutorial 38
39. “Linking” Data goes
beyond govt
Government Data is
currently over ½ the cloud in
size (~17B triples), 10s of
thousands of links to other
data (within and without)
2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL
http://linkeddata.org/ Tutorial
9 July 2012 IOGDC Open Data 39
40. 5 Star Data
2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL
9 July 2012 IOGDC Open Data Tutorial 40
41. Creating an Open Data
Ecosystem
2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL
9 July 2012 IOGDC Open Data Tutorial 41
42. Sustaining Data Publishing
• A steward (someone tied to the business process)
– Defines the resource
– Purpose, audience, content, context, policies, operations,
metrics, and access method
– Manages the resource
– Measures performance
– Identifies needs and opportunities
– Applies and enforces policies
– Oversees daily operations
– An open data project then is an enabler to creating the
environment in which the processes can occur
2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL
9 July 2012 IOGDC Open Data Tutorial 42
43. Embedding Publishing
• There are three ways to look at creating a sustainable
open data initiative
– Step 1: Process: Oriented on the way in which people do
their day-to-day work in the organization (the how and
why)
– Step 2: Services: Focused on who will help people share
their knowledge and who will maintain tools and processes
(the who)
– Step 3: Systems: Are the IT infrastructure and tools
necessary to deliver the processes and services efficiently
and effectively to the end users (the what and where)
2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL
9 July 2012 IOGDC Open Data Tutorial 43
44. Creating Community
• Communities are public-facing
spaces that present data,
information, and subject matter
knowledge about a single topic
from many organizations in one
place
– The topics for communities can be
chosen based on priorities from the
public, departments based on their
mission, or issues of national
importance
2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL
9 July 2012 IOGDC Open Data Tutorial 44
45. Community Vision
• These questions help to guide early discussions
– Vision: What will the community connection and collaboration look
like in the future?
– Leaders: Who will help to lead the community?
– Participants: Who will participate?
– Outcome: What are the expected outcomes, metrics, and
measurements that will show success? How will this community work
to improve the lives of citizens?
– Functionality: What types of activities will be conducted on the site
(forums, blogs, wikis, ranking, rating, challenges, or apps)?
– Content: What content should be displayed
– Interactivity: What ways will the community interact with the leaders,
with each other, and with the public?
2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL
9 July 2012 IOGDC Open Data Tutorial 45
46. Open Communities
Community
✓
Developers
Open Data ✓
Semantic Web ✓
Health ✓
Law ✓
Energy ✓
Education ✓
Ocean ✓
Safety ✓
Manufacturing ✓
Business ✓
Ethics ✓
Consumer
Research and Development
Cities
+ many more…
2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL
9 July 2012 IOGDC Open Data Tutorial 46
47. Supporting Global Events
Japanese tsunami, earthquake,
and radiation monitoring
Restore the Gulf:
Deepwater Horizon
Response
2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL
9 July 2012 IOGDC Open Data Tutorial 47
48. Health.Data.gov
Champion: Todd Park
Chief Technology Officer
US Dept. of Health and
Human Services
Apps Forums
Blogs
Challenges
2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL
9 July 2012 IOGDC Open Data Tutorial 48
49. Publicizing Data to Innovators
• Challenges and code‐a-thons
(health2challenge.org)
• Many innovator “meetups” and
conferences
• Annual health data-paloozas
• Over 139 applications
• 50 new businesses
• Thousands of lives improved
each day
• 1700 attendees at the Health
Data Palooza in 2012
2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL
9 July 2012 IOGDC Open Data Tutorial 49
50. Creating Apps That Improve Lives:
Asthmapolis
2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL
9 July 2012 IOGDC Open Data Tutorial 50
51. Creating Apps That Save
Lives: iTriage and Hospital
Compare
2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL
9 July 2012 IOGDC Open Data Tutorial 51
52. Use Case: Economic
• Healthtap
– Provides free, reliable, and
independent health information
through the use of open data
and by building the world’s
largest medical expert network
tailored for young mothers and
their children
– Won 2010 Health 2.0 challenge
– Success invigorated expansion
and new hiring at HealthTap
– Reaches to Data.gov datasets
(along with others)
2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL
9 July 2012 IOGDC Open Data Tutorial 52
53. Ensuring Use and Economic
Benefits
European Union market: EUR 28 billion in
2008 (Vickery)
£6 billion in the UK (Newbury, Bently and
Pollock 2008)
The Open Data Institute
2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL
9 July 2012 IOGDC Open Data Tutorial 53
54. Used in Collaborative Planning and
Governance Processes (from CTG SunyA)
STEP 1 STEP 2 STEP 3 STEP 4 STEP 5 STEP 6
Describe Identify and Identify and Rate Identify Summarize the Review Open
Initiative Prioritize the Public Value Stakeholder Public Value Government Public
Stakeholders Interests Assessments Value Portfolio
Initiative
Description: Identify a full list Identify the type For each Establish a Review public value
of initiative and level of stakeholder and summary assessments
Title stakeholders Public Value the each change assessment for across all initiatives
Purpose initiative is mechanism the initiative for to inform decision
Supporting Prioritize by expected to describe how each public value making.
strategic goals rating each create for each of the expected type across all
Program or policy stakeholder as the primary value will be the primary Does our open
area A, B, or C stakeholders produced stakeholders government
Tactics portfolio, taken as a
Public Value is Change All summary whole, optimize our
expressed in terms mechanisms include assessments are resources and
of: impacts on: populated into capabilities while
Economic value Efficiency the Portfolio meeting our
Social Value Effectiveness Review mission and
Political Value Intrinsic Workbook delivering
Strategic Value Enhancement (Step 6) maximum public
Quality of Life Transparency value to all
Value Collaboration stakeholders?
Ideological Value Participation
Stewardship
Value
Initiative One-Five Workbooks
Portfolio Review
Workbook
2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL
55. What Can You Do With
Open Data?
• What are the uses of open data you have seen
in your city, country, or organization?
• Write down on the post-it notes one or more
of the following:
– What has been done with open data
– What could be done with open data
– What are the challenges you are facing
– What have been your successes
2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL
9 July 2012 IOGDC Open Data Tutorial 55
56. Additional Topics
• Licensing, provenance, languages
• Metadata design (international)
• Trust – government data is controversial, who
controls it?
• Scaling – over 1M datasets and growing fast
– How to search, store, link, translate, and archive
• Versioning and updating
• Visualization beyond the single dataset
• Boundaries of open data
2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL
9 July 2012 IOGDC Open Data Tutorial 56
57. Questions
2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL
9 July 2012 IOGDC Open Data Tutorial 57
58. Summary and Next Steps
• Join a community
– W3C eGovernment Interest Group
• http://www.w3.org/egov/wiki/Main_Page
– Open Data Innovation Network on LinkedIn
• http://bit.ly/ODNetwork
2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL
9 July 2012 IOGDC Open Data Tutorial 58
Editor's Notes
A parent has a child who is illAsk questions online at HealthTapFind a hospital and compare (Hospital Compare)That doctor recommends GPS-powered inhaler (Asthmapolis)Monitor asthma levels at school through Public School RecordsKnow in advance the best places to play, how to get to school, and how to plan your dayThe data delivered through the 172 agencies participating in Data.gov eases the burden on families in caring for a sick childMore importantly, the data as it’s aggregated empower communities to make changes that improve the qualiy of life of citizens(ReLeaf plants trees in areas identified by Together We Breathe as danger areas for asthma sufferersCities see hot spots that trigger asthma problems for their citizens
Uses asthma patients aggregated GPS notations to create hot spots in communities where there are asthma issuesChanges individual behaviorFrom 65% daily incidence to 25% daily incidence of inhaler uses over a six month study