1. {Open|Big|Linked} Data
Enabling better policy, services and cost efficiency in Government
Version 0.5
Pia Waugh
Director of Coordination and Gov 2.0
Technology and Procurement Division
Office of the Australian Government CTO
1
2. Benefits to Government in Opening Data
Cuts red tape
• Improves efficiencies in sharing data across government and with public
• Proactive automated publishing rather than manual retrospective approach
Improves Government Operations
• Enables collaboration and consistency across government and with public
• Improves policy analysis, development, implementation and reporting
• Improves service delivery by enabling thematic personalised approach to info
including mobile services that leverage cloud hosted data and automated APIs
• Improves data quality through enabling verifiable public contributions
Innovation
• Enables innovation and new opportunities in government, industry and research
• Enables greater capacity for public to contribute meaningfully to public policy
2
4. The APS Policy Landscape
Others:
• Publishing Public Sector Information & National Standards Framework
• Open Public Sector Information: From Principles to Practice Report
• Declaration of Open Government
• Gov 2.0 Taskforce Report
• Statement of IP Principles for Government (CC-BY)
• Ahead of the Game
• Digital Transition Policy (Archives) & Accessibility Policy
• Emerging Open Research Policies
• Open Government Partnership (TBD)
4
6. Policies Components
APS:
• Permissive copyright – CC-BY as the default
• Open by default
• Support reuse and innovation
• More public engagement
• Better use of data for government policy and service development
States/Territories add:
• Procurement – open by design
• Reporting – dashboards
• Departmental strategies
6
7. APS ICT Strategy
Declaration of open govt
Online engagement
Online interaction
APS use of social media
Connected service delivery Open public sector info
ICT investment framework
Skills and capability
APS ICT Strategy
2012-2015
Operational efficiency ($1.8B)
Improved agency capability
Whole-of-govt approaches
Coordinated procurement
Benchmarking
promoting better government through
the innovative and strategic use of ICT
Using ICT to increase public sector
and national productivity by:
Enabling better service delivery
Improving the efficiency of
government operations
Supporting open engagement to
better inform decisions
addressing today’s challenges…
Public
expectations
Productivity
performance
Technology
advances
Agencies
pressures
Influenced by innovative digital
private sector services,
broadband availability,
smartphone take-up, social
media and blogs
Global economic impacts,
increasing global competition,
reduced resource demand,
demographic changes,
environmental constraints
Mobile, broadband, cloud
computing, virtualisation, big
data analytics and other
emerging technologies
Meeting outcomes and
expectations, funding
challenges, efficiency dividends,
organisational change, and
achieving ICT delivery targets
Australian Government Information Management Office
7
8. Big Data Strategy
VISION
PRINCIPLES
ACTIONS
Enhanced services
Data is a national asset
Develop better practice guidance
New services and business
partnership opportunities
Privacy by design
Identify and report on barriers to
adoption of big data analytics
Improved policy
development
Data integrity and the
transparency of process
Protection of privacy
Skills, resources and
capabilities will be shared
Leveraging Government’s
investment in ICT
Collaboration with industry
and academia
Enhancing open data
Enhance skills and experience in
big data analysis – initiation and
support of pilot projects
Develop guide to responsible
data analytics
Develop guidance to enable
agencies to create information
asset registers
Monitor technical advances
Australian Government Information Management Office
8
9. Privacy and confidentiality
• Custom API approach that confidentialises on the fly (eg, ABS)
• Deidentification of data – to appropriate level
• Aggregation
• Leveraging existing processes for researcher (unit level) access rather
than conflating open data discussions
• Privacy Commissioner as point of reference and support
• Avoiding common identifiers across multiple datasets
9
10. Open by Design
Building proactive publishing into:
• Systems
• Processes
• Procurement
• Planning
• Records management
Leveraging open data through:
• Public APIs
• Analysis tools and datavis
• Internal processes looking for external sources
10
12. Other Data Projects
• Spatial
• Geosciences
• Research
• Sensor
• Realtime (eg Transport)
• Census/Statistics
• Cultural
• Data about government
• International:
Aid/Extractive Industries
12
13. Loads of Tools Available
• Publishing tools – CKAN, Socrata, bespoke
• Automation – FME, Kettle
• Data visualisation – Tableau, SuperDataHub,
SpatialKey
• Analysis – R, domain specialist software, Palantir
• API development
• Application development
• Linked data tools
• Metadata tools
13
14. New and Old Skills Required
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
Publishing
Automation
Metadata/linked data
API development
Plumbing
Data visualisation
Analysis and statistics
Policy development
Public consultation
Online skills
14
15. Some Challenges
• Legislative
• Culture
• Systems
• Reactive vs proactive
• Metadata/semantic context
• Too much data
• Real time vs historic
• Definitions and common references
• Limited skills and over specialisation
15
16. data.gov.au
Free, cloud based, highly scalable platform for hosting government data.
Staged approach
1. Publishing (2013)
Improving the functionality and ease of
publishing for agencies with training and
documentation
2.
Value realisation (Early 2014)
Providing useful front end tools for data.gov.au
including data visualisation and analysis tools
3.
Data quality (Late 2014)
Looking at ways to provide agencies the ability
to accept iterative data improvements in a
verifiable way
Features
• Federated search making data
and data services easier to find
• Manual and automated publishing
options
• API access to government data
• Easy to publish, download and
interact with data online
• Basic data visualisation capability
16
17. All the pieces are in place,
we need people to put the puzzle together
17