2. …so what are we talking about…
Introductions
Do You Open Data?
Defining Open Data
How Do I Participate?
Q&A
3. Who Am I?
• Architect @ Dell Technologies since 2011
• From St. Paul, MN
• Longtime Software Engineer/Architect
• Business and Software Shops
• Individual Contributor and Leadership Roles
• All phases from project conception to implementation
• Sideline projects at home
• Music Lover
• 10+ years online radio station of personal music
• Contact Info
• Email: scott [dot] sosna [@] buddhadata [dot] com
• LinkedIn: https://www.linkedin.com/in/scott-sosna
• Slideshare: https://www.slideshare.net/ScottSosna
5. Citymapper
Initially for buses in London
Creator struggled getting from point-to-point in London
Now Supports 39 major world cities
Released 2012 (iOS), 2013 (Android)
Early adopter of TfL real-time Bus/Tube APIs
Live bus travel data saves as much as £58 million a year, by helping
Londoners and the city’s many visitors plan better routes and avoid
long waits at bus stops.
Center for Public Impact, March 2016
6. Zillow
Integrates real estate data with other public data sources to provide
nuanced view for consumers, buyers and sellers
Data available but difficult to access for average person
Levels playing field for buyers/sellers/builders, allows informed
decisions
Open Data includes
Property valuations/taxes and real estate transactions
US Census information
Local crime statistics
7. Flight Aware
All aspects of Aviation
Global Flight Tracking
Live Flight Data
Airport Delays
Airport Flight Information Displays
Weather Maps
Flight Planning
Government Data Sources
Federal Aviation Administration
8. Charles Schwab
Financial Services
Investing
Wealth management
Banking
Trading
Real-time (almost) market information
Historical market and economic indicators
Government Data Sources
Department of the Treasury
Securities and Exchange Commission
12. Definitions
Open data is data that can be freely used, shared, and built-on by anyone,
anywhere, for any purpose. Open Knowledge International
Open data is data that can be freely used, re-used, and redistributed by anyone
– subject only, at most, to the requirement to attribute and share alike. Open Data
Handbook
Open data is the idea that some data should be freely available to everyone to
use and republish as they wish, with restrictions from copyright, patents or other
mechanisms of control. Wikipedia
Daten öffentlich, frei verfügbar und nutzbar zu machen für mehr Transparenz,
Innovation und Effizienz — das ist Open Data. Open Data Switzerland
13. Open Data Essentials
Legally Open
Published data must be placed in the public domain or under liberal terms of use with
minimal restrictions
Technically Open
Published data must be in electronic formats that are machine readable and preferably
non-proprietary, so that anyone can access and use the data using common, freely
available software tools. Data must be publicly available and accessible on a public
server, without password or firewall restrictions
14. Open Data Guidelines and Principles
Completeness
Primacy
Timeliness
Ease of Physical and Electronic Access
Technically Open
Machine Readable
Technically Open
Non-discrimination
Legally Open
Commonly owned or open standard
Technically Open
Licensing
Legally Open
Permanence
Usage Costs
15. Using Open Data
Access Methods
File downloads
Synchronous API Calls
RESTful
SOAP
Asynchronous
File copies
emails
Example Data Formats
XML
JSON
CSV
XSL/XSLV (spreadsheets)
PDF
RDF (Resource Description Framework)
GeoJSON
OpenDocument
HTML
16. …and the exceptions…
Personally-Identifiable Data
Health Records
Financial Information
Legal Proceedings
Government
Proceedings, pending legal, negotiating
positions, etc.
Materials subject to copyright, patent,
trademark, confidentiality, trade secrets
Employee records, internal directories, IT
system info, proprietary applications
Undue financial/administrative burden
17. Open Data Open Government?
Maybe, Maybe Not…
http://myeinsteinjob.blogspot.com/2014/03/the-difference-between-open-
government.html
https://www.opengovpartnership.org/theme/open-data
http://blogs.worldbank.org/ic4d/differences-between-open-government-data-and-
open-government-data
https://medium.com/@digidickinson/whats-the-difference-between-open-data-and-
open-government-data-8a28eb525d2a
19. Open Data Categories
• Agriculture
• Climate
• Consumer
• Crime and Justice
• Defense
• Ecosystems
• Education
• Energy
• Environment
• Finance
• Government Spending
• Health
• Housing
• Intellectual Property
• Labor and Wages
• Manufacturing
• Maritime
• Ocean
• Pollution
• Public Safety
• Science & Research
• Society
• Transport
20. Global Open Data Index
Global Open Data Index
Scores countries on their Open Data initiatives
21. Open Data Census By City
United States Cities
Australian Cities
Canadian Cities
Germany Cities
Index of Places
22. Demos
Transport for London
National Institute of Health
Gene, search human AFF
Pollution
Honk Kong, air quality, download data from lower right
Finance
24. Problems With Open Data
Too Little Data
Many countries collect but don’t publish their data (Open Data Barometer)
Too Much Data
2600+ Open Data portals world-wide
Costs to collect, to publish, to share, to maintain, to standardize, to process
feedback, etc
Usage Fees
Data Quality
38. Links and References
Open Knowledge International
Open Data Handbook
Open Data 500 Global Network
Centre for Public Impact
California Public Records Act (1968)
California Public Records Act (2004)
Crowd-sourced Open Data
39. Open Data Example Usage
Charles Schwab
CityMapper
Councilmatic
CrashMap UK
Data Machen Schule
FlightAware
New York Times
Open Railway Map
Open Street Map
Safe Eats
Transport for London
Zillow
California Public Records Act (1968): “This definition is intended to cover every conceivable kind of record that is involved in the governmental process and will pertain to any new form of record-keeping instrument as it is developed. Only purely personal information unrelated to 'the conduct of the public's business' could be considered exempt from this definition, i.e., the shopping list phoned from home, the letter to a public officer from a friend which is totally void of reference to governmental activities.”
US Post Office Zip Codes
Data
The data on this website comes from the Chicago City Council legislation site, a system built by Granicus using their Legistar Legislative Management Suite. The Office of the City Clerkmanages the data, and on Councilmatic, each piece of legislation provides a link to its source for reference.
Daily, DataMade collects data from Legistar and the Legistar Web API, which we store using the Open Civic Data standard and platform. Built in collaboration with The Sunlight Foundation, Google, Granicus, and Open North, Open Civic Data standardizes information about people, organizations, events, and bills at any level of government.
Anyone can use the OCD API! This searchable government-transparency tool contains several useful endpoints:
bills: legislation introduced to City Council and their status and sponsors
events: past and upcoming Council and Committee meetings
people: the 50 Aldermen of Chicago and their Committee affiliations
organizations: groups of Aldermen who consider and debate legislation
votes: decisions made by specific Aldermen on bills
https://www.europeandataportal.eu/en/highlights/open-data-schools
A further example comes from Germany where a collaborative Open Data school project was launched in Moers. In Moers, a town with approximately 100,000 inhabitants in Germany’s federal state of North Rhine-Westphalia, the local administration and the Open Knowledge Foundation set out to explore the potential of Open Data in the classroom. The project -- DatenmachenSchule – aims to help develop of a series of software applications, together with the Adolfinum Gymnasium and the students of the Rhine-Whaal University of Applied Sciences. Like the Northern Irish project, the idea here as well is to make school lessons more interactive by employing Open Data.