Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.
Toward a
Frictionless
Data FuturePRESENTED BY
Jo Barratt
jo.barratt@okfn.org (@jobarratt/@okfnlabs)
AT
3rd Research Data N...
International non-profit founded in 2004
Who we are
● Vision
o A world where open knowledge is ubiquitous, enabling
citize...
Widely adopted - over 20 national governments
and 60+ local governments & cities
ckan.org/instances
£4m in 15mins
Frictionless Data is…
● Lightweight specifications for “packaging” datasets
● Integrations for loading datasets into tools...
The Problem
Treemap of issues …
Legal barriers
(open data, sharing
agreements etc)
Data Quality
Hard to find
Interoperability
No tool ...
Cargo loading ~1955
Manual, Slow, Costly
(and Dangerous)
Data is Shipping Pre-
Containerization
Containerization
Standards
- Standards (a few, simple ones)
- Tools (primarily for integration)
- Documentation
- Datasets
Data
Containeriz...
Key Principles
1. Simplicity
2. Web Oriented
3. Existing Tools
4. Open
5 Distributed
Tabular Data Package
Data Package
JSON Table
Schema
CSV
http://frictionlessdata.io/guides/tabular-data-package/
http://fri...
Tooling …
Data Package
Tabular Data Package
JSON Table
Schema
CSV
Tool
e.g. import to R, SQL etc
Tool
e.g. data checking a...
Validation
http://goodtables.okfnlabs.org
Validation
Continuous Validation
● If you’re working in a group, you need continuous validation… for data!
● In < 1 hour, we integrat...
http://uk-25k.datadashboards.io/
Platform Integrations
Partners
View more: http://frictionlessdata.io/partners/
Dataship
YOUR
RESEARCH
ORG
● Project website: http://frictionlessdata.io/
● Specifications: http://specs.frictionlessdata.io/
● GitHub: https://githu...
● Newsletter: http://frictionlessdata.io/get-
involved/#newsletter
● Follow @okfnlabs on Twitter (#frictionlessdata)
Towards a frictionless data future
Prochain SlideShare
Chargement dans…5
×

Towards a frictionless data future

533 vues

Publié le

Presented at the Research Data Network workshop, St Andrews, 30 Nov 2016

Publié dans : Formation
  • Soyez le premier à commenter

  • Soyez le premier à aimer ceci

Towards a frictionless data future

  1. 1. Toward a Frictionless Data FuturePRESENTED BY Jo Barratt jo.barratt@okfn.org (@jobarratt/@okfnlabs) AT 3rd Research Data Network - St Andrews University - 30 November 2016 Licensed under cc-by v3.0 (any jurisdiction)
  2. 2. International non-profit founded in 2004 Who we are ● Vision o A world where open knowledge is ubiquitous, enabling citizens and organizations to create insights that drive change on global and local challenges, combat injustice and inequality and hold governments and corporations to account. ● Mission o Open up all essential, public interest information and see it used to create insight that drives positive change o Build communities, tools and skills to empower individuals and organizations to use open information to create insights that drive change.
  3. 3. Widely adopted - over 20 national governments and 60+ local governments & cities ckan.org/instances
  4. 4. £4m in 15mins
  5. 5. Frictionless Data is… ● Lightweight specifications for “packaging” datasets ● Integrations for loading datasets into tools and platforms relevant to researchers The Goals... ● Introduce a significant, measurable improvement in how research data is shared, consumed, and analyzed. ● Make it easier to maintain and improve data quality.
  6. 6. The Problem
  7. 7. Treemap of issues … Legal barriers (open data, sharing agreements etc) Data Quality Hard to find Interoperability No tool integration
  8. 8. Cargo loading ~1955 Manual, Slow, Costly (and Dangerous)
  9. 9. Data is Shipping Pre- Containerization
  10. 10. Containerization
  11. 11. Standards - Standards (a few, simple ones) - Tools (primarily for integration) - Documentation - Datasets Data Containerization http://www.flickr.com/photos/photohome_uk/1494590209/
  12. 12. Key Principles 1. Simplicity 2. Web Oriented 3. Existing Tools 4. Open 5 Distributed
  13. 13. Tabular Data Package Data Package JSON Table Schema CSV http://frictionlessdata.io/guides/tabular-data-package/ http://frictionlessdata.io/guides/data-package/ http://frictionlessdata.io/guides/json-table-schema/ Tabular Data Package
  14. 14. Tooling … Data Package Tabular Data Package JSON Table Schema CSV Tool e.g. import to R, SQL etc Tool e.g. data checking ala GoodTables
  15. 15. Validation http://goodtables.okfnlabs.org
  16. 16. Validation
  17. 17. Continuous Validation ● If you’re working in a group, you need continuous validation… for data! ● In < 1 hour, we integrated elements (datapackage.json + Python libraries + GoodTables API) to support continuous data validation
  18. 18. http://uk-25k.datadashboards.io/
  19. 19. Platform Integrations
  20. 20. Partners View more: http://frictionlessdata.io/partners/ Dataship YOUR RESEARCH ORG
  21. 21. ● Project website: http://frictionlessdata.io/ ● Specifications: http://specs.frictionlessdata.io/ ● GitHub: https://github.com/frictionlessdata/ ● User Stories: http://frictionlessdata.io/user-stories/
  22. 22. ● Newsletter: http://frictionlessdata.io/get- involved/#newsletter ● Follow @okfnlabs on Twitter (#frictionlessdata)

×